fpga-archive-reports-record · EAGLE4 FPGA archive
Why 7 cards gave +8.9 % and not more: per-phase rank, 4-card F3t vs 7-card record (2026-11-16)
Program 1: ONE Llama-3.2-3B 2-bit EAGLE4 drafter. 4-card = ATT TP2 F3t 0xE4B713F9 x2 + MLPV 0x1342 x2; 7-card = ATT TP4 x4 + MLPV M1361 x2 (talon2, H200 verify).
Motto: a wall is a measured constraint to engineer around, toward the fastest flashable PDI.
Read-only research worker for box1-two-v2. No card, lock, build, flash or kill. talon2 was read as carrick (one tar of four COMBO3 serve logs into box1 scratch).
User question (verbatim): "Any reason why that the increase in card did not bring as much improv as expected., compare per phase timing in .rpt files for a detailed which got faster and which got slower..... in terms of ranked form"
Tags: [M] measured (artifact + md5 named), [D] derived by arithmetic from [M], [P] projection / hypothesis (also collected in the bar below).
Short paths: R/ = reports/results/DRAFTER_SPEED_RESEARCH_20261107/, T/ = reports/results/TRACK1_RECORD_F3T_20261109/.
Calculations and hypothesis (everything that is not a measurement)
- Cycle -> time: x 15.151 ns (drafter
clock_reset_clk_usr_0, 66.001 MHz) on both sides. - 4-card per tree =
occupancy2.att/mlpv_dwell_us_per_tree(FSM sampling, 4.18 us period). 7-card per tree = median of 20dbg_observecensus passes x 15.151 ns. ATT busy (7c) = AP_BUSY - PIPE_WAIT; "uncounted" = busy - phase counters - PIPE_IO. MLPV "G8 proper" = G8 counter - TREE (the G8 counter spans TREE,PER_PHASE_CYCLES_BY_IMAGE_20261110.md:73). - "New waiting" = tree issue->done p50 - ATT busy (MLPV-only time, both parked, issue / read-back).
- Serve step: ms/step = Total Time / Total Verify Step; tree path = (host_ahead_ns + join_wait_ns) / 2,500; A = ms/step - tree path (same instrument as
R/W4_SERVE_BOUND_20261115.md:34-38). - GPU verify inside A = 3.452 ms, measured once with cuda events on F3t at d7 (
reports/results/BATCH_BUBBLES_20261109/INTERLEAVE_BATCHVERIFY_SILICON_20261109.md:76); assumed equal at d11 and on the 7-card set (same target, same V20 verify graph). Not measured on the 7-card set. - COMBO3 card deltas on top of 0x13BE (no census exists): G5_P2W 8 = -495 cyc/item at ctx 566 (unit law,
R/DESIGN_FABLE_ATT_ITEM_20261112.md:12,21) => -157.5 us per d11 tree; R3_EARLY = -353 cyc/item (4-rank sim, same doc:14) => -112.3 us per d11 tree. Sum -269.8 us of ATT busy [P]. - Card -> 0 bound: 4c step >= A + host_ahead = 5.870 ms => <= 1,175 tok/s; COMBO3 >= 5.686 ms => <= 1,215 tok/s [P].
- "Ideal halving" (4 ranks halve ALL ATT busy: -1,478 instead of -604 us/tree), passed through at the measured 84 % (tree / ATT) and the measured 0.52 (serve tree path / census tree, 0x13BE vs F3t): about -0.38 ms more per step => ~990 tok/s on the 0x13BE base [P].
- Lever sizes in section 6 are per-item cycle savings measured on other lineages or in sim, multiplied by 21 items; how much reaches the tree is [P].
0. Answer first
.rptfiles have no cycle counts. A Vivado report gives period, slack, path and area only (R/../PER_PHASE_CYCLES_BY_IMAGE_20261110.md:8). What the.rptfiles DO say: the drafter clock is 66.001 MHz / 15.151 ns on both sets, so fmax contributes 0 to the 4c->7c delta (section 2). Per-phase timing below is from the on-card counters (7c) and the FSM-sampling chain census (4c).- Drafter speed (card us per full d11 tree, 21 items) went 3,384.05 -> 2,877.70 us = -15.0 % (4c F3t+0x1342 vs 7c 0x13BE x4 + M1361 x2) [M]. Doubling the ATT ranks cut ATT busy by only 20.4 % (2,956.5 -> 2,352.6 us), not 50 %, and 84 % of that reached the tree (-506 of -604); the other +98 us is new waiting on the MLPV side.
- Served step 7.812 -> 7.187 ms (-0.625 ms, -8.0 %) = 883.12 -> 961.43 tok/s (+8.9 %) [M]. Of the -0.625 ms, -0.445 ms is the card tree path and -0.181 ms is host-side "A". The card owns only the tree path, which was 31.4 % of the 4-card step; GPU verify (3.45 ms [M d7]) + host (~1.9 ms [D]) are the other 68.6 % and do not change with the card count.
- The bare 4->7 card change was SLOWER: B4 x4 + 0x134D = 859.06 vs 883.12 (-2.7 %) [M]. The +8.9 % came afterwards, roughly half from host levers that remove per-card MMIO (hx7, HX_O7, HX_S7) and half from new card images (G6_REISSUE, G8_OWN, HCUT, COMBO3) (section 5).
- Why the ATT did not halve (ranked): G5 is context sweeps (3 per group at 1 row/cycle), not heads: -27 %; G4 pads 768-lane Q/K/V to 512-lane tiles + a fixed 253-cycle prologue: -28 %; G7G keeps the serial R3 + pre-MLP norm (844 cyc/item): -21 %; G1 walks the full vector on every rank: +6 %; G6 got longer (+12 %) because the 3,072-lane all-reduce crosses two dependent QSFP hops; ATT seam/link time +26 %. The MLPV is still 2 cards: TREE 708 us/tree and G9 270 us/tree are unchanged; only G8 was split (G8_OWN, -314 us).
1. Sources [M]
| what | file | md5 |
|---|---|---|
| 4c card census d11 (F3t 0x13F9 x2 + 0x1342 x2, md 11, w2, K_C 64, 40 trees) | T/config_sweep/census/chain_llama3b_f3t_w2d11_kc64_185202.analysis.json.gz, key occupancy2 |
5b11b7f6 |
7c card census d11 (ATT 0x13BE x4 + MLPV M1361 x2, md 11, w2, K_C 64, ctx 566, 100 trees, 20 census passes, items_per_tree_counted 21/21) |
R/tree4_rca_20261114/m1361_census_md11.json (pairing named at R/TREE4_SILICON_RCA_20261114.md:69-71) |
3230661e |
| 7c census md 7 (0x13BC x4 + 0x1360), cross-check | R/tree4_rca_20261114/bc_1360_census_md7.json |
6cbab262 |
| 4c serve d11 fa3 (the two rounds behind 883.12: 882.38 / 883.87, 2,843 steps, @6.90) | T/config_sweep/serve3/hp_llama3b_PQC_w2d11_humaneval80x256_n1_182607_…log.gz :454,486-490 and …_183008_…log.gz :454,486-490 |
9c9b32fe / f11c5ff1 |
| 7c serve d11 fa3, record 0x13BE + M1361 (2 rounds, lfc + FRESH) | R/W4_SERVE_BOUND_20261115.md table row "rec W2" (STEP_DECOMP.tsv e3943d74) |
– |
| 7c serve d11 fa3, COMBO3 0x13D6 + M1361 (t1/t2/t3 + KV1H2C-off off1, boot 79cc9d1b) | talon2 runs/acwin_20261116/d6/serve/acw_hx7s7_kc64_{t1,t2,t3,off1}_…MEASUREONLY_….log, decomposed with R/w4bound_20261115/w4_step_decomp.py (e4c895a6); box1 scratch copy only |
t1 adee0bf6, t2 0e06f0cc, t3 c92c1f57, off1 a0319bb5 |
| COMBO3 tok/s rows | R/ATT_COMBO_SILICON_20261116.md (COMBO3 table, t1-t3 = 960.23 / 960.97 / 963.10, mean 961.43) |
– |
2. Clock and timing from the .rpt files [M]
| image | role | drafter clock | FINAL_TRUE WNS / WHS | source |
|---|---|---|---|---|
| F3t 0xE4B713F9 | 4c ATT | clock_reset_clk_usr_0 15.151 ns / 66.001 MHz |
+0.040 / +0.011 | T/timing_gates/impl_1/route_report_timing_summary_0.rpt.gz:197 (period), :237 (usr_0 0.000 / +0.006 with margin); R/F3T_SILICON_20261109.md:4 |
| 0x1342 | 4c MLPV | 66 MHz drafter + 132 MHz array (ARRAY_FAST_CLK 1) | – | R/MLPV_AND_STEP_BREAKDOWN_E2b.md:15 |
| COMBO3 0xE4B713D6 | 7c ATT | 66 MHz (PDI …_0xE4B713D6_66m_d2e737d7.pdi) |
+0.044 / +0.010, 0/0 failing | R/ADJUDICATION_0x13D6_0x13DB_20261116.md:14,57 |
| M1361 (0x1361 margin re-route) | 7c MLPV | 66 MHz drafter + 2x array | +0.041 / +0.016 | reports/results/FOURCARD_DESIGN_20261016/4C1_DELIVERY_TRACKER.md:1682 |
| Both censuses count 15.151 ns cycles. No clock moved between the two sets; the scaling question is entirely cycles and waits. |
3. Card per-phase table, one full d11 tree (21 items), us per tree
4c = F3t x2 + 0x1342 x2 (chain census) · 7c = 0x13BE x4 + M1361 x2 (counters, ATT0@17 = the slowest-G6 rank, MLPV0@50). "share" = delta / 7,812 us (the 4c served step).
| # | card | phase | 4c us [M] | 7c us [M] | delta 7c-4c | delta % of phase | share of 4c step |
|---|---|---|---|---|---|---|---|
| 1 | MLPV | G8 exact rescore (proper) | 551.83 | 237.71 | -314.12 | -56.9 % | -4.02 % |
| 2 | ATT | G5 attention | 841.34 | 610.48 | -230.86 | -27.4 % | -2.96 % |
| 3 | ATT | G4 QKV | 630.03 | 452.12 | -177.91 | -28.2 % | -2.28 % |
| 4 | ATT | G7U | 302.43 | 177.22 | -125.21 | -41.4 % | -1.60 % |
| 5 | ATT | G7G (+ serial R3 + pre-MLP norm) | 561.34 | 444.48 | -116.86 | -20.8 % | -1.50 % |
| 6 | ATT | G7P / SILU | 80.74 | 42.32 | -38.42 | -47.6 % | -0.49 % |
| 7 | MLPV | GLM | 73.49 | 71.91 | -1.58 | -2.1 % | -0.02 % |
| 8 | MLPV | G12 kick | 8.01 | 7.00 | -1.01 | – | -0.01 % |
| 9 | MLPV | G11 | 148.05 | 148.27 | +0.22 | 0 % | 0.00 % |
| 10 | MLPV | G9 down | 268.89 | 270.45 | +1.56 | +0.6 % | +0.02 % |
| 11 | MLPV | TREE (G12 scan + top-K merge) | 701.70 | 707.61 | +5.91 | +0.8 % | +0.08 % |
| 12 | ATT | G1 (+R1) | 136.94 | 145.09 | +8.15 | +5.9 % | +0.10 % |
| 13 | MLPV | INGEST + XCH + LVL_SEL/OUT (7c: PIPE_IO 170.54 + uncounted 81.54) | 243.25 | 252.08 | +8.83 | +3.6 % | +0.11 % |
| 14 | ATT | G6 o-proj + TP all-reduce (QSFP hops inside) | 195.07 | 218.14 (ranks 218.1 / 215.7 / 210.4 / 206.3) | +23.07 | +11.8 % | +0.30 % |
| 15 | ATT | seam / link: EMBG + EXPORT + INGEST (7c: PIPE_IO 191.64 + uncounted 71.07) | 208.60 | 262.71 | +54.11 | +25.9 % | +0.69 % |
| 16 | tree | new waiting = tree - ATT busy (MLPV-only, both parked, issue / read-back) [D] | 427.56 | 525.14 | +97.58 | +22.8 % | +1.25 % |
| ATT | ATT busy | 2,956.49 | 2,352.56 | -603.93 | -20.4 % | -7.73 % | |
| MLPV | MLPV busy (mostly hidden under the ATT) | 1,995.22 | 1,695.03 | -300.19 | -15.0 % | – | |
| tree | issue -> done p50 (the drafter speed) | 3,384.05 | 2,877.70 | -506.35 | -15.0 % | -6.48 % |
Per item [D]: ATT 140.8 -> 112.0 us; MLPV 95.0 -> 80.7 us. 4c joint occupancy [M]: both busy 1,798.1, ATT only 1,178.1, MLPV only (exposed) 210.5, both parked 35.9, outside the window 162.6 (occupancy2.joint_us_per_tree, window 3,221.48). No 7c joint split exists (section 7).
Instrument check: the two instruments agree to <0.3 % on unchanged MLPV phases at d7 (G9 852 vs 850, TREE 2,222 vs 2,224 cyc/item, R/../PER_PHASE_CYCLES_BY_IMAGE_20261110.md:78,82); here G9 / G11 / TREE again agree to <1 %.
COMBO3 0x13D6 on top of row "7c" [P, no census]: G5 -157.5 us, G7G -112.3 us (R3_EARLY) => ATT busy ~2,083 us/tree, ATT item ~99 us vs MLPV item 80.7 us + hand-offs. KV1H2C measured on the serve only: tree path 2.112 (off1) -> 2.010 ms (t1-t3) [M, n = 1 vs 3].
4. Step budget per verify step, d11 fa3 w2, humaneval 80x256 (ms)
| term | 4c F3t + 0x1342 (883.12) | 7c 0x13BE + M1361 (938.44) | 7c COMBO3 + M1361 (961.43) | delta 4c -> COMBO3 | tag |
|---|---|---|---|---|---|
| ms / step | 7.812 | 7.364 | 7.187 | -0.625 | [M] |
| tree path = host_ahead + join_wait | 2.455 (31.4 %) | 2.194 (29.8 %) | 2.010 (28.0 %) | -0.445 | [M] |
| - host_ahead (host work under the card) | 0.512 | 0.511 | 0.510 | -0.002 | [M] |
| - join_wait (host blocked on the card reply) | 1.943 | 1.683 | 1.501 | -0.442 | [M] |
| A = step - tree path | 5.358 (68.6 %) | 5.170 | 5.176 (72.0 %) | -0.181 | [M] |
| - GPU verify forward | 3.452 (44.2 %) | 3.452 | 3.452 (48.0 %) | 0 | [M] d7 F3t only; [D] elsewhere |
| - host rest of A (UpdateResult, accept, KV import, arm, root image, doorbell, per-request) | 1.906 (24.4 %) | 1.718 | 1.724 (24.0 %) | -0.181 | [D] |
| card full tree (census, for reference; serve trees stop early) | 3.384 | 2.878 | NOT-MEASURED | -0.506 (to 0x13BE) | [M] |
| Steps 2,843 (4c) vs 2,839 (7c), accept 6.90 vs 6.91, tokens 19,615 both: the tok/s gain (x1.0887) equals the step gain (x1.0870) to 0.2 %. |
5. The 4->7 ladder: where the +8.9 % actually came from (d11 fa3, same workload) [M]
| set | tok/s | delta | lever class | source |
|---|---|---|---|---|
| 4c F3t x2 + 0x1342 x2 | 883.12 | – | – | T/config_sweep/serve3/*182607*, *183008* |
| 7c B4 x4 + 0x134D x2 (no 7c host levers) | 859.06 | -24.06 (-2.7 %) | more cards only | R/C2_6CARD_TP4_20261110.md:14,62 |
| + hx7 (host: fewer pre-doorbell MMIO) | 879.57 (A/B vs 859.47) | +20.1 | host | reports/results/SESSION_CONTEXT_BACKUP_20261111.md:10 |
| 0x13BC (G6_REISSUE) + 0x1360 | 890.30 (4-run mean) | +10.7 | card | R/BA_0x13BA_SILICON_20261112.md:38 |
| 0x1361 MLPV G8_OWN | 902.77 | +12.5 | card | R/MLPV_G8_OWN_SILICON_20261112.md:24 |
| + HX_O7 (host: trims post-tree reads) | 927.65 | +24.9 | host | R/DESIGN_FABLE_START_HALF_O_20261112.md:3 |
| + HX_S7 v | 932.71 | +5.1 | host | R/ATT_13C3_SILICON_20261114.md:7 |
| 0x13BE HCUT ATT + M1361 | 938.94 | +6.2 (+7.1 vs same-boot parent BELL 931.82) | card | R/ADJUDICATION_0x13D5_20261113.md:129 |
| COMBO3 0x13D6 (G5_P2W 8 + R3_EARLY + KV1H2C) + M1361 | 961.43 | +22.5 | card | R/ATT_COMBO_SILICON_20261116.md COMBO3 table |
| Cross-day rows; steps between rows are indicative, not paired A/Bs, except where named. Sum: host levers ~+50, card images ~+52, the card count itself -24. | ||||
The 7c host cost per card is measured: 54 register reads per poll pass vs 28 on 4c, 1.75-2.15 us per posted write + read (R/DESIGN_FABLE_OUTSIDE_WINDOW_O_20261112.md:41-46), and KV_LOAD (427.8 us) + stage_cfg..doorbell (348.1 us) scale with ATT ranks (same :53-58). So the host levers partly repay what the extra cards cost. |
6. Why not more, and the top 3 levers
Facts [M]:
- The card owns 31.4 % of the 4c step (tree path 2.455 of 7.812 ms). The 7c set removed 0.445 ms of it = 18 % of the tree path.
- Inside the card, the full d11 tree fell 15.0 %. ATT busy fell 20.4 %, not 50 %:
- G5 = 3 context sweeps per head group at 1 row/cycle,
attention_head_assembly_pipe.sv:518,543,564; halving heads removes 1 of 4 sweeps (PER_PHASE_CYCLES_BY_IMAGE_20261110.md:158). - G4 = 6 padded tiles instead of 4.5 + A8Q 230 + RET_SER 23 cycles per pass (
same :157). - G7G keeps 844 serial cycles per item (R3 + norm, R3_EARLY 0 on 0x13BE) (
same :160). - G1 does not scale (full-vector norm on every rank) (
same :163). - G6 is the 3,072-lane x 48-bit all-reduce over two dependent QSFP hops; on B4 silicon it ran 472 cyc/item above the ideal-link xsim (
same :69,159).
- G5 = 3 context sweeps per head group at 1 row/cycle,
- The MLPV pair did not grow: TREE (707.6 us), G9 (270.5), G11 + GLM + G12 (227.2) are unchanged to <1 %. G8_OWN halved G8, but G8 sat mostly under the ATT.
- The ATT item (112.0 us) now nearly meets the MLPV item (80.7 us) plus the two hand-offs, so +97.6 us/tree of new waiting appeared.
- Clock: 66 MHz on every image in both sets.
Top 3 levers by modelled card us per d11 tree [P]:
- MLPV TREE (largest unchanged phase, 707.6 us/tree; the MLPV is now co-critical).
- TREE4 measured 2,224 -> 1,124 cyc/item on talon2 silicon (
R/TREE4_SILICON_RCA_20261114.md:74) = -350 us of MLPV time per d11 tree. - The tree gain is capped by the new waiting (525 us/tree) and grows once COMBO3's ATT cuts land; modelled -100..-200 us/tree.
- Blocker: the T4 image's wrong trees are an implementation defect upstream of G8 (that doc, sec. "Answer first" 5), not a TREE4 defect.
- TREE4 measured 2,224 -> 1,124 cyc/item on talon2 silicon (
- ATT serial prologues: NORMFUSE (-387 cyc/item) + G1FUSE (-390) + A8Q_OVL (-194) (
R/DESIGN_FABLE_ATT_ITEM_20261112.md:51-53).- Together -971 cyc/item x 21 = -309 us of ATT busy per tree; at the measured 84 % pass-through, about -260 us/tree.
- NORMFUSE needs R3_EARLY (now in COMBO3). G1FUSE costs +256 DSP and has an open auditor HOLD on its own lineage.
- G6 all-reduce link time. B4's 472 cyc/item over the ideal link x 21 = up to -150 us/tree; G6 + seam/link grew +77 us/tree here.
- Needs the per-rank first-beat timestamp to split hop latency, skew and retransmit. Then a TP_BEAT / one-hop reduce on TP4.
Not card levers, stated as facts: GPU verify 3.452 ms and the host rest of A ~1.72 ms are 72 % of the COMBO3 step. The card -> 0 bound is ~1,215 tok/s.
7. NOT-MEASURED, and how to capture each on talon2
- COMBO3 0x13D6 per-phase census (needed to replace the [P] row in section 3).
ac_steps.sh(4b69ec38) has no census phase. Use the census block ofR/m136Bwin_20261116/ship/m4_steps.sh:156-168(29092fd3), as root, under the window lock.cd $KD6 && OWNER=<window>-census bash ~/eagle4_talon2_4c1/lock/card_run.sh env \ EAGLE4_4C1_TOPOLOGY=7card EAGLE4_4C1_CTRLFSM=0 EAGLE4_4C1_CUDAPACK=0 EAGLE4_7C_RING_CHECK=0 \ EAGLE4_4C1_CARD_BDFS=$BDFS EAGLE4_4C1_BOOT_ROOT=$BOOT_ROOT EAGLE4_4C1_VERIFY_LOG=$EAGLE4_4C1_VERIFY_LOG \ EAGLE4_PAIR_EAGER=1 EAGLE4_PAIR_EARLY_A=1 EAGLE4_PAIR_SPEC0=0 EAGLE4_PAIR_TREE_STOP=0 \ /usr/bin/python3 -I $KD6/scripts/4c1/c4.py --model llama3b bench --disc $RUN/discover.json \ --md 11 --width 2 --kc 64 --trees 100 --census 20 --out $RUN/census/bench_P11w2_md11_w2.json- Do it after the clean golden of the boot. A second arm
--md 7matchesbc_1360_census_md7.json. - Two bench calls of 100 trees each are a few minutes, inside the ~10-min window.
- Do it after the clean golden of the boot. A second arm
- 7-card joint occupancy (ATT parked vs MLPV exposed, i.e. the split of the +97.6 us).
c4_chain --set fsmcannot run on 4 ATT ranks (R/C2_6CARD_TP4_20261110.md:96). Extending it is tooling work, not a window job. - GPU verify forward on the 7-card set. One serve round with
VTIMING=1(the cuda-event instrument behind the 3.452 ms,INTERLEAVE_BATCHVERIFY_SILICON_20261109.md:69-76). It is instrumented and slower, so run it as a separate row. - G6 hop split (latency / rank skew / retransmit). This needs the per-rank first-beat timestamp in RTL (
PER_PHASE_CYCLES_BY_IMAGE_20261110.md:206).
8. HYPOTHESIS: one drafter on all 15 cards (talon2 7 + talon1 8), single stream [D/P, not a measurement]
User (verbatim, via box1-two-v2): "If it helps, we might be able to use all 15 cards as a drafter?" Scope: ONE drafter, never 2 drafters. Everything in this section is a projection. The inputs are the measured per-phase numbers above plus each phase's RTL closed form. Nothing here was run.
8.1 Topology assumed
- ATT TP8 on talon1: 8 cards, a 3-hop hypercube all-reduce, so no all-reduce crosses the chassis. Each rank has half the heads and columns of TP4.
- MLPV x7 on talon2, next to the H200 and the host that publishes the tree.
- One new QSFP crossing per hand-off, each way (ATT seam -> MLPV, level tokens MLPV -> ATT).
- Every step the talon2 host must reach talon1's cards. The KV rows, root input, arm, doorbell, polls and SoA reads go to cards it has no PCIe path to today. That needs a talon1 host process plus a LAN RPC, or card-to-card forwarding over QSFP, which is new RTL.
8.2 Which phases shrink at TP8 (per ATT item, cycles; 0x13BE TP4 base = measured)
| phase | TP4 measured | measured 2->4 ratio | naive (ratio again) | mechanism TP8 [D from RTL closed form] | why |
|---|---|---|---|---|---|
| G1 | 456 | 1.06 | 483 | 456 (fixed) | full-vector norm on every rank |
| G4 | 1,421 | 0.72 | 1,020 | 829 | 384-lane Q/K/V = 1 tile each: 3 x 192 + 253 prologue (2 of 512 lanes idle per 4) |
| G5 | 1,919 | 0.73 | 1,393 | 1,919 (fixed) | 4 heads on 8 engines is still ONE group = 3 context sweeps; fewer heads buys nothing below 8/rank |
| G6 + all-reduce | 686 | 1.12 | 767 | 884 (longer) | array 144 -> 72; exchange 542 over 2 hops -> ~813 over 3 hops |
| G7G | 1,397 | 0.79 | 1,106 | 1,193 | array halves; the 844 serial R3 + norm stays |
| G7U | 557 | 0.59 | 327 | 349 | array halves; the 157 prologue stays |
| G7P | 133 | 0.52 | 70 | 67 | halves |
| seam / link (PIPE_IO + uncounted) | 826 | 1.26 | 1,040 | 826 (best) / 1,040 (realistic) | grew 26 % per doubling, measured |
| ATT item | 7,394 (112.0 us) | 6,204 (94.0 us) | 6,523 (98.8 us), -11.8 % | the second doubling buys about half of the first doubling's -20 % | |
| With COMBO3's levers on top (G5_P2W 8 -495, R3_EARLY -353 [P]), the ATT item is 99.2 us at TP4 and 86.0 us at TP8 (89.2 us with the seam growth). |
8.3 What stays fixed
- GPU verify 3.452 ms + host rest of A ~1.72 ms = 5.18 ms per step. That is 72 % of COMBO3's step.
- The card -> 0 bound is ~1,215 tok/s with any number of cards.
- MLPV phases not sharded in RTL today:
- TREE 2,224 cyc/item;
- G9 850;
- G11 + GLM + G12 714;
- PIPE_IO + uncounted 792.
- Only G8 is split, by token parity (G8_OWN).
- The tree's fixed ends: F = 427.6 us per tree for issue / root / tail / read-back (4c fit, below).
- The hand-off H = 40.6 us per item pair. Fitted [P] from tree = F + 21 x max(a, (a + m + H)/2), the pipeline form of
DESIGN_FABLE_ATT_ITEM_20261112.md:19, on the 4c and 7c census. - Clock: 66 MHz.
8.4 Projection (d11 w2, per verify step; base = COMBO3 7c modelled tree 2,743 us, measured step 7.187 ms)
Card-tree deltas reach the serve at the measured 0.516 (serve tree path / census tree, 0x13BE vs F3t).
| case | ATT item a | MLPV item m | H | tree item | full tree | step | tok/s |
|---|---|---|---|---|---|---|---|
| COMBO3 7c (model base) | 99.2 | 80.7 | 40.6 | 110.2 (loop-bound) | 2,743 | 7.187 [M] | 961.43 [M] |
| Best: TP8 + TREE / G8 / G9 split ideally over 7 MLPVs (new RTL) + 0 host cost | 86.0 | 39.4 | 40.6 | 86.0 (ATT-bound) | 2,233 | 6.924 | ~998 |
| Best + context-split G5 (2 engines per head, new RTL) | 76.0 | 39.4 | 40.6 | 78.0 | 2,065 | 6.838 | ~1,010 |
| Realistic: TP8 with seam growth, only G8 split further, 2 x 3.6 us cross-chassis hop, host +0.1 / +0.2 / +0.3 ms | 89.2 | 72.6 | 47.8 | 104.8 (loop-bound) | 2,628 | 7.23 / 7.33 / 7.43 | ~956 / 943 / 930 |
| Read it as follows: |
- Best case: +4 to +5 % (961 -> ~1,000) for 8 more cards, and it needs two new RTL features (MLPV vocab sharding of TREE / G9, context-split G5).
- Realistic case: flat to -3 %. The ATT gets faster, but the drafter becomes MLPV-loop-bound (TREE is not sharded), so the step barely moves (-0.06 ms). The host cost of reaching another chassis is then larger than the gain.
8.5 Break-even link latency
- Card link (QSFP between chassis): best case is ATT-bound with 27.3 us of slack per item.
- Each extra us per crossing adds 0.5 us per item until (a + m + H + 2d) / 2 meets COMBO3's item of 110.2 us.
- Break-even d ≈ 27 us per crossing.
- Today's QSFP hop costs ~3.6 us (B4 G6: 472 cyc/item over ideal across 2 hops,
PER_PHASE_CYCLES_BY_IMAGE_20261110.md:69). Cable length adds ~5 ns/m. - So the inter-chassis QSFP link is not the risk.
- Host / control path (talon2 host -> talon1 cards): the whole best-case gain is 0.26-0.35 ms per step.
- Any per-step cross-chassis RPC, KV push or extra poll latency above ~0.26 ms makes 15 cards slower than 7.
- Today it needs the LAN plus a second host process.
- Measured anchor for the scale: the bare 4->7 change cost host time and went -2.7 % before the host levers recovered it (section 5).
- Net: the 15-card win is gated on MLPV sharding of TREE (the largest unsharded phase), not on more ATT ranks. Without it the extra ATT ranks are hidden behind the MLPV loop.