# Rethinking the 7-card drafter from scratch: what the ideal architecture is, and what it buys (2026-11-17)

Program 1: ONE drafter on the talon2 V80s, single stream, H200 verifies. Two drafters are covered: Llama-3.2-3B bitnet_721 (record 961.43 tok/s @ 6.91, GPU-only 841.27) and Qwen3.8-27B GPTQ-2 W2A16 (140.15 vs GPU-only 326.83).
Motto: a wall is a measured constraint to engineer around, toward the fastest flashable PDI.
Fable architecture agent for box1-two-v2. Read-only study and a python model. No build, flash, card access, kill, move or removal.
User question (verbatim): "If you rethink entirely the optimal architecture for the 7 cards, what's the ideal way to run these models compared to the current one?"
Tags: **[M]** measured (file:line), **[D]** arithmetic on [M], **[P]** projection or assumption. Every [P] is collected in section 2.
Short paths: `R/` = `reports/results/DRAFTER_SPEED_RESEARCH_20261107/`, `Q/` = `reports/results/QWEN_GPTQ2_20261116/`, `F/` = `reports/results/FOURCARD_DESIGN_20261016/`, `RR/` = `reports/results/`.
Model: `R/arch_rethink_20261117/arch_model.py` (md5 3882bc44), output `arch_model.out` (md5 8294e164). Re-run: `python3 arch_model.py`.

## 0. Answer first

1. **The ideal is not "more tensor-parallel". It is three things in this order: (i) hide the card tree under the GPU verify, (ii) do one batched pass per tree level instead of one pass per tree node, (iii) only then spread every matrix over all cards on a faster array.** Uniform tensor parallel by itself (candidate A) is worth about +0.2 % (6 cards) to +1 % (7 cards) on Llama [P].
2. **Why A alone is nearly worth nothing.** Today's 4 + 2 role split is a two-stage pipeline that is already close to balanced: the attention cards work on node B while the MLP cards finish node A. A uniform image puts both stages on every card, so each node costs the full serial chain. On Llama that is 8,140 cycles per node at 6 cards against 8,240 measured today [P vs M]. A balanced pipeline and a uniform split use the same silicon equally well.
3. **Llama is not MAC-bound.** The array is 2,832 of the 11,860 busy cycles per node (24 %) [D]. The rest is context sweeps (G5), serial norm and quantiser walks, the cross-card sum, the tile-bound vocabulary scan and the hand-offs. So more cards or DSP lanes move little; removing per-node fixed work moves a lot. **Qwen is MAC-bound** (81 % of the ATT node, 77 % of the MLPV node [M]), so B matters there.
4. **Ceiling.** The cards own 2.010 ms of the 7.187 ms Llama step [M]. With a zero-time tree the step is 5.686 ms = 1,215 tok/s [D]. Any architecture lives between 961 and 1,215.
5. **What the full ideal buys on Llama [P]:** one image on 7 cards, level-batched, array at 16,384 MAC per drafter cycle per card, prologue fusions and TREE4: d11 tree 2,622 → about 1,490 us, **about 1,046 tok/s (+8.8 %)**; with a 2x array, 1,296 us and about 1,063. Evolving today's role split with the levers already in flight reaches about 993-1,008 [P]. So the rethink itself is worth roughly +4 to +5 % over the evolved role split.
6. **Qwen.** The ideal card tree is about 1.7-2.5 ms per d7 tree against 5.82 ms measured today [P vs M]. That alone does not reach GPU-only, because the round is 36.5 ms against 21.9 and the tree is only 8.2 ms of it [M]. **Qwen needs draft-ahead (D)**: the verify is 19.14 ms [M], long enough to hide the whole card path. At the fitted humaneval hit rate of about 0.84 [P] the round falls to about 25 ms; a hit rate of 1 gives 22.8 ms, i.e. GPU-only parity. Beating GPU-only then comes from depth: the cards can draft d11-d15 inside the verify window, which the GPU drafter cannot do for free.
7. **Draft-ahead on Llama is marginal**: measured hit rate 15.3 % at d11 (26.0 % at d7) [M]. At zero miss cost that is +3.3 %; at the 0.3 ms miss cost of the earlier study it is break-even (break-even hit rate 0.167) [D].
8. **Binding term once the array shrinks:** on Llama the G5 context sweeps (21 % of a batched level), then the array (27 % at 16,384, 16 % at 32,768), then the two all-reduces (16-19 %) [P]. The all-reduce is not the first wall, but it is the one that grows with N and with a bad topology: 150 → 1,200 cycles per all-reduce moves the result from 1,074 to 1,045 tok/s [P].
9. **Corrections to the brief.** "DSP58 70 % idle" is not what the images show: ATT 0x13BE uses 48.6 % (51 % free), MLPV M1361 uses 69.2 % (31 % free) [M]. Card 99 is unused for cabling and tooling reasons, not a hardware fault (section 1.2).

## 1. Measured inputs

### 1.1 Inter-card link [M]
| quantity | value | source |
|---|---|---|
| PHY per cage | 4 x 25.78 Gb/s Aurora 64B/66B; raw 99.84 Gb/s per direction, PRBS31 600 s | `RR/SEVENCARD_BOARD_20261026/TALON2_QSFP_MAP_20261110.md:56-57`; `F/A2A_0x125D_AC18_REPORT.md:28` |
| beat format | 16 lanes x 48 bit = 768 b; a 3,072-lane vector = 192 beats | `verilog/forks/lutgemm_att_d9_tp4_20261108/link4/tp_ar4_hc.sv:24`; `RR/PER_PHASE_CYCLES_BY_IMAGE_20261110.md:175` |
| link logic clock as built | 200 MHz, 4 cycles per beat = 0.758 beat per drafter cycle; 192 beats ≈ 253 drafter cycles | `RR/PER_PHASE_CYCLES_BY_IMAGE_20261110.md:111`; `R/G6_TP4_EXCHANGE_RCA_20261110.md:101,105` |
| all-reduce of 3,072 lanes, unit xsim of the real engines, 500 ns link | K4 one step 291 cycles; hypercube two steps 360 (+69); +66 cycles per +500 ns of link | `F/TP4_DESIGN_20261018.md:10-16`; `R/ATT_TP4_20261108.md:67` |
| G6 on silicon, 2 ranks (F3t) | 636 cycles per node, array 288 | `RR/PER_PHASE_CYCLES_BY_IMAGE_20261110.md:57,175` |
| G6 on silicon, 4 ranks before / after G6_REISSUE | 1,213 (B4) → 686 (0x13BE), 693 (COMBO3); array 144, **exchange ≈ 549** | `PER_PHASE...:57`; `R/SCALING_4C_VS_7C_PHASE_RANK_20261116.md:66,164`; `R/dw_sweep_combo3_20261116/census_combo3_P11w2.table.txt` (G6 14,552 / 21) |
| one-way hop latency of the shipped stack on silicon | **NOT-MEASURED**; the real-exchange testbench matches silicon at about 406 ns | `R/G6_TP4_EXCHANGE_RCA_20261110.md:115`; `R/G6_REISSUE_FIX_20261110.md:130` |
| small-message round trip on silicon (prototype link, 402 MHz) | 32 B 0.707 us (47 cycles); 6 KiB 1.20 us; 18,432 B 2.19 us | `reports/qsfp_payload_20260908/feasibility_20260909/test_1179_20260910_0056/README.md:11-23` |
| ATT → MLPV hand-off | fitted 40.6 us per node pair (≈ 2,680 cycles); seam 826 cycles per ATT node, 792 per MLPV node | `R/SCALING_4C_VS_7C_PHASE_RANK_20261116.md:168,179,182` |
| retransmits while serving | about 1 per verify step per ATT; cause = 16-deep cage RX FIFO; time cost NOT-MEASURED | `R/C2_6CARD_TP4_20261110.md:118-125`; `R/G6_TP4_EXCHANGE_RCA_20261110.md:131-161` |
| engine slots per card | 4 cages, 3 engine slots (NE = 3); a 4th needs a CSR re-layout | `link4/tp_link4.sv:27`; `R/ATT_TP4_20261108.md:33` |

### 1.2 Topology on talon2, and card 99 [M]
- 13 cables (`TALON2_QSFP_MAP_20261110.md:24-39`). ATT ranks 17 / 3d / bd / e1 form a hypercube (17-3d, bd-e1, 17-bd, 3d-e1). Each rank has one stage cable to an MLPV (17→50, bd→50, 3d→63, e1→63). 50-63 is the MLPV pair link. Two old 4-card cables remain (17-63, 3d-50).
- Every cage on the six drafter cards is occupied. **Card 99 has two cables only (to bd.c3 and e1.c0) and two free cages (c1, c2)** (`:38-39,50`).
- **Why 99 is never opened:** (a) the 7-card kit's allow-list is the six drafter cards; card 7 is "read once and nothing ever writes" (`scripts/4c1/c4_bar.py:52-53`); (b) the ATT all-reduce is a 4-rank hypercube, a fifth rank has no place in it (`R/ATT_TP4_20261108.md:34`); (c) 99 reaches neither MLPV, and the one design that used it (G8 on 99) needs two new cables and a user decision (`R/DESIGN_FABLE_7CARD_V2_20261110.md:127,145`). The card itself read AMC READY on 10-23 (`F/4C1_DELIVERY_TRACKER.md:1103`) and ran an ATT image then (`:1188`). The 11-07 interfacing note still lists 99 as "not ours" (`docs/TALON2_INTERFACING_20261107.md:34`); that ownership line should be confirmed before any plan spends it.

### 1.3 Per-card resources [M] (V80: 321,776 SLICE, 10,848 DSP58, 1,925 URAM, 3,741 BRAM36, 18,870 SLL per cut)
| image | SLICE device | SLICE per die (0 / 1 / 2) | DSP58 | URAM | BRAM | SLL 1↔0 | binding | source |
|---|---|---|---|---|---|---|---|---|
| ATT 0x13BE (COMBO3's parent; no util exists for 0x13D6) | 81.85 % | 98.12 / 68.37 / 77.67 % | 5,267 (48.6 %), 89 % on die 0 | 744 (38.7 %) | 47.7 % | 50.4 % | SLICE on die 0 | `Q/SURVEY_PRECISION_AREA_20261116.md:163` |
| MLPV M1361 | 89.6 % | 81.33 / 93.44 / 94.55 % | 7,504 (69.2 %) | 512 (26.6 %) | 27.7 % | 78.2 % | SLICE on dice 1-2, then DSP on die 1 | `SURVEY...:164`; `R/MLPV_G8_OWN_BUILD_20261111.md:114-120` |
| Qwen ATT 0x139C (L256) | – | 87.13 / 73.06 / 5.12 % | 2,757 | 913 | 1,651.5 | 69.7 % | SLICE die 0 | `Q/BUILDS_20261116.md:34,168-169` |
| Qwen MLPV 0x139D (L256) | – | 97.85 / 93.17 / 88.30 % | 3,458 (31.9 %) | 512 | 1,048 | 60.4 % | SLICE die 0 (over the 95 % gate) | `Q/BUILDS_20261116.md:43,151-152,185` |

DSP owners [M]: ATT attention 3,416 (8 engines x 395) + array 1,635; MLPV array 5,120 (20 per lane) + GLM 2,048 + G8 84 (`SURVEY...:183-188,229`).
Array today [M] (`RR/GEMM_REVISION_20261117/GEMM_REVISION_REQUEST_20261117.md:38-42`): Llama ATT 512 x 16 x 2 at 66 MHz (LUT lane); Llama MLPV 256 x 16 x 2 at 132 MHz (DSP lane); Qwen both 256 x 16 x 2 at 66 MHz. That is 16,384 MAC per drafter cycle per card on Llama and 8,192 on Qwen [D].

### 1.4 Weights per phase [M] (`Q/weight_listing_20261117/*.tsv`)
| phase | Llama bitnet_721 (tsv line) | Qwen GPTQ-2 W2A16 (tsv line) |
|---|---|---|
| G4 q, k, v | 3 x [3072, 6144] = 56.62 M (`:14,16,17`) | 3 x 10240 x 5120 = 157.29 M (`:30,38,42` qweight rows) |
| G6 o | 9.44 M (`:15`) | 26.21 M |
| G7 gate, up | 2 x 25.17 M (`:10,11`) | 2 x 83.89 M |
| G9 down | 25.17 M (`:9`) | 83.89 M |
| **body** | **141.56 M = 35.4 MB at 2 bit** | **435.16 M = 108.8 MB + scales** |
| exact lm_head (HBM) | [128256, 3072] BF16, 788 MB (`:6`) | 2-bit [320, 248320], 317.8 MB + scales |
| cheap lm_head | built offline (not in the file) | rank 128 x 248,320, 2-bit (`efficient_lm_head.qweight [8, 248320]`) |
Heads: Llama 32 x 96, Qwen 40 x 128 (`Q/WHY_SLOWER_LAYERS_PHASES_20261117.md:90`). The whole Llama body is 123 URAM-equivalents per sixth; weight memory is not a wall for a uniform split.

### 1.5 Per-node cycles today [M]
Llama, COMBO3 0xE4B713D6 x4 + M1361 x2, d11 W2, 21 nodes, census `R/dw_sweep_combo3_20261116/census_combo3_P11w2.table.txt` (JSON md5 a0603abb, `R/DW_SWEEP_7CARD_COMBO3_20261116.md:45-49`). Cycles per node = cycles per tree / 21.

| ATT | cyc | of which array [D] | MLPV | cyc | of which array [D] |
|---|---:|---:|---|---:|---:|
| G1 | 456 | 0 | G9 | 850 | 768 |
| G4 | 1,421 | 1,152 | G11 | 466 | 0 |
| G5 (ctx 566, P2W 8) | 1,420 | 0 | GLM + G12 kick | 248 | 0 |
| G6 + all-reduce | 693 | 144 | TREE (251 tiles x 8) | 2,224 | tile-bound |
| G7g (+R3, norm) | 1,044 | 384 | G8 tail | 737 | HBM-port-bound |
| G7u | 557 | 384 | PIPE_IO + uncounted | 792 | 0 |
| G7p | 133 | 0 | | | |
| PIPE_IO + uncounted | 809 | 0 | | | |
| **busy** | **6,533** | **2,064** | **busy** (0x13BE census) | **5,327** | **768** |

Tree: **2.622 ms, 238.3 us per level, 124.8 us = 8,240 cycles per node** [M]. Array closed forms: `RR/PER_PHASE_CYCLES_BY_IMAGE_20261110.md:173-181`. MLPV phases: `R/SCALING_4C_VS_7C_PHASE_RANK_20261116.md:59-65`.
Qwen (xsim of the image sources, ideal link; no silicon census exists): ATT 15,085 root / 15,757 child, array 10,720; MLPV 15,616, G9 5,133, TREE 6,816 (`Q/WHY_SLOWER_LAYERS_PHASES_20261117.md:37-52`).

### 1.6 Step split [M]
| | Llama record (humaneval 80, d11 W2) | Qwen GPTQ-2 (he4 + mt4, d7 W2) |
|---|---|---|
| step / round | 7.187 ms | 36.50 ms (GPU-only 21.9) |
| GPU verify | 3.451 ms p50, measured on this set | 19.14 ms |
| card path | tree path 2.010 = host_ahead 0.510 + join_wait 1.501 | card RPC 12.54 = KV rows 3.84 + tree 8.18 (issue→done 5.82) + IPC 0.52 |
| rest | host 1.72 ms [D] | ≈ 4.8 ms [D] |
| source | `R/SCALING...:81-88`; `R/DW_SWEEP_7CARD_COMBO3_20261116.md:50-53` | `RR/SESSION_CONTEXT_BACKUP_20261117.md:97` |
Qwen humaneval 80: 140.15 @ 7.4166 (52.9 ms per round including prefill) vs 326.83 @ 7.3993 (22.6 ms) (`SESSION_CONTEXT_BACKUP_20261117.md:86-87`).
Host start half on 7 cards: verify result → root doorbell 1,130 us p50, of which KV_LOAD 357 and stage → doorbell 306 (`R/DESIGN_FABLE_START_HALF_O_20261112.md:253-306`).

**The 0.516 factor.** It is a ratio of deltas between two sets: (2.455 − 2.194) / (3.384 − 2.878) = 0.516 (`R/SCALING...:82,71`) [D]. It is not explained in any doc. Two other measured readings exist: served trees hold 15.9 of 21 nodes because of TREE_STOP = 0.757 (`R/W4_SERVE_BOUND_20261115.md:124`); 0x13BE → COMBO3 card-only gave 0.32 (tree path 2.194 → 2.112 with KV1H2C off, census 2.878 → 2.622; n = 1). I use 0.516 and quote the 0.32-0.757 range where it matters.

## 2. Calculations and hypotheses (everything that is not a measurement)

### 2.1 Model
One batch of W sibling nodes on N identical cards, T = MAC per drafter cycle per card:
`batch = once + W x per_sibling`, with `per_sibling = MAC / (N x T) + G7p + tree scan + G8` and `once = G1 + quantiser prologues + R3/norm + G5 + 2 x all-reduce + G11 + GLM + merges + level select`.
Assumptions [P], each tied to a measured part:
- P1. Megatron split: QKV, gate, up split by output; o and down split by input with one all-reduce each. Two wide syncs per node, plus two narrow ones (top-K merge, G8 result). Today has one all-reduce, one pair exchange, and two wide hand-offs.
- P2. All-reduce over 6 or 7 cards = today's 4-rank exchange, 549 cycles, because a 4-cage card reaches every other card in two hops (octahedron at 6, circulant at 7). Needs the 4th engine slot. Batching W adds 192 beats per extra sibling. Sensitivity 150-1,200 in section 3.3.
- P3. Fixed parts carry over at today's measured size: G1 456, A8Q prologues 253 / 157 / 157 / 82, R3 + norm 491 + 12, G11 466, GLM 226, level select + embedding gather 461 (F3t parts, `PER_PHASE...:61,100`).
- P4. G5: 5-6 heads per card is one group of 8 engines = 1,420 cycles. Two siblings in one batch need two groups = 4 sweeps (x 4/3). Qwen: 6-7 heads per card is one group, where today's 10 heads per rank are two.
- P5. Vocabulary scan splits by N at 8 cycles per 256-lane tile (4 with TREE4, measured 2,019 → 1,124, `R/TREE4_SILICON_RCA_20261114.md:74`). G8 candidates split by N (today by 2).
- P6. The hand-off seam (809 + 792 cycles) disappears; narrow merges cost 120 + 60 cycles (silicon small-message round trip is 47).
- P7. "fuse" = G1FUSE −390, NORMFUSE −387, A8Q_OVL −194 (`R/DESIGN_FABLE_ATT_ITEM_20261112.md:48-55`); none is on silicon yet.
- P8. tok/s = 961.43 x 7.187 / step; step moves by 0.516 x the census tree change.
- P9. Area: a unified card must hold attention, array, vocabulary scan, GLM and G8 together. Today the ATT is at 98 % SLICE on its hub die without the last three, and the MLPV at 94 % without attention. **T = 16,384 (256 lanes at 132 MHz, or 512 at 66) is the plausible unified array; T = 32,768 is area-unproven.** DSP for a unified card with a 256-lane DSP array and 6 engines: 5,120 + 2,370 + 2,048 + 84 + ~450 ≈ 10,070 of 10,848 [D]: too tight, so the unified array is the LUT lane.
- P10. Today's role split follows tree = F + 21 x max(a, (a + m + H) / 2) (`R/SCALING...:182`); with the COMBO3 inputs it gives 2,741 us against 2,622 measured, so it is scaled by 0.957.
- Qwen-specific: all-reduce 677 (320 beats), pre-MLP norm 1,377, tile time 14 cycles (6,803 / 485 tiles), G8 2,000, context 300 rows, G5 = sweeps x ctx + 307 (fit of 612 at ctx 50 and 990 at ctx 112).

### 2.2 Draft-ahead arithmetic
- Llama: on a hit the join wait (1.501 ms) disappears, because the tree (2.0-2.6 ms) fits under the 3.45 ms verify. step = 7.187 − h x 1.501 + (1 − h) x c_miss. Break-even h = c_miss / (1.501 + c_miss).
- Qwen: a hit removes the card RPC and its injection, 13.68 ms of the 36.50 ms round. Hit rate on humaneval is a fit, not a histogram: a constant per-level miss of 2.2 % reproduces accept 7.4166 at d7, giving P(full depth) 0.86 and hit 0.84. The same fit on the he4 + mt4 mix gives 0.20, against the measured 21.7 % full-depth share (`Q/D7_HOSTGAP_20261117.md:183`), so the fit form is reasonable.
- A pre-drafted tree is rooted on the drafter's own hidden state, not on the target's. The measured single-chain cost of that kind of rooting is large when used as a root replacement (cos 0.174, `RR/BOLD_SPEC_AHEAD_ANALYSIS_20260810.md:12`); used as a continuation of the same tree it is what every deeper level already does. The accept of continuation trees is NOT-MEASURED.

## 3. Results of the model [P]

### 3.1 Llama, per level (two nodes) and per d11 tree; today = 238.3 us per level, 2,622 us, 961.43 [M]
| architecture | N | T | level us | tree us | tok/s (0.516) | range (0.32-0.757) |
|---|---:|---:|---:|---:|---:|---|
| A uniform, one node at a time | 6 | 16,384 | 246.7 | 2,590 | 963.6 | – |
| A | 7 | 16,384 | 236.1 | 2,479 | 971.4 | 968-976 |
| A + B (2x array) | 6 | 32,768 | 224.8 | 2,361 | 979.8 | – |
| A + B | 7 | 32,768 | 217.4 | 2,283 | 985.4 | 976-997 |
| A + C (level batch) | 6 | 16,384 | 173.3 | 1,857 | 1,017.3 | – |
| A + C | 7 | 16,384 | 162.8 | 1,746 | 1,026.0 | 1,001-1,059 |
| A + C + fuse + TREE4 | 6 | 16,384 | 148.4 | 1,588 | 1,038.5 | – |
| **A + C + fuse + TREE4** | **7** | **16,384** | **139.3** | **1,492** | **1,046.3** | 1,012-1,091 |
| A + B + C + fuse + TREE4 | 6 | 32,768 | 126.6 | 1,359 | 1,057.3 | – |
| A + B + C + fuse + TREE4 | 7 | 32,768 | 120.6 | 1,296 | 1,062.6 | 1,022-1,118 |
| same at 166 MHz | 7 | 41,300 | 116.8 | 1,255 | 1,066.0 | – |
| card time zero | – | – | 0 | 0 | 1,215 | – |

Today's role split with the same levers (law P10):
| step | a | m | tree us | tok/s |
|---|---:|---:|---:|---:|
| COMBO3 + M1361 | 6,533 | 5,327 | 2,622 [M] | 961.4 [M] |
| + NORMFUSE, G1FUSE, A8Q_OVL | 5,562 | 5,327 | 2,474 | 971.8 |
| + ATT array at 132 MHz (−853, xsim, `R/ATT_FASTCLK_20261117.md:45`) | 4,709 | 5,327 | 2,344 | 981.0 |
| + TREE4 | 4,709 | 4,227 | 2,177 | 993.2 |
| + hand-off halved | 4,709 | 4,227 | 1,973 | 1,008.4 |

Reading:
- **A: per node 8,140 (N 6) / 7,791 (N 7) cycles against 8,240 today.** The array shrinks from 2,832 to 1,440 / 1,234 and both seams go, but the node now pays the attention chain and the MLP chain in series.
- **B on Llama is small** (−720 cycles per node at N 6): the array is a quarter of the node.
- **C is the large term**: the fixed work (about 3,000 cycles of norms, walks, GLM, level select) and the latency part of both all-reduces are paid once per level instead of once per node. A level goes from 2 x 7,791 to 10,743 cycles (N 7).
- C has been slower on silicon so far (W2 +26 %, only W4 rt2 −6.1 %) because the MLPV releases the batch one node after another (`R/DESIGN_FABLE_SIBW_W4_REVIVE_20261113.md:302-316`). A unified image has no second role to serialise the batch. That is the real argument for unifying, more than the weight split.
- C costs area: +945 LUT per sibling lane; 512 lanes x W2 placed at 97.57 % SLICE (`R/F5_SIBW_L512_20261108.md:95,133,196-198`). The model above therefore keeps the lane count and pays 2x array time per level; a W2-wide array would cut a further 1,234 cycles per level at T = 16,384.

### 3.2 Where a batched level goes (N 7, T 16,384, fuse + TREE4; 9,196 cycles)
G5 1,893 (21 %) · array 2 x 1,234 (27 %) · two all-reduces 1,482 (16 %) · vocabulary scan 2 x 288 · G8 2 x 211 · other once-per-level 2,208.
At T 32,768 the order is G5 (24 %), all-reduces (19 %), array (16 %). **G5 is the binding term**; its lever is rows per cycle (P2W) or context-split engines, not cards.

### 3.3 All-reduce sensitivity (N 7, T 32,768, batched, fuse + TREE4)
| cycles per all-reduce | 150 | 300 | 549 | 800 | 1,200 |
|---|---:|---:|---:|---:|---:|
| share of the level | 9.5 % | 13.2 % | 18.6 % | 23.4 % | 30.1 % |
| tok/s | 1,073.9 | 1,069.7 | 1,062.6 | 1,055.6 | 1,044.6 |
Floor from measured parts: 192 beats at 0.758 beat per cycle = 253 cycles of serialisation at the 200 MHz link clock. Levers: link logic at 400 MHz, 24-bit partial sums (96 beats), K4-style one-step sum where cabling allows (−69).

### 3.4 Qwen, d7 W2 tree (13 nodes); today: issue→done 5.82 ms in serve [M]
| architecture | N | T | level us | tree ms |
|---|---:|---:|---:|---:|
| A, today's lanes | 6 / 7 | 8,192 | 548.8 / 497.1 | 3.57 / 3.23 |
| A + C | 7 | 8,192 | 418.1 | 2.76 |
| A + B (132 MHz) | 6 / 7 | 16,384 | 414.7 / 382.1 | 2.70 / 2.48 |
| A + B + C + fuse + tile floor | 7 | 16,384 | 260.0 | 1.72 |
| A + B (512 lanes or DSP lanes too) + C + fuse + tile floor | 7 | 32,768 | 202.5 | 1.35 |
- Here A does help against the measured serve (the Qwen loop runs nearly serial: 965 us per level against ATT 229 + MLPV 237 + 237, `Q/D7_TREE_LEVERS_20261117.md:335-339`). Against a well-pipelined role split it is break-even again: max(a, (a + m + H) / 2) = 17,026 cycles per node against 18,111 uniform at N 6 [D].
- B is the Qwen lever: array is 44 % of the level even at T 16,384. The measured building blocks: MLPV DSP lane at 132 MHz shipped on Llama; A16 DSP lane fits 256 lanes on an MLPV (`Q/SURVEY_PRECISION_AREA_20261116.md:98-118`); array-only route at 166 MHz +0.041 (`RR/ARRAY_POC_20260923.md:164-166`); ATT LUT array at 132 MHz routed +0.118 out of context (`R/ATT_FASTCLK_20261117.md:72-74`). Missing RTL: a per-column zero point in the DSP lane (`Q/DESIGN_ARCH_SHIFT.md:207-208`).
- A unified Qwen card halves G5: 6-7 heads per card is one engine group.

## 4. (D) Drafting the next tree under the verify

### 4.1 Hit rates [M]
| | d7 | d11 | source |
|---|---:|---:|---|
| Llama, all drafts accepted (FPGA d11 with TREE_STOP: 22.96 %) | 50.0 % | 24.4 % | `R/accept_gap_20261112/evidence/level_model.out:1-11`; `R/DESIGN_FABLE_ACCEPT_GAP_7C_20261112.md:223-245` |
| accept ends on the best-path leaf | 30.4 % | 17.4 % | `RR/VERIFY_OVERLAP_SPECULATIVE_TREE_20261110.md:49-61` |
| **hit: also bonus token = drafter top-1 there** | **26.0 %** | **15.3 %** | same |
| hit with top-2 continuations | 28.0 % | 16.5 % | same |
| Qwen he4 + mt4, full depth | 21.7 % | – | `Q/D7_HOSTGAP_20261117.md:183` |
| Qwen humaneval 80, full depth | 0.86 [P fit] | – | section 2.2 |
The other 77 % of Llama steps end at a node whose bonus token is not in the tree (41.9 % at a forwarded node whose two children both miss, `R/DESIGN_FABLE_ACCEPT_NOLEVEL_20261112.md:122-130`); nothing drafted in advance can be reused there.

### 4.2 Gain as a function of hit rate [D]
Llama, tok/s:
| miss cost | h 0.153 | h 0.204 | h 0.26 | h 0.4 | h 0.6 | break-even h |
|---|---:|---:|---:|---:|---:|---:|
| 0 | 993 | 1,004 | 1,017 | 1,049 | 1,099 | 0 |
| 0.1 ms | 981 | 993 | 1,006 | 1,040 | 1,092 | 0.062 |
| 0.3 ms | 958 | 971 | 984 | 1,021 | 1,079 | 0.167 |
Qwen round (36.50 ms today, GPU-only 21.9): h 0.23 → 33.4 ms; h 0.5 → 29.7; h 0.84 → 25.0 (1.46x today's speed); h 1 → 22.8.

### 4.3 Verdict
- **Llama: a secondary lever.** +3.3 % at the measured 15.3 % only if abandoning a tree is free on the card. It and a faster tree are substitutes: with a 1,400 us tree the same hit rate adds about 2 %.
- **Qwen: the primary lever.** The verify window is 5.5x longer than on Llama and the card path is 37 % of the round. Depth is the second half of it: at the fitted 2.2 % per-level miss, accept would be 10.7 at d11 and 13.6 at d15 against 7.42 at d7 [P]. Cards can draft those levels inside the 19 ms window; that is how the cards pass GPU-only rather than tie it.
- Earlier studies of this idea ended NO-GO or unbuilt for three engineering reasons, all solvable in RTL: the tree writer could not abandon a tree, the leaf's hidden state was not exported, and the KV rows of accepted tokens arrive after the verify (`RR/VERIFY_OVERLAP_SPECULATIVE_TREE_20261110.md:63-93,112-136`). A "continue" item that extends the running tree below its best leaf removes the first two; lazy row ingest during the next verify removes the third.

## 5. The ideal, stated once
One image on every card, rank negotiated at power-on (the RING8 goal; 7-card autoneg converges in 18.25 us in xsim, `F/PG1_EXPERIMENTS_20261022.md:13`). Each card holds 1/N of every matrix, its own heads with their KV, 1/N of the vocabulary scan and a full copy of the exact head in HBM for its share of the candidates. One card faces the host; the others are fed over QSFP, so host time no longer grows with the card count (the bare 4 → 7 change cost −2.7 % for exactly that reason, `R/SCALING...:95`). The tree runs one batched pass per level. While the H200 verifies, the cards ingest the previous step's rows and keep extending the tree below its best leaf.
Compared with today: same number of wide syncs per node, no hand-off, no idle role, one image instead of two, all 7 cards used, and batching is not blocked by a second role. The cost is area: attention plus vocabulary scan plus GLM on one card at 256 lanes, against a SLICE wall of 94-98 % on the fullest die today.

## 6. Migration in image-sized steps
| # | step | image | modelled gain [P] | first measurement that confirms or kills it |
|---|---|---|---|---|
| 0 | No image: censuses that the model stands on | – | – | (a) per-rank first-beat timestamp inside G6, to split hop latency, skew and retransmit (never measured); (b) Qwen silicon census (`--census 20`), none exists; (c) Qwen humaneval accept histogram and a GPU-only depth sweep d7 / d11 / d15 on the GPTQ-2 drafter. (c) decides whether section 4.3's depth claim is real, with no card |
| 1 | Qwen draft-ahead, host + "continue" item | MLPV + ATT loop sequencer, no array change | round 36.5 → about 25-30 ms | replay of captured Qwen trees: hit rate and accept of continuation trees. Kill if the humaneval hit rate is below about 0.4 |
| 2 | Fast array on both roles (B) | Llama ATT AFC8 0xE4B713EF / 0x137F (building); Qwen MLPV at 132 MHz with the per-column zero | Llama +1 %; Qwen tree −25 to −30 % | AFC8 routed WNS, then its census: G4 1,421 → about 914 per node |
| 3 | MLPV vocabulary scan + G8 on more cards: card 99 as a third MLPV shard (two cables to 50 and 63, or relay through bd / e1) | MLPV image with a 3-way shard map | Llama: TREE 2,224 → about 1,560, G8 737 → 490 per node, ≈ +1 % (the MLPV is co-critical); Qwen more (TREE is 44 % of its MLPV node) | joint census: does tree issue→done fall by more than a third of the MLPV saving? If not, the loop is ATT-bound and this step waits |
| 4 | Level batching on the role split, with the batch release fixed (dynamic groups) | ATT W2 column path (B3 / B5 lineage) + MLPV two-column release | level 238 → about 190 us [P from the −29 % 2-child xsim, `R/SEVENCARD_PROJECTION_20261109.md:51`] | W2 census must be below serial 238 us per level; today's W2 is +26 %. Kill for the role split if the MLPV still serialises |
| 5 | Unified card, fit probe only | out-of-context place of attention (6 engines) + 256-lane array + scan + GLM + G8 | none (a probe) | worst-die SLICE under 95 % and SLL under 18,870. Kill A if it does not place at 256 lanes |
| 6 | Unified image on 6 cards (17, 3d, bd, e1, 50, 63), existing cables + 4th engine slot | one image, rank by negotiation | Llama about 1,017-1,038; Qwen tree about 2.2-2.7 ms | 6-rank all-reduce on silicon under 800 cycles, and a level below 190 us |
| 7 | Seventh card and single host-facing card | same image, 2 cables for 99 (user's hands) | Llama +0.8 % over step 6; Qwen −10 % tree | host start half (1,130 us today) flat against the 6-card value |
Steps 1-4 keep today's two images and are each useful alone. Step 5 costs no flash and decides whether 6 and 7 are worth building. If step 5 fails, the evolved role split of section 3.1 (about 1,000 on Llama) plus step 1 on Qwen is the plan.

## 7. NOT-MEASURED, stated plainly
- One-way hop latency and retransmit time cost of the shipped link on silicon.
- Utilisation of COMBO3 0x13D6 itself (its parent 0x13BE is used above).
- Any Qwen silicon per-phase census; any Qwen humaneval accept histogram; any GPU-only Qwen depth sweep.
- Accept of a tree rooted as a continuation of the previous tree.
- A routed unified card at any lane count; a routed 256-lane x K_PAR 32 array.
- 7-card joint occupancy (how much of the MLPV is exposed), so step 3's gain is a model.
