Talon2 cabling map
11 measured cables; 2 to add (3d c1-e1 c1, bd c1-50 c1) for the 6/7-card run
Three ways to run language models on Alveo V80 FPGAs, side by side, with their measured progress day by day.
Demo 2026-11-10. The product is the drafter: a speculative-decoding draft model running on Alveo V80 cards, with the H200 only verifying. Every number on this tab is a measured value; the line under each table names its source file.
| what | image / config | measured | where |
|---|---|---|---|
| 4-card record, Llama-3.2-3B | ATT F3t 0xE4B713F9 (D9 L512 LUT-GEMM + G5_OVL + G6_REPLAY_DB, FINAL_TRUE WNS +0.040 / WHS +0.011) + MLPV 0x1342; w2 d11 fa3 VTOK 20 K_C 64 theta -2.5, PQ+C | 883.12 tok/s @ 6.90 (882.38 / 883.87, 2,843 steps, 19,615 tokens) | talon2, 11-09 |
| same, depth 7 (record config) | w2 d7 fa4 | 831.14 / 830.20 @ 5.89; 9-round anchor mean 828.06 | talon2, 11-09 |
| same, 15 demo prompts | w2 d11 | 1,059.3 aggregate; d7 953.0 | talon2, 11-09 |
| Qwen3.8-27B drafter, same ATT image (switched by registers) | w2 d7 K_C 64 | 249.62 / 249.04 @ 7.14; GPU-only humaneval 256.67 @ 7.13 | talon2, 11-09 / 11-10 |
| GPU-only best, same day, same H200 | exec-mode -1, d9 fa3 | 837.65 / 832.60 @ 6.63 (d7 fa3 826.06 / 821.41; d11 fa3 823.62; d11 fa4 774.57) | talon2, 11-09 |
| F3t + MLPV 0x134C (today) | w2 d11 fa3 | 880.44 (881.71 / 879.16 / 881.37 / 879.52); goldens == record | talon2, 11-10 |
| box1 demo set (E2b 0xE4B713E2 + 0x1342) | w2 d11 fa3, demo launcher | 820.20 @ 6.90 humaneval-80; 951.19 demo prompts; PQ+C 836.00 / 990.88 | box1, 11-10 |
Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md (md5 5a972bc8) · reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md (3b4f135e) · reports/results/TRACK1_RECORD_F3T_20261109/PER_PROMPT_BENCH_20261109.md (d1e0101e) · reports/results/TRACK1_DAILY_BLOG · reports/results/TALON2_NIGHT_TEST_PLAN_20261110.md (ac3d4759) · reports/results/FOURCARD_DESIGN_20261016/BOX1_E2B_D11_20261110.md (16b7eea0)
The drafter speed is our silicon: the time the cards take to build one draft tree, measured by the cards' own counters. GPU verify (3.45 ms per step) happens after the tree is published and is not part of the drafter.
| card time per draft tree (card only, no host) | d | items | issue -> done p50 (us) | source |
|---|---|---|---|---|
| E2b 0xE4B713E2 + 0x1342 (4 cards) | 7 | 13 | 2,422.11 | F3T_SILICON_20261109.md |
| F3t 0xE4B713F9 + 0x1342 (4 cards), window run | 7 | 13 | 2,220.41 | same |
| F3t + 0x1342, sweep census start / end | 7 | 13 | 2,233.82 / 2,238.64 | F3T_CONFIG_SWEEP_20261109.md 4 |
| F3t + 0x1342 | 9 | 17 | 2,774.44 | same |
| F3t + 0x1342 | 11 | 21 | 3,384.05 | same |
| 6-card TP4 B4 + 0x134D, bench (no tree stop) | 7 | 13 | 2,160 | C2_6CARD_TP4_20261110.md |
| 6-card TP4 B1 + 0x134D, bench | 7 | 13 | 2,428 | C3_B1_6CARD_20261110.md |
| 6-card B4, in serving: tree_server issue -> done (incl. 0.348 ms pre) | 7 | up to 13 (tree stop on) | 2,688 | C2 stepprof sp2 |
Card-process issue -> done per tree while serving humaneval and the demo prompts (stepprof), plus a full-depth bench census. Goldens 0c0bf3bb / bfad1366 == record; d11 texts identical to the talon2 d11 record; bench 100/100 trees, 0 discarded.
| box1 E2b 0xE4B713E2 + MLPV 0x1342, 11-10 18:03-18:11Z | d11 humaneval-80 | d7 humaneval-80 | d11 15 demo prompts |
|---|---|---|---|
| card time per tree in serving, card-process issue -> done, mean / p50 (ms) | 3.658 / 3.969 (n 2,797) | 3.056 / 2.964 (n 3,289) | 4.290 / 4.284 (n 451) |
| of which the on-card loop, launch -> done, mean / p50 (ms) | 2.896 / 3.229 | 2.207 / 2.245 | 3.270 / 3.403 |
| full-depth bench tree, no tree stop, issue -> done p50 | 3.685 ms: 21 forwards, 42 nodes, 175 us/forward | 2.449 ms: 13 forwards, 26 nodes, 188 us/forward | same d11 bench tree (the bench is per depth, not per prompt set) |
| drafted tokens per tree sent to verify | 19.00 (54,017 / 2,843) | 19.00 | 19.00 |
| accept length (incl. bonus token) | 6.899 | 5.892 | 8.724 |
| whole verify step, host wall per FPGA step, mean / p50 (ms) | 8.330 / 8.406 | 7.868 / 7.460 | 9.575 / 9.045 |
| end to end in this window (STEPPROF on) / clean reference | 779.52 / 820.20 tok/s | 703.49 / 763.25 | 869.10 / 951.19 |
box1 per phase at d11 (attention card gold, MLP / vocab card bronze):
| box1 phase, one full-depth tree (median of 20, dbg_observe, 66 MHz) | d11 us | d7 us |
|---|---|---|
| ATT G1 norm | 145.1 | 89.8 |
| ATT G4 QKV | 635.4 | 393.3 |
| ATT G5 attention | 1,219.4 | 752.5 |
| ATT G6 o-proj + TP exchange (01 / 42) | 199.0 / 195.7 | 124.6 / 121.0 |
| ATT G7g gate (+ R3/norm) | 566.7 | 350.8 |
| ATT G7u up | 299.4 | 185.3 |
| ATT G7p SiLU | 83.0 | 51.4 |
| ATT sum | 3,148 | 1,948 |
| MLPV G9 down | 270.5 | 167.4 |
| MLPV G11 norm | 148.3 | 91.8 |
| MLPV GLM + G12 | 78.9 | 48.8 |
| MLPV TREE select | 707.6 | 438.1 |
| MLPV G8 exact rescore (G8 - TREE) | 551.9 | 348.0 |
| MLPV sum (pipelined under the ATT) | 1,757 | 1,094 |
| tree issue -> done p50 | 3,685 | 2,449 |
Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/DRAFT_SPEED_BOX1_20261110.md (md5 ed5b4527) · reports/results/DRAFTER_SPEED_RESEARCH_20261107/draft_speed_box1_20261110/
attention card (ATT) MLP / vocab / tree card (MLPV, runs mostly under the ATT). F3t d7.
| phase (us per tree, F3t + 0x1342, w2 K_C 64, 40 full-depth trees) | d7 | d11 |
|---|---|---|
| ATT G1 | 83.5 | 136.9 |
| ATT G4 QKV | 390.2 | 630.0 |
| ATT G5 attention | 523.4 | 841.3 |
| ATT G6 o-proj + TP exchange | 125.2 | 195.1 |
| ATT G7G | 351.1 | 561.3 |
| ATT G7U | 186.4 | 302.4 |
| ATT SILU + LVL_EMBG | 88.4 | 144.8 |
| ATT INGEST + EXPORT | 73.2 | 144.5 |
| ATT busy | 1,821.5 | 2,956.5 |
| MLPV G9 | 167.8 | 268.9 |
| MLPV G11 + GLM + G12 | 138.5 | 229.6 |
| MLPV TREE | 437.7 | 701.7 |
| MLPV G8 exact rescore | 343.2 | 551.8 |
| MLPV INGEST + XCH + LVL_SEL/OUT | 152.9 | 243.3 |
| MLPV busy (mostly hidden under the ATT) | 1,240.0 | 1,995.2 |
| MLPV only (exposed) | 216.9 | 210.5 |
| tree issue -> done p50 | 2,233.8 | 3,384.1 |
Same phases on the first 6-card TP4 set (4 attention ranks + 2 MLPV):
| 6-card TP4 B4 0xE4B713B4 + 0x134D, d7, per-tree dbg_observe counters | us |
|---|---|
| ATT G1 | 89.8 |
| ATT G4 QKV | 279.9 |
| ATT G5 attention | 376.7 |
| ATT G6 o-proj + TP exchange (ranks 0..3) | 239.0 / 234.2 / 230.6 / 226.7 |
| ATT G7G | 275.2 |
| ATT G7U | 109.7 |
| ATT G7P | 26.2 |
| ATT sum of phases | 1,396.5 |
| MLPV G9 | 167.4 |
| MLPV G11 + GLM + G12 | 140.6 |
| MLPV TREE | 438.1 |
| MLPV G8 exact rescore | 339.8 |
| tree issue -> done p50 (bench, 100 trees) | 2,160 |
Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md · reports/results/DEMO_SPEED_AUDIT_20261110.md (3cb6ce81) 1.3 · reports/results/DRAFTER_SPEED_RESEARCH_20261107/C2_6CARD_TP4_20261110.md (4a9c7b74)
| term (d7) | measured | source |
|---|---|---|
| GPU verify forward (target, 20 tokens, bs 1, CUDA events p50) - outside the drafter | 3.452 ms 4-card F3t; 3.491-3.496 ms 6-card B4 | INTERLEAVE_BATCHVERIFY_SILICON_20261109.md; host model |
| host waits on the card per step (join counter) | 1.330 ms (d11 1.937) 4-card; 1.588 ms 6-card | record log counters; C2 stepprof |
| host work overlapping the card | 0.506 ms (d11 0.515), 4-card | record log host_ahead_ns |
| host chain, UpdateResult end -> ATT doorbell | 0.95 ms (E2b PQ+C profile) | HOST_PRECARD_OPT.md |
| step wall per FPGA step, p50 | 7.109 ms (mean 7.203), 6-card B4 CUDAPACK 1 | C2 stepprof |
| whole run: time / verify steps | 23.60 s / 3,329 steps, 4-card F3t record | record SUMMARY |
Source: reports/results/DEMO_SPEED_AUDIT_20261110.md 1.2 · reports/results/DRAFTER_SPEED_RESEARCH_20261107/DESIGN_7CARD_V2_HOST_MODEL_20261110.md (ecb804d9) · reports/results/DRAFTER_SPEED_RESEARCH_20261107/C2_6CARD_TP4_20261110.md
| run (11-10) | cards | result | verdict | source |
|---|---|---|---|---|
| C2: first 6-card TP4 drafter | ATT B4 0xE4B713B4 x4 + MLPV 0x134D x2 | d11 859.06 @ 6.91, d7 796.13 @ 5.89; full-depth d7 tree 2.160 ms (4-card 2.234); every golden gate PASS, 0 discarded trees, RING-H1 identical 123/123 and 171/171; d7 texts md5 78512f8d == 4-card record in 13/13 rounds; 0 link restarts | works end to end; still below the 4-card record | C2_6CARD_TP4_20261110.md |
| C3: 6-card, ATT B1 | ATT B1 0xE4B713B1 x4 + 0x134D x2 | d11 829.30, d7 773.57; G6 338-354 us vs B4 227-239; tree 2.428 ms | slower than B4; B4 kept | C3_B1_6CARD_20261110.md |
| W1: TPB R1 | ATT 0xE4B713FB + MLPV 0x134C | d11 863.46 @ 6.90 (861.05 / 863.57 / 865.76), d7 813.73, demo 1,031.54 @ 8.72, Qwen 248.40 @ 7.14; goldens == record | -2.2 % vs F3t; F3t back | 4C1_DELIVERY_TRACKER.md 16:18-16:33Z |
| W3: F3t + MLPV 0x134C | ATT F3t + 0x134C | d11 880.44 @ 6.90; d13 879.69 @ 7.08, d15 864.77 @ 7.16 (1 round each) | 0x134C == 0x1342 within noise | TALON2_NIGHT_TEST_PLAN_20261110.md |
| GPU memory as the DMA source (qdma_p2p.ko) | card 50, then serving on F3t | H2C p50, n=200: 48 KB 50.0 us (DtoH + pwrite) -> 29.6 us P2P; 1 MiB 365.7 -> 125.4 us (8.36 GB/s); in serving 882.15 vs 880.44, read-back compare passed 3/3 | correct; neutral at today's payload sizes | qdma_p2p_talon2_20261110/ |
| QDMA multi-queue / multi-card writes | 50, 63, 17, 3d | 48 KB writes: 29,010 calls/s (1 card, 1 queue) -> 106,683 (1 card, 4 queues) -> 202,181 (4 cards x 2 queues) | concurrent writes cost ~1 call latency, not N | 4C1_DELIVERY_TRACKER.md 17:57Z |
| box1 E2b at depth 11 | E2b + 0x1342 (box1) | d11 820.20 @ 6.90 vs d7 763.25 (+7.5 %); 15 demo prompts 951.19 vs 883.99; texts 15/15 identical d7 vs d11 | box1 demo now serves d11 | BOX1_E2B_D11_20261110.md |
| C1: 7-card v2 item-parallel attention (RTL + xsim) | fork c1_itempar_20261110 (F3t + 0x134D parents) | tb_c1 PASS: TAIL seam digests identical to F3t mode, K/V of all 4 HEADs == golden for all 7 nodes, level 27,613 -> 23,900 cycles (-13.4 %) loop modes off; EAGER + TREE_STOP -8.4 %; 5/5 RED arms fired | xsim only; no build yet | C1_ITEMPAR_RTL_20261110.md |
Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/C2_6CARD_TP4_20261110.md · reports/results/DRAFTER_SPEED_RESEARCH_20261107/C3_B1_6CARD_20261110.md (9862103d) · reports/results/DRAFTER_SPEED_RESEARCH_20261107/C1_ITEMPAR_RTL_20261110.md (e5852d20) · reports/results/DRAFTER_SPEED_RESEARCH_20261107/DESIGN_FABLE_7CARD_V2_20261110.md (d57ffb3e) · reports/results/FOURCARD_DESIGN_20261016/4C1_DELIVERY_TRACKER.md
Same H200, same humaneval-80 x256 workload, same configuration on both sides (tree width 2, depth 15, 20 verify tokens, fa4): H200 + 4-card V80 drafter 870.10 tok/s @ 7.12 vs H200 GPU-only 700.71 @ 6.92.
| Target | Llama-3.2-3B-Instruct, fp16, one H200 NVL verifies |
| Drafter | EAGLE4 (bitnet_721 weights) |
| Workload | humaneval, 80 prompts × 256 new tokens, batch 1, greedy |
| Tree | width 2, depth 15, 20 verify tokens |
| Draft attention | fa4 |
| Serving | SGLang 0.5.16, CUDA graphs on |
| FPGA arm | exec-mode 1, 4 × Alveo V80 (2 × ATT 0xE4B713F9, 2 × MLPV 0x1342), K_C 64, tree-stop −2.5 |
| GPU-only arm | exec-mode −1, the H200 drafts for itself |
Same H200, same configuration on both arms: H200 + 4-card V80 drafter 250.27 tok/s @ 7.14 vs H200 GPU-only 256.67 @ 7.13 (0.975×); the 27B target alone decodes 61.08 tok/s. Measured talon2 2026-11-10 (FPGA repeat of 249.62, 11-09).
| Target | Qwen3.8-27B, one H200 NVL verifies |
| Drafter | Qwen EAGLE drafter on the same ATT card image, switched by registers |
| Workload | humaneval, 80 prompts × 256 new tokens, batch 1, greedy |
| Tree | width 2, depth 7, 20 verify tokens |
| Draft attention | fa3 |
| Serving | SGLang 0.5.16, CUDA graphs on |
| FPGA arm | exec-mode 1, 4 × Alveo V80 (2 × ATT 0xE4B713F9, 2 × MLPV 0x1342) |
| GPU-only arm | exec-mode −1, the H200 drafts for itself |
| Target alone | no speculative decoding |
F3t serving texts 80/80 identical to E2b on both models in both rounds; golden lines md5 0c0bf3bb (Llama) / 7aceae28 (Qwen) == record. The 6-card set reproduces the 4-card d7 texts (md5 78512f8d) in 13/13 rounds. The d11 fa3 winner is byte-deterministic across rounds (0/80 texts, 0/80 verify counts differ); GPU-only d11 fa4 is not (2/80 texts, 31/80 verify counts).
Per-phase cycle counters on the cards themselves (dbg_observe / FSM occupancy), not models: G5_OVL cut G5 attention 751.55 -> 519.98 us per tree and the tree 2,422 -> 2,220 us, and every other ATT and MLPV phase stayed within +/-4 us - the lever saved exactly what it was built to save.
First 6-card TP4 boot (4 attention ranks + 2 MLPV): all golden gates PASS, 0 discarded trees, cross-card RING-H1 sums identical 123/123 and 171/171, 33 served rounds, 0 link restarts on any engine; ATT phases fell 22-41 % (G4 -28 %, G5 -28 %, G7U -41 %).
The same ATT image serves the Llama-3.2-3B drafter and the Qwen3.8-27B drafter, switched by registers at runtime: Qwen 249.62 @ 7.14 on F3t, with golden lines identical to the record.
qdma_p2p.ko (companion module, stock QDMA driver untouched): 48 KB H2C 29.6 us vs 50.0 us through host memory, 1 MiB 125.4 vs 365.7 us; correct in serving (read-back compare 3/3).
c4_chain --set fsm occupancy, the 6-card column from per-tree dbg_observe counters (c4_chain cannot run 4 ATT ranks); the G8 counter spans TREE, so G8 = G8 - TREE.Each column is one project. Every row is a day: the blocks under a track are what that track produced that day, newest day on top.
11 measured cables; 2 to add (3d c1-e1 c1, bd c1-50 c1) for the 6/7-card run
Version 7: bridge cable measured on both ends, talon1 PCIE8 cage 1 to talon2 PCIE6 cage 1; link not up yet
Where every byte goes in one draft step of the F3t record pair, replayed from the measured 40-tree census: the tree is ATT-bound and PIPE_WAIT is two MLPV round trips.
Four talon2 V80 cards draft the tree; per step they need the accepted token ids and the target's 3 aux hidden states per token, and return W+W^2(D-1) nodes. A target-FPGA verifier can drive them through the existing card-server process (software only) or, after RTL work, over a direct QSFP link.
One-screen summary: what the 4-card drafter is, what a target-model verifier sends and gets back each step, the two ways to connect, and the key driver files.
11 directions measured and below the F3t record; failure cause and revival conditions listed
F3t (G5 overlap) cuts card G5 by 30.8 % and the tree p50 by 8.3 % (Llama) / 8.5 % (Qwen). Llama serving rises from 812.22-816.61 (E2b, 7 rounds) to 830.09-831.14, outside the noise band; the Qwen change (247.62-249.33 -> 249.05-249.61) is inside it. Outputs are byte-identical to E2b. FINAL_TRUE WNS +0.040 / WHS +0.011 ns.
Cards add about 1% power under decode load; idle share, not per-token power, drives the energy cost.
Daily log of Track 1 progress, code changes, test results and mechanisms. Newest day first.
Fundraising deck + white paper: 4 V80 drafter beside an H200 - drafter forward 2.1x faster and 2.3x lower energy than the H200 drafting itself; box1 record 839.65 tok/s; talon2 11-04 GPU-only faster end to end (826.58 vs 723.09), host-side gap
One dual-model ATT image (0xE4B7129C) + MLPV 0x128B serve both Llama-3.2-3B and the Qwen3.8-27B drafter, switched by registers at runtime, on box1 and on talon2. On the same day and the same H200 (talon2, humaneval 80x256) the 4-card drafter reaches the GPU drafter's accept but is slower than GPU+EAGLE (241.24 vs 251.20 at w2 d7, 216.35 vs 248.23 at w4 d7). v1's lead over the GPU drafter came from throttled 2026-10-20 box1 GPU references and is withdrawn.
With the step-21491 EAGLE3 drafter, Qwen3.8-27B decodes 4.1x faster than the target alone on code (humaneval) and 1.9-2.0x on chat (mt_bench). Moving the drafter from the GPU to 4 V80 FPGA cards keeps 0.96x of the GPU-drafter speed at the 2x7 tree with 74/80 identical outputs. All numbers are measured the same day on the same talon2 H200.
Wave E (ATT KV-commit + R3_EARLY + NORMFUSE, MLPV G11F) runs on all four cards after cold AC#28 with every QSFP link up on its own and boot kits verified 4/4; it measures 518.0 tok/s mean at mt_bench w2 d7 (wave B 513.20) and 857.93-860.83 at humaneval d12, accept identical to wave C at every depth. Golden is bf16-only (measure-only). Next: a second drafter, ATT pair + MLPV-solo 0xE4B712F5.
0x130E (v8 + ring link fix) is a strict improvement and is the new default talon image. CP2BS2 fixed every DSP lane (probe2 clean 8/8), but the done-right 0x126D image is wrong from token 0 on real programs (Qwen and Llama-8B), so the Qwen exactness gap is an image-level functional bug, now under RTL-sim root cause.
Corner prohibit cleared the port-12 lanes on 8/8 cards; lane 7.21 remains, so Qwen runs 2.2x faster but wrong. HBM realized ~229 GB/s/card vs 614 design / 819 device; SLR2 at 94% SLICE is the congestion die.
Best measured batch-1 result is 513.20 tok/s mean (wave B) against 475.0-494.7 H200-only reference; measure-only regime, no exactness claim. box1 data file pending.
Every measured result of the three tracks as of 2026-10-21, failures included. Measured only; projections are on the progress post. Track 1 box1 data pending, from the delivery note.
Current builds, gates and ETAs for 10-21 to 10-23 with projected batch-1 numbers. All projected or derived; nothing here is measured.
Track 2 batch-1: Qwen3.8-27B 17.8-18.9 greedy, up to 21.1 with EAGLE3; Llama-1B 415. Five silicon failures listed, including Llama-8B ring error at token 21.
FOLDRING8 0xE4B712F4 runs 5,837-5,845 tok/s batch-1 bit-exact on 8 cards; the URAM image 0x12F8 failed on silicon with a constant token.
R3_EARLY and NORMFUSE cut G7G by 756 cycles per item bit-exact in xsim; probes B3, B4 and B5 (B4 + G5 on the 132 MHz clock) are building, all quarantined; G1FUSE does not pay with the fold on.
Four ATT cuts and the MLPV LVL_SEL cut are unit/chain-gated GREEN with REDs failing and strap-0 identity; the full-drafter gate-3 arms have no verdict yet (boot preload). Both images carry the RESUME TP link engine: flash both cards of a TP pair together.
Host mode EAGLE4_4C1_FREELEAF_D1 is built and gated; card A/B pending. RTL every-level form in progress.
The two ATT features merge with one RTL line; unit, law, md7, elab and host gates pass with RED arms failing; full-drafter replay and timing pending.
mt_bench W2 d7-d15: FPGA accept 4.5-5.6% below GPU fa3; humaneval MAXNEW 32 rows are capped
Theta and per-level width are flat on mt_bench d7; the drafter's rank-3.. children as free verify leaves add +2.3% (root), +3.7% (depth<=1), +5.0% (every level) accept; hedge h(2,2) >= 0.116 on card trees.
K3 passes the texts gate. K1 cuts the verify graph 10.8% but flips one mt_bench near-tie (HF fp32 picks K1's token); adoption is the user's ruling. K2/K4 Triton GEMMs lose to cuBLAS.
The 4C1 ATT card now elaborates a max-shape attention (16 slots x 128) and switches between the Llama 16x96 and Qwen 12x128 geometry, rope table, softmax scale, KVI block size, eps, SubLN and e_norm fixes from a MODEL_CFG register block at 0x200. Every modular gate is green; the PDI build waits for a qualifying host.
Source and stage differ from 0x1288 only in the four KVCOMMIT files; host-side offline gates pass with RED arms failing; timing pending.
Depth 8-12 at W=2 raises accept to 4.06 at flat step time; W=3 deep loses tok/s. Sweep stopped at w3 d10 V27 (ATT0 0x1CC[1] item error); cold AC needed.
The optimum stays at w2 d7; W=3 hides the cards (margin >= 1.3 ms), W=4 at d6-d7 nearly exposes them. Sweep stopped at a card fault (MLPV 0x1F4).
GLM 256 lanes removes the GLM tail (227 -> 32 cycles); the SUMSQ fold saves 10 cycles. Two images building; no routed timing yet.
The unattributed level time is on-card (seam path 37.6, G8 23.1, LVL_SEL 15.6, LVL_EMBG 15.5, return hop 8.8 us); at w2 siblings overlap and the level is ATT-bound; D1 is not what hides card cuts, the GPU verify is.
v1.2 cuts the MLPV item from 113 to 72 us and the card loop per level by 7-11%, but the record cell moves only +0.3% to +2.2% (inside one sd).
The verify graph has no launch bubbles; its time is GEMM (68%), attention (17%) and norms (6.5%). VQSTRIDE drops two dead kernels per layer, bit-exact; default off.
Our drafter wins accept whenever a tree is allowed (up to 4.22 mt_bench, 7.40 humaneval) but the native MTP head out-accepts it at equal chain depth (d7 3.70 vs 2.85); tok/s at batch 1 is a tie with MTP.
Weights + tables + goldens + host profile for Qwen mode are ready; RTL items: 14-bit scale frac, e_norm wrap on tokens 11/13/15, cheap-head K_C.
URAM is not the limit for a bigger drafter; every shape change needs a rebuild, and the cost is card time on a serial tree-then-verify chain.
HEAD_PAR 16 / G5_HPS 1 halves the G5 slope bit-exact; the full image closes once the KVIFAN ch0 300 MHz class is pinned to SLR0.
Both host cuts are exact and measured; after LEANATTN the T phase is GPU-bound (5.0-5.6 ms H200 span), so further T gains need its GPU work split across card windows.
Two of six margin builds finished and clear +0.03/+0.010; four in route; two timing-RTL builds launched/queued.
Host path and RTL agree bit-for-bit offline and in xsim; silicon A/B waits for the 0xE4B712D1 flash.
VERIFY_PRE stays on the card-published fast path at theta -2.5 and wins; the 4C1 launcher now defaults the five levers on (plain launch 476.67 @ 3.86).
Gate is now label-bound: run_4c1.sh maps 0x1288+0x1287 -> wave-A ref, 0x1288+0x128B -> wave-B ref, and refuses (rc 3) a ref built for other images. References land when the co-sims finish.
The verify's host work runs under the card wait behind a GPU wait on a card-written seq word; step p50 6.350 -> 5.794 ms. Recommended for the record cell.
Adopt theta -2.5 for the 4C1 path (+2.5% mean, paired +11.75, 5/8 rounds); texts unchanged. Golden depth >= 1 still FAILs identically at -2.5.
Three sequencing straps plus 128 GLM lanes cut G11+GLM from 1,656 to 692 cycles per item, bit-exact against the 0x128B RTL in a full-drafter xsim whose base arm reproduces the silicon census (842/814); RED arms bite; OOC lmdown 128 lanes +1.932/+0.015 ns; full-card build running.
The unexplained 0.35 is the host TREE_STOP lever (85%) and the fa4 lever-Y first drafts (12%); survivor selection, verify pick and KV/RoPE at depth are not the gap.
A one-ctypes-access register path in the card process (same mapping, every guard kept) cut the tree pre-launch 0.306 -> 0.232 ms and KV_LOAD 0.237 -> 0.193 ms per step and raised end-to-end throughput 7.0% mean over 6 rotated rounds with identical texts and per-request meta. Replacing the pickled socket with a memfd ring is exact but did not speed the reply on silicon. The host verify launch (~0.4 ms) is untouched.
On the 4-card drafter (ATT 0x1288 + MLPV 0x1287), no variant beat the current stack. fa3 raises accept by 0.02 but loses 6% tok/s; turning the first GPU draft off lowers accept to 3.62; K_C 128/256 run on the cards but accept does not rise (3.64/3.63). Measure-only (depth golden unproven).
T and D steps shrink by 0.7-0.8 ms each; other steps +0.12 ms and admission +0.2 ms cancel it; stays default OFF.
2-bit weights cost 0 (the GPU already runs them); card head at K_C 64 -0.08; all card formats -0.04; card silicon 3.65 is 0.35 below its GPU numerics twin.
The cards need only the prompt K/V from the GPU. Accept is preserved by both on-card variants; the speed of A is bound by per-item host choreography today.
D1 and Y_DEFER are exact and faster; prefix-resident KV with the SGLang radix cache is slower (target prefill +3 ms) and drifts 1 of 50 texts.
The 4-card system (ATT 0xE4B71288 x2 + MLPV 0xE4B71287 x2) runs 441.8 tok/s mean (6 runs) vs the 2-card record 350.16 at identical accept 3.65. The draft phase dropped 46%; the attention side is 1.5x faster per item, the MLP/lm-head side only 1.04x (128 lanes @66 MHz vs the 2-card TAIL's 256 @75 MHz). Measure-only: golden at depth vs bf16 unproven; silicon == RTL through tree level L1.
RTL and host mode are gate-green offline; silicon numbers pending the two PDIs.
Binding = 3 serial passes x 2 head groups. HP16/HPS1 halves the slope; OOC cost +2,360 DSP, +176 URAM. Not built.
The counter is state-gated and correct. Fold ON gives 1,027 zero-wait; silicon's extra 47-70 cycles is consistent with host-row fold-fetch wait (unmeasured on card).
Cards can compute prompt K/V; a W=8 sweep is 2,399 vs 2,324 array cycles. W=2 fits the 0x1288 ATT; W=4/8 at LANES 256 do not (SLICE).
Sibling-per-pair is sound but pays only if the pair round shrinks (hop instead of host relay) or the tree widens; first measurement: wave A w1 vs w2 repeats at d7 and d4.
Start with the KV_LOAD autopilot; GPU-to-card stores need one probe, card-to-GPU push is blocked in the current images.
w2 d7 is the measured best; w3 d7 is the only shape with a path above it and needs two unmeasured parameters; width >= 4 does not pay at depth <= 7.
Batch-1 median of 3 repeats for every model served on talon image 0xE4B71262 (v8, VPU4 on); all exact vs int sim or golden.
Real Llama models run tensor-parallel on all 8 talon V80 cards over the QSFP ring, bit-exact to the integer simulator.
Index of read-only build-report browsers; every file linkable by line.
Read-only copies of the 1262_v8_s2 build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 1261_v6_s1b build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the ure_v2e build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 1263_v9b_s1 build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 1262_v8_s1 build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 125F_kitB build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 125F_kitBM build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the council-2026-10-17 build reports, linkable by line (?f=<run>/<file>#L219-L230).
ATT and MLPV failed only on the link clock; MLPV u_tree_level +0.003 ns misses the +0.03 release gate; ATT GEMV array +0.234 ns is its tightest drafter path.
Read-only copies of the 125F_s1c build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 1256_s1r build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 1254_s2_GOOD build reports, linkable by line (?f=<run>/<file>#L219-L230).
Every number on this page comes from build s3LAattf2: its staged RTL, its synth log, its placed utilization and its post-placement checkpoint. The GEMV phases are URAM-read bound by construction; the build failed timing only on the link clock (fixed in the relaunch).
v8 0xE4B71262 (C_COLS 2 @225, GT island, crumbc fanout) closed timing (+0.003 / true +0.093, 0 failing) and passes on silicon: HBMS ladder 52/52 on all 8 talon cards, ring BIST 16/16 PASS 0 errors
The 4C1 shift moves G7 off the HEAD, halves every weight-bound phase per card with 48-bit partial reduces over QSFP, and replaces the host relay with a 6 KB card-to-card seam; SLR1 (all QSFP GTMs) is the die to keep below 80%.
Record pair: cards 28% of the step; HEAD GEMV-bound (G4+G7 78%), TAIL TREE+G9-bound (81%); 4-card slicing P1-P4 in simulation.
Pick a track and an image to see its block diagram, what it added, its timing and its measured tok/s.
The flashed HBMS image 0xE4B71254 serves real Llama models end to end on silicon with no new PDI: one card up to 8B, eight cards pipeline-parallel for 70B; decode is bound by W8 weight streaming at ~230 GB/s per card.
Every talon card serves 3B bit-exact (54.8-57 tok/s each); one model across 8 cards waits on the HBMS all-reduce to ring bridge (0xE4B71256).
Batch-1 decode on the 0x1254 s2 engine is bound by the 32-port weight-stream beat rate (230 GB/s), not HBM (stall 3.7-3.9%) or DSPs (3 of 4 columns idle).
Same placement idea, different drafting algorithm, model-size ratio and memory system. Their claim (2-10x vs GPU spec decode, measured+modeled) and our batch-1 tok/s are not comparable numbers.
The single-image LUTRAM fold is correct on silicon and its compute time matches simulation; it is one layer's projections, so it cannot generate text by itself.
The drafter's ternary body linears accept BitNet-style int8 activations with no accept loss on the pinned GPU command, so its GEMVs can use the integer (int8 x ternary) contract that constant folding (Track T) assumes.
Running. Causal-fixed v3 goldens for both target models pass every Tier-Q check on the capture device with REDs caught; four numeric contracts handed to HQ and adopted; Tier-E hook ready for the first RING8 builds.
Per-op goldens (every phase G1..G8 + TREE) at ctx 16..564, md 1/2/3, for 4+4 prompts, plus the A->B / B->C / C->A seam regions as .bin/.mem; the checker catches every planted defect.
Reusable AXI4/AXI-Lite/AXIS checkers + randomized BFMs; every check proven to fire. control_regs.sv passes all 60 ABI vectors but answers B/R in the same cycle as the request handshake (A3.3.1 strict dependency FAIL).
Numbers only; the W2 accuracy bar is the USER's call.
Running. Umbrella page of the 8-card FULL TARGET program: Track T -> ternary-make-possible-20261011, Track Q -> qwen27b-ring8-20261011. HQ build plan v2 sets the gates and the L0-L7 ladder; next: L0 HBM readout on card 61, R8F re-floorplan, ring cabling card i P4 -> i+1 P3.
Running (reopened as the Track Q living page). Shape, schema and shell facts measured; HQ e8 transport xsim-exact under faults. Next: R8F re-floorplan + lanes-vs-HBM winner -> 0x1250 C4 build on the laptop.
Program 2: 8-card FULL TARGET; Track Q. Design + shell facts measured; no image built yet (Qwen PDI builds owned by the laptop copy of box1-golden, after the 3STAGE report).
Running. Silicon: 0xE4B71258 bit-exact on V80 card 42. OOC: 4 single-image layers per card close 400 MHz in two SLRs, leaving SLR2 free, so TriLM-99M fits 4 cards. Next: shell integration, on-chip activation path, ring.
Works bit-exact; the wall is area (~1.2 LUT per nonzero bit-serial). Road to a PDI: folded unit PDI, one-layer-per-card routability, sparsity-trained tiny folds; URAM-resident path for 2B4T.
Program 2: 8-card FULL TARGET; Track T. Re-aimed per user: (1) TriLM-99M fully folded on 8 cards, (2) a constant-folded ternary drafter co-resident with the HBM target. BitNet-2B-4T stays the URAM-resident ternary reference.
Diff from the 10-05 architecture post: what each 4-card variant became, the measured verdicts, and why CL4 ring + shape-generic + HBM streaming are the next bets.
Perfetto traces per hardware block from xsim + box1 hostprof; the draft step is a serial host->cards->GPU chain, so overlap beats engine speed-up.
Animated walk-through of the EAGLE4 V80 drafter: HEAD/TAIL split, 256-lane DSP58 GEMV, KV import, LEVEL_PIPE/FG8, 75 MHz TAIL record 339.12 tok/s [M], QSFP D2/CL4 4-card paths [P], and runtime shapes.
Curated pages and files to review with partner hardware leads, grouped by the roadmap decision each one feeds.
Weekly architecture reference: current HEAD/TAIL datapaths, memories, clocks and links, with a selectable timeline of every major shift.
On-chip speculative-decoding drafter: 1.15x (MT-Bench) / 1.33x (HumanEval) vanilla H200 at batch 1; FPGA evaluation system, SGLang, switchable target model
41.58 MiB 2-bit body on-chip (URAM), fp16 lm-head and embeddings in HBM; 4 V80s hold ~0.9B 2-bit weights, ~5x today
A new HEAD (HX1, E3b packed-DSP lanes) and TAIL (FSL, LUTRAM seams) took batch-1 to 323.30 tok/s, exact. The 307.67 miss was a slow-boot host state, now classified in <1 s. SLR1 is the wall; EVAC, FSL and I1 free it.
Host-side fixes and two image generations took the pair from 175.9 to 307.67 tok/s batch-1 (+75%). Depth 3 now beats depth 2 on both benches. The remaining gap to the GPU (~63% of it) is per-level FPGA cost and the strictly serial GPU/FPGA step.
Priming, not depth, was the accept problem; KV import fixed accept at 1.5-1.7x speed. Then host glue, burst seam export and pipelined attention took the pair from 188 to 268 tok/s. What remains is per-level cost (~1.80 ms/level vs <1.72 needed for d3 > d2) and the GPU verify.
Best 1-round pair: A5+F4 KVI+FASTGLUE4 mt w2 d2 268.01 tok/s @2.55 [measured]; 6 pairs catalogued with w x d sweeps.
SGLang never reuses tree KV; the pair needs priming (cheap via PRIME_BURST) plus a host slot fix; RTL child RoPE is +1 (accept-neutral).
Two default-off host levers (RPC_PIPE, CPU_PIN) are byte-identical in mock with 4 REDs caught; they remove host scheduling only, so G < 3 ms still needs SGLang verify work. Silicon A/B pending.
SEAM_WR_PIPE removes the r4 seam write fanout (unit A/B PASS, RED FAILs, scope census confirms placement of the arm). The HEAD clock still cannot rise until attention is pipelined (67.3 MHz ceiling); image 0xE4B711E5 (CD + fix, 66 MHz) and the full-HEAD gate are in flight.
On silicon at steady load, the vocab shortlist gives no gain at width 2 (156-157 tok/s at every tier vs 156.44 full) and at most +3% at width 3 (noise level), at a cost of 2-4 extra verify steps out of ~875 on mt_bench and -2.5% accept on humaneval. FR-Spec shortens the TAIL's TREE walk, but the HEAD is the busier card. Re-test once a faster HEAD (CD/E1/E3) is on silicon. K_C: keep 64.
Priming costs about 1 ms per row on the pair and the HEAD computes for only about 0.23 ms of that; the rest is the host round trip per row. PRIME_BURST runs N rows from one doorbell, with the card loading each next row itself. The xsim verdict is pending.
Launched. The routed builds will answer (1) whether the 256-lane GEMV card still fits with the hop link in the netlist and (2) the open question: does the state card's SLR1 hold attention + the link + the Aurora core. Not serving images.
No single track is 'the' bottleneck any more: the array clock (CD, routed), the HEAD seam/KV-only levers (gated, committed), and the pair's tok/s (112.73 @2.22, +10.4%) all moved today. The two open threads that gate the next silicon step are the width-3 accept shortfall (regime, not proven defect) and the 100 MHz fmax census (G7 seams, not the array, are 60% of the failures).
The HEAD census at 100 MHz is complete and bucketed by block. The G7-up and G7-gate FF activation seams dominate the failing-endpoint count (218,700 + 104,720 = 60% of 542,754), ahead of attention (66,480, but the single worst path at -4.850 ns) and the GEMV array (32,201, -3.878 ns). A faster whole-drafter clock is therefore a seam fan-out + attention-combine project, not primarily an array project — consistent with the array's own 132/133 MHz domain (image CD) already closing cleanly on its ow
Running the array alone at exactly 2x the drafter clock, fed by a per-die BRAM activation cache, is bit-exact and cuts the HEAD item from 33,010 to 18,610 clk in the real drafter. What was pending at the last update — whether the full HEAD image closes timing in context — is now answered: CD is ROUTED, closing both clocks with positive margin, and even freed some resources (LUT 998,363 vs C8's 1,039,733). Silicon flash and the resulting pair tok/s are still pending the user.
Per-die activation replicas + a W-wide serialised return remove the August inter-die-wire wall for the array (measured at W=4, 821 SLL). The new wall is die area: 2,038 LUT per lane-column measured, so W=8 x 128 lanes is ~157% of the routable LUT budget (derived). Once the GEMV sweep is batched, the state card's per-child G12/TREE/G8 tail dominates a level. As of 2026-09-24 ~16:50 EDT the user-mandated target W=8 has NOT been placed at any lane count: the W=8 L64 flow was just admitted on Oleg's
Both halves of the engine split reproduce the single-card drafter bit-for-bit in xsim (md=2, ctx 16, OSPREY 32×96), RED arms fire, and the link transport delivers every hop exactly under back-pressure. No synthesis, timing or silicon yet; the pair speed-up is still a model. Nothing here has regressed as of 2026-09-24 ~16:50 EDT: the separate step-3 joint two-card sim (other agent, fork engine_split_pair_20260923, rebased on this fork's f8f62fb / the OUT-DMA latch fix) shows its old un-rebased co
Adopt the LUT-GEMM (g=2) lane: exact, cycle-neutral, −138K LUT at LANES 128, and now confirmed on the routed netlist — 171 MHz fmax vs 174 (A2 alone) / 177 (broadcast base), so the LUT saving costs no clock that matters at the 66 MHz ship clock. Do not adopt the systolic activation chain for clock at LANES 128. L256 systolic-vs-broadcast: sys8 synth landed, its impl and the base L256 pair are still in the mesh queue.
Next pair = HEAD E3 + TAIL 0xE4B711F1 (CF + T1), gate-green changes only; building
The round-1 HOSTFAST=1 result (+4.8%, 50/50 identical) stands and is the launcher default. Two later missions (I7, I8) added a CUDA-event timing instrument that decomposes every step into V (target-verify GPU), D (tree+KV span) and G (host glue), then measured G directly instead of estimating it: G = 8.58 ms median / 6.00 p10 on the default path. Four new byte-exact cuts (root-image + prep-row CUDA graphs, a cost-fallback fast path, a single-transfer capture) take G to 6.28/5.03, and with the ex
The comparator is ready and its own self-test is green, but no card data has been captured yet — whether the pair's depth-3 accept gap (1.97->2.08 vs GPU's 2.64->3.19) is an op-level defect or the unprimed regime is still an open question until the pending silicon capture runs.
Both levers are byte-exact in every gate, including all 10 full-HEAD arms (29/29 items, 928/928 KV rows, every RED fails exactly as predicted). RTL is committed (verilog 0220bf7, parent 2a6ea01a). Mode 1 (256-bit port alone) removes ~44% of the silicon export time because 1,216 per-cell B round trips dominate; page-capped bursts (mode 2) reach the model's ~40 us, but sit in R2 PIPE_EXP_MULTI's risk class (the 0x1190 accept-1.00 confound, also byte-identical in xsim), so mode 1 is the first image
The two-card protocol works end to end (24 items, 72-node tree, 0 CRC/seq/overflow) and the full-data bit-exact gate is GREEN at L=0 and L=40 (RTL committed verilog f29beae). Hop 3 must carry GLM + G11 (211 wire cycles). The GEMV item FIFO must hold K_F=8. RED arms red_flip and red_tag were harvested 16:10 EDT and both FAIL exactly as designed (non-vacuity proven); red_crd, red_crd1 and the fifo4 deadlock proof are still running on the laptop (slow host, in GEMV weight init since 2026-09-23 21:3
Narrow trees lose less accept than the pair model assumed (width 3 keeps 95.5-97.8% at depth 2, falling only to 90.7-93.4% at depth 15). Grid is COMPLETE: 132/132 runs (fa3 40/40, fa4 40/40, anchors, repeats, controls, bs2, E3 split, contamination re-runs). d2 w3 remains the pair's modelled optimum in every scenario and fa4-retention re-run, with a larger margin (+37% vs the model's original +27-35%). GPU bs2 raises the bar ~1.3-1.5x at unchanged accept; batch-2 verify only costs +6-13% over bat
The pair's drafter datapath is bit/rank-correct against the GPU golden (Test B). The accept shortfall vs GPU (2.22 vs 2.78 baseline; 1.97/2.08 vs 2.64/3.19 at width 3, depth 2/3) is explained (92% at width 8, and closed almost entirely at width 3 by Test A) by one mechanism: SGLang's GPU drafter always attends a fully-primed prompt KV, while the pair's default catch-up-only regime starts every request with an EMPTY drafter KV. Cheap KV priming is the lever, not an RTL fix; its own silicon A/B (c
Width 3 works end to end on silicon: same HEAD, new TAIL 0xE4B711CF, one AXI-lite register. The pair gains +32% tok/s at width 3 (111.1 -> 146.8 tok/s, mt_bench cap-32, unprimed) even though accept drops more than the GPU-only comparator predicts (1.97 vs a 2.10 GPU-shaped expectation). The extra 6% of accept is explained, not measured, by the pair's known unprimed-regime weakness (it loses proportionally more when only the top-3 of 8 ranks survive). W=2 and W=4 measured too (1.58 and 2.07); the
HEAD 0xE4B711C8 (HEAD_PAR 8) + TAIL 0xE4B711CF (TISEQ + TREE_P 16 + runtime width) raised the matched pair benchmark from 102.13 to 112.73 tok/s at unchanged accept 2.22 (+10.4%), by making the HEAD's attention phase ~5.7x faster and the TAIL's tree walk ~1.47x faster on silicon. Width and depth are now characterized end to end: tok/s peaks at width 2 (153.4 @ 1.83), width 3 is close behind (146.8 @ 1.97, +32% tok/s over width 8 for -11% accept) and is the current default candidate; depth costs
Pieces 1 (table builder) and 3 (host mock) remain GATED/committed as before. Piece 2 (HBM fetch RTL) is now COMMITTED (verilog f792e27) with a clean run4 unit gate (latency sweep, parallel abort-drain, RED fired) and a first OOC synthesis census. The full-drafter xsim gate that would prove the fold end-to-end has NOT landed yet: 4 arms have been queued on the box1 mesh since 05:16-05:20 and were still in session-init at 08:00. No HEAD-level or silicon claim is made.
M2 (LANES 256 + E3 packed-DSP lane + locality + RET_SER + TILE_PIPE + FOLD) is bit-exact, closes 166 MHz and needs 2.16× fewer cycles; the drafter around it was not run at 166 MHz. M3 P&R is running (synth phase, box1, checked 2026-09-24 16:47 EDT, not yet routed); DOT_DSP L128 and K_PAR32 L128 P&R are still queued/pending.
On the placed HEAD 0x11C9 every GEMV lane's weight bank is on its own die, but all three activation broadcast replicas and every result consumer sit in SLR1, so most GEMV traffic crosses a die boundary; attention fills 40,205 of SLR1's 98,416 drafter SLICE and the TAIL carries an unused copy of it.
Every research page, newest update first.
Version 7: bridge cable measured on both ends, talon1 PCIE8 cage 1 to talon2 PCIE6 cage 1; link not up yet
Daily log of Track 1 progress, code changes, test results and mechanisms. Newest day first.
11 measured cables; 2 to add (3d c1-e1 c1, bd c1-50 c1) for the 6/7-card run
Where every byte goes in one draft step of the F3t record pair, replayed from the measured 40-tree census: the tree is ATT-bound and PIPE_WAIT is two MLPV round trips.
Four talon2 V80 cards draft the tree; per step they need the accepted token ids and the target's 3 aux hidden states per token, and return W+W^2(D-1) nodes. A target-FPGA verifier can drive them through the existing card-server process (software only) or, after RTL work, over a direct QSFP link.
One-screen summary: what the 4-card drafter is, what a target-model verifier sends and gets back each step, the two ways to connect, and the key driver files.
11 directions measured and below the F3t record; failure cause and revival conditions listed
Pick a track and an image to see its block diagram, what it added, its timing and its measured tok/s.
F3t (G5 overlap) cuts card G5 by 30.8 % and the tree p50 by 8.3 % (Llama) / 8.5 % (Qwen). Llama serving rises from 812.22-816.61 (E2b, 7 rounds) to 830.09-831.14, outside the noise band; the Qwen change (247.62-249.33 -> 249.05-249.61) is inside it. Outputs are byte-identical to E2b. FINAL_TRUE WNS +0.040 / WHS +0.011 ns.
Fundraising deck + white paper: 4 V80 drafter beside an H200 - drafter forward 2.1x faster and 2.3x lower energy than the H200 drafting itself; box1 record 839.65 tok/s; talon2 11-04 GPU-only faster end to end (826.58 vs 723.09), host-side gap
Cards add about 1% power under decode load; idle share, not per-token power, drives the energy cost.
0x130E (v8 + ring link fix) is a strict improvement and is the new default talon image. CP2BS2 fixed every DSP lane (probe2 clean 8/8), but the done-right 0x126D image is wrong from token 0 on real programs (Qwen and Llama-8B), so the Qwen exactness gap is an image-level functional bug, now under RTL-sim root cause.
With the step-21491 EAGLE3 drafter, Qwen3.8-27B decodes 4.1x faster than the target alone on code (humaneval) and 1.9-2.0x on chat (mt_bench). Moving the drafter from the GPU to 4 V80 FPGA cards keeps 0.96x of the GPU-drafter speed at the 2x7 tree with 74/80 identical outputs. All numbers are measured the same day on the same talon2 H200.
One dual-model ATT image (0xE4B7129C) + MLPV 0x128B serve both Llama-3.2-3B and the Qwen3.8-27B drafter, switched by registers at runtime, on box1 and on talon2. On the same day and the same H200 (talon2, humaneval 80x256) the 4-card drafter reaches the GPU drafter's accept but is slower than GPU+EAGLE (241.24 vs 251.20 at w2 d7, 216.35 vs 248.23 at w4 d7). v1's lead over the GPU drafter came from throttled 2026-10-20 box1 GPU references and is withdrawn.
Wave E (ATT KV-commit + R3_EARLY + NORMFUSE, MLPV G11F) runs on all four cards after cold AC#28 with every QSFP link up on its own and boot kits verified 4/4; it measures 518.0 tok/s mean at mt_bench w2 d7 (wave B 513.20) and 857.93-860.83 at humaneval d12, accept identical to wave C at every depth. Golden is bf16-only (measure-only). Next: a second drafter, ATT pair + MLPV-solo 0xE4B712F5.
Corner prohibit cleared the port-12 lanes on 8/8 cards; lane 7.21 remains, so Qwen runs 2.2x faster but wrong. HBM realized ~229 GB/s/card vs 614 design / 819 device; SLR2 at 94% SLICE is the congestion die.
Best measured batch-1 result is 513.20 tok/s mean (wave B) against 475.0-494.7 H200-only reference; measure-only regime, no exactness claim. box1 data file pending.
Track 2 batch-1: Qwen3.8-27B 17.8-18.9 greedy, up to 21.1 with EAGLE3; Llama-1B 415. Five silicon failures listed, including Llama-8B ring error at token 21.
FOLDRING8 0xE4B712F4 runs 5,837-5,845 tok/s batch-1 bit-exact on 8 cards; the URAM image 0x12F8 failed on silicon with a constant token.
R3_EARLY and NORMFUSE cut G7G by 756 cycles per item bit-exact in xsim; probes B3, B4 and B5 (B4 + G5 on the 132 MHz clock) are building, all quarantined; G1FUSE does not pay with the fold on.
Four ATT cuts and the MLPV LVL_SEL cut are unit/chain-gated GREEN with REDs failing and strap-0 identity; the full-drafter gate-3 arms have no verdict yet (boot preload). Both images carry the RESUME TP link engine: flash both cards of a TP pair together.
Host mode EAGLE4_4C1_FREELEAF_D1 is built and gated; card A/B pending. RTL every-level form in progress.
The two ATT features merge with one RTL line; unit, law, md7, elab and host gates pass with RED arms failing; full-drafter replay and timing pending.
mt_bench W2 d7-d15: FPGA accept 4.5-5.6% below GPU fa3; humaneval MAXNEW 32 rows are capped
Our drafter wins accept whenever a tree is allowed (up to 4.22 mt_bench, 7.40 humaneval) but the native MTP head out-accepts it at equal chain depth (d7 3.70 vs 2.85); tok/s at batch 1 is a tie with MTP.
HEAD_PAR 16 / G5_HPS 1 halves the G5 slope bit-exact; the full image closes once the KVIFAN ch0 300 MHz class is pinned to SLR0.
Theta and per-level width are flat on mt_bench d7; the drafter's rank-3.. children as free verify leaves add +2.3% (root), +3.7% (depth<=1), +5.0% (every level) accept; hedge h(2,2) >= 0.116 on card trees.
Weights + tables + goldens + host profile for Qwen mode are ready; RTL items: 14-bit scale frac, e_norm wrap on tokens 11/13/15, cheap-head K_C.
K3 passes the texts gate. K1 cuts the verify graph 10.8% but flips one mt_bench near-tie (HF fp32 picks K1's token); adoption is the user's ruling. K2/K4 Triton GEMMs lose to cuBLAS.
The 4C1 ATT card now elaborates a max-shape attention (16 slots x 128) and switches between the Llama 16x96 and Qwen 12x128 geometry, rope table, softmax scale, KVI block size, eps, SubLN and e_norm fixes from a MODEL_CFG register block at 0x200. Every modular gate is green; the PDI build waits for a qualifying host.
URAM is not the limit for a bigger drafter; every shape change needs a rebuild, and the cost is card time on a serial tree-then-verify chain.
Source and stage differ from 0x1288 only in the four KVCOMMIT files; host-side offline gates pass with RED arms failing; timing pending.
Depth 8-12 at W=2 raises accept to 4.06 at flat step time; W=3 deep loses tok/s. Sweep stopped at w3 d10 V27 (ATT0 0x1CC[1] item error); cold AC needed.
The optimum stays at w2 d7; W=3 hides the cards (margin >= 1.3 ms), W=4 at d6-d7 nearly exposes them. Sweep stopped at a card fault (MLPV 0x1F4).
GLM 256 lanes removes the GLM tail (227 -> 32 cycles); the SUMSQ fold saves 10 cycles. Two images building; no routed timing yet.
The unattributed level time is on-card (seam path 37.6, G8 23.1, LVL_SEL 15.6, LVL_EMBG 15.5, return hop 8.8 us); at w2 siblings overlap and the level is ATT-bound; D1 is not what hides card cuts, the GPU verify is.
v1.2 cuts the MLPV item from 113 to 72 us and the card loop per level by 7-11%, but the record cell moves only +0.3% to +2.2% (inside one sd).
The verify graph has no launch bubbles; its time is GEMM (68%), attention (17%) and norms (6.5%). VQSTRIDE drops two dead kernels per layer, bit-exact; default off.
Both host cuts are exact and measured; after LEANATTN the T phase is GPU-bound (5.0-5.6 ms H200 span), so further T gains need its GPU work split across card windows.
Two of six margin builds finished and clear +0.03/+0.010; four in route; two timing-RTL builds launched/queued.
Host path and RTL agree bit-for-bit offline and in xsim; silicon A/B waits for the 0xE4B712D1 flash.
VERIFY_PRE stays on the card-published fast path at theta -2.5 and wins; the 4C1 launcher now defaults the five levers on (plain launch 476.67 @ 3.86).
Gate is now label-bound: run_4c1.sh maps 0x1288+0x1287 -> wave-A ref, 0x1288+0x128B -> wave-B ref, and refuses (rc 3) a ref built for other images. References land when the co-sims finish.
The verify's host work runs under the card wait behind a GPU wait on a card-written seq word; step p50 6.350 -> 5.794 ms. Recommended for the record cell.
Adopt theta -2.5 for the 4C1 path (+2.5% mean, paired +11.75, 5/8 rounds); texts unchanged. Golden depth >= 1 still FAILs identically at -2.5.
Batch-1 median of 3 repeats for every model served on talon image 0xE4B71262 (v8, VPU4 on); all exact vs int sim or golden.
Three sequencing straps plus 128 GLM lanes cut G11+GLM from 1,656 to 692 cycles per item, bit-exact against the 0x128B RTL in a full-drafter xsim whose base arm reproduces the silicon census (842/814); RED arms bite; OOC lmdown 128 lanes +1.932/+0.015 ns; full-card build running.
The unexplained 0.35 is the host TREE_STOP lever (85%) and the fa4 lever-Y first drafts (12%); survivor selection, verify pick and KV/RoPE at depth are not the gap.
A one-ctypes-access register path in the card process (same mapping, every guard kept) cut the tree pre-launch 0.306 -> 0.232 ms and KV_LOAD 0.237 -> 0.193 ms per step and raised end-to-end throughput 7.0% mean over 6 rotated rounds with identical texts and per-request meta. Replacing the pickled socket with a memfd ring is exact but did not speed the reply on silicon. The host verify launch (~0.4 ms) is untouched.
On the 4-card drafter (ATT 0x1288 + MLPV 0x1287), no variant beat the current stack. fa3 raises accept by 0.02 but loses 6% tok/s; turning the first GPU draft off lowers accept to 3.62; K_C 128/256 run on the cards but accept does not rise (3.64/3.63). Measure-only (depth golden unproven).
T and D steps shrink by 0.7-0.8 ms each; other steps +0.12 ms and admission +0.2 ms cancel it; stays default OFF.
2-bit weights cost 0 (the GPU already runs them); card head at K_C 64 -0.08; all card formats -0.04; card silicon 3.65 is 0.35 below its GPU numerics twin.
The cards need only the prompt K/V from the GPU. Accept is preserved by both on-card variants; the speed of A is bound by per-item host choreography today.
D1 and Y_DEFER are exact and faster; prefix-resident KV with the SGLang radix cache is slower (target prefill +3 ms) and drifts 1 of 50 texts.
The 4-card system (ATT 0xE4B71288 x2 + MLPV 0xE4B71287 x2) runs 441.8 tok/s mean (6 runs) vs the 2-card record 350.16 at identical accept 3.65. The draft phase dropped 46%; the attention side is 1.5x faster per item, the MLP/lm-head side only 1.04x (128 lanes @66 MHz vs the 2-card TAIL's 256 @75 MHz). Measure-only: golden at depth vs bf16 unproven; silicon == RTL through tree level L1.
RTL and host mode are gate-green offline; silicon numbers pending the two PDIs.
Binding = 3 serial passes x 2 head groups. HP16/HPS1 halves the slope; OOC cost +2,360 DSP, +176 URAM. Not built.
The counter is state-gated and correct. Fold ON gives 1,027 zero-wait; silicon's extra 47-70 cycles is consistent with host-row fold-fetch wait (unmeasured on card).
Cards can compute prompt K/V; a W=8 sweep is 2,399 vs 2,324 array cycles. W=2 fits the 0x1288 ATT; W=4/8 at LANES 256 do not (SLICE).
Sibling-per-pair is sound but pays only if the pair round shrinks (hop instead of host relay) or the tree widens; first measurement: wave A w1 vs w2 repeats at d7 and d4.
Start with the KV_LOAD autopilot; GPU-to-card stores need one probe, card-to-GPU push is blocked in the current images.
w2 d7 is the measured best; w3 d7 is the only shape with a path above it and needs two unmeasured parameters; width >= 4 does not pay at depth <= 7.
Real Llama models run tensor-parallel on all 8 talon V80 cards over the QSFP ring, bit-exact to the integer simulator.
Index of read-only build-report browsers; every file linkable by line.
Read-only copies of the 1262_v8_s2 build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 1261_v6_s1b build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the ure_v2e build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 1263_v9b_s1 build reports, linkable by line (?f=<run>/<file>#L219-L230).
v8 0xE4B71262 (C_COLS 2 @225, GT island, crumbc fanout) closed timing (+0.003 / true +0.093, 0 failing) and passes on silicon: HBMS ladder 52/52 on all 8 talon cards, ring BIST 16/16 PASS 0 errors
Read-only copies of the 1262_v8_s1 build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 125F_kitB build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 125F_kitBM build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the council-2026-10-17 build reports, linkable by line (?f=<run>/<file>#L219-L230).
ATT and MLPV failed only on the link clock; MLPV u_tree_level +0.003 ns misses the +0.03 release gate; ATT GEMV array +0.234 ns is its tightest drafter path.
Read-only copies of the 125F_s1c build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 1256_s1r build reports, linkable by line (?f=<run>/<file>#L219-L230).
Read-only copies of the 1254_s2_GOOD build reports, linkable by line (?f=<run>/<file>#L219-L230).
Every number on this page comes from build s3LAattf2: its staged RTL, its synth log, its placed utilization and its post-placement checkpoint. The GEMV phases are URAM-read bound by construction; the build failed timing only on the link clock (fixed in the relaunch).
The 4C1 shift moves G7 off the HEAD, halves every weight-bound phase per card with 48-bit partial reduces over QSFP, and replaces the host relay with a 6 KB card-to-card seam; SLR1 (all QSFP GTMs) is the die to keep below 80%.
Record pair: cards 28% of the step; HEAD GEMV-bound (G4+G7 78%), TAIL TREE+G9-bound (81%); 4-card slicing P1-P4 in simulation.
Running. Silicon: 0xE4B71258 bit-exact on V80 card 42. OOC: 4 single-image layers per card close 400 MHz in two SLRs, leaving SLR2 free, so TriLM-99M fits 4 cards. Next: shell integration, on-chip activation path, ring.
The flashed HBMS image 0xE4B71254 serves real Llama models end to end on silicon with no new PDI: one card up to 8B, eight cards pipeline-parallel for 70B; decode is bound by W8 weight streaming at ~230 GB/s per card.
Every talon card serves 3B bit-exact (54.8-57 tok/s each); one model across 8 cards waits on the HBMS all-reduce to ring bridge (0xE4B71256).
Batch-1 decode on the 0x1254 s2 engine is bound by the 32-port weight-stream beat rate (230 GB/s), not HBM (stall 3.7-3.9%) or DSPs (3 of 4 columns idle).
The single-image LUTRAM fold is correct on silicon and its compute time matches simulation; it is one layer's projections, so it cannot generate text by itself.
Same placement idea, different drafting algorithm, model-size ratio and memory system. Their claim (2-10x vs GPU spec decode, measured+modeled) and our batch-1 tok/s are not comparable numbers.
Numbers only; the W2 accuracy bar is the USER's call.
The drafter's ternary body linears accept BitNet-style int8 activations with no accept loss on the pinned GPU command, so its GEMVs can use the integer (int8 x ternary) contract that constant folding (Track T) assumes.
Running. Causal-fixed v3 goldens for both target models pass every Tier-Q check on the capture device with REDs caught; four numeric contracts handed to HQ and adopted; Tier-E hook ready for the first RING8 builds.
Running. Umbrella page of the 8-card FULL TARGET program: Track T -> ternary-make-possible-20261011, Track Q -> qwen27b-ring8-20261011. HQ build plan v2 sets the gates and the L0-L7 ladder; next: L0 HBM readout on card 61, R8F re-floorplan, ring cabling card i P4 -> i+1 P3.
Running (reopened as the Track Q living page). Shape, schema and shell facts measured; HQ e8 transport xsim-exact under faults. Next: R8F re-floorplan + lanes-vs-HBM winner -> 0x1250 C4 build on the laptop.
Diff from the 10-05 architecture post: what each 4-card variant became, the measured verdicts, and why CL4 ring + shape-generic + HBM streaming are the next bets.
Perfetto traces per hardware block from xsim + box1 hostprof; the draft step is a serial host->cards->GPU chain, so overlap beats engine speed-up.
Animated walk-through of the EAGLE4 V80 drafter: HEAD/TAIL split, 256-lane DSP58 GEMV, KV import, LEVEL_PIPE/FG8, 75 MHz TAIL record 339.12 tok/s [M], QSFP D2/CL4 4-card paths [P], and runtime shapes.
Curated pages and files to review with partner hardware leads, grouped by the roadmap decision each one feeds.
Weekly architecture reference: current HEAD/TAIL datapaths, memories, clocks and links, with a selectable timeline of every major shift.
On-chip speculative-decoding drafter: 1.15x (MT-Bench) / 1.33x (HumanEval) vanilla H200 at batch 1; FPGA evaluation system, SGLang, switchable target model
41.58 MiB 2-bit body on-chip (URAM), fp16 lm-head and embeddings in HBM; 4 V80s hold ~0.9B 2-bit weights, ~5x today
Host-side fixes and two image generations took the pair from 175.9 to 307.67 tok/s batch-1 (+75%). Depth 3 now beats depth 2 on both benches. The remaining gap to the GPU (~63% of it) is per-level FPGA cost and the strictly serial GPU/FPGA step.
A new HEAD (HX1, E3b packed-DSP lanes) and TAIL (FSL, LUTRAM seams) took batch-1 to 323.30 tok/s, exact. The 307.67 miss was a slow-boot host state, now classified in <1 s. SLR1 is the wall; EVAC, FSL and I1 free it.
Priming, not depth, was the accept problem; KV import fixed accept at 1.5-1.7x speed. Then host glue, burst seam export and pipelined attention took the pair from 188 to 268 tok/s. What remains is per-level cost (~1.80 ms/level vs <1.72 needed for d3 > d2) and the GPU verify.
Best 1-round pair: A5+F4 KVI+FASTGLUE4 mt w2 d2 268.01 tok/s @2.55 [measured]; 6 pairs catalogued with w x d sweeps.
SGLang never reuses tree KV; the pair needs priming (cheap via PRIME_BURST) plus a host slot fix; RTL child RoPE is +1 (accept-neutral).
Two default-off host levers (RPC_PIPE, CPU_PIN) are byte-identical in mock with 4 REDs caught; they remove host scheduling only, so G < 3 ms still needs SGLang verify work. Silicon A/B pending.
SEAM_WR_PIPE removes the r4 seam write fanout (unit A/B PASS, RED FAILs, scope census confirms placement of the arm). The HEAD clock still cannot rise until attention is pipelined (67.3 MHz ceiling); image 0xE4B711E5 (CD + fix, 66 MHz) and the full-HEAD gate are in flight.
On silicon at steady load, the vocab shortlist gives no gain at width 2 (156-157 tok/s at every tier vs 156.44 full) and at most +3% at width 3 (noise level), at a cost of 2-4 extra verify steps out of ~875 on mt_bench and -2.5% accept on humaneval. FR-Spec shortens the TAIL's TREE walk, but the HEAD is the busier card. Re-test once a faster HEAD (CD/E1/E3) is on silicon. K_C: keep 64.
Priming costs about 1 ms per row on the pair and the HEAD computes for only about 0.23 ms of that; the rest is the host round trip per row. PRIME_BURST runs N rows from one doorbell, with the card loading each next row itself. The xsim verdict is pending.
Launched. The routed builds will answer (1) whether the 256-lane GEMV card still fits with the hop link in the netlist and (2) the open question: does the state card's SLR1 hold attention + the link + the Aurora core. Not serving images.
No single track is 'the' bottleneck any more: the array clock (CD, routed), the HEAD seam/KV-only levers (gated, committed), and the pair's tok/s (112.73 @2.22, +10.4%) all moved today. The two open threads that gate the next silicon step are the width-3 accept shortfall (regime, not proven defect) and the 100 MHz fmax census (G7 seams, not the array, are 60% of the failures).
The HEAD census at 100 MHz is complete and bucketed by block. The G7-up and G7-gate FF activation seams dominate the failing-endpoint count (218,700 + 104,720 = 60% of 542,754), ahead of attention (66,480, but the single worst path at -4.850 ns) and the GEMV array (32,201, -3.878 ns). A faster whole-drafter clock is therefore a seam fan-out + attention-combine project, not primarily an array project — consistent with the array's own 132/133 MHz domain (image CD) already closing cleanly on its ow
Running the array alone at exactly 2x the drafter clock, fed by a per-die BRAM activation cache, is bit-exact and cuts the HEAD item from 33,010 to 18,610 clk in the real drafter. What was pending at the last update — whether the full HEAD image closes timing in context — is now answered: CD is ROUTED, closing both clocks with positive margin, and even freed some resources (LUT 998,363 vs C8's 1,039,733). Silicon flash and the resulting pair tok/s are still pending the user.
Per-die activation replicas + a W-wide serialised return remove the August inter-die-wire wall for the array (measured at W=4, 821 SLL). The new wall is die area: 2,038 LUT per lane-column measured, so W=8 x 128 lanes is ~157% of the routable LUT budget (derived). Once the GEMV sweep is batched, the state card's per-child G12/TREE/G8 tail dominates a level. As of 2026-09-24 ~16:50 EDT the user-mandated target W=8 has NOT been placed at any lane count: the W=8 L64 flow was just admitted on Oleg's
Both halves of the engine split reproduce the single-card drafter bit-for-bit in xsim (md=2, ctx 16, OSPREY 32×96), RED arms fire, and the link transport delivers every hop exactly under back-pressure. No synthesis, timing or silicon yet; the pair speed-up is still a model. Nothing here has regressed as of 2026-09-24 ~16:50 EDT: the separate step-3 joint two-card sim (other agent, fork engine_split_pair_20260923, rebased on this fork's f8f62fb / the OUT-DMA latch fix) shows its old un-rebased co
Adopt the LUT-GEMM (g=2) lane: exact, cycle-neutral, −138K LUT at LANES 128, and now confirmed on the routed netlist — 171 MHz fmax vs 174 (A2 alone) / 177 (broadcast base), so the LUT saving costs no clock that matters at the 66 MHz ship clock. Do not adopt the systolic activation chain for clock at LANES 128. L256 systolic-vs-broadcast: sys8 synth landed, its impl and the base L256 pair are still in the mesh queue.
Next pair = HEAD E3 + TAIL 0xE4B711F1 (CF + T1), gate-green changes only; building
The round-1 HOSTFAST=1 result (+4.8%, 50/50 identical) stands and is the launcher default. Two later missions (I7, I8) added a CUDA-event timing instrument that decomposes every step into V (target-verify GPU), D (tree+KV span) and G (host glue), then measured G directly instead of estimating it: G = 8.58 ms median / 6.00 p10 on the default path. Four new byte-exact cuts (root-image + prep-row CUDA graphs, a cost-fallback fast path, a single-transfer capture) take G to 6.28/5.03, and with the ex
The comparator is ready and its own self-test is green, but no card data has been captured yet — whether the pair's depth-3 accept gap (1.97->2.08 vs GPU's 2.64->3.19) is an op-level defect or the unprimed regime is still an open question until the pending silicon capture runs.
Both levers are byte-exact in every gate, including all 10 full-HEAD arms (29/29 items, 928/928 KV rows, every RED fails exactly as predicted). RTL is committed (verilog 0220bf7, parent 2a6ea01a). Mode 1 (256-bit port alone) removes ~44% of the silicon export time because 1,216 per-cell B round trips dominate; page-capped bursts (mode 2) reach the model's ~40 us, but sit in R2 PIPE_EXP_MULTI's risk class (the 0x1190 accept-1.00 confound, also byte-identical in xsim), so mode 1 is the first image
The two-card protocol works end to end (24 items, 72-node tree, 0 CRC/seq/overflow) and the full-data bit-exact gate is GREEN at L=0 and L=40 (RTL committed verilog f29beae). Hop 3 must carry GLM + G11 (211 wire cycles). The GEMV item FIFO must hold K_F=8. RED arms red_flip and red_tag were harvested 16:10 EDT and both FAIL exactly as designed (non-vacuity proven); red_crd, red_crd1 and the fifo4 deadlock proof are still running on the laptop (slow host, in GEMV weight init since 2026-09-23 21:3
Narrow trees lose less accept than the pair model assumed (width 3 keeps 95.5-97.8% at depth 2, falling only to 90.7-93.4% at depth 15). Grid is COMPLETE: 132/132 runs (fa3 40/40, fa4 40/40, anchors, repeats, controls, bs2, E3 split, contamination re-runs). d2 w3 remains the pair's modelled optimum in every scenario and fa4-retention re-run, with a larger margin (+37% vs the model's original +27-35%). GPU bs2 raises the bar ~1.3-1.5x at unchanged accept; batch-2 verify only costs +6-13% over bat
The pair's drafter datapath is bit/rank-correct against the GPU golden (Test B). The accept shortfall vs GPU (2.22 vs 2.78 baseline; 1.97/2.08 vs 2.64/3.19 at width 3, depth 2/3) is explained (92% at width 8, and closed almost entirely at width 3 by Test A) by one mechanism: SGLang's GPU drafter always attends a fully-primed prompt KV, while the pair's default catch-up-only regime starts every request with an EMPTY drafter KV. Cheap KV priming is the lever, not an RTL fix; its own silicon A/B (c
Width 3 works end to end on silicon: same HEAD, new TAIL 0xE4B711CF, one AXI-lite register. The pair gains +32% tok/s at width 3 (111.1 -> 146.8 tok/s, mt_bench cap-32, unprimed) even though accept drops more than the GPU-only comparator predicts (1.97 vs a 2.10 GPU-shaped expectation). The extra 6% of accept is explained, not measured, by the pair's known unprimed-regime weakness (it loses proportionally more when only the top-3 of 8 ranks survive). W=2 and W=4 measured too (1.58 and 2.07); the
HEAD 0xE4B711C8 (HEAD_PAR 8) + TAIL 0xE4B711CF (TISEQ + TREE_P 16 + runtime width) raised the matched pair benchmark from 102.13 to 112.73 tok/s at unchanged accept 2.22 (+10.4%), by making the HEAD's attention phase ~5.7x faster and the TAIL's tree walk ~1.47x faster on silicon. Width and depth are now characterized end to end: tok/s peaks at width 2 (153.4 @ 1.83), width 3 is close behind (146.8 @ 1.97, +32% tok/s over width 8 for -11% accept) and is the current default candidate; depth costs
Pieces 1 (table builder) and 3 (host mock) remain GATED/committed as before. Piece 2 (HBM fetch RTL) is now COMMITTED (verilog f792e27) with a clean run4 unit gate (latency sweep, parallel abort-drain, RED fired) and a first OOC synthesis census. The full-drafter xsim gate that would prove the fold end-to-end has NOT landed yet: 4 arms have been queued on the box1 mesh since 05:16-05:20 and were still in session-init at 08:00. No HEAD-level or silicon claim is made.
M2 (LANES 256 + E3 packed-DSP lane + locality + RET_SER + TILE_PIPE + FOLD) is bit-exact, closes 166 MHz and needs 2.16× fewer cycles; the drafter around it was not run at 166 MHz. M3 P&R is running (synth phase, box1, checked 2026-09-24 16:47 EDT, not yet routed); DOT_DSP L128 and K_PAR32 L128 P&R are still queued/pending.
On the placed HEAD 0x11C9 every GEMV lane's weight bank is on its own die, but all three activation broadcast replicas and every result consumer sit in SLR1, so most GEMV traffic crosses a die boundary; attention fills 40,205 of SLR1's 98,416 drafter SLICE and the TAIL carries an unused copy of it.
Every measured result of the three tracks as of 2026-10-21, failures included. Measured only; projections are on the progress post. Track 1 box1 data pending, from the delivery note.
Current builds, gates and ETAs for 10-21 to 10-23 with projected batch-1 numbers. All projected or derived; nothing here is measured.
Per-op goldens (every phase G1..G8 + TREE) at ctx 16..564, md 1/2/3, for 4+4 prompts, plus the A->B / B->C / C->A seam regions as .bin/.mem; the checker catches every planted defect.
Program 2: 8-card FULL TARGET; Track Q. Design + shell facts measured; no image built yet (Qwen PDI builds owned by the laptop copy of box1-golden, after the 3STAGE report).
Reusable AXI4/AXI-Lite/AXIS checkers + randomized BFMs; every check proven to fire. control_regs.sv passes all 60 ABI vectors but answers B/R in the same cycle as the request handshake (A3.3.1 strict dependency FAIL).
Works bit-exact; the wall is area (~1.2 LUT per nonzero bit-serial). Road to a PDI: folded unit PDI, one-layer-per-card routability, sparsity-trained tiny folds; URAM-resident path for 2B4T.
Program 2: 8-card FULL TARGET; Track T. Re-aimed per user: (1) TriLM-99M fully folded on 8 cards, (2) a constant-folded ternary drafter co-resident with the HBM target. BitNet-2B-4T stays the URAM-resident ternary reference.
Related articles
Our own AVED-style wiki for the V80: click a component or a tool, read what it is, how we use it, our measured numbers, the pitfalls, and links to AMD docs and our scripts.
Batch-1 decode on the 0x1254 s2 engine is bound by the 32-port weight-stream beat rate (230 GB/s), not HBM (stall 3.7-3.9%) or DSPs (3 of 4 columns idle).
The flashed HBMS image 0xE4B71254 serves real Llama models end to end on silicon with no new PDI: one card up to 8B, eight cards pipeline-parallel for 70B; decode is bound by W8 weight streaming at ~230 GB/s per card.
Running. Umbrella page of the 8-card FULL TARGET program: Track T -> ternary-make-possible-20261011, Track Q -> qwen27b-ring8-20261011. HQ build plan v2 sets the gates and the L0-L7 ladder; next: L0 HBM readout on card 61, R8F re-floorplan, ring cabling card i P4 -> i+1 P3.
Both new hosts are fully provisioned and admitted to the mesh (auto-pick still held back until this page's checks landed); the toolchain and license path both trace back through asgard, so an asgard reboot restarts the license tunnel automatically while a box1 cold-AC does not affect these hosts at all.
41.58 MiB 2-bit body on-chip (URAM), fp16 lm-head and embeddings in HBM; 4 V80s hold ~0.9B 2-bit weights, ~5x today
The round-1 HOSTFAST=1 result (+4.8%, 50/50 identical) stands and is the launcher default. Two later missions (I7, I8) added a CUDA-event timing instrument that decomposes every step into V (target-verify GPU), D (tree+KV span) and G (host glue), then measured G directly instead of estimating it: G = 8.58 ms median / 6.00 p10 on the default path. Four new byte-exact cuts (root-image + prep-row CUDA graphs, a cost-fallback fast path, a single-transfer capture) take G to 6.28/5.03, and with the ex
The two-card protocol works end to end (24 items, 72-node tree, 0 CRC/seq/overflow) and the full-data bit-exact gate is GREEN at L=0 and L=40 (RTL committed verilog f29beae). Hop 3 must carry GLM + G11 (211 wire cycles). The GEMV item FIFO must hold K_F=8. RED arms red_flip and red_tag were harvested 16:10 EDT and both FAIL exactly as designed (non-vacuity proven); red_crd, red_crd1 and the fifo4 deadlock proof are still running on the laptop (slow host, in GEMV weight init since 2026-09-23 21:3
Wave E (ATT KV-commit + R3_EARLY + NORMFUSE, MLPV G11F) runs on all four cards after cold AC#28 with every QSFP link up on its own and boot kits verified 4/4; it measures 518.0 tok/s mean at mt_bench w2 d7 (wave B 513.20) and 857.93-860.83 at humaneval d12, accept identical to wave C at every depth. Golden is bf16-only (measure-only). Next: a second drafter, ATT pair + MLPV-solo 0xE4B712F5.
Track 2 batch-1: Qwen3.8-27B 17.8-18.9 greedy, up to 21.1 with EAGLE3; Llama-1B 415. Five silicon failures listed, including Llama-8B ring error at token 21.
Running (reopened as the Track Q living page). Shape, schema and shell facts measured; HQ e8 transport xsim-exact under faults. Next: R8F re-floorplan + lanes-vs-HBM winner -> 0x1250 C4 build on the laptop.
Diff from the 10-05 architecture post: what each 4-card variant became, the measured verdicts, and why CL4 ring + shape-generic + HBM streaming are the next bets.
Weekly architecture reference: current HEAD/TAIL datapaths, memories, clocks and links, with a selectable timeline of every major shift.
The laptop remains a verified, ready full-PDI build host, and everything D6 needs is staged there with no further work. But D6 itself has been queued twice and admitted neither time: the peak-50 waiter was abandoned by main at 07:51 and resubmitted at peak 46 (informed by D2's measured 38.3 GB build peak), and that second waiter was itself abandoned at 10:07:45 while still waiting on memory behind 8 long-running xsim_full_drafter jobs (492-669 min old) plus 3 headlever_xsim jobs. No D6 placement
Pieces 1 (table builder) and 3 (host mock) remain GATED/committed as before. Piece 2 (HBM fetch RTL) is now COMMITTED (verilog f792e27) with a clean run4 unit gate (latency sweep, parallel abort-drain, RED fired) and a first OOC synthesis census. The full-drafter xsim gate that would prove the fold end-to-end has NOT landed yet: 4 arms have been queued on the box1 mesh since 05:16-05:20 and were still in session-init at 08:00. No HEAD-level or silicon claim is made.
Program 2: 8-card FULL TARGET; Track Q. Design + shell facts measured; no image built yet (Qwen PDI builds owned by the laptop copy of box1-golden, after the 3STAGE report).
Agentic operations: the build mesh and hosts, agent sessions, memory and coordination. Not research results.
Living page, re-pushed at each state change. Owners report state changes to HQ (hq-junior-2).
Sessions inherit reasons, not just rules: skills replace what the user had to remember; a dreamer condenses history on demand; nothing raw is deleted without the user's plan.
Living page, re-pushed at each roster change. Source of truth: /shared/rnd_hq/hq_daily/collab/roster.tsv.
L256 is the recommended GEMV-card width (routed, PDI exists); a drafter clock above 66 MHz is a seam fan-out + attention-combine project, not an array project — the array alone already closes 132-177 MHz in isolation.
The laptop remains a verified, ready full-PDI build host, and everything D6 needs is staged there with no further work. But D6 itself has been queued twice and admitted neither time: the peak-50 waiter was abandoned by main at 07:51 and resubmitted at peak 46 (informed by D2's measured 38.3 GB build peak), and that second waiter was itself abandoned at 10:07:45 while still waiting on memory behind 8 long-running xsim_full_drafter jobs (492-669 min old) plus 3 headlever_xsim jobs. No D6 placement
Live since 16:43 EDT: submit/cancel/adopt CLI, append-only ledger, 60 s dispatcher timer on box1, input push + output fetch-back, idle-host work requests. First real placements: two array-PoC arms moved off a saturated box1 onto the new 251 GB VMs.