A 2-bit drafter on V80 FPGAs, beside the H200 that verifies it.
measured talon2, 2026-11-16: COMBO3 0xE4B713D6 × 4 + M1361 0xE4B71361 × 2, W2 d11, rounds 960.23 / 960.97 / 963.10, texts 1b0779cf = record · DRAFTER_SPEED_RESEARCH_20261107/ATT_COMBO_SILICON_20261116.md (COMBO3 serve rows) · GPU-only: KC_RAISE_PLAN_7CARD_20261116.md:131; BEST_VS_BEST_GPU_20261112.md:4.
Live chat: the same prompt through FPGA drafting, GPU + EAGLE, or GPU only.
demo.yshicarrick.com runs one H200 with four V80 drafter cards (box1). A mode switch picks who drafts; auto-run loops benchmark prompts and keeps a running tally.
measured box1, 2026-11-10, engine E2b + MLPV 0x1342, w2 d11, cap 256; token-weighted means · DEMO_AUTORUN_BOX1_20261110.md:40-41 (Llama), :87 (Qwen); screenshot DEMO_AUTORUN_BOX1_20261110_shots/01_running.png (md5 e8fe8095…).
Same H200, same prompts, same tree: the FPGA-drafted run was faster.
measured talon2, Llama-3.2-3B, humaneval 80 × 256, batch 1, greedy, depth 15 width 2, 20 verify tokens, fa4, SGLang 0.5.16; FPGA arm ATT 0xE4B713F9 × 2 + MLPV 0x1342 × 2 · FPGA_VS_H200_20261110/SOURCES.md (Results + same-day tables; logs md5 d75ab93e…, 86239804…). Screenshot SITE_DESIGN_AUDIT_20261110/demo_1280_light.png.
Seven cards: 961.43 tok/s at accept 6.91, every reply identical to the reference.
| round | tok/s | accept | verify steps | texts (80) |
|---|---|---|---|---|
| t1 | 960.23 | 6.91 | 2,839 | 1b0779cf = record |
| t2 | 960.97 | 6.91 | 2,839 | 1b0779cf = record |
| t3 | 963.10 | 6.91 | 2,839 | 1b0779cf = record |
| mean | 961.43 | 6.91 |
measured talon2 2026-11-16, humaneval 80 × 256, W2 d11 fa3 · ATT_COMBO_SILICON_20261116.md (COMBO3 serve rows t1-t3) · KC_RAISE_PLAN_7CARD_20261116.md:131 (GPU-only) · SCALING_4C_VS_7C_PHASE_RANK_20261116.md §3 (tree), §4 (step). The ratio to GPU-only is in the calculations bar.
One drafter split over two card roles: ATT does attention, MLPV does the MLP, vocabulary and tree.
Block diagram from the Architecture Book for its newest Track-1 image, F3t 0xE4B713F9 (4 cards): solid blocks are built, dashed are not in this image, gold = what this image added. Live, clickable book (66 images): hardware.research.yshicarrick.com/p/eagle4-arch-book/.
measured rendered 2026-11-18 from /p/eagle4-arch-book/ v5 (arch_book.json sha256 8acbaeda…), image "F3t G5_OVL ATT (0x13F9)". The book has no entry yet for the 7-card images on the next slide.
Two images, flashed on six cards: four ATT ranks share every matrix, two MLPV cards finish each node.
| ATT COMBO3 0xE4B713D6 (4 cards) | MLPV M1361 0xE4B71361 (2 cards) | |
|---|---|---|
| role | G1 embedding + norm, G4 QKV, G5 attention, G6 o-proj with a 4-rank all-reduce over QSFP, G7 gate/up | G9 down, G11 norm, GLM low-rank, G12 vocabulary scan + TREE top-K, G8 exact fp16 rescore |
| levers in this image | 0x13BC + G5_P2W 8 + R3_EARLY 1 + KV1H2C 1 + HANDOFF_CUT 1 | MLPV_G8_OWN 1 (G8 per d11 tree 551.83 → 237.71 µs vs the 4-card MLPV) |
| GEMV array | 512 lanes × 16 × 2, 66 MHz, LUT-GEMM 2-bit lanes | 256 lanes × 16 × 2, 132 MHz, DSP lanes |
| resources (one card) | 0x13BE parent: SLICE 81.85 % (die 0 98.12 %), DSP58 5,267 (48.6 %), URAM 744 (38.7 %), BRAM 47.7 % | SLICE 89.6 %, DSP58 7,504 (69.2 %), URAM 512 (26.6 %), BRAM 27.7 % |
| timing (final, true) | setup +0.044 ns / hold +0.010 ns, 0 failing endpoints | margin rescue e24deb3f of the same netlist |
| FINAL PDI md5 | d2e737d7c7134cb0d317345a6e405b1b | e24deb3f6d1de5e5b41964eecc1be7e9 |
measured ADJUDICATION_0x13D6_0x13DB_20261116.md:11-14 (levers, PDI, timing) · ARCH_RETHINK_7CARD_20261117.md §1.3 (resources; no utilization report exists for 0x13D6 itself, so its parent 0x13BE is shown), §1.3 "Array today" · SCALING_4C_VS_7C_PHASE_RANK_20261116.md §3 row 1 (G8) · scripts/4c1/c4_regmap.py EXPECT_PDI_MD5 (M1361).
One server: seven V80 cards (six used) and one H200, the cards joined by QSFP cables.
measured 13 point-to-point QSFP cables, each 4 × 25.78 Gb/s Aurora 64B/66B · SEVENCARD_BOARD_20261026/TALON2_QSFP_MAP_20261110.md:24-50 · ARCH_RETHINK_7CARD_20261117.md §1.1-1.2 (card 99 is not opened: allow-list, 4-rank hypercube, no MLPV cable).
The host sends one root; the cards build the tree level by level; the H200 verifies it in one pass.
- Host: SGLang with the FPGA drafter hook (
--speculative-exec-mode 1) and a card worker that drives the six cards over PCIe / QDMA. - Per node: the four ATT ranks each run 1/4 of G1..G7 and all-reduce over the hypercube; the result goes over the stage links to the MLPV pair, which runs G9..G12, TREE and G8 and picks the next level.
- Per tree: 21 nodes for a width-2, depth-11 tree, then the token tree goes back over PCIe and the H200 verifies it.
measured SCALING_4C_VS_7C_PHASE_RANK_20261116.md §3 (one full d11 tree = 21 items; ATT / MLPV phase split) · ARCH_RETHINK_7CARD_20261117.md §1.1 (stage cables, all-reduce) · FPGA_VS_H200_20261110/SOURCES.md (exec-mode 1).
Per-engine time for one full depth-11 tree: attention and QKV lead on ATT, the vocabulary scan on MLPV.
| card | phase | 4 cards (µs) | 7 cards (µs) |
|---|---|---|---|
| ATT | G5 attention | 841.34 | 610.48 |
| ATT | G4 QKV | 630.03 | 452.12 |
| ATT | G7G (+ serial R3 + pre-MLP norm) | 561.34 | 444.48 |
| ATT | seam / link (EMBG + EXPORT + INGEST) | 208.60 | 262.71 |
| ATT | G6 o-proj + TP all-reduce | 195.07 | 218.14 |
| ATT | G7U | 302.43 | 177.22 |
| ATT | G1 (+R1) | 136.94 | 145.09 |
| ATT | G7P / SiLU | 80.74 | 42.32 |
| MLPV | TREE (G12 scan + top-K merge) | 701.70 | 707.61 |
| MLPV | G9 down | 268.89 | 270.45 |
| MLPV | INGEST + XCH + LVL_SEL/OUT | 243.25 | 252.08 |
| MLPV | G8 exact rescore | 551.83 | 237.71 |
| MLPV | G11 | 148.05 | 148.27 |
| MLPV | GLM | 73.49 | 71.91 |
| ATT busy | 2,956.49 | 2,352.56 | |
| MLPV busy | 1,995.22 | 1,695.03 | |
| tree issue → done, p50 | 3,384.05 | 2,877.70 |
measured µs per full d11 tree (21 items) from on-card counters; 4 cards = F3t 0xE4B713F9 × 2 + MLPV 0x1342 × 2, 7 cards = 0x13BE × 4 + M1361 × 2 (COMBO3's parent; no census exists for COMBO3) · SCALING_4C_VS_7C_PHASE_RANK_20261116.md §3 table (rows 1-15, totals).
Of the 7.187 ms step, the host waits on the cards for 1.501 ms; the GPU verify and host work are the rest.
| term (ms per verify step) | 4 cards (883.12 tok/s) | 7 cards, record (961.43) |
|---|---|---|
| step | 7.812 | 7.187 |
| tree path = host_ahead + join_wait | 2.455 | 2.010 |
| · host work under the card (host_ahead) | 0.512 | 0.510 |
| · host blocked on the card reply (join_wait) | 1.943 | 1.501 |
| rest of the step (GPU verify + host) | 5.358 | 5.176 |
| GPU verify forward (cuda events, d7, 4-card F3t) | 3.452 | not measured on 7 cards |
measured d11 fa3 w2, humaneval 80 × 256; steps 2,843 vs 2,839, accept 6.90 vs 6.91 · SCALING_4C_VS_7C_PHASE_RANK_20261116.md §4 · GPU verify: BATCH_BUBBLES_20261109/INTERLEAVE_BATCHVERIFY_SILICON_20261109.md:76. The split of the remainder is in the calculations bar.
Everything needed to reproduce the record run.
| target / drafter | Llama-3.2-3B-Instruct (fp16) on the H200 NVL · EAGLE4 2-bit drafter bitnet_721 on the cards |
| workload | humaneval, 80 prompts × 256 new tokens, batch 1, greedy |
| tree | width 2, depth 11, 20 verify tokens (VTOK 20), K_C 64 candidates, tree-stop theta −2.5 |
| GPU side | SGLang, draft attention fa3, FASTGLUE 4, CUDAPACK 1 |
| host levers | HX_MMIO7 / HX_O7 / HX_S7 (fewer per-card MMIO round trips), KV1H2C on |
| cards | ATT ranks 17 / 3d / bd / e1 = COMBO3 0xE4B713D6; MLPV 50 / 63 = M1361 0xE4B71361; drafter clock 66.001 MHz |
| checks before the timed rounds | identity 6 / 6 (build-id registers), golden fingerprint 51524c4f PASS, exact-tooth PASS × 2, zero KV-reader aborts |
measured DW_SWEEP_7CARD_COMBO3_20261116.md:8-10 (bench line) · ATT_COMBO_SILICON_20261116.md (14:24 bring-up, 14:25:12 golden, serve rows) · SCALING_4C_VS_7C_PHASE_RANK_20261116.md:11 (clock) · FPGA_VS_H200_20261110/SOURCES.md (target, drafter weights).
Every report behind these slides is in the FPGA archive.
- FPGA archive (reports, schematics, demo files, raw data, md5 of every file): /p/eagle4-fpga-archive/
- Architecture Book (interactive): /p/eagle4-arch-book/
- Record: ATT_COMBO_SILICON_20261116.md · scaling: SCALING_4C_VS_7C_PHASE_RANK_20261116.md · rethink: ARCH_RETHINK_7CARD_20261117.md
- Demo: DEMO_AUTORUN_BOX1_20261110.md · head-to-head: FPGA_VS_H200_20261110/SOURCES.md
All paths are under reports/results/ (DRAFTER_SPEED_RESEARCH_20261107/ for the record, scaling and rethink reports).