fpga-archive-reports-process · EAGLE4 FPGA archive
01 — Project and architecture (what the thing is, how time is spent)
The project in one paragraph
EAGLE4 speculative decoding for Llama-3.2-3B (target, on an H200 GPU) with a 2-bit EAGLE4 drafter running on AMD
Alveo V80 FPGAs (Versal HBM, AVED 25.1 shell, QDMA driver — NOT XRT). The drafter proposes a token tree; SGLang
verifies it on the GPU (--speculative-exec-mode 1 = FPGA hook, -1 = GPU-only). The FPGA drafter is hand-written
SystemVerilog (HLS is legacy — never touch HLS). Weights are 2-bit (GPTQ-style, OSPREY 32×96 lineage for the live
images; checkpoint bitnet_721, bf16 source md5 649f79c0…), resident in on-chip URAM ("Layout B": zero HBM weight
traffic), scales in BRAM, only the fp16 exact lm-head (788 MB) lives in HBM for the G8 rescore gather.
Phases of one drafter forward (CLAUDE.md has the canonical table)
G1 RMSNorm → R1 → G4 QKV GEMV (6144→9216) → G5 attention (24 heads, D128, KV cache: K URAM, V BRAM) → G6 o-proj
→ R3 → G7g/G7u gate/up (3072→8192) → G7p SiLU·up → R7 → G9 down (8192→6144) → G11 RMSNorm → GLM low-rank →
G12 cheap lm-head over 128,256 vocab (1,002 tiles) → TREE top-K → G8 exact fp16 rescore (gather from HBM).
Dims: H=3072, 24 Q heads / 24 KV heads (MHA, not GQA; k_proj.qweight [384,3072] is GPTQ packing, not 3 KV heads).
Engines vs phases (the user asked for this distinction explicitly)
- Shared 2-bit GEMV array (one engine, time-multiplexed by
stage_sel: G4=0, G6=1, G7g=2, G7u=3, G9=4, G12=5): 128 lanes × K_PAR 16 × 2 read ports; cycles = N·K/4096 (exact to the cycle on 8 stage-points). Wallace 3:2 CSA adder tree (csa_tree16_int20: 16→11→8→6 reg, 6→4→3→2 reg, DSP58 CPA; latency 3, II=1). Tile overlapTILE_PIPEalready takes G4/G9 to 99.91% of the bandwidth floor. - Dedicated engines: attention (G5; HEAD_PAR selects 1 or 8 parallel head engines), fp16 rescore (G8), low-rank GEMV (GLM), TREE top-K, RMSNorm, residual adders, SiLU.
- On a single card the 2-bit array is ~77% of card time. On the pair it is 71.4% of the HEAD's busy time.
The two-card PAIR (live today)
- HEAD card 61:00.0 runs G1–G7u (the "front half"); TAIL card 41:00.0 runs G7p–G8 + TREE ("back half").
- They are relayed THROUGH THE HOST over PCIe: HBM rings, 16 slots, item 77,824 B, bases
0x41_4010_0000/0x42_B000_0000, stride0x14000. Doorbells:0x180 pipe_in_seq,0x184 pipe_out_seq,0x1C0 pipe_consumed. - PERSIST images: arm
0x20= tree depth (md) and0x158 = 0x2FBEFORE the first AP_START after boot; some ABI is latched at the first AP_START (memoryfeedback_persist_latches_md_abi_at_first_apstart).0x154= KV-session reset pulse, HEAD-owned (scripts/pair_sglang_bridge.py:75).0x1CC= runtime tree width (TAIL, only on the new CF TAIL). Fold CSRs0x1F0/0x1F4/0x1F8/0x1FC/0x178(only in the fold fork). - W2 KV-only fix (image
0xE4B711C9): the HEAD samples the KV-only bit while parked inS_PIPE_WAIT(top_fsm.sv:273), so catch-up rows run KV-only instead of full trees. That took the pair 57.15 → 102.13 tok/s (trees per request 1,531 → 734). Every future HEAD must be KV-only-capable and be listed inKVONLY_HEAD_BUILDS(scripts/oneflip_relay_driver.py:2290; C8 and CD were added tonight). - Host driver chain:
scripts/pair_sglang_run.sh→pair_sglang_bridge.py→oneflip_relay_driver.py(RelayDriverP2) +fpga_tree_marshaller.py;cost_draft_tree_fpga_eaglefit.pyhooks SGLang. MOCK paths exist for testing.
Measured time budget of the pair (the numbers everything else is built on)
Per-phase counter census on HEAD 0x11C9 (1 request, reports/results/PAIR_KVONLY_HEAD_0x11C9_SILICON_20260923.md):
HEAD 205.38 ms/request = G4 70.67 · G5 56.55 · G7g 34.11 · G7u 30.45 · G6 11.42 · G1 2.18.
TAIL 135.90 ms = TREE 87.53 · G9 28.18 · GLM 14.18 · G11 3.67 · G7p 2.26 · G12 0.08 (G8 raw 117.25 overlaps TREE).
Per-step waterfall (mt_bench, 25.0 ms/step; reports/results/PAIR_HOST_OVERHEAD_20260923.md): HEAD computing 47%
(L1 8 child items 10.1 ms, root 1.2, KV catch-up 0.5), GPU verify 22% (5.5 ms, both cards idle), host glue ~20%,
TAIL-only 6%. HOSTFAST=1 = +4.8% (now the launcher default).
Hardware facts that bind every design
- V80 = 2.57M LUT, 5.15M FF, 10,848 DSP58, 3,741 BRAM36, 1,925 URAM, 321,776 SLICE, 3 SLRs, 18,870 SLLs per boundary.
- SLICE (and SLL for wide designs) binds, not URAM. Attention occupies 40,205 of 98,416 drafter SLICE in SLR1 on the HEAD.
- HBM: 2×16 GB stacks, 32 pseudo-channels of 1 GB each (each PC addresses only its own 1 GB), ~25 GB/s per PC,
HBM NoC ports along the top of SLR2, AVED requests only 250 MB/s QoS per PC. Aperture
GMn_BASE = 0x40_0000_0000 + n·0x4000_0000. - Whole drafter runs at ONE ~66 MHz clock. The HEAD's worst path is attention (14.47 ns, 85% routing).
All 3 broadcast replicas are XDC-pinned to SLR1 (
eagle4_drafter_bcast_slr1_pblock.xdc), so 123/128 lanes are fed across a die — that pin, not broadcasting itself, causes the 13.3 ns array path on the HEAD. - OOC-synth slack does NOT predict routed timing (I retracted a "200 MHz" claim tonight because of this).