fpga-archive-reports-process · EAGLE4 FPGA archive

Source docs/handoff_20260924/01_PROJECT_AND_ARCHITECTURE.md · source md5 caded5ba2825dec5c9b00f526451a4f8 · archived 2026-11-18

Download PDFOpen PDFRaw markdown

01 — Project and architecture (what the thing is, how time is spent)

The project in one paragraph

EAGLE4 speculative decoding for Llama-3.2-3B (target, on an H200 GPU) with a 2-bit EAGLE4 drafter running on AMD Alveo V80 FPGAs (Versal HBM, AVED 25.1 shell, QDMA driver — NOT XRT). The drafter proposes a token tree; SGLang verifies it on the GPU (--speculative-exec-mode 1 = FPGA hook, -1 = GPU-only). The FPGA drafter is hand-written SystemVerilog (HLS is legacy — never touch HLS). Weights are 2-bit (GPTQ-style, OSPREY 32×96 lineage for the live images; checkpoint bitnet_721, bf16 source md5 649f79c0…), resident in on-chip URAM ("Layout B": zero HBM weight traffic), scales in BRAM, only the fp16 exact lm-head (788 MB) lives in HBM for the G8 rescore gather.

Phases of one drafter forward (CLAUDE.md has the canonical table)

G1 RMSNorm → R1 → G4 QKV GEMV (6144→9216) → G5 attention (24 heads, D128, KV cache: K URAM, V BRAM) → G6 o-proj → R3 → G7g/G7u gate/up (3072→8192) → G7p SiLU·up → R7 → G9 down (8192→6144) → G11 RMSNorm → GLM low-rank → G12 cheap lm-head over 128,256 vocab (1,002 tiles) → TREE top-K → G8 exact fp16 rescore (gather from HBM). Dims: H=3072, 24 Q heads / 24 KV heads (MHA, not GQA; k_proj.qweight [384,3072] is GPTQ packing, not 3 KV heads).

Engines vs phases (the user asked for this distinction explicitly)

The two-card PAIR (live today)

Measured time budget of the pair (the numbers everything else is built on)

Per-phase counter census on HEAD 0x11C9 (1 request, reports/results/PAIR_KVONLY_HEAD_0x11C9_SILICON_20260923.md): HEAD 205.38 ms/request = G4 70.67 · G5 56.55 · G7g 34.11 · G7u 30.45 · G6 11.42 · G1 2.18. TAIL 135.90 ms = TREE 87.53 · G9 28.18 · GLM 14.18 · G11 3.67 · G7p 2.26 · G12 0.08 (G8 raw 117.25 overlaps TREE). Per-step waterfall (mt_bench, 25.0 ms/step; reports/results/PAIR_HOST_OVERHEAD_20260923.md): HEAD computing 47% (L1 8 child items 10.1 ms, root 1.2, KV catch-up 0.5), GPU verify 22% (5.5 ms, both cards idle), host glue ~20%, TAIL-only 6%. HOSTFAST=1 = +4.8% (now the launcher default).

Hardware facts that bind every design