# 01 — Project and architecture (what the thing is, how time is spent)

## The project in one paragraph
EAGLE4 speculative decoding for Llama-3.2-3B (target, on an H200 GPU) with a 2-bit EAGLE4 drafter running on AMD
Alveo V80 FPGAs (Versal HBM, AVED 25.1 shell, QDMA driver — NOT XRT). The drafter proposes a token tree; SGLang
verifies it on the GPU (`--speculative-exec-mode 1` = FPGA hook, `-1` = GPU-only). The FPGA drafter is hand-written
SystemVerilog (HLS is legacy — never touch HLS). Weights are 2-bit (GPTQ-style, OSPREY 32×96 lineage for the live
images; checkpoint `bitnet_721`, bf16 source md5 `649f79c0…`), resident in on-chip URAM ("Layout B": zero HBM weight
traffic), scales in BRAM, only the fp16 exact lm-head (788 MB) lives in HBM for the G8 rescore gather.

## Phases of one drafter forward (CLAUDE.md has the canonical table)
G1 RMSNorm → R1 → **G4 QKV GEMV (6144→9216)** → **G5 attention (24 heads, D128, KV cache: K URAM, V BRAM)** → G6 o-proj
→ R3 → G7g/G7u gate/up (3072→8192) → G7p SiLU·up → R7 → G9 down (8192→6144) → G11 RMSNorm → GLM low-rank →
G12 cheap lm-head over 128,256 vocab (1,002 tiles) → TREE top-K → G8 exact fp16 rescore (gather from HBM).
Dims: H=3072, 24 Q heads / 24 KV heads (MHA, not GQA; `k_proj.qweight [384,3072]` is GPTQ packing, not 3 KV heads).

## Engines vs phases (the user asked for this distinction explicitly)
- **Shared 2-bit GEMV array** (one engine, time-multiplexed by `stage_sel`: G4=0, G6=1, G7g=2, G7u=3, G9=4, G12=5):
  128 lanes × K_PAR 16 × 2 read ports; cycles = N·K/4096 (exact to the cycle on 8 stage-points). Wallace 3:2 CSA
  adder tree (`csa_tree16_int20`: 16→11→8→6 reg, 6→4→3→2 reg, DSP58 CPA; latency 3, II=1). Tile overlap `TILE_PIPE`
  already takes G4/G9 to 99.91% of the bandwidth floor.
- **Dedicated engines:** attention (G5; HEAD_PAR selects 1 or 8 parallel head engines), fp16 rescore (G8), low-rank
  GEMV (GLM), TREE top-K, RMSNorm, residual adders, SiLU.
- On a single card the 2-bit array is ~77% of card time. On the pair it is 71.4% of the HEAD's busy time.

## The two-card PAIR (live today)
- HEAD card 61:00.0 runs G1–G7u (the "front half"); TAIL card 41:00.0 runs G7p–G8 + TREE ("back half").
- They are relayed THROUGH THE HOST over PCIe: HBM rings, 16 slots, item 77,824 B, bases `0x41_4010_0000` /
  `0x42_B000_0000`, stride `0x14000`. Doorbells: `0x180 pipe_in_seq`, `0x184 pipe_out_seq`, `0x1C0 pipe_consumed`.
- PERSIST images: arm `0x20` = tree depth (md) and `0x158 = 0x2F` BEFORE the first AP_START after boot; some ABI is
  latched at the first AP_START (memory `feedback_persist_latches_md_abi_at_first_apstart`). `0x154` = KV-session
  reset pulse, HEAD-owned (`scripts/pair_sglang_bridge.py:75`). `0x1CC` = runtime tree width (TAIL, only on the new
  CF TAIL). Fold CSRs `0x1F0/0x1F4/0x1F8/0x1FC/0x178` (only in the fold fork).
- **W2 KV-only fix** (image `0xE4B711C9`): the HEAD samples the KV-only bit while parked in `S_PIPE_WAIT`
  (`top_fsm.sv:273`), so catch-up rows run KV-only instead of full trees. That took the pair 57.15 → **102.13 tok/s**
  (trees per request 1,531 → 734). Every future HEAD must be KV-only-capable and be listed in `KVONLY_HEAD_BUILDS`
  (`scripts/oneflip_relay_driver.py:2290`; C8 and CD were added tonight).
- Host driver chain: `scripts/pair_sglang_run.sh` → `pair_sglang_bridge.py` → `oneflip_relay_driver.py`
  (RelayDriverP2) + `fpga_tree_marshaller.py`; `cost_draft_tree_fpga_eaglefit.py` hooks SGLang. MOCK paths exist for testing.

## Measured time budget of the pair (the numbers everything else is built on)
Per-phase counter census on HEAD `0x11C9` (1 request, `reports/results/PAIR_KVONLY_HEAD_0x11C9_SILICON_20260923.md`):
HEAD 205.38 ms/request = G4 70.67 · G5 56.55 · G7g 34.11 · G7u 30.45 · G6 11.42 · G1 2.18.
TAIL 135.90 ms = TREE 87.53 · G9 28.18 · GLM 14.18 · G11 3.67 · G7p 2.26 · G12 0.08 (G8 raw 117.25 overlaps TREE).
Per-step waterfall (mt_bench, 25.0 ms/step; `reports/results/PAIR_HOST_OVERHEAD_20260923.md`): HEAD computing 47%
(L1 8 child items 10.1 ms, root 1.2, KV catch-up 0.5), GPU verify 22% (5.5 ms, both cards idle), host glue ~20%,
TAIL-only 6%. HOSTFAST=1 = +4.8% (now the launcher default).

## Hardware facts that bind every design
- V80 = 2.57M LUT, 5.15M FF, 10,848 DSP58, 3,741 BRAM36, 1,925 URAM, 321,776 SLICE, 3 SLRs, 18,870 SLLs per boundary.
- SLICE (and SLL for wide designs) binds, not URAM. Attention occupies 40,205 of 98,416 drafter SLICE in SLR1 on the HEAD.
- HBM: 2×16 GB stacks, 32 pseudo-channels of 1 GB each (each PC addresses only its own 1 GB), ~25 GB/s per PC,
  HBM NoC ports along the top of SLR2, AVED requests only 250 MB/s QoS per PC. Aperture `GMn_BASE = 0x40_0000_0000 + n·0x4000_0000`.
- Whole drafter runs at ONE ~66 MHz clock. The HEAD's worst path is attention (14.47 ns, 85% routing).
  All 3 broadcast replicas are XDC-pinned to SLR1 (`eagle4_drafter_bcast_slr1_pblock.xdc`), so 123/128 lanes are fed
  across a die — that pin, not broadcasting itself, causes the 13.3 ns array path on the HEAD.
- OOC-synth slack does NOT predict routed timing (I retracted a "200 MHz" claim tonight because of this).
