EAGLE4 Drafter — Verilog Architecture (live status)

Progress ladder

V1 ✓ Inventory
V2 ✓ MAC + xsim
V3 ✓ MAC ooc-synth
V4 ✓ Dequant
V5a ✓ Lane functional
V5b ✓ Wallace wired
V5b-scale ✓ Q1.15 scale
V5c ✓ LANES=16 array
V5d ✓ URAM weight bank
V6 ✓ OS systolic
V7 ✓ IS systolic (structural)
V8 ✓ Constfold (negative)
V9 ✓ K=3072 + V9.5 streaming fix
Phase 3 ✓ K-scan all 5 shapes
Phase 4 ✓ 9 layer primitives built & PASS
Phase 5a ✓ Top control plane (AXI-Lite + 11-stage FSM)
Phase 5b ✓ gemv_stage_array_uram LANES=128 K=3072 integrated
Phase 6 ✓ Vivado P&R → full-drafter PDIs built (URAM 88.83% binding)
Silicon ✓ M7 correctness (all 19 stages golden, ordered 8/8)
Silicon ✓ WIDE G8 correct-AND-fast (0xE4B7110E: 6.96 ms/fwd, 4.9×)
Silicon ✓ Host fast-path E2E 5.46 tok/s (1.71×)
SEL_PIPE ✓ 100 MHz OOC-MET (+0.087, reclock unblocked)
Batch-tree PDI 0xE4B7110F (B=2) building → depth-N

2-bit matrix multiplies in scope

#GEMMK (in)N (out)Shape classqweight bytes2-bit?
G4qQKV-Q61443072M_in4.5 MBYES
G4kQKV-K61443072M_in4.5 MBYES
G4vQKV-V61443072M_in4.5 MBYES
G6O proj30723072S (square)2.25 MBYES
G7gMLP gate30728192M_out6.0 MBYES
G7uMLP up30728192M_out6.0 MBYES
G9MLP down81926144L12.0 MBYES
G12LM head vocab256128256V (skew)8.0 MBYES
G8fused gather-dot (fp16 exact rescore, HBM)3072512 rows top-K_C788 MB fp16 (HBM)—SILICON 8/8 (WIDE-256, 0xE4B7110E)

Total qweight footprint: 47.75 MB. SHIPPED CHOICE (Layout B, 2026-07-09): ALL body 2-bit qweight — including G9 down — lives in URAM (zero HBM traffic at compute); the packing that made it fit is the LANES=128 × K_PAR=16 array binding at URAM 1,710 / 88.83%. The only weights in HBM are the exact fp16 lm-head (788 MB) that G8 gathers. The earlier "G9 stays HBM-streamed" was the pre-Layout-B feasibility note and is superseded. Evidence: reports/evidence/wide-g8-silicon/WIDE_G8_SILICON_GAIN.md.

Module catalog

ℹ️ This lists the early PRIMITIVE building blocks (MAC cells, gptq2 unpack, GEMV variants) — accurate and still the foundation. The silicon-shipped top-level modules that compose them are, for reference: g8_rescore_p.sv (the WIDE-256 gather+dot, 0xE4B7110E), kv_journal.sv (per-branch ancestor K/V for depth-N), stage_io_buffer_ff.sv (FF activation seams — kills the cross-SLR BRAM-strand bug class), and the batch path in eagle4_drafter.sv (BODY_BATCH_B=2, sibling weight-reuse, building now as 0xE4B7110F). See docs/PDI24_GOLDEN_REFERENCE.md for the full phase→file map.
ModuleRoleStatusxsimSource
mac_2bit_shift_add.sv One 2-bit MAC cell, multiplier-free (LUT carry only) ooc-synth ✓ 1000/1000 src/primitives/mac_2bit_shift_add.sv
mac_2bit_mult.sv Same I/O, uses *; synth chooses LUT vs DSP58 ooc-synth ✓ 1000/1000 src/primitives/mac_2bit_mult.sv
gptq2_dequant16.sv 16-lane combinational unpack + zero subtract (consumes one int32 of qweight per cycle, produces 16 signed 3-bit weights) ooc-synth ✓ 1024/1024 src/primitives/gptq2_dequant16.sv
gemv_systolic_ws.sv Weight-stationary GEMV (parameterized K, N, LANES, DEPTH) pending — (V5)
gemv_systolic_os.sv Output-stationary GEMV pending — (V6)
gemv_systolic_is.sv Input-stationary GEMV pending — (V7)
gen_constfold_gemv.py Compile-time generator: weights baked into Verilog (no weight memory, no multiplier) pending — (V8)

Lane architecture (one of 128)

The whole drafter array is built by stamping out one lane 128 times in parallel. Each lane owns one output column of every matrix. The diagram below shows what's inside one lane, and where each piece sits on the V-ladder.

One PE lane — what gets stamped out 128 times

Reading the diagram top-to-bottom: per cycle, the lane reads one packed-int32 from its URAM bank (16 raw 2-bit weights) and the matching activation chunk (16 INT16 values, broadcast from a shared bus). gptq2_dequant16 unpacks the weights, 16 parallel MAC cells compute partial products, the tree-reduce sums them, the INT32 accumulator chases across K_PAR-sized chunks, and finally a single fp32 multiply applies the per-group scale. The result is one output value for this lane's column. Repeat over all chunks for a full GEMV, with all 128 lanes working in parallel.

Implemented hardware (datapaths)

What got built so far is two variants of a single 2-bit MAC cell. The two diagrams below trace the actual datapath each Vivado synthesis produced, with measured LUT counts per sub-block. Critical path (1.19–1.29 ns at 7 logic levels) shown as a red dashed overlay.

mac_2bit_shift_add — multiplier-free, explicit 3-term decomposition

mac_2bit_shift_add datapath

Three conditional shift-add terms (-4·a·w[2], +2·a·w[1], +a·w[0]) summed in a 3-input adder. All branches forced into LUT carry via (* USE_DSP="no" *). The -4a term is the most expensive (~22 LUT) because of the explicit negate; the sum tree is another ~17 LUT.

mac_2bit_mult — let synth pick (it chose LUT carry too)

mac_2bit_mult datapath

Single $signed(a) * $signed(w_s) expression. Vivado decomposes the 16×3-bit multiply into LUT carry chains too — DSP58 is too coarse for this width. Per-cell wins on area (53 vs 77 LUT) because synth's auto-decomposition is tighter than my hand-rolled shift-add tree at this size. Logic-level depth and post-route timing are within 0.1 ns of the shift_add variant.

Why both still exist: at LANES≥16 systolic chains the explicit shift-add structure controls fanout (each lane is independent, no shared multiplier scheduling) and pairs cleanly with the LUT-carry accumulator chain that gives 128-tap inner products a single-cycle II. Whether mult retains its per-cell win at LANES-scale is one of the open questions to confirm at V5/V6.

Measurements (V80 part xcv80-lsva4737-2MHP-e-S, ooc-synth)

ℹ️ These are per-PRIMITIVE ooc-synth numbers (single MAC cell, one GEMV lane, one CSA tree). They are accurate for what they measure and kept as the building-block ledger — but they are not the full-design performance. For the whole-drafter measurements see: WIDE G8 PDI 0xE4B7110E [SILICON 2026-07-09] — 6,964.8 µs/forward, M7 8/8, 4.9× vs the 33.84 ms scalar baseline, URAM 1,710/88.83% binding (reports/evidence/wide-g8-silicon/WIDE_G8_SILICON_GAIN.md); and SEL_PIPE full-top OOC @100 MHz = +0.087 MET [OOC 2026-07-10] (the reclock enabler; module OPT1F −3.452 → SELPIPE +0.426; verilog/forks/sel_pipe/build/ooc_top_selpipe_100m/timing_100m.rpt). Full silicon ledger: EAGLE4_MEASURED_RESULTS.md.
ModuleCLB LUTDSP58FFData path delayLogic levelsxsim correctness
mac_2bit_mult 53 0 21 1.19 ns 7 1000/1000 PASS
mac_2bit_shift_add 77 0 21 1.29 ns 7 1000/1000 PASS
gptq2_dequant16 (16-lane comb) 58 0 0 0.675 ns 3 1024/1024 PASS
gemv_lane_int_pilot (V5a, K=64, K_PAR=16, no pipeline) 1703 0 ~280 10.21 ns 54 200/200 PASS
csa_tree16_int20 (SOTA Wallace 3:2 CSA + DSP58 CPA, standalone) 322 1 ~80 0.896 ns 3 500/500 PASS
gemv_lane_int_pilot_wallace (V5b — full lane with Wallace tree wired in) 1547 1 — 2.817 ns 13 200/200 PASS
gemv_lane_qscale (V5b-scale — + Q1.15 scale apply on DSP58) 1,556 3 — 2.848 ns 13 200/200 PASS
gemv_array_ws (V5c — LANES=16 of gemv_lane_qscale in parallel) 24,896 48 — 2.849 ns 13 3200/3200 PASS
weight_bank_uram (V5d primitive, DEPTH=4096, DATA_W=32) 1 0 — 1.051 ns 1 (URAM288E5) measured
gemv_lane_uram (V5d integration: URAM-fed lane) 1,558 3 1 URAM 3.277 ns 10 200/200 PASS

V5d URAM inference confirmed. Vivado maps the (* ram_style = "ultra" *) register array to exactly 1 URAM288E5 block — no LUTRAM fallback, no BRAM. URAM read goes through 1 logic level (the URAM cell itself) at 1.051 ns. Plenty of margin to wire into the lane's qweight feed: 1.051 ns (URAM) + 2.849 ns (current lane) ≈ 3.9 ns total — needs ONE register stage between URAM read and dequant for 300 MHz close. Next sub-task: V5d integration step.

V5c locks linear scaling. 16 lanes synthesizes at exactly 16× the single-lane cost (24,896 = 1,556 × 16, 48 DSP = 3 × 16) AND the critical path is unchanged (+0.001 ns). The 16-way activation broadcast adds essentially zero delay. Production projection (128 lanes) updates to 199,168 LUT / 384 DSP — verified linear up to 16×, the remaining unknown is the 16→128 fanout jump (next decision point at V9).

The full lane with the Wallace tree wired in now closes 300 MHz (0.51 ns slack) AND uses 9% fewer LUTs than V5a. Critical path 3.6× shorter, logic levels 4.2× shallower. Cost: 1 DSP58 (out of 10,848) for the final CPA, and 3 extra pipeline cycles of latency (throughput unchanged at II=N_CHUNKS=4). This unlocks K_PAR=32 / K_PAR=64 production scaling that V5a's timing made impossible.

The Wallace tree wins big. Replaces the dumb CPA tree in V5b/V5c/V6/V7. 322 LUT (48% less than my V5a's tree portion), 1 DSP, 0.896 ns critical path (9× faster), 3 logic levels (17× shallower). 3-stage pipeline, II=1, bit-exact. Closes 300 MHz with 2.4 ns slack — could run at 700+ MHz. This unlocks the Aggressive production tier (K_PAR=32/64) that V5a's timing made unreachable.

V5a functional PASS but timing fails 300 MHz — 10.21 ns combinational dominated by the K_PAR=16 inline tree-reduce of INT24 adders (27 LUTCY1 + 10 LUTCY2 + 11 LOOKAHEAD8). Need 2–3 pipeline register stages in the tree OR drop K_PAR to 4–8 (V5b). Good news: 1703 LUT × 128 lanes = 218K LUT ≈ 8.5% V80 — matches the Production-sizing extrapolation exactly. Compute-density math holds; pipelining is purely a timing-closure cost.

Per-lane dequant cost is 3.6 LUT (58 LUT / 16 lanes). At 0.675 ns combinational depth, the dequant + MAC pair fits in one ~3.33 ns clock cycle at 300 MHz with margin. Dequant overhead is essentially free on the 2-bit weight path.

Surprise: Vivado declined the DSP58 even when I wrote $signed(a) * $signed(w_s) — for a 16×3-bit MAC the DSP58 is too coarse, so synth chose LUT carry chains in both cases. The mult variant is smaller than my explicit shift-add (53 vs 77 LUT) — Vivado's automatic decomposition beat my hand-rolled one at this size. The explicit shift-add still wins at larger fan-out, where forcing structure controls the route — confirm when we get to V5 LANES=16 systolic.

Phase 4 — non-GEMV layer primitives (all xsim-validated)

⚠️ "xsim-validated" here = validated in ISOLATION. Two of these primitives carry known ctx_len≥2 integration bugs that their per-primitive gates (all run at ctx_len=1) did NOT catch — the honest lesson of this project: (1) rope_pair_rotate — a cos/sin-vs-rotate latency skew corrupts attention head-0 for any ctx_len≥2 (masked because every unit gate ran ctx_len=1 or compared two matched-cadence instances; memory project_rope_head0_ctxlen_ge2_latency_bug); (2) the attention score-dot truncates every Q·K to INT20 → overflows pervasively at real σ≈0.93 for ctx_len≥2 → wrong softmax winner (memory project_int20_attnscore_overflow_ctxlen_ge2). Both are latent in the flashed lineage and are a measured contributor to accept 1.68-vs-2.13. Root cause of the miss: A/B-twin TBs are blind to bugs SHARED by both arms (feedback_ab_twin_gate_cancels_shared_bug). Fix + ctx≥2 real-magnitude gates are queued for the next respin. The COST/xsim columns below are accurate for the isolated primitive.
ModuleRoleOriginxsimCost (where measured)
rmsnorm_scale_int.sv 1/sqrt(mean(x²)+ε) scale via isqrt + divide FSM (~57 cyc/call) TALOS-V2 lift 200/200 438 LUT / 0 DSP / 2.64 ns (closes 300 MHz)
rope_pair_rotate.sv One (i, i+D_HEAD/2) rotation pair: 4 muls + 2 adds, Q4.12 new (uses cos/sin LUT) 200/200 ~2-4 DSP / pair · ~50 LUT
exp_pwl_q12.sv 32-entry PWL exp(-x) for softmax TALOS-V2 lift 200/200 ~50 LUT combinational
sigmoid_pwl_q12.sv 64-entry PWL sigmoid for SiLU new (computed from math.exp) via silu_mul ~60 LUT combinational
silu_mul.sv gate × sigmoid(gate) × up = SiLU output, 2-stage pipeline new 200/200 2 DSP / lane · ~150 LUT
kv_head_uram.sv Per-head KV cache: dual-port URAM, append-write + random read new 1024/1024 ~16 URAM/head packed-4 · ~5 LUT control
attn_score_dot.sv Q · K[p] dot product reusing Wallace tree primitive new (composes csa_tree16_int20) 200/200 16 DSP + ~400 LUT (D_HEAD=16 pilot)
score_max_tracker.sv + softmax_exp_sum.sv Softmax pass-1 (running max) + pass-2 (sum of exp_pwl) new 20/20 groups ~80 LUT / pair

All 8 Phase 4 layer primitives are xsim-PASS, with TALOS-V2 lifts validated bit-exact against numpy mirrors. Total Phase 4 LUT cost so far is < 1K — these are tiny relative to the 164K LUT GEMV array. Remaining Phase 4 work is the assembly: softmax pass-3 (divide each exp by sum) + per-head attention loop (orchestrate kv_head_uram + attn_score_dot + max_tracker + exp_sum + V combine). Phase 5 = top-level drafter FSM composing all of this.

Scaling to a working drafter step

⚠️ SUPERSEDED BY SILICON (2026-07-09). The 300 MHz projections and pilot PE-sizing table below are the early feasibility model and are kept for the per-cell ooc-synth analysis (which is still accurate). The actual shipping drafter is measured, not projected: the full 15-stage forward runs on V80 silicon at ~50–55 MHz (not 300), the golden PE array is LANES=128 × K_PAR=16 (K_PAR can't widen — 32/64 ooc-round to 1,920 URAM over the 1,925 wall), binding resource is URAM 1,710 / 88.83%, and the measured per-forward is 6,964.8 µs (WIDE G8 PDI 0xE4B7110E, N=60 median, ctx=1 — reports/evidence/wide-g8-silicon/WIDE_G8_SILICON_GAIN.md). That is 10.6× slower than a GPU forward (658 µs) — the FPGA's win is correctness (M7 8/8 exact rescore) + perf/W + the offload-appliance role, not raw latency. E2E throughput is 5.46 tok/s (host fast-path) vs the GPU's ~105–125. Read the pilot table below as history; the live numbers are in Measurements above and EAGLE4_MEASURED_RESULTS.md / docs/TOKS_ROADMAP_FOR_OPUS.md.

The per-cell baseline (measured)

One 2-bit MAC cell at fp16/INT16 activation: 53 LUT, 0 DSP58, 21 FF, 1.19 ns. Multiply this by the parallel-MAC count and add the supporting hardware (weight memory, accumulator chain, scale apply, control, DMA) to get the full picture.

The drafter step workload

Per drafter step (1 forward through G1..G12), the 8 two-bit GEMVs total ~134 million MACs:

GEMMK × NMACsqweight bytes
G4_q + G4_k + G4_v6144 × 3072 × 356.6 M13.5 MB
G6_o3072 × 30729.4 M2.25 MB
G7g + G7u3072 × 8192 × 250.3 M12.0 MB
G9_down8192 × 614450.3 M12.0 MB
G12_lm256 × 12825632.8 M8.0 MB
Total—~199 M47.75 MB

(I rounded the starter-kit's "~134M" up here because G9 outputs 2H not H — the 2HS=true chunking. Either way the order of magnitude is the same.)

The PE array sizing knob

A WS systolic GEMV is fundamentally LANES_N (output cols processed in parallel) × K_PAR (input dim consumed per cycle per col). Each lane owns a slice of qweight in its own URAM bank. The drafter-step total cycles ≈ Σ over GEMVs of (K/K_PAR) × (N/LANES_N).

ConfigLANES_N × K_PARParallel MACs MAC LUTScale-apply DSPURAM (weights) Step cyclesStep time @ 300 MHzvs GPU (0.658 ms)
Pilot (V5)16 × 464 ~3.4 K (0.13% LUT)16 (0.15%)~16 (1.2%) ~3.1 M~10 ms15× slower
Modest64 × 161024 ~54 K (2.1%)64 (0.6%)~64 (5%) ~194 K~0.65 mstie
Production128 × 324096 ~217 K (8.4%)128 (1.2%)~128 (10%) ~49 K~0.16 ms4× win
Aggressive256 × 328192 ~434 K (16.9%)256 (2.4%)~256 (20%) ~24 K~0.08 ms8× win

These are compute-bound estimates assuming the MAC array is fed every cycle. Real cycle counts add ~10-20% overhead for accumulator chain pipeline fill (1 register stage per 4 K_PAR), per-output scale apply (a single fp32 multiply via DSP58, 1 cycle per output), and inter-GEMV layer transitions (RMSNorm + RoPE + SiLU + KV-cache writes, ~10 µs total). After overhead, Production gives ~0.20 ms per step — still 3× the GPU baseline.

What scales WITH compute, what doesn't

ResourcePer-cellProduction (4096 MACs)V80 budgetHeadroom
LUT53217 K2.57 M (~1.8 M post-shell)8× still left
DSP580128 (scale-apply only)10,84885× still left
URAM (qweight)0~128 (≈ 4.5 MB per array)1,925 (67.7 MB)drafter total 48 MB → 71% — fits with margin
BRAM (scales)0~403,741 RAMB36 (16.84 MB)also room for G9 here if desired
Activation b/w (1 act/cycle, broadcast)32 bits/cycle~0.6 GB/s~32 GB/s PCIe Gen5trivial
Weight b/w (per-lane URAM bank)—~1.2 TB/s aggregateURAM ≈ 27 TB/s aggregatecomfortable if banked

The real bottlenecks (the V5 pilot must surface these)

  1. Activation broadcast fanout. At LANES_N=128 the activation goes to 128 PE columns × 16 K_PAR muxes = ~2000 sinks. Naïve fanout blows up routing and timing. Mitigation: H-tree or 2-stage register pipeline on the broadcast bus.
  2. K_PAR accumulator chain timing. Summing K_PAR=32 partial products in one cycle = 32-element fp32 add tree ≈ 5–8 ns logic depth. Won't close at 300 MHz. Mitigation: register every 4–8 stages of the tree (latency ↑ ~3 cycles, no throughput loss).
  3. Per-lane URAM bank fragmentation. qweight is [K/16, N] int32 = 192×3072 for G6. Splitting across 128 lanes means each lane owns 24 rows × 192 cols = a fractional URAM. URAMs don't sub-divide cleanly. Mitigation: 4-lane URAM sharing with dual-port arbitration, or bond URAMs into 288-bit rows (G12 already needs this).
  4. Scale-apply DSP scheduling. 128 outputs in flight × 1 scale-apply each = 128 fp32 multiplies/cycle in the output FIFO drain. Mitigation: stagger output completion across LANES_N so only ~LANES_N/8 DSPs are active per cycle on the drain path.
  5. The integration class hangs from V0/V3/V5. Any of the above turning into >5 ARVALID drivers per m_axi bundle gives us the V0 deadlock. Mitigation: single top-level kernel, single m_axi bundle for activations, separate m_axi bundle for KV cache, NO inter-IP HBM handoff.

The constant-folded path (V8)

Where systolic puts weights in URAM and streams activations, constfold bakes the weights into the bitstream as combinational adder trees — 0 URAM, 0 weight bandwidth, 0 multiplier. For an output column at K=3072 with sparse 2-bit weights (~50% zero after pruning), the per-output cost is ~K/2 = ~1500 LUT.

GEMMN (outputs)~LUT per outputTotal LUTFits V80?
G6 (K=3072, N=3072)3072~1500~4.6 Mno (2.6× over)
G12 (K=256, N=128256)128256~128~16.4 Mno (9× over)
One G12 column at K=2561~128~128demo only

So constfold doesn't scale to any full drafter-step GEMV. Its useful niche is small constant ops — RoPE/gamma multipliers, RMSNorm constants, fixed permutations. We'll measure ONE constfold column at V8 as a proof-of-concept, then defer it: the systolic path is what scales to a real drafter step.

Memory hierarchy (where each tensor lives)

Corrected V80 budget per Vivado 2026-05-20: the real V80 (xcv80-lsva4737-2MHP-e-S) has 16.84 MB BRAM (3,741 RAMB36) and 67.7 MB URAM (1,925 blocks) — ~80% more than the starter kit's table. The drafter's 48 MB qweight fits cleanly on-chip in multiple ways:

✅ SILICON-CONFIRMED (2026-07-09): Layout B (the winner row) is not just the design choice — it is proven on the card: WIDE PDI 0xE4B7110E runs the whole body with all 2-bit qweight (incl. G9) in URAM, zero HBM compute traffic, M7 8/8, URAM 1,710/88.83% binding. Only the 788 MB fp16 lm-head lives in HBM (G8 gather). The A/C rows below are the rejected alternatives, kept for the rationale.

LayoutURAM useBRAM useHBM compute trafficComplexity
B — All weights in URAM ~48 MB (71%) ~3 MB scales (18%) 0 GB/s simplest — zero ARVALID during compute
A — G9 in BRAM, rest in URAM ~36 MB (53%) ~15 MB (90%) 0 GB/s BRAM >80% util risks timing
C — Block 4 pattern (G9 HBM-streamed) ~36 MB (53%) ~3 MB (18%) ~75 GB/s silicon-validated but adds m_axi bundle

Layout B was not viable under the wrong (smaller) URAM budget — that's why Block 4 went with C. With the real budget it's the obvious win: single top-level kernel, no HBM compute traffic, no streaming buffers, zero ARVALID drivers during compute — which collapses the integration-hang class (V0/V3/V5 root cause) to nothing.

Per-tensor placement (Layout B):

TensorSizeWhereWhy
G4_q/k/v qweight13.5 MBURAMHot, ~370 URAMs at native 72-bit. Read 1×/forward × 3 GEMVs.
G6_o qweight2.25 MBURAMSquare, hot. ~60 URAMs.
G7g + G7u qweight12.0 MBURAMWide-output, big MACs. ~330 URAMs.
G12 qweight8.0 MBURAMTiny K=256, huge N — bandwidth not capacity. ~220 URAMs.
G9 qweight12.0 MBHBM-streamedDoesn't fit if everything else is in URAM. Same call Block 4 makes today.
All scales (fp16)~3 MBBRAMTiny, dense, multi-port.
KV cache5–20 MBURAM hot + HBM cold spillDepends on max_ctx.
Embedding table788 MBHBMNo alternative — far too big for on-chip.
RoPE / gamma constants<100 KBLUTROM or BRAMTruly constant, tiny — constfold-able.

URAM subtotal: ~36 MB qweight + KV-hot + bias ≈ 38–43 MB in 45 MB usable URAM (margin). G9 stays HBM-streamed at ~75 GB/s bandwidth, comfortably under a 2-stack budget of ~110 GB/s.

Why not all-HBM

Why not BRAM for G9?

Why not all-URAM

One weight-load path (Layout B — locked 2026-05-20)

With the corrected V80 budget, the cleanest architecture is everything URAM-resident, no streaming buffer:

  1. All 8 two-bit qweight tensors pre-loaded at boot into per-lane URAM banks (one bank per PE column). 48 MB total → 71% URAM utilization.
  2. All scales (~3 MB) in BRAM as per-group lookup tables.
  3. KV cache hot in remaining URAM; cold spill to HBM if context exceeds URAM allotment.
  4. Zero ARVALID drivers during compute. The only m_axi access during forward is the embedding lookup at the start (one read). The integration-hang class (V0/V3/V5 from too many m_axi bundles fighting NoC arbitration) collapses to nothing.

Risk noted: 71% URAM utilization is higher than the v23 timing-failure threshold (55%). v23 was on KV cache (high-rate read/write); read-only weight ROMs should close timing more easily, but unverified at scale. Fallback B2: if URAM-only fails timing, split G9 (12 MB) into BRAM (~71% BRAM util), dropping URAM to 53% — see project_verilog_layout_b_locked.md in memory for full reasoning.

The honest summary

End-state criterionTargetRealistic at "Production" config
Per-forward latency< 0.658 ms (beat GPU drafter)~0.20 ms (3× win) if bottlenecks 1–4 close
Bit-exact vs C++ refYES on every GEMVYES (already proven on per-cell MAC)
Resource fit on V80< 80% any single resource~10% LUT, ~10% URAM, ~1% DSP
Single top-level kernel (avoids integration hang class)YESdesignable but unverified
Energy per token (V80 75W vs H200 700W)9× lower power-per-tokenunmeasured

What gets confirmed at V5 (K=N=64 pilot, LANES=16, K_PAR=4):

Those four numbers tell us whether the Modest → Production extrapolation holds. If V5 lands LUT × 16 ≈ 850 LUT/lane, accumulator slack > +0.5 ns at 300 MHz, and II=1, we're on track for sub-0.3 ms drafter steps.

Prior art — TALOS-V2 (local clone)

The user cloned github.com/y344shi/TALOS-V2-learn into cleanup_EAGLE/TALOS-V2-learn/ — a 1-layer 27-token Karpathy microGPT in SystemVerilog on Cyclone V (Q4.12 fixed-point). It is small, clean, and the synthesis pattern is directly applicable. Full lessons in TALOS_V2_LESSONS.md.

TALOS fileLinesVerdictWhat we take
rtl/src/systolic_matvec16_tile.sv67port as V5 backboneWS systolic FSM (start/busy/done), per-lane acc, parameterized LANES/DATA/ACC. Replace flat-weight bus with gptq2_dequant16, widen K_PAR=16.
rtl/src/include/microgpt_exact_core_math.svh117lift helperssat16, mul_q12, isqrt64, rms_scale_from_sum, and especially the 32-entry exp_neg_q12 PWL LUT for future softmax.
rtl/src/processing_element.sv40reference patternClean MAC + pass-through structure for V6/V7 OS/IS systolic forwarding.
rtl/sim/tb_microgpt_core.sv—methodologyLoad golden hex, drive RTL, compare — same flow we already use.
rtl/python/export_weights.py—pattern$readmemh hex flow — adapt for GPTQ-2bit qweight + fp16 scale ROMs.
INDEX_FOR_EAGLE_PORT.md7672nd-opinion port planSection-by-section reusability + 6-phase port roadmap from a prior-context agent. Cross-check our plan against it.

Critical reframe for V5: instead of writing gemv_lane_int_pilot.sv from scratch, port TALOS's systolic_matvec16_tile.sv (60 lines lifted) and add (a) gptq2_dequant16 integration, (b) K_PAR=16 tree-reduce, (c) generalized COLS for our K=64 pilot. Maybe 80 lines new on top of 60 lifted — half the work of the from-scratch path.

What TALOS does NOT help with: GPTQ-2bit (handled in V4), Versal primitives (Cyclone V is different chip family), AXI4-Lite + QDMA host stack (TALOS uses Avalon + JTAG), K_PAR widening, multi-group sequencing, fp32 scale apply, URAM bank-per-lane Layout B, tree decoding. Those are still our delta.

Decision log

DateDecision / findingSource
2026-07-10 Real GPU traces caught a latent depth-N KV-replay bug no synthetic gate could see: journal replay rd_slot=branch_s never re-keys across survivor commits, but re-parenting is the COMMON case (5/8 non-identity per commit) → depth≥2 would replay a SIBLING's ancestor KV. Fix kv_anc_lane.sv gated on real trees (PASS 0 err; RED arm 120/504). RULE: depth-N gates must drive real/adversarially-re-parented trees, never self-consistent synthetics. reports/results/DEPTHN_GPU_TRACE_CAPTURE.md
2026-07-10 Batch-tree PDI 0xE4B7110F (B=2) built + flashed (route WNS +0.024 MET; SLL cleared; preflash PASS; FLASH_EXIT=0, awaiting cold-AC). Depth-0 expected bit-identical to 0xE4B7110E — an ENABLER proof for depth-N, not a tok/s change. Also: jewel per-SLR purity measured FALSE (E1=1,024 URAM > any SLR's 677/624/624) → E1 spans SLR0-top+SLR1+2×SLR2 rows. reports/PDI_0xE4B7110F_BATCH_TESTPLAN.md · EAGLE4_MEASURED_RESULTS.md
2026-07-09 WIDE G8 PDI 0xE4B7110E is the first correct-AND-fast drafter on silicon. M7 all 19 stages golden, ordered 8/8; 6,964.8 µs/forward (N=60 median, ctx=1) = 4.9× over the 33.84 ms scalar baseline. HONEST: 4.9× is below the 28× xsim model (consume-bound fabric); still 10.6× slower than a GPU forward (658 µs). Correctness anchor every speed build must reproduce. reports/evidence/wide-g8-silicon/WIDE_G8_SILICON_GAIN.md
2026-07-09 Host fast-path lifts E2E 3.20 → 5.46 tok/s (1.71×), host-only (C MMIO store loop + in-process pread; no RTL/flash). Accept 1.78 unchanged, M7 8/8 intact. CORRECTION: forward is ctx-dependent (6.97 ms @ctx0 → 17.6 ms @ctx620, G5 ctx-scan) — this, not host glue, caps E2E. reports/results/HOST_FASTPATH_RESULT.md
2026-07-10 SEL_PIPE breaks the reclock wall: full-top in-context OOC @100 MHz = +0.087 ns MET, 0 failing endpoints (baseline −2.935; module OPT1F −3.452 → SELPIPE +0.426). Bit-exact, cycle-identical, URAM 88.83% preserved. New worst cone = tree→g8 seam, not the selector. Post-route @100 MHz NOT yet proven (OOC optimistic ~3.3 ns derate). URAM-OREG measured a no-op (both +5.291) → deprioritized. verilog/forks/sel_pipe/build/ooc_top_selpipe_100m/timing_100m.rpt
2026-07-09 Accept 1.78 → 2.13 gap = DEPTH, not fidelity. Ours today = depth-0 width-8 (accept 1.78); GPU = depth-3 tree width-8 (accept 2.13). Batch-tree PDI 0xE4B7110F (B=2, sibling weight-reuse, depth-0 bit-identical) building to enable affordable depth-N. Prior measurement: batch-B is URAM/BRAM-neutral but LUT-expensive (+233K/unit) → batch-4-on-all-3-engines is LUT-infeasible; B=1 default, opt-in. docs/TOKS_ROADMAP_FOR_OPUS.md · commit 04016a8
2026-05-20 Verilog exploration opened; 8 GEMVs across 5 shape classes catalogued; numerical contract pinned to gptq2_gemv. 2BIT_MM_INVENTORY.md
2026-05-20 Two MAC variants land xsim PASS on 1000 stimuli vs Python golden. Both are bit-exact. build/log/xsim.log
2026-05-20 ooc-synth: mac_2bit_mult 53 LUT / 0 DSP / 1.19 ns vs mac_2bit_shift_add 77 LUT / 0 DSP / 1.29 ns. Synth automatically infers LUT-carry for 16×3-bit MACs even when written with *. Explicit shift-add deferred until LANES-scale validation. build/log/util_mac_2bit_*.rpt
2026-05-20 Block 4 "0.139 ms / 4.73× faster than GPU" silicon claim confirmed FALSE by user. Real per-forward is csynth ~5.5 ms. Paper headline framing pending revision (see ~/.claude/.../memory/project_block4_retraction_scope.md). conversation
2026-05-20 V80 resource correction: real budget per Vivado is BRAM 16.84 MB / URAM 67.7 MB (1,925 blocks) — ~80% more than starter-kit's smaller-SKU numbers. The "48 MB doesn't fit URAM" claim earlier was wrong. build/log/util_mac_2bit_*.rpt · memory project_v80_actual_resources.md
2026-05-20 Layout B locked. All 48 MB qweight in URAM (71%), scales in BRAM, zero HBM compute traffic. Single top-level kernel; integration-hang class (V0/V3/V5) eliminated by construction. V5+ systolic designs assume per-lane URAM banks — drop the streaming-buffer plan. user decision · memory project_verilog_layout_b_locked.md
2026-05-20 TALOS-V2 prior-art clone surfaced. Lift systolic_matvec16_tile.sv as V5 backbone; lift exp_neg_q12 PWL LUT for future softmax. gemv_lane_int_pilot.sv from-scratch start absorbed/discarded in favor of TALOS port. memory reference_talos_v2_prior_art.md · verilog/docs/TALOS_V2_LESSONS.md

Open questions

✅ MOSTLY ANSWERED BY SILICON since this list was written (kept as the historical record): fp32-vs-INT32 accumulator → shipped INT32 + post-scale, proven M7 8/8 bit-exact vs eagle4_ref; activation dtype → INT16 seams with re-provisioned fracs, 8/8 on real inputs (MATCH-REALINPUT); PE array dimensions → golden locked at LANES=128 × K_PAR=16 (K_PAR can't widen — URAM wall); activation broadcast cost → real at 128 lanes: the cross-SLR BCAST tree caused the SLL route wall, solved by the BalancedSLR subdirective (not H-tree redesign); per-group dequant scope → per-group scale in BRAM shipped (scale_bram + gemv_lane_qscale), bit-exact. The LIVE open questions now live in docs/TOKS_ROADMAP_FOR_OPUS.md §5 (reclock fallback, depth-vs-denominator order, FF-seam waiver, speed-vs-batch line split) and docs/DESIGN_KV_ONCHIP_DEPTHN.md §7 (journal mux fan-in, MAX_CTX audit).
  1. Per-group dequant scope. Scale per-group (inside inner loop) or per-output (after full sum)? Per-output is fewer fp multiplies but loses per-group precision — measure rtol degradation.
  2. fp32 vs INT32 accumulator. HLS uses INT32 + post-scale-cast. fp32 matches reference exactly but costs more. Measure both.
  3. Activation dtype. Reference is fp16 in / fp16 out; internal can be fp32. INT8 activation shrinks the MAC but introduces an extra error source.
  4. PE array dimensions. LANES × DEPTH is the main resource-vs-throughput knob.
  5. Activation broadcast cost. Above ~16 lanes, fanout becomes a timing concern; consider H-tree or systolic activation distribution.

Next planned step

Raise end-to-end tokens/s from the 5.46 tok/s silicon anchor per the multi-night lever plan in docs/TOKS_ROADMAP_FOR_OPUS.md (5.46 → ~20–30 tok/s modeled — each [MODELED] rung is replaced by a silicon measurement as it builds). Ordered levers:

  1. Reclock to 100 MHz. SEL_PIPE closes the module + full-top in-context OOC at 100 MHz (+0.087 ns MET, 0 failing endpoints; baseline was −2.935). Fold SELPIPE into a reclock PDI and let post-route body-WNS be the arbiter (OOC is optimistic vs post-route). ~1.7–1.8× on the whole forward if it holds.
  2. Batch-tree (B=2) PDI 0xE4B7110F — ✅ SILICON-VALIDATED (2026-07-10, T1–T5 ALL PASS): depth-0 regression ordered 8/8 + all 19 CRCs (bit-identical to 0xE4B7110E); batch-test 4/4 — one URAM weight read fed BOTH sibling MAC sets on silicon; timing 6.965 ms (+0.07%, no LUT penalty); endurance 100/100 bit-identical; SGLang 5.75 tok/s @ accept 1.78 (no-regression). The sibling weight-reuse architecture EXISTS in silicon — depth-N builds on it. reports/PDI_0xE4B7110F_BATCH_TESTPLAN.md §2. Placement note: the batch triggered a wholesale SLR rebalance — device SLICE 92.66% is the new wall (not LUT 58%); the +0.024 margin owner is an AVED-shell reset recovery net (congestion canary, watchlisted) — A3: reports/results/BATCH_LUT_PRESSURE_A3.md.
  3. Depth-N tree decode — closes the accept 1.78 → 2.13 gap (the gap is DEPTH, not datapath fidelity).
  4. G8 throughput — TREE_P8 + 32-channel HBM G8 gather (the forward is now ~4 ms G8 + ~3 ms body/tree; G8 is no longer the lone bottleneck).

No [MODELED] number ships as achieved — the nightly cadence exists to replace each with a [SILICON] measurement, and any over-claim is corrected the moment a measurement contradicts it.

How this page stays current

This file is generated and refreshed by the fpga-eagle4-accelerator agent on demand. The maintenance prompt lives at verilog/docs/STATUS_UPDATE_PROCEDURE.md — it tells the agent which source files, synth reports, sim logs, and memory entries to scan, and how to merge them into this HTML.

Trigger an update whenever: (a) a new .sv module lands, (b) make synth_* produces new util/timing reports, (c) make * xsim run flips PASS↔FAIL, (d) a measurement at a new shape class arrives, or (e) the user makes a decision worth recording in the decision log.

EAGLE4 / Llama-3.2-3B / GPTQ-2bit / V80 Versal HBM / Vitis 2025.1.
Numerical contract: reference/include/eagle4_gemm_abstract.h.
Hard rules: reports/VERILOG_STARTER_KIT.md §11.
Memory: ~/.claude/projects/.../memory/MEMORY.md.