Every stage of the drafter, in the order the hardware runs it. Each block's
shape encodes its operation class; the label under the name is the matrix it computes
[out×in]; the line between blocks is annotated with the vector width flowing; a
small colored tag marks where the weights/data live (URAM / BRAM / HBM / host). Two distinct
silicon engines do the heavy lifting — see the color of each block and the note below the diagram.
Numbers are verified against reference/include/eagle4_gemm_abstract.h.
Block width is schematic (not to time-scale) — for the time-proportional silicon
datapath see the silicon datapath page.
G8's HBM gather (751 MiB) is the wall-clock bottleneck
(56–93% of the forward) → NOT the bottleneck: G8 = 10.93% of AP-BUSY at the
shipped operating point (md=2, K_C=64); 49.4% only at K_C=512, which is not shipped. The MAC count was never the
issue either — but neither is the gather, at the point the image actually runs.
§ enginesTwo engines, two precisions, two silicon fabrics
2-bit body GEMV engine URAM-resident weights
Every body projection — G4, G6, G7g, G7u, G9, G12 — is a matrix–vector
product of 2-bit GPTQ weights (group-size 128, fp16 per-group scales) against INT16 activations.
Runs on a Wallace 3:2 CSA-tree LUT MAC array (u_array); DSP58 declines at
2-bit granularity, so the reduction is pure LUT. A single DSP58 per lane applies the
Q1.15 dequant scale. LANES=128 output columns × K_PAR=16 reduction. All weights live in
URAM — zero HBM traffic at compute time.
INT16×INT16 DSP58 attention engine separate fabric
Attention (G5) is a different precision — INT16×INT16 dot products
(attn_score_dot) mapped onto DSP58, time-multiplexed across the 24 heads.
HEAD_PAR=6 replicates the engine 6× for a 6.00× head-parallel speedup. K-cache
lives in URAM, V-cache in BRAM; there is no weight matrix — the operands are the
per-branch KV context. Different precision ⇒ different silicon, kept off the GEMV array.
Norm (G1/G11), residual adds, SiLU (G7p) and the TREE funnel are lightweight
scalar/elementwise logic on neither heavy engine. fc (fp16 root fuse) runs off-FPGA on the
host/GPU and only at depth-0; depth≥1 reuses the parent's for_reasoning and skips it.
§ weightsPer-operation compute & memory requirements
Sizes re-derived as params × bits / 8. 2-bit = params/4 bytes;
fp16 = params×2 bytes. MiB = 1,048,576 bytes.
| Stage | Operation | Matrix [out×in] | Dtype | Params | Size | Resident | MAC / fwd |
|---|---|---|---|---|---|---|---|
| fc | root fuse (3×h → 1) | [3072×9216] | fp16 | 28.31 M | 54.0 MiB | host/GPU | 28.3 M* |
| G1 | RMSNorm (input emb) | γ[3072] | fp16 | 3,072 | 6 KiB | BRAM | — |
| G4 | QKV projection GEMV | [9216×6144] | 2-bit | 56.62 M | 13.50 MiB | URAM | 56.6 M |
| G5 | attention, 24-head (MHA) | KV [ctx×24×128] | INT16 | — | ∝ ctx | K URAM V BRAM | ∝ ctx |
| G6 | O-projection GEMV | [3072×3072] | 2-bit | 9.44 M | 2.25 MiB | URAM | 9.4 M |
| G7g | gate GEMV | [8192×3072] | 2-bit | 25.17 M | 6.00 MiB | URAM | 25.2 M |
| G7u | up GEMV | [8192×3072] | 2-bit | 25.17 M | 6.00 MiB | URAM | 25.2 M |
| G7p | SiLU(gate)·up | elementwise [8192] | INT16 | — | — | FF/regs | — |
| G9 | down GEMV | [3072×8192] | 2-bit | 25.17 M | 6.00 MiB | URAM | 25.2 M |
| G11 | final RMSNorm | γ[3072] | fp16 | 3,072 | 6 KiB | BRAM | — |
| GLM | lmdown low-rank proj | [256×3072] | fp16 | 0.79 M | 1.50 MiB | URAM/BRAM | 0.8 M |
| G12 | cheap lm-head logits | [128256×256] | 2-bit | 32.83 M | 7.83 MiB | URAM | 32.8 M |
| TREE | top-K_C survivor select | K_C=512 ← 128256 | — | — | — | on-chip | — |
| G8 | exact rescore = gather + dot | [128256×3072] | fp16 | 394.0 M | 751 MiB (788 MB) |
HBM | 1.57 M** |
| Total 2-bit body weight (G4+G6+G7g+G7u+G9+G12) | 41.58 MiB | URAM | 175.2 M‡ | ||||
* fc is fp16 and runs off-FPGA (host/GPU), only at the depth-0 root — not a per-forward
FPGA cost.
** G8 multiplies only the K_C=512 gathered rows (512×3072 = 1.57 M MAC), but must
fetch 751 MiB (788 MB) of fp16 lm-head from HBM — it is memory-bound, and this
gather (not the MAC) dominates wall-time at
56–93% of the forward → costs 10.93% of AP-BUSY at the shipped
operating point (md=2, K_C=64), 49.4% at the unshipped K_C=512.
‡ Total body MAC/forward ≈ 175.2 M (G4+G6+G7g+G7u+G9+GLM+G12, attention excluded —
it scales with context). The 2-bit body weights (≈ 41.6 MiB) fit entirely in URAM, so the
only HBM traffic in the whole forward is G8's exact-rescore gather.