EAGLE4 weight residency

Where every tensor of the current drafter checkpoint lives across the two V80 cards, how much room is left, and how large a model four cards could hold. Labels: M measured, D derived, P projected.

The checkpoint

One transformer layer with Llama-3.2-3B dimensions: hidden 3072, 24 attention heads × 128, MLP 8192, vocabulary 128,256. The QKV input is the concatenation of the token embedding and the previous hidden state (2 × 3072). The down projection keeps both output halves: to_logits and for_reasoning, which seeds the next tree level. Source: eagle4-dashboard/site/weight_residency.md.

Tensor by tensor

TensorShapeFormatSizeMemoryCard
QKV projection6144 → 92162-bit, group 12813.50 MiBURAMHEAD
o-projection3072 → 30722-bit2.25 MiBURAMHEAD
MLP gate3072 → 81922-bit6.00 MiBURAMHEAD
MLP up3072 → 81922-bit6.00 MiBURAMHEAD
MLP down8192 → 61442-bit6.00 MiBURAMTAIL
Cheap vocabulary head256 → 128,2562-bit7.83 MiBURAMTAIL
2-bit body41.58 MiBabout 174M weights, all on-chip
K cachectx × 24 × 128int16, post-RoPE6 KiB/tokenURAMHEAD
V cachectx × 24 × 128int166 KiB/tokenBRAMHEAD
Dequantization scalesone per 128 weightsQ3.13~1.4 MiBBRAMboth
RMSNorm gains × 43072 eachQ4.1224 KiBBRAMboth
Low-rank LM projection (GLM)3072 → 256Q1.151.50 MiBBRAMTAIL
Exact fp16 lm-head (rescore)128,256 × 3072fp16788 MBHBMTAIL (two copies on G8 N=4 TAILs: 1.58 GB)
Embedding table128,256 × 3072bf16788 MBHBMHEAD (child-branch rows, depth ≥ 1)
The invariant. During a forward pass no weight bit moves. The only off-chip traffic is the lm-head rescore gather and the child-branch embedding rows.

Capacity per V80 card

MemoryBlocksPer blockTotalSource
URAM1,92536 KiB67.7 MiBplaced utilization report M
BRAM (36 Kb tiles)3,7414.5 KiB16.4 MiBplaced utilization report M
HBM32 GBAMD V80 spec

URAM in use on placed images, per die (SLR0 / SLR1 / SLR2) M, from /shared/rnd_hq/2026-09-28/VIVADO_SLR_SUMMARY_full.txt:

ImageURAM % SLR0 / 1 / 2BRAM % SLR0 / 1 / 2
HX1 HEAD (record)64.40 / 62.18 / 60.2618.49 / 79.17 / 73.50
FB2 TAIL (FS/FSL lineage)70.16 / 26.44 / 30.7728.34 / 62.67 / 8.33

The TAIL has spare URAM (about 460 blocks free in SLR1 M). SLR1 logic, not memory, is the wall on both cards: slices are at 98–100% there.

How large a model four cards could hold

QuantityValueLabel
On-chip memory, 4 cards (URAM + BRAM)4 × 84 MiB = 336 MiBD
Realistically fillable with weights (~70%, leaving room for KV cache, scales, routing)~235 MiBP
2-bit weights that fit, after the 6.25% scale overhead~0.9 BP
Versus today's body (41.58 MiB, ~174M weights)~5×D
Cost of one extra 3072-wide layer (QKV 3072→9216, o, gate, up, down)~27 MiBD
Cost of one wider layer (hidden 4096, MLP 14336)~45 MiBD
So, at 2 bits: today's layer plus about 6 more 3072-wide layers, or 1–2 wider layersP
At 4-bit weightshalf of the aboveD
HBM, 4 cards (lm-head, embeddings, anything streamed)128 GBAMD spec

The real limit at batch 1 is latency, not capacity. Each added layer adds roughly one more HEAD-sized pass per tree level, about 380 µs per item today M (2026-09-28/tracks/tail_algo/TAIL_ALGO.md). A deeper drafter only pays off if acceptance rises by more than that cost. Streaming layer weights from HBM instead of holding them on-chip would cost milliseconds per forward and breaks the design.

Compiled 2026-09-29 by hq-junior from weight_residency.md, the placed utilization reports and HQ findings. Silicon numbers belong to the pair in use on 2026-09-29 (HX1 HEAD + FSL TAIL).

Ask HQ about this page

For the project lead: ask for supporting data, a clarification or a re-test. The answerer can only read HQ records and queue a re-test for the lab; it cannot build, flash or change anything. Answers arrive within a few minutes.