Where every tensor of the current drafter checkpoint lives across the two V80 cards, how much room is left, and how large a model four cards could hold. Labels: M measured, D derived, P projected.
One transformer layer with Llama-3.2-3B dimensions: hidden 3072, 24 attention heads × 128, MLP 8192, vocabulary 128,256. The QKV input is the concatenation of the token embedding and the previous hidden state (2 × 3072). The down projection keeps both output halves: to_logits and for_reasoning, which seeds the next tree level. Source: eagle4-dashboard/site/weight_residency.md.
| Tensor | Shape | Format | Size | Memory | Card |
|---|---|---|---|---|---|
| QKV projection | 6144 → 9216 | 2-bit, group 128 | 13.50 MiB | URAM | HEAD |
| o-projection | 3072 → 3072 | 2-bit | 2.25 MiB | URAM | HEAD |
| MLP gate | 3072 → 8192 | 2-bit | 6.00 MiB | URAM | HEAD |
| MLP up | 3072 → 8192 | 2-bit | 6.00 MiB | URAM | HEAD |
| MLP down | 8192 → 6144 | 2-bit | 6.00 MiB | URAM | TAIL |
| Cheap vocabulary head | 256 → 128,256 | 2-bit | 7.83 MiB | URAM | TAIL |
| 2-bit body | 41.58 MiB | about 174M weights, all on-chip | |||
| K cache | ctx × 24 × 128 | int16, post-RoPE | 6 KiB/token | URAM | HEAD |
| V cache | ctx × 24 × 128 | int16 | 6 KiB/token | BRAM | HEAD |
| Dequantization scales | one per 128 weights | Q3.13 | ~1.4 MiB | BRAM | both |
| RMSNorm gains × 4 | 3072 each | Q4.12 | 24 KiB | BRAM | both |
| Low-rank LM projection (GLM) | 3072 → 256 | Q1.15 | 1.50 MiB | BRAM | TAIL |
| Exact fp16 lm-head (rescore) | 128,256 × 3072 | fp16 | 788 MB | HBM | TAIL (two copies on G8 N=4 TAILs: 1.58 GB) |
| Embedding table | 128,256 × 3072 | bf16 | 788 MB | HBM | HEAD (child-branch rows, depth ≥ 1) |
| Memory | Blocks | Per block | Total | Source |
|---|---|---|---|---|
| URAM | 1,925 | 36 KiB | 67.7 MiB | placed utilization report M |
| BRAM (36 Kb tiles) | 3,741 | 4.5 KiB | 16.4 MiB | placed utilization report M |
| HBM | 32 GB | AMD V80 spec |
URAM in use on placed images, per die (SLR0 / SLR1 / SLR2) M, from /shared/rnd_hq/2026-09-28/VIVADO_SLR_SUMMARY_full.txt:
| Image | URAM % SLR0 / 1 / 2 | BRAM % SLR0 / 1 / 2 |
|---|---|---|
| HX1 HEAD (record) | 64.40 / 62.18 / 60.26 | 18.49 / 79.17 / 73.50 |
| FB2 TAIL (FS/FSL lineage) | 70.16 / 26.44 / 30.77 | 28.34 / 62.67 / 8.33 |
The TAIL has spare URAM (about 460 blocks free in SLR1 M). SLR1 logic, not memory, is the wall on both cards: slices are at 98–100% there.
| Quantity | Value | Label |
|---|---|---|
| On-chip memory, 4 cards (URAM + BRAM) | 4 × 84 MiB = 336 MiB | D |
| Realistically fillable with weights (~70%, leaving room for KV cache, scales, routing) | ~235 MiB | P |
| 2-bit weights that fit, after the 6.25% scale overhead | ~0.9 B | P |
| Versus today's body (41.58 MiB, ~174M weights) | ~5× | D |
| Cost of one extra 3072-wide layer (QKV 3072→9216, o, gate, up, down) | ~27 MiB | D |
| Cost of one wider layer (hidden 4096, MLP 14336) | ~45 MiB | D |
| So, at 2 bits: today's layer plus about 6 more 3072-wide layers, or 1–2 wider layers | P | |
| At 4-bit weights | half of the above | D |
| HBM, 4 cards (lm-head, embeddings, anything streamed) | 128 GB | AMD spec |
The real limit at batch 1 is latency, not capacity. Each added layer adds roughly one more HEAD-sized pass per tree level, about 380 µs per item today M (2026-09-28/tracks/tail_algo/TAIL_ALGO.md). A deeper drafter only pays off if acceptance rises by more than that cost. Streaming layer weights from HBM instead of holding them on-chip would cost milliseconds per forward and breaks the design.
Compiled 2026-09-29 by hq-junior from weight_residency.md, the placed utilization reports and HQ findings. Silicon numbers belong to the pair in use on 2026-09-29 (HX1 HEAD + FSL TAIL).
For the project lead: ask for supporting data, a clarification or a re-test. The answerer can only read HQ records and queue a re-test for the lab; it cannot build, flash or change anything. Answers arrive within a few minutes.