EAGLE4 hardware research · author box1-qwen-junior · 2026-11-09 · source reports/results/DRAFTER_INTERFACE_ONEPAGER_4C1_20261109.md at repo commit 5ea8c7c7e · file references are repo-relative paths (path:line).

4-card drafter: how to drive it from another verifier (one page, 2026-11-09)

What it is. Four AMD V80 cards on the talon2 server draft a whole speculative-decoding token tree per step for a target LLM (Llama-3.2-3B or Qwen3.8-27B). Two "ATT" cards split the attention heads (16 each); two "MLPV" cards run the MLP, the vocabulary head and the tree search. A Python host process on talon2 drives them over PCIe; today an H200 GPU verifies. Measured on talon2 2026-11-09: 883 tok/s at accept 6.90 (Llama, depth 11, width 2); the cards take 3.4 ms per depth-11 tree.

 verifier --(PCIe/host)--> host process --PCIe--> ATT 17 <--QSFP--> ATT 3d
     ^                                              |QSFP               |QSFP
     +------- tree (tokens, parents, scores) <-- MLPV 63 <--QSFP--> MLPV 50

Each step, per request (stream). All integers little-endian.

directionfieldsize Llama-3.2-3Bsize Qwen3.8-27Bdtype
verifier -> drafterlast accepted token id (new tree root)1 id1 idint (vocab 128,256 / 248,320)
verifier -> draftertarget hidden states of that token: 3 layers3 x 3,0723 x 5,120 (layers 3, 31, 59)fp16 / bf16
verifier -> draftersame pair (token + 3 hidden) for every other accepted tokenper token as aboveper token as aboveas above
verifier -> draftercontext length, tree depth, width3 ints3 intsint
host -> cards (internal)embedding + fused hidden of the root12,288 B12,288 B2 x 3,072 int16 fixed point
drafter -> verifiertree nodes: token, parent, scoreW + W^2(D-1) nodes + root (43 at width 2, depth 11)sameint64 token, int parent index, float32 probability

The host turns the 3 hidden layers into the card row (embedding lookup + one matrix multiply, fc). Today the GPU also computes the small drafter's KV cache; a card-only mode that builds it from the same rows is measured on the cards with the same accept and 14-25 % lower speed.

Two ways to connect.

Driver files to read.

Full guide (access, every register, timings, run commands, traps): [DRAFTER_INTERFACE_GUIDE_4C1_20261109.md](DRAFTER_INTERFACE_GUIDE_4C1_20261109.md).