EagleEagle Livereading card…

Speculative decoding on four FPGAs

An EAGLE drafter runs on four AMD V80 cards on box1 (an attention pair and an MLP/vocabulary pair, two cards each) and proposes a tree of tokens at once; an H200 checks the tree against the target model in one step. One FPGA image serves two targets, switched at runtime: Llama-3.2-3B and Qwen3.8-27B. Chat with it below and watch the bursts.

H200 + 4 FPGAs (our drafter)870.10tok/saccept 7.12. Llama-3.2-3B, humaneval 80 × 256, width 2, depth 15, 20 verify tokens, fa4, same H200.
H200 + EAGLE (drafting for itself)700.71tok/saccept 6.92, same configuration.
FPGA drafter vs H200 + EAGLE1.24×fastersame prompts, same tree, same H200; only the drafter differs.
Drafts on four V80s, verifies on the H200. Greedy decoding.

Ask anything, then watch how the answer arrives.

Each gold block is one streamed step. Wide blocks mean the draft guessed several tokens right at once.
Attention on two V80s

The attention pair splits the drafter's heads across two cards (12 Qwen heads or 16 Llama heads each), with low-bit weights held in on-chip memory and the prompt's keys and values imported from the GPU.

MLP and vocabulary on two V80s

The second pair finishes the layer, scores the vocabulary and grows the draft tree. The four cards pass partial sums over direct QSFP links.

Verify on the H200

The GPU runs the target (Llama-3.2-3B or Qwen3.8-27B) once over the whole tree and keeps the longest correct branch. The output is exactly what the GPU alone would write.