Llama-3.2-3B · EAGLE4 tree drafting · talon2 · 2026-11-09/10

4 V80 FPGAs draft 1.24× faster than the H200

Same target model, same prompts, same tree settings, same H200 doing the verify. The only difference: who builds the draft tree, four Alveo V80 cards or the H200 itself. Humaneval, 80 prompts × 256 new tokens, tree depth 15, width 2, 20 verify tokens, fa4 draft attention backend on both sides.

H200 + 4 × V80 drafter
870.10tok/s
accept 7.12 · 2,754 verify steps · 19,618 tokens
H200 alone (drafts on the GPU)
700.71tok/s
accept 6.92 · 2,829 verify steps · 19,563 tokens

The deeper the tree, the bigger the lead

H200 + 4 × V80 drafterH200 aloneLabel = FPGA ÷ H200

Every run in the chart

TreeDraft backendFPGA tok/sFPGA runsFPGA acceptH200 tok/sH200 runsH200 acceptFPGA ÷ H200
Hardware. talon2: one H200 NVL + four AMD Alveo V80 cards. Attention cards 17/3d run image 0xE4B713F9, MLP cards 50/63 run 0x1342 (11-09) or 0x134C (11-10, depth 13/15 fa3 rows; same golden trees as 0x1342). SGLang 0.5.16, CUDA graphs on, one request at a time, greedy.
Same flags both sides. --draft-tree-depth --draft-tree-width 2 --verify-tokens-num --speculative-draft-attention-backend; FPGA = --speculative-exec-mode 1, H200 alone = -1. The FPGA side also uses a card-side tree-stop threshold (−2.5), which has no GPU equivalent.
Draft backend. With fa4 the H200's own tree drafting accepts fewer tokens (depth 15: 6.92 vs 7.34 with fa3). With fa3 the depth-15 lead is 1.17× (866 vs 740 tok/s).
Sources. FPGA: F3T_CONFIG_SWEEP_20261109.md §1 and the 11-10 sweep (tracker line 1585). H200 alone: same report §2 and demo_cfg_20261110/gpu3.out (11-10 19:35–19:44Z). Values with several runs are means.