Six AMD Alveo V80 FPGAs run a 2-bit EAGLE-style drafter and send a token tree to one NVIDIA H200, which verifies it in a single pass. Target model: Llama-3.2-3B. The comparison that matters is the same H200 running the same EAGLE method on its own. Below the dashed box: measured numbers, plus the stated drafter energy per forward (derived, method labelled); every other ratio is inside the box.
| Dataset | vs best GPU-only | vs GPU-only d11 (matched) | vs no speculation | GPU-only vs no spec. |
|---|---|---|---|---|
| HumanEval (code) | 1.112× vs d9 | 1.138× | 3.06× | 2.75× |
| MT-Bench (chat) | 1.144× vs d9 | 1.228× | 1.82× | 1.59× |
| GSM8K (math) | 1.100× vs d9 | 1.142× | 2.87× | 2.61× |
| Alpaca (instruct) | 1.147× vs d9 | 1.242× | 1.73× | 1.51× |
| Natural Q. (QA) | 1.188× vs d9 | 1.290× | 1.40× | 1.18× |
| Arithmetic mean (5 datasets) | 1.138× | 1.208× | 2.18× | 1.93× |
| Geometric mean (5 datasets) | 1.138× | 1.207× | 2.08× | 1.83× |
| Dataset (80 prompts) | FPGA + H200, d11 | H200 only, EAGLE d9 | H200 only, EAGLE d11 | H200, no speculation | Identical texts /80FPGA–AR · GPU d9–AR · FPGA–GPU d9 |
|---|---|---|---|---|---|
| HumanEval (code) | 933.1 @ 6.91n=2 | 838.8 @ 6.63n=2 | 820.3 @ 7.05n=2 | 305.4 n=2 | 79 · 74 · 75 |
| MT-Bench (chat) | 557.8 @ 3.67n=2 | 487.6 @ 3.80n=2 | 454.1 @ 3.85n=2 | 306.1 n=2 | 71 · 71 · 70 |
| GSM8K (math) | 877.7 @ 6.46n=2 | 797.6 @ 6.30n=2 | 768.3 @ 6.61n=2 | 306.1 n=2 | 77 · 77 · 77 |
| Alpaca (instruct) | 530.5 @ 3.46n=2 | 462.5 @ 3.61n=2 | 427.2 @ 3.62n=2 | 306.2 n=2 | 76 · 74 · 76 |
| Natural Q. (QA) | 430.4 @ 2.71n=2 | 362.3 @ 2.81n=2 | 333.6 @ 2.81n=2 | 306.6 n=2 | 73 · 70 · 72 |
tok/s = mean of n runs (HumanEval FPGA: 934.48, 931.81); @ = tokens accepted per verify step; same H200 in every column, run order randomised, SM clock 1,785 MHz throughout. Identical texts = prompts with byte-identical greedy output; no two different configurations match on every prompt, GPU-only included. Every configuration repeats its own texts exactly between rounds (FPGA round 2 vs round 1: 80/80, 80/80, 80/80, 80/80, 80/80).
| Dataset | 6 V80s, W | FPGA mJ / forward | H200 W (GPU-only d9) | GPU d9 mJ / forward | GPU d9 ÷ FPGA | GPU d11 mJ / fwd | d11 ÷ FPGA |
|---|---|---|---|---|---|---|---|
| HumanEval (code) | 323.5 | 44.3 | 262.9 | 97.3 | 2.20× | 97.6 | 2.20× |
| MT-Bench (chat) | 322.6 | 44.2 | 264.3 | 97.8 | 2.21× | 96.7 | 2.19× |
| GSM8K (math) | 324.4 | 44.4 | 262.7 | 97.2 | 2.19× | 95.6 | 2.15× |
| Alpaca (instruct) | 321.5 | 44.0 | 263.4 | 97.4 | 2.21× | 96.6 | 2.19× |
| Natural Q. (QA) | 321.2 | 44.0 | 265.6 | 98.3 | 2.23× | 97.8 | 2.22× |
| 5-dataset mean | 322.7 | 44.2 | 263.8 | 97.6 | 2.21× | 96.9 | 2.19× |
Method: time per forward × device run-mean power, per device, never summed (white paper 11-04 method). FPGA 0.137 ms/forward (measured on the benchmark boot) × six-V80 run-mean W; GPU 0.37 ms/forward (11-04 value) × H200 GPU-only run-mean W, which includes verify. Powers measured, energies derived; inputs, sensitivity and the per-session view in the box.
| Configuration | H200 board power | 6 V80 cards, sum | H200 SM clock, MHz | runs |
|---|---|---|---|---|
| FPGA drafter + H200 verify (d11) | 243.2 W | 322.6 W (51-57 W per card) | 1,785 | 10 |
| H200 alone, EAGLE d9 | 263.8 W | 318.8 W (51-56 W per card) | 1,785 | 10 |
| H200 alone, EAGLE d11 | 261.8 W | 318.7 W (51-56 W per card) | 1,785 | 10 |
| H200 alone, no speculation (AR) | 335.4 W | 318.9 W (51-56 W per card) | 1,785 | 10 |
H200: nvidia-smi, 1 Hz. V80: on-card Total_Power, ~2 s, six cards summed; idle-but-powered in GPU-only and AR runs. Host CPUs, fans and PSU not included.
| Work / system | Model (size) | Headline as published | Stream | Hardware, power, price |
|---|---|---|---|---|
| EAGLE4 on Eagle Talon (this sheet) | Llama-3.2-3B, 2-bit drafter, 16-bit target | 933.1 tok/s @ τ 6.91 (HumanEval, d11) measured | batch 1 | 6× AMD Alveo V80 + 1× H200 NVL; power measured (above) |
| EAGLE-3 (2025)[1] | Llama-3.1-8B | 4.44× vs AR (HumanEval 4.85×); τ 6.23 (HumanEval 6.74). In SGLang: 373.25 tok/s vs 158.34 no-spec | batch 1 | 1× H100 (SGLang table) |
| Medusa (2024)[3,2] | Vicuna-7B/13B | 2.2× (Medusa-1), 2.3-3.6× (Medusa-2); τ 2.44 (Vicuna-13B) | batch 1 | GPU |
| EAGLE-2 (2024)[2] | Vicuna-13B; LLaMA3-8B | 4.04× mean (Vicuna-13B); 3.46× MT-Bench (L3-8B); τ 4.65 / 4.53 | batch 1 | GPU (not named) |
| SpecInfer (ASPLOS'24)[4] | LLaMA / OPT family | 1.5-2.8× vs other serving systems, not vs AR | serving | multi-GPU |
| Lookahead decoding (2024)[5,2] | LLaMA-2 / CodeLlama | up to 1.8× MT-Bench; τ 1.62 (Vicuna-13B) | batch 1 | 1 GPU |
| Cerebras Inference[6,7,8,9] | Llama-3.1-8B, 16-bit Llama-3.1/3.3-70B + spec | 1,800 tok/s (8B) 2,100 tok/s (70B, spec. decoding) | per user | CS-3 wafer, 23 kW per system (vendor); est. $2-4M |
| Groq LPU[10,11,12,13] | Llama-3-8B; Llama-3.1-8B Llama-3.3-70B specdec | 877 tok/s (AA, 2024-04); ~560 tok/s (docs, 2026-11) 1,665 tok/s | per user | GroqCard 275 W TDP; 576 chips for a 70B (2023) |
| SambaNova SN40L[14,15] | Llama-3-8B, 16-bit | 1,084 tok/s (AA) | per user | 16-chip rack, 10 kW typical (vendor) |
| NVIDIA + TensorRT-LLM spec. decoding[16,17] | Llama-3.3-70B FP8 Llama-4 Maverick 400B MoE | 181.74 tok/s, 1B draft (3.55× vs 51.14) >1,000 tok/s/user | batch 1 per user | 1× H200 (600 W) DGX B200, 8 GPUs |
| FlightLLM (FPGA'24)[18] | LLaMA2-7B, ~3.5-bit | 92.5 tok/s (VHK158); 6.0× energy efficiency vs V100S (U280) | batch 1 | Versal VHK158 / Alveo U280, whole model on FPGA |
| DFVG (closest prior art)[19] | not retrieved | 3.26× speedup (index snippet only, unverified) | n/a | FPGA drafts, GPU verifies |
| HeteroReason (2026)[20] | reasoning models | 1.01-1.42× latency vs GPU; 1.25-1.57× energy efficiency | n/a | FPGA draft, GPU target |
Batch-1 speed falls with model size: a 3B number is not comparable with an 8B or 70B one. "Per user" = speed one request sees; vendor numbers date-drift. Paper speedups are vs autoregressive decoding on each paper's stack (SpecInfer: vs serving systems). AA = Artificial Analysis. Compare τ on the same dataset only.