Eagle EAGLE4 drafter
Specification sheet · measured 2026-11-13
FPGA speculative-decoding drafter · single-user speed

EAGLE4 on Eagle Talon

Six AMD Alveo V80 FPGAs run a 2-bit EAGLE-style drafter and send a token tree to one NVIDIA H200, which verifies it in a single pass. Target model: Llama-3.2-3B. The comparison that matters is the same H200 running the same EAGLE method on its own. Below the dashed box: measured numbers, plus the stated drafter energy per forward (derived, method labelled); every other ratio is inside the box.

933.1tok/s
Llama-3.2-3B, one user, HumanEval 80×256, FPGA drafter + H200, measured
838.8tok/s
same H200, GPU-only EAGLE, its best tree depth, same session, measured
6.91
tokens accepted per verify step (τ), HumanEval, tree depth 11, measured
44.2mJ
mJ per drafter forward, six V80s, vs 97.6 mJ on the H200 = 2.21× less (5-dataset mean; per device, never summed)

Calculations and assumptions

Speedup = measured FPGA tok/s ÷ measured comparator tok/s, per dataset (same H200, same session, batch 1)

Datasetvs best GPU-onlyvs GPU-only d11 (matched)vs no speculationGPU-only vs no spec.
HumanEval (code)1.112× vs d91.138×3.06×2.75×
MT-Bench (chat)1.144× vs d91.228×1.82×1.59×
GSM8K (math)1.100× vs d91.142×2.87×2.61×
Alpaca (instruct)1.147× vs d91.242×1.73×1.51×
Natural Q. (QA)1.188× vs d91.290×1.40×1.18×
Arithmetic mean (5 datasets)1.138×1.208×2.18×1.93×
Geometric mean (5 datasets)1.138×1.207×2.08×1.83×

Drafter energy: per-forward inputs, per-session view (HumanEval, depth 11), caveats

System (H200 + drafter cards), cost

Measured single-user throughput, by dataset

Dataset (80 prompts)FPGA + H200, d11H200 only, EAGLE d9H200 only, EAGLE d11H200, no speculationIdentical texts /80FPGA–AR · GPU d9–AR · FPGA–GPU d9
HumanEval (code)933.1 @ 6.91n=2838.8 @ 6.63n=2820.3 @ 7.05n=2305.4 n=279 · 74 · 75
MT-Bench (chat)557.8 @ 3.67n=2487.6 @ 3.80n=2454.1 @ 3.85n=2306.1 n=271 · 71 · 70
GSM8K (math)877.7 @ 6.46n=2797.6 @ 6.30n=2768.3 @ 6.61n=2306.1 n=277 · 77 · 77
Alpaca (instruct)530.5 @ 3.46n=2462.5 @ 3.61n=2427.2 @ 3.62n=2306.2 n=276 · 74 · 76
Natural Q. (QA)430.4 @ 2.71n=2362.3 @ 2.81n=2333.6 @ 2.81n=2306.6 n=273 · 70 · 72

tok/s = mean of n runs (HumanEval FPGA: 934.48, 931.81); @ = tokens accepted per verify step; same H200 in every column, run order randomised, SM clock 1,785 MHz throughout. Identical texts = prompts with byte-identical greedy output; no two different configurations match on every prompt, GPU-only included. Every configuration repeats its own texts exactly between rounds (FPGA round 2 vs round 1: 80/80, 80/80, 80/80, 80/80, 80/80).

Drafter energy per drafter forward

Dataset6 V80s, WFPGA mJ / forwardH200 W (GPU-only d9)GPU d9 mJ / forwardGPU d9 ÷ FPGAGPU d11 mJ / fwdd11 ÷ FPGA
HumanEval (code)323.544.3262.997.32.20×97.62.20×
MT-Bench (chat)322.644.2264.397.82.21×96.72.19×
GSM8K (math)324.444.4262.797.22.19×95.62.15×
Alpaca (instruct)321.544.0263.497.42.21×96.62.19×
Natural Q. (QA)321.244.0265.698.32.23×97.82.22×
5-dataset mean322.744.2263.897.62.21×96.92.19×

Method: time per forward × device run-mean power, per device, never summed (white paper 11-04 method). FPGA 0.137 ms/forward (measured on the benchmark boot) × six-V80 run-mean W; GPU 0.37 ms/forward (11-04 value) × H200 GPU-only run-mean W, which includes verify. Powers measured, energies derived; inputs, sensitivity and the per-session view in the box.

System power, H200 + drafter cards (secondary; run means, measured)

ConfigurationH200 board power6 V80 cards, sumH200 SM clock, MHzruns
FPGA drafter + H200 verify (d11)243.2 W322.6 W (51-57 W per card)1,78510
H200 alone, EAGLE d9263.8 W318.8 W (51-56 W per card)1,78510
H200 alone, EAGLE d11261.8 W318.7 W (51-56 W per card)1,78510
H200 alone, no speculation (AR)335.4 W318.9 W (51-56 W per card)1,78510

H200: nvidia-smi, 1 Hz. V80: on-card Total_Power, ~2 s, six cards summed; idle-but-powered in GPU-only and AR runs. Host CPUs, fans and PSU not included.

Where we lead, and where we don't

Where we lead

  • Single-user speed on the same H200: faster than the best GPU-only EAGLE on all 5 datasets: HumanEval (933.1 vs 838.8 tok/s), MT-Bench (557.8 vs 487.6 tok/s), GSM8K (877.7 vs 797.6 tok/s), Alpaca (530.5 vs 462.5 tok/s), Natural Q. (430.4 vs 362.3 tok/s).
  • Acceptance: τ 6.91 on HumanEval at depth 11, in the band of EAGLE-3's published HumanEval τ 6.74[1] (8B target: not a like-for-like ranking).
  • Drafting adds little power in this box: run averages: the six V80s drew 323 W while serving vs 319 W idle, and the H200 drew 243 W vs 264 W when it drafted itself. Most of the cards' power is their idle floor.
  • Energy per drafter forward: six V80s 44.2 mJ vs H200 97.6 mJ, 2.21× less (5-dataset mean; per device, never summed; method in the box).
  • Small envelope vs wafer/rack systems: six 190 W cards + one 600 W GPU vs 23 kW per Cerebras CS-3[8] or 10 kW per SambaNova rack[15] (larger models).
  • Reprogrammable: a new drafter image loads per card in minutes; the drafter follows the model.

Where we don't

  • Per drafting session: no FPGA energy advantage per draft step (about equal at depth 11; box).
  • System energy per token: the FPGA system draws 566 W of accelerator power vs 264 W for the H200 alone; GPU-only uses about half the system energy per output token (box).
  • Cost per tok/s: six V80s add ~$57k list to a ~$31-35k GPU; GPU-only gives more tok/s per dollar (box).
  • Absolute per-user speed: Cerebras (1,800 tok/s)[6] and SambaNova (1,084 tok/s)[14] run 8B models at or above our 3B speed, on far bigger systems.
  • Scope: measured on Llama-3.2-3B at batch 1 only; multi-user throughput and 8B/70B targets not measured here.
  • Exact text: 68-77 of 80 outputs per dataset byte-identical to GPU-only (GPU-only d9 vs d11 also differ).

Competitor landscape (top 10 by citations / deployment, plus closest prior art)

Work / systemModel (size)Headline as publishedStreamHardware, power, price
EAGLE4 on Eagle Talon (this sheet)Llama-3.2-3B, 2-bit drafter, 16-bit target933.1 tok/s @ τ 6.91 (HumanEval, d11) measuredbatch 16× AMD Alveo V80 + 1× H200 NVL; power measured (above)
EAGLE-3 (2025)[1]Llama-3.1-8B4.44× vs AR (HumanEval 4.85×); τ 6.23 (HumanEval 6.74). In SGLang: 373.25 tok/s vs 158.34 no-specbatch 11× H100 (SGLang table)
Medusa (2024)[3,2]Vicuna-7B/13B2.2× (Medusa-1), 2.3-3.6× (Medusa-2); τ 2.44 (Vicuna-13B)batch 1GPU
EAGLE-2 (2024)[2]Vicuna-13B; LLaMA3-8B4.04× mean (Vicuna-13B); 3.46× MT-Bench (L3-8B); τ 4.65 / 4.53batch 1GPU (not named)
SpecInfer (ASPLOS'24)[4]LLaMA / OPT family1.5-2.8× vs other serving systems, not vs ARservingmulti-GPU
Lookahead decoding (2024)[5,2]LLaMA-2 / CodeLlamaup to 1.8× MT-Bench; τ 1.62 (Vicuna-13B)batch 11 GPU
Cerebras Inference[6,7,8,9]Llama-3.1-8B, 16-bit
Llama-3.1/3.3-70B + spec
1,800 tok/s (8B)
2,100 tok/s (70B, spec. decoding)
per userCS-3 wafer, 23 kW per system (vendor); est. $2-4M
Groq LPU[10,11,12,13]Llama-3-8B; Llama-3.1-8B
Llama-3.3-70B specdec
877 tok/s (AA, 2024-04); ~560 tok/s (docs, 2026-11)
1,665 tok/s
per userGroqCard 275 W TDP; 576 chips for a 70B (2023)
SambaNova SN40L[14,15]Llama-3-8B, 16-bit1,084 tok/s (AA)per user16-chip rack, 10 kW typical (vendor)
NVIDIA + TensorRT-LLM spec. decoding[16,17]Llama-3.3-70B FP8
Llama-4 Maverick 400B MoE
181.74 tok/s, 1B draft (3.55× vs 51.14)
>1,000 tok/s/user
batch 1
per user
1× H200 (600 W)
DGX B200, 8 GPUs
FlightLLM (FPGA'24)[18]LLaMA2-7B, ~3.5-bit92.5 tok/s (VHK158); 6.0× energy efficiency vs V100S (U280)batch 1Versal VHK158 / Alveo U280, whole model on FPGA
DFVG (closest prior art)[19]not retrieved3.26× speedup (index snippet only, unverified)n/aFPGA drafts, GPU verifies
HeteroReason (2026)[20]reasoning models1.01-1.42× latency vs GPU; 1.25-1.57× energy efficiencyn/aFPGA draft, GPU target

Batch-1 speed falls with model size: a 3B number is not comparable with an 8B or 70B one. "Per user" = speed one request sees; vendor numbers date-drift. Paper speedups are vs autoregressive decoding on each paper's stack (SpecInfer: vs serving systems). AA = Artificial Analysis. Compare τ on the same dataset only.

System configuration and method

Target model (verify)Llama-3.2-3B-Instruct, float16, 1× NVIDIA H200 NVL; SGLang 0.5.16, CUDA graphs, greedy (temperature 0).
DrafterEAGLE-style one-layer drafter, 2-bit weights on-chip in the FPGAs; tree depth 11, width 2, 64-candidate shortlist, 20 verify tokens.
FPGAs used6 × AMD Alveo V80: 4 attention + 2 MLP/vocabulary cards (images 0xE4B713BC, 0xE4B71361). The server holds 7; the 7th was not opened in any run.
ServerASUS ESC8000-E11 4U, 2 × Intel Xeon Gold 6426Y, Ubuntu 24.04 LTS.
ComparatorsGPU-only: EAGLE entirely on the H200, same drafter checkpoint in 16-bit, FA3, depth 9 and 11, width 2, 20 verify tokens. No speculation: same H200, target and SGLang, plain decoding.
Method5 datasets × 80 prompts × 256 tokens, batch 1, 2 rounds, random order; tok/s = tokens ÷ benchmark time. Best GPU-only = faster of d9/d11; FPGA fixed d11. Competitor numbers as published.
Power limits and pricesV80 190 W TDP, $9,495 list[21]; H200 NVL 600 W max, $31-35k estimate[22]. Host server (needed by both) excluded from every ratio.

Sources

  1. [1] EAGLE-3, Li et al., arXiv 2503.01840 v3 (2025-04-23), Tables 1, 3, 4EAGLE-3, arXiv 2503.01840 — arxiv.org
  2. [2] EAGLE-2, Li et al., arXiv 2406.16858 (2024-06-24), Tables 1-2EAGLE-2, arXiv 2406.16858 — arxiv.org
  3. [3] Medusa, Cai et al., arXiv 2401.10774 (v3 2024-06-14)Medusa, arXiv 2401.10774 — arxiv.org
  4. [4] SpecInfer, Miao et al., ASPLOS'24, arXiv 2305.09781SpecInfer, arXiv 2305.09781 — arxiv.org
  5. [5] Lookahead decoding, Fu et al., arXiv 2402.02057 (2024-02-03)Lookahead, arXiv 2402.02057 — arxiv.org
  6. [6] Cerebras Inference launch blog (2024-08-27)Cerebras blog 2024-08-27 — www.cerebras.ai
  7. [7] Cerebras Inference 3x faster blog (2024-10-24)Cerebras blog 2024-10-24 — www.cerebras.ai
  8. [8] Cerebras CS-3 press release, 23 kW (2024-03-13)Cerebras CS-3 PR 2024-03-13 — www.cerebras.ai
  9. [9] The Next Platform, CS-3 price estimate (2024-03-14)NextPlatform 2024-03-14 — www.nextplatform.com
  10. [10] Groq blog, Llama 3 8B/70B, Artificial Analysis numbers (2024-04)Groq blog 2024-04 — groq.com
  11. [11] GroqCloud docs, llama-3.1-8b-instant (accessed 2026-11-13)GroqCloud docs, 2026-11-13 — console.groq.com
  12. [12] GroqCloud docs, llama-3.3-70b-specdec (2024-12)GroqCloud specdec docs 2024-12 — console.groq.com
  13. [13] GroqCard data page, 275 W TDPGroqCard data page — wow.groq.com
  14. [14] SambaNova press release, Llama 3 8B (2024-05-29)SambaNova PR 2024-05-29 — sambanova.ai
  15. [15] SambaRack SN40L-16 data sheet (10 kW typical)SambaRack SN40L data sheet — sambanova.ai
  16. [16] NVIDIA blog, Llama 3.3 70B speculative decoding on H200 (2024-12-17)NVIDIA blog 2024-12-17 — developer.nvidia.com
  17. [17] NVIDIA blog, DGX B200 >1,000 TPS/user Llama 4 Maverick (2025-05-22)NVIDIA blog 2025-05-22 — developer.nvidia.com
  18. [18] FlightLLM, Zeng et al., FPGA'24, arXiv 2401.03868FlightLLM, arXiv 2401.03868 — arxiv.org
  19. [19] DFVG (draft on FPGA, verify on GPU), ACM DL 10.1145/3779212.3790153 (full text not retrieved)DFVG, doi 10.1145/3779212.3790153 — dl.acm.org
  20. [20] HeteroReason, arXiv 2609.28717 (2026-09-23)HeteroReason, arXiv 2609.28717 — arxiv.org
  21. [21] AMD Alveo V80: $9,495 MSRP (2024-05-17); 190 W max (AMD product page)AMD V80 price/TDP 2024-05-17 — wccftech.com
  22. [22] NVIDIA H200 NVL price estimates $31-35k (accessed 2026-11-13); 600 W maxH200 NVL price est., 2026-11-13 — www.trgdatacenters.com
  23. [23] Speculative Decoding: Performance or Illusion?, arXiv 2601.11580, MLSys'26Spec. decoding illusion?, arXiv 2601.11580 — arxiv.org