EAGLEEAGLE CORP
EAGLEEAGLE CORP
01 · EAGLE talon · 2026-11-19

Fast FPGA drafting for H200 serving.

The talon is a box of eight AMD V80 FPGA cards that does the drafting half of EAGLE speculative decoding for H200 GPU servers. The H200 keeps the target model and only verifies; the talon proposes the token trees. Same output as the GPU alone, sooner.

1,000.67 tok/sLlama-3.2-3B, one user: H200 + 6 V80 cards vs 696.42 for the H200 alone, ×1.44
324.74 tok/sQwen3.8-27B, one user: H200 + 4 V80 cards vs 289.49 for the H200 alone, ×1.12
1,235.47 tok/sLlama-3.2-3B, 8 requests in flight on one card set and one H200 (the H200 alone serves more at 8 requests; see “Where the H200 wins”)

measured talon2 server, 2026-11-19: humaneval 80 prompts × 256 new tokens, greedy; accept length 6.91 vs 6.90 (Llama) and 10.42 vs 10.46 (Qwen), so the speed-up comes from faster drafting, not from accepting more tokens per step. All 80 replies are identical to the reference, three rounds each. Everything marked [P] below is projected.

02 · What a talon is

A drafting box beside your GPUs, not a replacement for them.

A talon: one box of 8 V80 FPGA cards drafting for several H200 GPU serverstalon: 8 AMD V80 FPGA cardsV802-bit drafterV802-bit drafterV802-bit drafterV802-bit drafterV802-bit drafterV802-bit drafterV802-bit drafterV802-bit drafterdrafter weights live on chip · cards linked by QSFP cablesH200 server Atarget model verifies the draft tree in one passH200 server Btarget model verifies the draft tree in one passH200 server Ctarget model verifies the draft tree in one passgold: draft token trees out · dashed: accepted tokens + next root back

EAGLE speculative decoding splits each step in two. A small drafter guesses a tree of next tokens; the large target checks the whole tree in one pass and keeps the longest correct branch. The result is exactly what the target would have written alone.

  • Drafter on FPGA: the 2-bit drafter lives in on-chip memory on every card, so drafting is a fixed, short, predictable time.
  • Target on H200: the GPU only verifies, the job it is best at.
  • Shared: the cards are idle while a GPU verifies, so one talon takes turns serving several GPU servers.
03 · Real today, measured

On silicon today: one user gets ×1.44 on Llama and ×1.12 on Qwen.

Llama-3.2-3B batch 1: H200 alone 696.42 vs H200 + 6 V80 1,000.67 tok/s
measured data (CSV)
Qwen3.8-27B batch 1: H200 alone 289.49 vs H200 + 4 V80 324.74 tok/s
measured data (CSV)

Batch 1 is what one user feels: tokens per second on their own reply. The Llama result reproduced on three separate power-ups of the server (1,000.67, 996.00 and 998.68 tok/s).

measured talon2, 2026-11-19, humaneval 80 × 256, greedy, depth-11 width-2 tree (Llama) and depth-15 width-2 tree (Qwen) on the cards; H200 alone = SGLang EAGLE with the GPU drafting, same target, same drafter weights.

04 · Why it is fast: drafting

Drafting is where the talon is strongest: ×23.7 the drafting rate of two H200s at small batch.

Pure drafting: 2 H200 measured vs talon projected, draft trees per second by trees in flight
[P] projected talon line; H200 line and the diamond measured. data (CSV)

Measured today

Today's six cards draft a full depth-15 tree in 3.13 ms: 319 trees/s, against 120.57 for an H200 drafting alone (×2.6). Today's cards draft one tree at a time, so the H200 overtakes them from about three trees in flight.

The talon design [P]

The talon drafts many trees at once (two 4-card sets taking turns) with the 4× card-time speedup built in. It stays ahead of two H200s at every batch from 1 to 128: ×23.7 at 1 tree, ×17.1 at 8, ×2.6 at 128.

Equal cost: one talon box vs two H200

Taking as a premise that one talon box costs the same as two H200s, these ratios are drafting throughput per unit cost. No prices are assumed.

draft trees in flight1248163264128
talon ÷ 2 H200, drafting throughput per unit cost [P]×23.7×23.7×22.4×17.1×10.6×5.9×3.6×2.6
same, against the H200's in-engine draft time×14.0×14.0×12.5×9.8×6.3×3.9×2.8×2.5

talon (8x V80): 8 V80 with the 4× card-time speedup built in: 2× clock (66→132 MHz, roadmap) × 2× misc (placeholder) [P]. H200 line = standalone drafter bench (same tree, same context of 566 tokens, no target loaded). Inside SGLang the H200 drafts one tree faster (4.91 ms), so the second row is the conservative comparison. Drafted tokens per tree = 29.

05 · One talon, many GPUs

While one GPU verifies, the talon drafts for the next: the cards are busy only 32% of each step.

Duty interleave: the talon drafts for server A while servers B and C verifytalon cardsH200 server AH200 server BH200 server Cverify + hostverify + hostdraft Averify + hostdraft Bverify + hostdraft Cverify + hostdraft Averify + hostdraft Bverify + hostdraft C0 ms2 ms4 ms6 ms8 ms10 ms12 msOne serving step = 6.9 ms; the cards are busy 2.2 ms of it (31.6%, measured on one server). Three servers share the same cards by taking turns [P].

Measured on one server: a Llama step takes 6.9 ms and the cards are busy for 2.2 ms of it (31.6%), so one card set has room for about 3 servers. On Qwen3.8-27B the GPU verifies longer and the cards are busy 18.7% of the step, room for about 5 servers per 4-card set.

The bigger the target, the more GPUs per talon

A longer verify on the GPU means a smaller busy share on the cards. A talon (8x V80, 4× card-time speedup built in) keeps 33 Qwen H200s drafted at batch 1 and 38 at batch 8 [P].

H200 GPUs one talon keeps drafted, Qwen3.8-27B, by batch per H200
[P] projected line; diamond derived from the measured busy share. data (CSV)

measured busy share = card time per step ÷ step time, from on-card counters and the serving rate (Llama record set, batch 1, 3 rounds; Qwen set, batch 1). Servers per set = floor(1 ÷ busy share); per talon (two sets) = floor(2 ÷ busy share per set) [P]. Interleaving three servers on one card set is a projection; one server per set is what has run.

06 · An 8×H200 node, with and without a talon

Add one talon to an 8×H200 node: ×1.82 at batch 1, up to ×3.26 at batch 64 [P].

8xH200 node by itself (measured) vs node + talon (projected), Llama
node by itself measured; node + talon [P] projected. data (CSV)
batch per H2001248163264128
users on the node81632641282565121,024
Llama-3.2-3B: node + talon ÷ node [P]×1.82×1.41×2.14×2.31×2.44×2.61×3.26×2.93
Qwen3.8-27B: node + talon ÷ node [P]×1.44×1.71×1.72×1.92×1.44×1.59×1.90×2.04

The H200s the talon feeds stop drafting and only verify; the others keep drafting for themselves. The Qwen node baseline above batch 1 comes from a verify-time model calibrated on measured H200 runs.

One talon + 6 H200 servers, Llama

batch per H2001248163264128
6 H200 by themselves, tok/s4,1797,2458,88914,80221,48728,03129,85435,116
the same 6 H200 + one talon, tok/s [P]7,59910,19419,06334,26052,36573,06397,330125,658
gain×1.82×1.41×2.14×2.31×2.44×2.61×3.26×3.58

talon (8x V80): 8 V80 with the 4× card-time speedup built in: 2× clock (66→132 MHz, roadmap) × 2× misc (placeholder) [P]. Node by itself = 8 × the measured single-H200 EAGLE rate at that batch.

07 · Roadmap multipliers

Two factors of two on the card time; the charts above include both.

[P]talon design at today's 66 MHz clock
card design in development
+ 2× drafter clock (132 MHz)
roadmap, timing not closed
+ 2× misc = 4× card time
placeholder factor; shown in every chart
Pure drafting vs 2 H200, 1 tree×5.9×11.9×23.7
Pure drafting vs 2 H200, 8 trees×4.3×8.5×17.1
Pure drafting vs 2 H200, 128 trees×0.65×1.30×2.61
8×H200 node gain, Llama, batch 1×1.10×1.36×1.82
8×H200 node gain, Llama, batch 8×1.25×1.87×2.31
8×H200 node gain, Llama, batch 64×1.46×2.06×3.26
8×H200 node gain, Qwen, batch 1×1.12×1.23×1.44
8×H200 node gain, Qwen, batch 8×1.47×1.63×1.92
H200s one talon feeds, Llama, batch 13510
H200s one talon feeds, Qwen, batch 1101933

2× drafter clock (66 → 132 MHz)

The cards' compute array already runs at 132 MHz; the rest of the drafter runs at 66 MHz. Closing the whole drafter at 132 MHz halves card time with the same cycle counts. Not closed yet.

2× misc

A placeholder for the remaining card-side and talon-side host savings: card time ÷ 2 again and the host's per-request cost ÷ 2. It is not tied to one specific change.

[P] projected every cell. Columns are cumulative: the last column (highlighted) is what the charts show. Pure drafting is Llama; ratios against two H200s drafting alone.

08 · Where the H200 wins today

What the talon does not do yet, in numbers.

  • Many users on today's cards. With 8 requests in flight, one card set + H200 serves 1,235.47 tok/s; the H200 alone at batch 8 serves 2,467.02. Today's card images are built for batch 1. At 8 in flight the host process driving the cards is 97% busy while the cards are about 37% busy: the host, not the cards, is the limit.
  • Pure drafting today. An H200 drafting alone overtakes today's cards from about three trees in flight (measured).
  • Without the clock roadmap. At today's 66 MHz the talon design would fall behind two H200s in pure drafting at 64 and 128 trees in flight (×0.90, ×0.65), and the 8×H200 Llama node gain at batch 2 would be ×0.98 [P].
  • Against the H200's in-engine draft time the talon's drafting lead at one tree is ×14.0, not ×23.7 [P].
  • Other drafters. With the Qwen GPTQ-2 drafter the H200 alone is faster today: 442.98 vs 255.69 tok/s at batch 1 (measured).
09 · How to read this page

Measured means it ran on silicon; [P] means a model calibrated on silicon.

Measured vs projected

  • measured read from a run on the talon2 server (7 V80 + 1 H200): every number in the hero, the two bar charts, today's diamonds and the H200 lines.
  • [P] every talon line. It comes from a timing model of the cards and the H200 that predicts measured Llama serving rounds it was not fitted on within −2.9 % to +1.0 %.
  • Ratios (×N) are the two plotted numbers divided, at the same batch.

Assumptions

  • Today's drafter clock is 66 MHz; 132 MHz is a roadmap target, not closed.
  • “2× misc” is a placeholder factor.
  • The talon line models the high-batch card design (8 V80 as two 4-card sets taking turns), which is in development and not yet on silicon.
  • Qwen memory wall: on one H200, batch × (1 + verified tokens) ≤ 440, so fewer tokens are verified per request at large batch, on both sides.
  • Same drafter weights on both sides; measured accept lengths match (6.91 vs 6.90 Llama, 10.42 vs 10.46 Qwen), and the projections keep the same tree on both sides.
  • Lossless: output equals the target's own greedy output.
  • Equal cost (talon box = two H200) is a premise; no prices are used.
Technical appendix: run configuration, card images and sources

Measured runs

Projection model (S0 / S1 / S2)

S0 = high-batch redesign with the fitting MLPV F6 (all three dies route with positive slack in out-of-context runs), 8 V80 = two 4-card TP2 sets; S1 = card wall time ÷ 2 (132 MHz, same cycles); S2 = card time ÷ 4 and talon-side host per-request cost ÷ 2. H200 verify and every H200 baseline unchanged. Charts here use S2 only (user ruling 2026-11-19). Calibration: Llama held-out serve errors −2.88 … +1.01 %; Qwen held-out −21.3 … −6.4 % (the model under-predicts the cards, i.e. conservative); per-tree census leave-one-out ±9.6 %.

Sources

whatpath (repo-relative)md5where
Measured serve rounds, Llama record set and 8-in-flight A/B, Qwen retestreports/results/DRAFTER_SPEED_RESEARCH_20261107/WINDOW_RESULTS_20261119.mdadf9dcda8294:100 (1,000.67), :190 (324.74), :222 (1,235.47), :141 (NB 8 probe 1,222.82)
H200-alone EAGLE baselines (Llama bs 1-128, Qwen bs 1-16)reports/results/HIGH_BATCH_REDESIGN_20261118/site_talon_vs_h200_20261119/data.json149980d12132llama.h200, qwen.h200, h200_accept
Model calibration and held-out errorsreports/results/HIGH_BATCH_REDESIGN_20261118/HB_CALIBRATION.mdca65ddf09022§2.1 (H200 alone llama B=1 696.42 :158)
Held-out tablescripts/hbcal/out/holdout.tsva3ad0651bdbcLlama serve held-out −2.88 … +1.01 %; Qwen held-out −21.3 … −6.4 % (model under-predicts the cards)
Card busy share (duty) per round, today's cards, drafter projectionsscripts/hbcal/out/projections.json1bdb87ddb75eduty_meas (ac2_rec, ac2_il8), drafter.today_llama, drafter.redesign_qwen
H200 drafter-only bench and in-engine draft spanreports/results/HIGH_BATCH_REDESIGN_20261118/drafter_only/results_20261119/DSPAN.tsv0f7bd3766288DBENCH.tsv (md5 def2c762…) + DSPAN.tsv
Projection model, talon (8x V80) with the 4× card-time speedup built in (S2)reports/results/HIGH_BATCH_REDESIGN_20261118/site_talon_vs_h200_20261119/f6_v9/data_v12.json68d02efaa9d1compute_v10.py / compute_v12.py (S0/S1/S2 = card time ÷1/÷2/÷4, host h ×1/×1/×0.5)
Per-chart point files used herereports/results/HIGH_BATCH_REDESIGN_20261118/site_talon_vs_h200_20261119/per_graph_csv/proj_pure_llama_v11_20261119.csv271d73ef38bf+ proj_1_node8_llama_v11, proj_1_node8_qwen_v11, proj_pure_qwen_v11
Qwen GPTQ-2 drafter: H200 alone 442.98 vs cards 255.69reports/results/HIGH_BATCH_REDESIGN_20261118/HB_QWEN_GPTQ2_REDESIGN.md0b5088d5f119:36, :102
Presentation style sourcereports/results/FPGA_ARCHIVE_20261118/presentation/eagle4_presentation_20261118.html407636da88a6palette, type, slide sections, brand mark

Charts and CSVs on this page: reports/results/HIGH_BATCH_REDESIGN_20261118/showcase_20261119/ (make_charts.py, build_page.py, showcase_numbers_v2.json with the md5 of every input; v2 = label “talon (8x V80)”). Fuller analysis: /p/eagle4-talon-vs-h200-eagle-20261119/ · presentation: /p/eagle4-presentation-20261118/.