EAGLEEAGLE CORP
AMD Alveo V80 FPGAs beside an NVIDIA H200

Keep the drafter's weights next to the math.EAGLE4: a 2-bit speculative-decoding drafter that lives inside FPGA on-chip memory.

870.10 vs 700.71 tok/s · 1.24×Llama-3.2-3B, humaneval 80 × 256, same H200, same configuration on both sides (tree width 2, depth 15, 20 verify tokens, fa4): H200 + our four-card drafter (accept 7.12) vs H200 + its own EAGLE drafter (accept 6.92) MEASURED
831.14 tok/s at depth 7accept 5.89; texts byte-identical (80/80) to the previous card image; card tree 2.234 ms MEASURED
820.20 / 951.19 tok/sthe public demo on box1 at depth 11: humaneval 80 × 256 / its 15 demo prompts MEASURED
Open the live demo ↗ The real box1 system: H200 + 4 × V80, Llama-3.2-3B at w2 d11, with an auto-run of the best prompts. Its live loop measured 1,000.1 tok/s @ 8.83 on 2 prompts and 977.7 tok/s @ 8.83 on 3 prompts today. The site asks for a login.

reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md:10-28 (answer), :40-77 (every point) · reports/results/TRACK1_RECORD_F3T_20261109/EVALUATION.md (rounds table §1); reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md · reports/results/FOURCARD_DESIGN_20261016/BOX1_E2B_D11_20261110.md (Answer table; demo final state) · live loop: reports/results/DEMO_AUTORUN_BOX1_20261110.md:114 (3 prompts, 18:55Z); USER_DIALOGUE_LOG.md:2094 (2 prompts, 19:0xZ live check)

01 · End to end · measured

Same prompts, same H200, same configuration: 870.10 vs 700.71 tok/s

The measured runs, not a re-enactment. Try the real system yourself on the live demo.

Matched end-to-end result, talon2, 2026-11-09: Llama-3.2-3B target, humaneval 80 × 256, batch 1, width 2, the same H200 for both systems.
systemconfigtok/saccepttexts
H200 + 4 V80 drafter (F3t + 0x1342)d15 fa4, VTOK 20870.107.12one round (11-09)
H200 GPU-only EAGLE, same configurationd15 fa4, VTOK 20700.716.92one round (11-10)
H200 + 4 V80 drafter (F3t + 0x1342)d11 fa3, VTOK 20883.12 (882.38 / 883.87)6.90the two rounds byte-identical (80/80)
H200 + 4 V80 drafter (F3t + 0x1342)d7 record config831.14 (831.14 / 830.20)5.89byte-identical to the previous image E2b (80/80)
H200 GPU-only EAGLE, its best configd9 fa3835.12 (837.65 / 832.60)6.63both rounds byte-identical (80/80)
H200 GPU-only EAGLEd7 fa3826.06 / 821.415.97both rounds byte-identical (80/80)
H200 GPU-only EAGLEd11 fa3823.627.05one round
Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md:10-28 (answer), :40-77 (every point); reports/results/TRACK1_RECORD_F3T_20261109/EVALUATION.md (rounds table §1); reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md
Open the live demo ↗ The real box1 system: H200 + 4 × V80, Llama-3.2-3B at w2 d11, with an auto-run of the best prompts. Its live loop measured 1,000.1 tok/s @ 8.83 on 2 prompts and 977.7 tok/s @ 8.83 on 3 prompts today. The site asks for a login.
The runs behind this table

Every row is a full humaneval 80 × 256 serve on talon2's H200 on 2026-11-09 (19,615 tokens for the FPGA rows). The target verifies every token. Beyond depth 11 the cards flatten (d12–d15: 874.74 / 873.77 / 870.10); the GPU-only drafter slows beyond depth 9 (d13 fa3 778.82).

Earlier, different workload (box1, 2026-10-20, earlier card images): humaneval 20 × 512, same tree w2 d15, 59 verify tokens: cards 846.20 @ 7.29 vs H200 + its own drafter 741.27 @ 7.97 (best rounds).

reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md:10-28 (answer), :40-77 (every point); §2 GPU-only comparators; reports/results/TRACK1_RECORD_F3T_20261109/EVALUATION.md (rounds table §1); reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md · 20 × 512: FOURCARD_DESIGN_20261016/4C1_DELIVERY_TRACKER.md:868; EXP_K1_V59_HUMANEVAL_20261020.md:68-72 · live loop: reports/results/DEMO_AUTORUN_BOX1_20261110.md:114 (3 prompts, 18:55Z); USER_DIALOGUE_LOG.md:2094 (2 prompts, 19:0xZ live check)

02 · Scoreboard

Measured results, both models, both servers

Each bar is a measured run, labelled with its dataset and tree shape.

Datasets, shapes and rounds behind each bar

Llama-3.2-3B, talon2, 2026-11-09. Depth 7: GPU-only 826.06 (fa3) vs cards 831.14. Best config each: GPU-only d9 835.12 vs cards d11 883.12.

box1 public demo, 2026-11-10. ATT E2b 0xE4B713E2 + MLPV 0x1342, depth 11, the demo's own launcher: humaneval 80 × 256 820.20 at 6.90; 15 demo prompts 951.19 at 8.72. With the PQ+C host levers: 836.00 / 990.88.

Qwen3.8-27B, talon2. humaneval 80 × 256, w2 d7: cards 249.62 at 7.14 (2026-11-09); GPU-only 256.67 at 7.13 (2026-11-10); target alone 61.08 (2026-10-23). On Qwen the cards are still below GPU-only.

reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md:10-28 (answer), :40-77 (every point) · reports/results/FOURCARD_DESIGN_20261016/BOX1_E2B_D11_20261110.md (Answer table; demo final state) · reports/results/FPGA_QWEN_ALLSETS_TEMPS_20261110/REPORT.md:9-20; reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md · reports/results/GPU_BASELINE_QWEN_20261109/REPORT.md:6-15 · reports/results/SESSION_CONTEXT_BACKUP_20261020.md:76 (talon2 hq-research T2 queue, 2026-10-23)

03 · Deeper trees

Deeper trees: the GPU peaks at depth 9, the cards keep gaining

Measured tok/s at depth 7, 9 and 11 on the same H200, fa3 verify on both sides.

826.06 · 835.12 · 823.62H200 + its own drafter, tok/s at d7 · d9 · d11 (accept 5.97 · 6.63 · 7.05)
829.92 · 870.61 · 883.12H200 + four V80 cards, tok/s at d7 · d9 · d11 (accept 5.91 · 6.51 · 6.90)
The runs

Llama-3.2-3B, humaneval 80 × 256, width 2, VTOK 20, fa3 verify on both sides, same H200, 2026-11-09. On the cards accept keeps rising while the tree grows from 2.234 ms (d7) to 3.384 ms (d11); beyond d11 it flattens (d12–d15, fa4: 874.74 / 873.77 / 870.10).

reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md:10-28 (answer), :40-77 (every point)

04 · Our drafter, one pass

One layer, ten stages, every weight already on chip

A token's state flows through the EAGLE4 drafter layer. Each stage reads its 2-bit weights from the UltraRAM beside it. Only the final exact rescore reaches out to card memory, for 64 rows.

G1current stage
2-bitbody + cheap lm-head weights, in on-chip UltraRAM
64 rowsof the 16-bit lm-head gathered per pass (K_C 64)
What each stage does

G1 RMSNorm of the input embedding · G4 Q, K, V projection · G5 attention over the KV cache · G6 output projection · G7 gate and up projections, SiLU — these run on the ATT pair · G9 down projection · G11 / GLM final norm and low-rank projection · G12 cheap 2-bit lm-head over all 128,256 words · TREE keeps the top-K candidates · G8 exact rescore: gathers those K rows of the 16-bit lm-head from card memory and picks the children — these run on the MLPV pair.

Measured per-phase time of one full d11 tree on box1: G5 attention 1,219.4 µs, G4 635.4, G7 gate 566.7, TREE 707.6, G8 551.9 µs (on-card counters, median of 20 trees).

reports/results/DRAFTER_SPEED_RESEARCH_20261107/DRAFT_SPEED_BOX1_20261110.md (Answer table; per-phase census) (per-phase census) · 788,004,864 B lm-head: tested_archive/weights/WEIGHTS.md:13

05 · The GPU way

The same weights, re-read for every tree level

On the H200 the drafter fills under 1% of memory, but nothing stays beside the cores. A depth-15 tree needs 15 forwards, and every forward streams all the drafter weights across again — on the same chip and the same memory bandwidth the target needs.

0 / 15weight passes this drafting step (depth 15)
1.071 GB16-bit drafter weights read per forward
60.4%of each GPU step is drafting: 7.254 of 12.0 ms (depth 15, width 4, humaneval 80) MEASURED
Why 15 passes, and why width does not add more

Each tree level depends on the guesses of the level above, so levels run one after another, each with its own kernel launches. Siblings inside a level are batched into one pass. A depth-15 tree is 14 tree levels plus one catch-up pass over the accepted words: 15 forwards per drafting step.

On box1's H200 (2026-11-10 clean pass, humaneval 20 × 256, CUDA events, p50) the served draft step is 2.528 / 2.572 ms at depth 7 and 3.933 / 3.903 ms at depth 11 (fa4 / fa3), plus a separate 0.40 ms extend per step.

15 forwards, 1.071 GB: reports/results/WHY_SLOW_ROOFLINE_20260902.md:84-91, :108 · 60.4%: reports/results/GPU_DRAFTER_VERIFY_SPLIT_BITNET721_D15_20260916.md:12 · d7 / d11: reports/results/GPU_DRAFT_ONLY_BOX1_20261110.md (Answer table; CUDA events, humaneval 20x256, w2 VTOK 20, bs 1)

06 · The FPGA way

Loaded once at power-up. Never streamed again.

The drafter file goes onto the cards one time. After that each tree level reads its weights where they sit, beside the matrix units (weight-stationary).

1.67 GBdrafter checkpoint file loaded at boot
0 Bbody weights read from HBM per pass
744 / 1,925UltraRAM blocks used by the ATT card image
What goes where

2-bit body weights and the cheap 2-bit lm-head sit in UltraRAM; scales and norms in block RAM; the KV cache in on-chip RAM. The 16-bit exact lm-head (788 MB) and the embedding table are written into the card's HBM at boot and touched only for shortlisted rows.

1,668,067,628 B 2-bit checkpoint: tested_archive/weights/WEIGHTS.md · URAM 744 of 1,925: reports/results/TRACK1_RECORD_F3T_20261109/timing_gates/impl_1/place_report_utilization_0.rpt (md5 4acb3ba6…) :43, :61, :95, :100, :113, :363-368, :393-413

07 · Four cards, one drafter

ATT pair, MLPV pair, QSFP cables between

Each tree level runs on the attention cards, hops to the MLP and lm-head cards, and returns. A state machine on the cards runs the levels back to back; the host is not involved per level, and only small activations cross the cables.

3,685 µsone full d11 tree, 21 forwards (box1 E2b, p50) MEASURED
2,234 µsone full d7 tree (talon2 F3t, p50) MEASURED
The split, per card

ATT ranks 0 and 1 (tensor-parallel pair): G1, G4, G5, G6, G7 gate / up and SiLU, with the G6 partial results exchanged over a QSFP link. MLPV 0 and 1: G9, G11 / GLM, G12, TREE and G8. On the d11 tree the ATT phases sum to 3,148 µs and the MLPV phases to 1,757 µs; the two pairs pipeline, so these overlap in time.

reports/results/DRAFTER_SPEED_RESEARCH_20261107/DRAFT_SPEED_BOX1_20261110.md (Answer table; per-phase census) · reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md (chain census, 40 trees, md 7 w2 K_C 64); TRACK1_RECORD_F3T_20261109/EVALUATION.md

08 · The drafter itself

One d7 draft tree: 2.234 ms on the cards, 2.528 ms + 0.399 ms extend on the H200 as served

Time to build one width-2 draft tree, GPU verify excluded. Two regimes, both measured: in the serving loop the cards build a tree in less time than the H200's draft + extend step; in an isolated back-to-back loop of draft() calls with no verify, the H200's draft call is faster than today's card tree.

depth 7

cards, F3t full d7 tree (talon2), no host glue2.234 ms
cards, E2b full d7 tree (box1), no host glue2.449 ms
H200 draft step as served (box1, fa4, clean pass)2.528 ms
H200 extend step as served (box1, fa4)0.399 ms
H200 isolated draft() loop (box1, fa4, flagged pass)1.753 ms

depth 11

cards, E2b full d11 tree (box1), no host glue3.685 ms
cards, serving tree issue→done mean (box1), with card-process host work3.658 ms
H200 draft step as served (box1, fa4, clean pass)3.933 ms
H200 extend step as served (box1, fa4)0.400 ms
H200 isolated draft() loop (box1, fa4, flagged pass)2.886 ms
4 × V80 cardsH200 as served in SGLangH200 isolated back-to-back loop, no verify
Where the timings come from

(a) H200 as served in the decode loop (box1, clean pass 18:44–18:49Z, humaneval 20 × 256, w2, VTOK 20, bs 1, CUDA events, p50): draft step d7 2.528 ms (fa4) / 2.572 (fa3), d11 3.933 / 3.903; each plus a separate extend of 0.399–0.406 ms.

(b) H200 isolated back-to-back loop (200 draft() calls on a fixed prefix, no verify; first pass, flagged in the report because it overlapped demo traffic, which could only slow the GPU): d7 1.753–1.759 ms (fa4), 1.786–1.926 (fa3); d11 2.886–2.894 (fa4), 2.927–2.938 (fa3).

Cards (no host glue): full d7 tree 2.234 ms (F3t, talon2, chain census of 40 trees, md 7, w2, K_C 64) and 2.449 ms (E2b, box1); full d11 tree 3.685 ms (E2b, box1). In serving, the card process's issue → done per tree is 3.658 ms mean / 3.969 p50 at d11 and includes its host work (pre 0.30, post 0.47, read 0.13 ms). The card serving numbers are on humaneval 80 × 256, the GPU cells on humaneval 20 × 256.

Earlier history (2026-10-19 images, mt_bench 50 × 32): H200 2.32 / 2.49 ms per tree; cards 1.93 ms including host glue, 1.29 ms on-card loop; per drafter forward 0.177 ms on the cards vs 0.37 ms on the H200.

reports/results/GPU_DRAFT_ONLY_BOX1_20261110.md (CLEAN answer table; FIRST PASS table) · reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md (chain census, 40 trees, md 7 w2 K_C 64); TRACK1_RECORD_F3T_20261109/EVALUATION.md · reports/results/DRAFTER_SPEED_RESEARCH_20261107/DRAFT_SPEED_BOX1_20261110.md (Answer table; per-phase census) · reports/results/FOURCARD_DESIGN_20261016/GPU_VS_FPGA_DRAFTER_20261019.md:7-17 (CUDA events, mt_bench 50x32, w2 d7)

09 · Energy · talon2 2026-11-04

Less energy per draft, but the cards' static power is where the edge goes

Measured power by device while serving. These arms ran the card images of 2026-11-04; the throughput has risen since, and power has not been re-measured.

Cards win · drafter forward
0.177 ms vs 0.37 ms

per drafter pass, 4 V80 vs H200 (box1, 2026-10-19, mt_bench 50 × 32 w2 d7; the H200 value is a depth-slope fit of CUDA events)

Cards win · energy per drafter forward
45 mJ vs 102 mJ

0.177 ms × 255.03 W (4 V80) vs 0.37 ms × 276.67 W (H200), each device alone, never summed

Cards win · H200 power while serving
227.0 W vs 276.7 W

H200 that only verifies vs H200 drafting for itself (Llama arms; Qwen 386.6 vs 433.3 W)

Cards win · H200 energy per token
0.314 J vs 0.335 J

H200 W ÷ tok/s, Llama (Qwen 1.637 vs 1.689 J)

Edge lost · V80 static power
251.1 W idle · 255.0 W drafting

the 4 V80s draw almost the same power idle and busy

Edge lost · host CPU
289.6 W vs 202.5 W

CPU package power, card arm vs GPU-only arm (RAPL)

Arms, sensors and the full table

Llama-3.2-3B and Qwen3.8-27B, humaneval 80 × 256, w2 d7, batch 1, 1 Hz sampling (AMC per V80, nvidia-smi, RAPL, IPMI DCMI). Chassis energy per token: Llama 1.535 J (cards) vs 1.293 J (GPU-only); Qwen 5.484 vs 4.821 J. Card duty while serving: busy at most 47.1% (Llama) and 14.9% (Qwen) of each step.

FUNDRAISING_POWER_PRICING_20261104/talon2_power_econ_analysis_20261104.jsonl (arms I0, LF2+LF3, LG1+LG2, QSP, QG1); TALON2_POWER_ECON_20261104.md:78-117 · forward times FOURCARD_DESIGN_20261016/4C1_DELIVERY_TRACKER.md:379

EAGLEEAGLE CORP
White paper · 2026-11-10 · v9

EAGLE4: A Speculative-Decoding Drafter Held in FPGA On-Chip Memory

Abstract

Speculative decoding speeds up a large language model by letting a small drafter propose a tree of tokens that the target model verifies in one pass. On a GPU the drafter and the target share one chip: in our GPU-only H200 profile, drafting takes 7.254 ms of a 12.0 ms step. EAGLE4 moves the EAGLE drafter onto four AMD Alveo V80 FPGAs. The drafter body is quantised to 2 bits and held in on-chip UltraRAM beside the multipliers, and the draft tree is decoded on chip, so the H200 only verifies. With Llama-3.2-3B as the target the system reaches 870.10 tok/s at accept 7.12, against 700.71 tok/s for the H200 running its own EAGLE drafter under the same configuration (humaneval 80 × 256, tree width 2, depth 15, fa4): 1.24× faster. At depth 7 it reaches 831.14 tok/s at accept 5.89. One full depth-7 tree takes 2.234 ms on the cards. The same ATT card image also drafts for Qwen3.8-27B (249.62 tok/s against 256.67 GPU-only).

1 Introduction

A language model generates one token per forward pass, so a single user's speed is set by how fast the hardware reads the model's weights. Speculative decoding [1][2] breaks this limit: a cheap drafter guesses several tokens, organised as a tree [7], and the target checks all of them at once and keeps the longest correct prefix. The text is exactly what the target would have produced. EAGLE [3][4][5] drafts from the target's own hidden features.

The drafter is small but sequential: each tree level depends on the level above. On a GPU a depth-15 tree costs 15 drafter forwards, and each forward streams 1.071 GB of 16-bit drafter weights from HBM (reports/results/WHY_SLOW_ROOFLINE_20260902.md:84-91, :108). EAGLE4 puts the drafter on dedicated FPGA cards, with its weights resident next to the arithmetic.

2 GPU compute style vs FPGA compute style

GPU. All weights live in HBM, off the compute die. Every drafter forward launches kernels that pull the drafter weights from HBM into the streaming multiprocessors, and the target uses the same SMs and the same HBM bandwidth. Draft levels and the verify pass run one after another on one timeline. Measured on box1's H200 as served (clean pass): draft step 2.528 / 2.572 ms at depth 7 and 3.933 / 3.903 ms at depth 11 (fa4 / fa3), plus a separate 0.40 ms extend per step (reports/results/GPU_DRAFT_ONLY_BOX1_20261110.md (Answer table; CUDA events, humaneval 20x256, w2 VTOK 20, bs 1)).

FPGA. The 2-bit drafter body and the cheap lm-head are loaded once at boot into UltraRAM (1,925 blocks per V80; 744 used by the ATT image), interleaved with the multiplier lanes: the weights stay put and only activations move (weight-stationary). Only the 16-bit exact lm-head lives in card HBM, and G8 gathers 64 shortlisted rows of it per pass. A tree state machine on the card runs level after level with no host round trip. The H200 only verifies. Measured: one full depth-7 tree 2.234 ms on the four cards (reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md (chain census, 40 trees, md 7 w2 K_C 64); TRACK1_RECORD_F3T_20261109/EVALUATION.md).

GPU: one chip, weights stream in HBM (off the compute die) target weights · drafter weights · KV cache every pass streaming multiprocessors (shared) drafter and target take turns on the same SMs one decode step, one timeline L1L2L3L4L5L6L7 verify each tree level = its own kernel launches + weight read d7 draft 2.53–2.57 ms + 0.40 extend d15: drafting 7.254 of 12.0 ms 1.071 GB of weights per forward H200, SGLang EAGLE (GPU-only), measured
FPGA: weights stay beside the math card HBM: 16-bit exact lm-head only G8 gathers 64 shortlisted rows per pass activations in, results out UltraRAM + multiplier lanes, interleaved 2-bit weights loaded once at boot one decode step, two timelines H200verify only 4 V80L1L2L3L4L5L6L7 on-chip tree FSM: no host trip per level d7 full tree 2.234 ms (F3t census p50) d11 full tree 3.685 ms (21 forwards) 175 µs per forward at d11 4 × V80 at 66 MHz, measured on silicon
drafter worktarget verifyUltraRAM (weights)multiplier laneGPU SM
Figure 1. Compute style. Left: the H200 streams drafter weights from HBM for every tree level and shares its SMs with the target. Right: on the V80 cards the weights stay in UltraRAM next to the multipliers, the tree is decoded on chip, and the H200 only verifies.Source: reports/results/FOURCARD_DESIGN_20261016/GPU_VS_FPGA_DRAFTER_20261019.md:7-17 (CUDA events, mt_bench 50x32, w2 d7); reports/results/GPU_DRAFTER_VERIFY_SPLIT_BITNET721_D15_20260916.md:12; reports/results/WHY_SLOW_ROOFLINE_20260902.md:84-91, :108; reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md (chain census, 40 trees, md 7 w2 K_C 64); TRACK1_RECORD_F3T_20261109/EVALUATION.md; reports/results/DRAFTER_SPEED_RESEARCH_20261107/DRAFT_SPEED_BOX1_20261110.md (Answer table; per-phase census)

3 Architecture

One drafter layer as a fixed pipeline. Hidden 3,072, MLP 8,192, vocabulary 128,256. Stages: input norm (G1), QKV projection (G4), attention over the KV cache (G5), output projection (G6), gate / up projections and SiLU (G7), down projection (G9), final norm and low-rank projection (G11 / GLM), a cheap 2-bit lm-head over the whole vocabulary (G12), on-chip top-K tree selection (TREE) and an exact 16-bit rescore of the shortlisted rows (G8).

Four cards, split by phase. An ATT pair runs G1 to G7 tensor-parallel across two cards and exchanges the G6 partial results over a QSFP link. An MLPV pair runs G9, G11 / GLM, G12, TREE and G8. The pairs are joined by QSFP links, and the children of each level go back to the ATT cards on those links. The host moves only the accepted tokens and the draft tree to and from the H200 over PCIe (Figure 2).

H200target model: verifies the draft treePCIe ⇄ host
ATT pair · 2 × V80 · tensor-parallel 2
ATT rank 0G1 norm · G4 QKV · G5 attention · G6 o-proj · G7 gate/up · SiLU
ATT rank 1same phases, other half of the heads and lanes
MLPV pair · 2 × V80
MLPV 0 / 1G9 down · G11 norm · GLM + G12 cheap lm-head · TREE select · G8 exact rescore
Figure 2. The 4-card system (2 ATT + 2 MLPV over QSFP) beside the H200. Phase assignment from the per-card counter census.Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/DRAFT_SPEED_BOX1_20261110.md (Answer table; per-phase census); reports/results/TRACK1_RECORD_F3T_20261109/EVALUATION.md (rounds table §1); reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md

Inside one ATT card. The V80 has three dies (SLRs). Attention and its KV cache fill SLR0's block RAM, so the 2-bit weight array is split into lane groups on SLR1 and SLR2. Each GEMV pass broadcasts activations from SLR0 to the groups and returns per-lane results; the QSFP transceivers sit in SLR1 (Figure 3, Table 3). The binding resource is SLICE on the hub die: 87.60% in SLR0.

SLR0attention + KV cache (BRAM) + full-width hubSLICE 87.60% · BRAM 89.19% · DSP 84.94%
SLR1weight-array group 1 + QSFP transceiversSLICE 67.61% · BRAM 40.58% · DSP 24.77%
SLR2weight-array groups 0 and 2SLICE 64.45% · BRAM 42.63% · DSP 30.13%
Figure 3. Per-die layout and utilisation of the ATT card F3t 0xE4B713F9 (post-place).Source: ATT per-die layout: gemv_stage_array_scaled_slr.sv:6-29; xdc_sllfix_e0/eagle4_drafter_dualclk_loc_pblock.xdc (E0/E2b lineage, read 2026-11-08); reports/results/TRACK1_RECORD_F3T_20261109/timing_gates/impl_1/place_report_utilization_0.rpt (md5 4acb3ba6…) :43, :61, :95, :100, :113, :363-368, :393-413
Table 3. ATT card F3t 0xE4B713F9, post-place resources (count, % of V80), and routed timing.
resourceused% of V80
LUT1,062,12541.26%
FF1,023,53519.88%
DSP585,26748.55%
BRAM362,194.558.66%
URAM74438.65%
SLICE (device)236,67673.55%
SLICE, worst die (SLR0)98,58087.60%
SLL SLR1↔SLR0 / SLR2↔SLR18,888 / 1,990 of 18,87047.10% / 10.55%
routed WNS / WHS+0.040 / +0.011 ns66 MHz drafter clock
Source: reports/results/TRACK1_RECORD_F3T_20261109/timing_gates/impl_1/place_report_utilization_0.rpt (md5 4acb3ba6…) :43, :61, :95, :100, :113, :363-368, :393-413; reports/results/TRACK1_RECORD_F3T_20261109/EVALUATION.md (FINAL_TRUE WNS +0.040 / WHS +0.011, three retimes)

4 Evaluation

Setup. Two servers: talon2 (H200 + seven V80, four or six used) and box1 (H200 + four V80, the public demo). Targets Llama-3.2-3B and Qwen3.8-27B. SGLang 0.5.16, batch 1, tree width 2. The baseline is the H200 running SGLang's own EAGLE drafter (GPU-only), measured on the same H200. Table 1 lists every end-to-end number.

Table 1. End-to-end results in this paper. Batch 1, width 2, humaneval 80 × 256 unless noted.
systemhost, dateconfigtok/saccept
H200 + 4 V80 (F3t + 0x1342)talon2, 11-09Llama, d15 fa4870.107.12
H200 GPU-only EAGLE, same configurationtalon2, 11-10Llama, d15 fa4700.716.92
H200 + 4 V80 (F3t + 0x1342)talon2, 11-09Llama, d11 fa3883.126.90
H200 + 4 V80 (F3t + 0x1342)talon2, 11-09Llama, d7 (record config)831.145.89
H200 GPU-only EAGLE, best configtalon2, 11-09Llama, d9 fa3835.126.63
H200 + 4 V80 (E2b + 0x1342), demo launcherbox1, 11-10Llama, d11820.206.90
same, 15 demo promptsbox1, 11-10Llama, d11951.198.72
same, PQ+C host leversbox1, 11-10Llama, d11 · demo prompts836.00 · 990.886.90 · 8.72
H200 + 6 V80 (TP4 ATT B4 + MLPV 0x134D)talon2, 11-10Llama, d11859.066.91
H200 + 4 V80 (F3t + 0x1342)talon2, 11-09Qwen3.8-27B, d7249.627.14
H200 GPU-only EAGLEtalon2, 11-10Qwen3.8-27B, d7256.677.13
Qwen3.8-27B target alonetalon2, 10-23no speculation61.081
H200 + 4 V80, same tree (earlier images)box1, 10-20Llama, humaneval 20×512, d15846.207.29
H200 GPU-only EAGLE, same treebox1, 10-20Llama, humaneval 20×512, d15741.277.97
Llama-3.2-3B target alonebox1, 10-20humaneval 20×512, no speculation301.411
Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md:10-28 (answer), :40-77 (every point); reports/results/TRACK1_RECORD_F3T_20261109/EVALUATION.md (rounds table §1); reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md; reports/results/FOURCARD_DESIGN_20261016/BOX1_E2B_D11_20261110.md (Answer table; demo final state); reports/results/DRAFTER_SPEED_RESEARCH_20261107/C2_6CARD_TP4_20261110.md (Verdict); reports/results/FPGA_QWEN_ALLSETS_TEMPS_20261110/REPORT.md:9-20; reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md; reports/results/GPU_BASELINE_QWEN_20261109/REPORT.md:6-15; reports/results/SESSION_CONTEXT_BACKUP_20261020.md:76 (talon2 hq-research T2 queue, 2026-10-23); FOURCARD_DESIGN_20261016/4C1_DELIVERY_TRACKER.md:868, :1171; EXP_K1_V59_HUMANEVAL_20261020.md:68-72

Throughput. Under the same configuration on both sides (width 2, depth 15, 20 verify tokens, fa4) the four cards reach 870.10 tok/s and the H200 drafting for itself 700.71 tok/s, 1.24× (Table 1). On the same H200 on the same day, the four cards at depth 11 reach 883.12 tok/s; the GPU-only drafter peaks at 835.12 tok/s at depth 9 and slows at greater depth (823.62 at d11, 778.82 at d13) (Figure 4). The depth ladder on the cards (Figure 5) shows why depth 11 wins: accept rises from 5.89 at d7 to 6.87–6.90 at d11 while the card tree grows from 2.234 to 3.384 ms.

H200 + 4 V80 · w2 d11 fa3883.12 tok/s
H200 + 4 V80 · w2 d7 (record config)831.14 tok/s
GPU-only · d9 fa3 (its best)835.12 tok/s
GPU-only · d7 fa3826.06 tok/s
GPU-only · d11 fa3823.62 tok/s
GPU-only · d11 fa4774.57 tok/s
H200 + 4 V80 drafterH200 GPU-only EAGLE
Figure 4. End-to-end throughput, Llama-3.2-3B, humaneval 80 × 256, batch 1, width 2, same H200 (talon2), 2026-11-09. FPGA d11 = mean of 882.38 / 883.87; GPU-only d9 = mean of 837.65 / 832.60. The GPU gets slower beyond depth 9; the cards keep gaining to depth 11.Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md:10-28 (answer), :40-77 (every point)
d5756.68 tok/s
d6793.79 tok/s
d7827.57 tok/s
d8850.36 tok/s
d9860.62 tok/s
d10869.95 tok/s
d11881.37 tok/s
d12874.74 tok/s
d13873.77 tok/s
d15870.10 tok/s

accept: d5 4.97 · d7 5.89 · d9 6.47 · d11 6.87 · d13 7.03 · d15 7.12

Figure 5. Depth ladder on the 4 cards (fa4 verify, VTOK 20, one round each). Accept keeps rising with depth while the card tree grows slowly, so throughput peaks at depth 11.Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md:10-28 (answer), :40-77 (every point)

Pure drafting speed. Table 2 gives the time for one draft tree. On the cards, one full depth-11 tree (21 forwards) takes 3.685 ms, 175 µs per forward; in serving the card time per tree averages 3.658 ms of an 8.330 ms step on box1. On the H200 the d11 draft step takes 3.933 ms (fa4) as served in the decode loop, plus a 0.400 ms extend, so in serving the cards build the tree in less time than the H200's draft + extend step. In an isolated back-to-back loop of draft() calls with no verify, the H200's draft call takes 2.886–2.894 ms, faster than today's card tree. Figure 6 breaks the depth-11 tree into phases: G5 attention is the largest phase (1,219.4 µs on ATT), and the ATT card bounds the tree.

Table 2. Pure drafting speed: time the drafter needs for one tree, GPU verify excluded.
drafterinstrument (context)depthtime per tree
4 × V80, F3t + 0x1342 (talon2, 11-09)card census, full-depth tree p50, no host glue72.234 ms
4 × V80, F3t + 0x1342 (talon2, 11-09)card census, full-depth tree p50, no host glue113.384 ms
4 × V80, E2b + 0x1342 (box1, 11-10)card bench, full-depth tree p50, no host glue72.449 ms
4 × V80, E2b + 0x1342 (box1, 11-10)card bench, full-depth tree p50, no host glue113.685 ms (21 forwards, 175 µs each)
4 × V80, E2b + 0x1342 (box1, 11-10)serving trees, card-process issue→done mean, includes its host work113.658 ms
H200 EAGLE drafter (box1, 11-10, clean pass)CUDA events, draft step as served in SGLang, humaneval 20×256, p50, fa4 / fa372.528 / 2.572 ms + 0.399 / 0.406 ms extend
H200 EAGLE drafter (box1, 11-10, clean pass)CUDA events, draft step as served in SGLang, humaneval 20×256, p50, fa4 / fa3113.933 / 3.903 ms + 0.400 / 0.404 ms extend
H200 EAGLE drafter (box1, 11-10, first pass, flagged: overlapped demo traffic)CUDA events, 200 back-to-back draft() calls on a fixed prefix, no verify, fa471.753–1.759 ms
H200 EAGLE drafter (box1, 11-10, first pass, flagged: overlapped demo traffic)CUDA events, 200 back-to-back draft() calls on a fixed prefix, no verify, fa4112.886–2.894 ms
4 × V80 vs H200, per drafter forward (10-19 images)on-card loop vs CUDA-event depth slope70.177 vs 0.37 ms
Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md:10-28 (answer), :40-77 (every point) (census); reports/results/DRAFTER_SPEED_RESEARCH_20261107/DRAFT_SPEED_BOX1_20261110.md (Answer table; per-phase census); reports/results/GPU_DRAFT_ONLY_BOX1_20261110.md (Answer table; CUDA events, humaneval 20x256, w2 VTOK 20, bs 1); reports/results/FOURCARD_DESIGN_20261016/GPU_VS_FPGA_DRAFTER_20261019.md:7-17 (CUDA events, mt_bench 50x32, w2 d7)
ATT · G1 norm145.1 µs
ATT · G4 QKV635.4 µs
ATT · G5 attention1,219.4 µs
ATT · G6 o-proj + exchange199.0 µs
ATT · G7 gate566.7 µs
ATT · G7 up299.4 µs
ATT · SiLU83.0 µs
MLPV · G9 down270.5 µs
MLPV · G11 norm148.3 µs
MLPV · GLM + G1278.9 µs
MLPV · TREE select707.6 µs
MLPV · G8 exact rescore551.9 µs
ATT cardMLPV card
Figure 6. Per-phase time of one full-depth d11 tree (21 forwards), median of 20 trees, on-card counters at 66 MHz (box1, ATT E2b + MLPV 0x1342). The ATT and MLPV cards pipeline, so the two sums overlap in time; tree issue→done p50 is 3,685 µs.Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/DRAFT_SPEED_BOX1_20261110.md (Answer table; per-phase census)

What made F3t. F3t is the previous ATT image (E2b) plus the G5 overlap engine. It cut G5 attention per tree by 30.8% and the whole tree by 8.3% (Llama) and 8.5% (Qwen), with every other phase unchanged (Figure 7). Same-day serving at the identical host config: F3t 831.14 / 830.09 tok/s against E2b 812.22–816.61 over 7 rounds.

G5 attention per tree · E2b751.55 µs
G5 attention per tree · F3t (G5 overlap)519.98 µs
tree issue→done · E2b2,422.11 µs
tree issue→done · F3t2,220.41 µs
F3t 0xE4B713F9E2b, previous ATT image
Figure 7. What made F3t: the G5 overlap engine runs one head group's score phase under the previous group's softmax / value phases. Llama, chain census of 40 trees at md 7, w2, K_C 64, talon2 2026-11-09. Every other phase changed by at most ±4 µs; Qwen shows the same −30.8% on G5.Source: reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md (chain census, 40 trees, md 7 w2 K_C 64); TRACK1_RECORD_F3T_20261109/EVALUATION.md

Qwen3.8-27B. The same ATT image, switched by registers, drafts for Qwen3.8-27B: 249.62 tok/s at accept 7.14 against 256.67 GPU-only and 61.08 for the target alone (Figure 8). On Qwen the cards are still below GPU-only.

H200 + 4 V80 (F3t)249.62 tok/s
GPU-only EAGLE256.67 tok/s
target alone61.08 tok/s
H200 + 4 V80GPU-only EAGLEno speculation
Figure 8. Qwen3.8-27B, humaneval 80 × 256, w2 d7, talon2 H200. FPGA 2026-11-09 (repeat 2026-11-10: 250.27); GPU-only 2026-11-10; target alone 2026-10-23.Source: reports/results/FPGA_QWEN_ALLSETS_TEMPS_20261110/REPORT.md:9-20; reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md; reports/results/GPU_BASELINE_QWEN_20261109/REPORT.md:6-15; reports/results/SESSION_CONTEXT_BACKUP_20261020.md:76 (talon2 hq-research T2 queue, 2026-10-23)

Live demo. box1's public demo (https://demo.yshicarrick.com, login required) serves Llama at w2 d11 on ATT E2b + MLPV 0x1342: 820.20 tok/s on humaneval 80 × 256 and 951.19 tok/s on its 15 demo prompts; a real 256-token request decoded at 1,079 tok/s at accept 8.83 (reports/results/FOURCARD_DESIGN_20261016/BOX1_E2B_D11_20261110.md (Answer table; demo final state)). Its auto-run of the best prompts measured 1,000.1 tok/s @ 8.83 on 2 prompts and 977.7 tok/s @ 8.83 on 3 prompts on 2026-11-10 (reports/results/DEMO_AUTORUN_BOX1_20261110.md:114 (3 prompts, 18:55Z); USER_DIALOGUE_LOG.md:2094 (2 prompts, 19:0xZ live check)).

Correctness. The target verifies every token, so the output is the target's. Between card images the draft trees are compared level by level: F3t equals E2b on 12/12 Llama trees at depths 15, 7 and 2 (56 levels) and 9/9 Qwen trees; serving texts are 80/80 byte-identical to E2b on both models. The two d11 rounds are byte-identical to each other (80/80). Against a 16-bit reference drafter the 2-bit trees differ beyond the first level, as expected from 2-bit weights (reports/results/TRACK1_RECORD_F3T_20261109/EVALUATION.md (rounds table §1); reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md).

5 Energy (measured on talon2, 2026-11-04)

Setup. talon2: 4 V80 (ATT pair + MLPV pair) + 1 H200 NVL; Llama-3.2-3B and Qwen3.8-27B, humaneval 80 × 256, w2 d7, batch 1. Power sampled at 1 Hz: each V80's AMC sensor, nvidia-smi for the H200, RAPL for the CPU package and IPMI DCMI for the whole chassis. These arms ran the card images of 2026-11-04 (Llama 723.09 tok/s on the cards vs 826.58 GPU-only that day); the speed has since risen to 831.14 / 883.12 tok/s, and power has not been re-measured. Energy = measured W × measured time; J/token = measured W ÷ measured tok/s.

Cards win · drafter forward
0.177 ms vs 0.37 ms

per drafter pass, 4 V80 vs H200 (box1, 2026-10-19, mt_bench 50 × 32 w2 d7; the H200 value is a depth-slope fit of CUDA events)

Cards win · energy per drafter forward
45 mJ vs 102 mJ

0.177 ms × 255.03 W (4 V80) vs 0.37 ms × 276.67 W (H200), each device alone, never summed

Cards win · H200 power while serving
227.0 W vs 276.7 W

H200 that only verifies vs H200 drafting for itself (Llama arms; Qwen 386.6 vs 433.3 W)

Cards win · H200 energy per token
0.314 J vs 0.335 J

H200 W ÷ tok/s, Llama (Qwen 1.637 vs 1.689 J)

Edge lost · V80 static power
251.1 W idle · 255.0 W drafting

the 4 V80s draw almost the same power idle and busy

Edge lost · host CPU
289.6 W vs 202.5 W

CPU package power, card arm vs GPU-only arm (RAPL)

Table 4. Measured power and energy per arm, talon2, 2026-11-04 (1 Hz means over each benchmark window). These arms ran the 4-card images of that day, which served Llama at 723.09 tok/s; power has not been re-measured on today's F3t image.
armtok/sH200 WCPU pkg Wchassis W (DCMI)H200 J/tokenchassis J/token
Llama-3.2-3B · H200 + 4 V80723.09227.0289.61,110.10.3141.535
Llama-3.2-3B · GPU-only826.58276.7202.51,068.60.3351.293
Qwen3.8-27B · H200 + 4 V80236.17386.6261.51,295.21.6375.484
Qwen3.8-27B · GPU-only256.58433.3204.01,236.9 (before a flash on other cards)1.6894.821
idle server080.4132.9751.900
Source: FUNDRAISING_POWER_PRICING_20261104/talon2_power_econ_analysis_20261104.jsonl (arms I0, LF2+LF3, LG1+LG2, QSP, QG1); TALON2_POWER_ECON_20261104.md:78-117
Table 5. Power of the 4 V80 drafter cards (AMC Total_Power sensors, talon2, 2026-11-04). The chassis also holds 3 other V80s, idle at ~119.5 W.
arm4 V80 W4 V80 J/token
idle server251.10
Llama · cards drafting255.00.353
Llama · GPU-only (cards idle)251.90
Qwen · cards drafting252.41.069
Source: FUNDRAISING_POWER_PRICING_20261104/talon2_power_econ_analysis_20261104.jsonl (arms I0, LF2+LF3, LG1+LG2, QSP, QG1); TALON2_POWER_ECON_20261104.md:78-117
Llama · H200, cards drafting0.314 J
Llama · H200 drafting itself0.335 J
Llama · 4 V800.353 J
Qwen · H200, cards drafting1.637 J
Qwen · H200 drafting itself1.689 J
Qwen · 4 V801.069 J
H200 in the card armH200 GPU-only4 V80 drafter cards
Figure 9. Energy per token by device (device W ÷ tok/s), talon2 2026-11-04. Each device is shown alone; the bars are not summed.Source: FUNDRAISING_POWER_PRICING_20261104/talon2_power_econ_analysis_20261104.jsonl (arms I0, LF2+LF3, LG1+LG2, QSP, QG1); TALON2_POWER_ECON_20261104.md:78-117
Llama · H200 + 4 V801.535 J
Llama · GPU-only1.293 J
Qwen · H200 + 4 V805.484 J
Qwen · GPU-only4.821 J
H200 + 4 V80GPU-only (7 idle V80s inside the same chassis)
Figure 10. Whole-server energy per token (IPMI DCMI input ÷ tok/s), talon2 2026-11-04, on that day's images (723.09 / 236.17 tok/s on the cards).Source: FUNDRAISING_POWER_PRICING_20261104/talon2_power_econ_analysis_20261104.jsonl (arms I0, LF2+LF3, LG1+LG2, QSP, QG1); TALON2_POWER_ECON_20261104.md:78-117
4 V80, idle251.1 W
4 V80, drafting255.0 W
host CPU, GPU-only arm202.5 W
host CPU, card arm289.6 W
Figure 11. Where the edge is lost: the V80s' power is almost all static, and the host CPU works harder for the card chain. talon2 2026-11-04.Source: FUNDRAISING_POWER_PRICING_20261104/talon2_power_econ_analysis_20261104.jsonl (arms I0, LF2+LF3, LG1+LG2, QSP, QG1); TALON2_POWER_ECON_20261104.md:78-117
Table 6. Card duty cycle while serving, steady decode steps, batch 1 (talon2, 2026-11-04). Card window = from the early tree send to the last card join, so busy is an upper bound.
modelsteady stepscard window msstep msH200 verify mscard busycard idle
Llama-3.2-3B3,2773.818.105.57≤ 47.1%≥ 52.9%
Qwen3.8-27B2,0914.0026.9420.20≤ 14.9%≥ 85.1%
Source: TALON2_POWER_ECON_20261104.md:104-117 (md5 0e9214f6…)

6 Work in progress

  • 6 cards (measured). The first 6-card drafter (four ATT ranks, TP4, + MLPV pair) passes every golden gate and serves 859.06 tok/s at accept 6.91 (d11) and 796.13 at 5.89 (d7); its d7 texts are byte-identical to the 4-card record. It is still below the 4-card record (reports/results/DRAFTER_SPEED_RESEARCH_20261107/C2_6CARD_TP4_20261110.md (Verdict)).
  • 7 cards (simulation only). The item-parallel design C1 (two complete attention sets, one tree item each, + MLPV pair) passes its xsim gates; whole-tree cycles fall by 8.4% (with tree stop) to 12.1% (loop modes off) in the testbench. It has not run on silicon (reports/results/DRAFTER_SPEED_RESEARCH_20261107/C1_ITEMPAR_RTL_20261110.md:79-90 (tb_c1 whole-tree cycles)).
  • Qwen. The FPGA drafter is 249.62 against 256.67 tok/s GPU-only on humaneval and below GPU-only on all eight prompt sets measured on 2026-11-10 (FPGA / GPU 0.961–0.975 in that report's table; reports/results/FPGA_QWEN_ALLSETS_TEMPS_20261110/REPORT.md:9-20; reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_SILICON_20261109.md).

7 Related work

Speculative decoding [1][2] and tree verification [7] are the basis of this work; EAGLE [3][4][5] and Medusa [6] are the leading drafter designs, all run on the same GPU as the target. FPGA LLM accelerators such as DFX [9] and FlightLLM [10] run the whole model on FPGAs; we keep the target on the GPU and move only the latency-bound drafter, with its weights on chip [11]. Low-bit weights follow GPTQ-style quantisation [8].

8 Conclusion

Holding a 2-bit EAGLE drafter in FPGA on-chip memory, split by phase over four V80 cards, gives 870.10 tok/s on Llama-3.2-3B with an H200 that only verifies, against 700.71 tok/s for the H200 drafting for itself under the same configuration: 1.24× faster. The card drafts a full depth-7 tree in 2.234 ms, and the same image drafts for Qwen3.8-27B.

References

  1. Y. Leviathan, M. Kalman, Y. Matias. Fast Inference from Transformers via Speculative Decoding. ICML 2023.
  2. C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, J. Jumper. Accelerating Large Language Model Decoding with Speculative Sampling. arXiv:2302.01318, 2023.
  3. Y. Li, F. Wei, C. Zhang, H. Zhang. EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty. ICML 2024.
  4. Y. Li, F. Wei, C. Zhang, H. Zhang. EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees. EMNLP 2024.
  5. Y. Li, F. Wei, C. Zhang, H. Zhang. EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test. arXiv:2503.01840, 2025.
  6. T. Cai, Y. Li, Z. Geng, H. Peng, J. D. Lee, D. Chen, T. Dao. Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads. ICML 2024.
  7. X. Miao et al. SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification. ASPLOS 2024.
  8. E. Frantar, S. Ashkboos, T. Hoefler, D. Alistarh. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. ICLR 2023.
  9. S. Hong et al. DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation. MICRO 2022.
  10. S. Zeng et al. FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs. FPGA 2024.
  11. AMD. Alveo V80 Compute Accelerator Data Sheet (DS1013).

Repository paths are relative to the project root. Builder: reports/results/WHITEPAPER_V7_20261110/build_v9.py; claims and sources: CLAIMS_V7_20261110.md. Earlier versions stay on the site at /v/1/ to /v/6/.