fpga-archive-presentation-20261110 · EAGLE4 FPGA archive
FPGA vs H200, same configuration: sources for 870.10 vs 700.71 tok/s (2026-11-10)
Program 1 (3-card/4-card Llama-3.2-3B drafter). Both arms ran on talon2, on the same H200 NVL.
Configuration (identical on both arms)
- Target: Llama-3.2-3B-Instruct, fp16. Drafter: EAGLE4 bitnet_721 (
fetched_weights/bitnet_721/bf16_ckpt). - Workload: humaneval, 80 prompts x 256 new tokens, batch 1, greedy (temperature 0).
- Tree:
--draft-tree-depth 15 --draft-tree-width 2 --verify-tokens-num 20. - Draft attention backend:
--speculative-draft-attention-backend fa4. - Serving: SGLang 0.5.16,
--enable-cuda-graph --enable-piecewise-cuda-graph. - FPGA arm:
--speculative-exec-mode 1on 4 x V80. Cards 17/3d run ATT F3t 0xE4B713F9, cards 50/63 run MLPV 0x1342. Card-only settings: K_C 64, tree-stop theta -2.5, host levers PQ+C. - GPU-only arm:
--speculative-exec-mode -1, so the H200 drafts for itself.
Results
| arm | tok/s | accept | verify steps | tokens | date (UTC) | log | md5 (uncompressed) |
|---|---|---|---|---|---|---|---|
| H200 + 4 V80 | 870.10 | 7.12 | 2,754 | 19,618 | 2026-11-09 17:28:50 | reports/results/TRACK1_RECORD_F3T_20261109/config_sweep/serve2/hp_llama3b_PQC_w2d15_humaneval80x256_n1_172850_MEASUREONLY_goldendepth_unproven_20261109_172850.log.gz |
d75ab93eb7d8af0dcea4eae8699840c6 |
| H200 GPU-only | 700.71 | 6.92 | 2,829 | 19,563 | 2026-11-10 19:42:39 | reports/results/FPGA_VS_H200_20261110/gpu_logs/cfg_g2_d15_fa4_20261110_194239.log (talon2 original: ~/eagle4_talon2_4c1/runs/cfgsweep_20261109/gpu/) |
8623980467edf16df1678d7bb02b70ce |
| ratio | 1.242x |
Same-day re-measure (2026-11-11, both arms, same H200, same config; ledger item 13)
| arm | round | tok/s | accept | verify steps | tokens | time (UTC) | log | md5 |
|---|---|---|---|---|---|---|---|---|
| H200 + 4 V80 | n1 | 870.06 | 7.12 | 2,755 | 19,618 | 04:40:38 | talon2 runs/sameday_llama_d15_20261111/serve/hp_llama3b_PQC_w2d15_humaneval80x256_n1_044038_*.log (run by box1-qwen-junior) |
fae79d650077d0474fe3b5713b3e5b08 |
| H200 + 4 V80 | n2 | 866.94 | 7.12 | - | - | 04:41:59 | same dir, ..._n2_044159_*.log |
ee4f9fd749a394eeb8f96c8e9279e06c |
| H200 GPU-only | ss1 | 700.47 | 6.88 | 2,851 | 19,618 | 04:44:00 | gpu_logs/cfg_g2_d15_fa4_ss1_20261111_044400.log |
f233a21c08050e33b75bacd59d829e07 |
| H200 GPU-only | ss2 | 701.77 | 6.90 | 2,835 | 19,563 | 04:45:57 | gpu_logs/cfg_g2_d15_fa4_ss2_20261111_044557.log |
e6a9dac6142d33d36a5a68c82687bda5 |
| ratio | 1.242x / 1.235x per round; 1.239x pooled | driver output gpu_logs/gpu5.out |
FPGA SUMMARY_llama3b.tsv in that dir: md5 1c2889f30c54ec5b6e41eeb7d684a831 (md5s reported by box1-qwen-junior).
The 11-09 FPGA (870.10) and 11-10 GPU (700.71) single rounds reproduce within 0.5% on one day. Both arms ran one after the other, ~4 min apart, in the same talon2 lock window. GPU arm args were checked in the log header (speculative_draft_attention_backend='fa4', speculative_exec_mode=-1, steps 15, topk 2, draft tokens 20, cuda graph on).
Same configuration with fa3 (GPU-only both rounds 11-10; FPGA both rounds 11-10 from the W5 sweep, tracker line 1585):
| arm | tok/s | accept | log | md5 |
|---|---|---|---|---|
| GPU-only fa3 r1 | 738.63 | 7.34 | gpu_logs/cfg_g2_d15_fa3_20261110_193550.log |
6f4da91ff4da58fab215011cfc24808b |
| GPU-only fa3 r2 | 740.44 | 7.34 | gpu_logs/cfg_g2_d15_fa3r2_20261110_193913.log |
864745803f38061d69b7b93c8cc4d79f |
| FPGA fa3 | 864.77 / 867.56 | 7.16 | talon2 TALON2_NIGHT_20261110/W5_F3T_SWEEP/ |
- |
Exact commands
- GPU-only:
gpu_cells.shcopytalon2:~/eagle4_talon2_4c1/cfgsweep_20261109/gpu_cells.shruns each point (g2_d15_fa4 15 2 20 fa4, filetalon2:~/eagle4_talon2_4c1/demo_cfg_20261110/gpu3.pts) through the reference celltalon2:~/eagle4_talon2_4c1/sibw/repo/scripts/humaneval_vs_mtbench_20261020/gpu_cell.sh. Those were run as carrick insideOWNER=box1-two-v2-gpucfg card_run.sh(19:35:50-19:44:00Z). The cell runs:engine_test.py --base-model-path .../Llama-3.2-3B-Instruct --fast-model-path .../bitnet_721/bf16_ckpt --bench-data-path .../humaneval/question.jsonl --temperature 0.0 --speculative-model EAGLE4 --model-data-type float16 --draft-tree-depth 15 --draft-tree-width 2 --verify-tokens-num 20 --max-running-requests 1 --max-requests-num 80 --mem-fraction-static 0.50 --max-new-tokens 256 --enable-cuda-graph --enable-piecewise-cuda-graph --speculative-exec-mode -1 --dp-size 1 --tp-size 1 --speculative-draft-attention-backend fa4 --sglang-version 0.5.16The driver output isgpu_logs/gpu3.out. - FPGA:
F3T_CONFIG_SWEEP_20261109.md§6 serve driver withDEPTH=15 WIDTH=2 VTOK=20 KC=64 BACKEND=fa4 EAGLE4_PAIR_TREE_STOP_THETA=-2.5 ARM_PINPAR=0. Its point file isconfig_sweep/serve2/point_d15v20_172850.drv(md5b14a69b53d786deae7f4c9e0f8b3db87,MD5_SOURCE.txt:206).
Caveats (facts)
- The FPGA run is labelled MEASURE-ONLY ("golden-depth not passed"): the card golden is checked at md 7/2, not at 15. The F3t record d11 rows carry the same label.
- The two arms are one round each, on different days (FPGA 11-09, GPU-only 11-10), on the same H200 in talon2. GPU clock during the GPU arm was SM 1,785 MHz at 63 C pre-run (log header).
- With fa4, the H200's own tree drafting accepts fewer tokens than with fa3 (6.92 vs 7.34 at d15).