Seven AMD Alveo V80 FPGAs and one NVIDIA H200 in one 4U server. The FPGAs run the speculative-decoding drafter, and the GPU verifies. A new FPGA image loads in minutes, so the drafter changes with the model.
| Chassis | ASUS ESC8000-E11, 4U rack, 8 dual-slot accelerator bays vendor |
|---|---|
| Dimensions / weight | 800 × 440 × 174.5 mm (D × W × H), about 27 kg net, barebone vendor |
| Host CPU | 2 × Intel Xeon Gold 6426Y, 16 cores each (64 threads) as installed |
| Host memory | 32 GB, as tested as installed |
| Storage | 3.5 TB system volume, as tested as installed |
| Power supplies | Up to 4 × 3,000 W Titanium vendor |
| Operating system | Ubuntu 24.04 LTS as installed |
| Expansion slots free | TBD |
| FPGA | 7 × AMD Alveo V80 (Versal HBM), passive cooling as installed |
|---|---|
| FPGA memory | 32 GB HBM2e per card, 819 GB/s peak per card vendor |
| FPGA networking | 4 × QSFP56 per card, 800 Gb/s aggregate per card vendor |
| FPGA host link | PCIe Gen5 ×8 ×8 or Gen4 ×16 per card vendor |
| FPGA card power | Up to 190 W TDP per card vendor; 57–68 W per card measured at idle |
| GPU | 1 × NVIDIA H200 NVL, 141 GB HBM3e, 600 W power limit as installed |
| Model | System | tok/s | tokens / step | Date |
|---|---|---|---|---|
| Llama-3.2-3B | 7 FPGAs + H200 (best configuration) | 889.97 | 6.91 | 2026-11-11 |
| Llama-3.2-3B | H200 alone (its best configuration) | 835.12 | 6.63 | 2026-11-09 |
| Llama-3.2-3B | 4 FPGAs + H200, tree depth 15 | 870.06 | 7.12 | 2026-11-11 |
| Llama-3.2-3B | H200 alone, tree depth 15 (same day) | 700.47 | 6.88 | 2026-11-11 |
| Qwen3.8-27B | 4 FPGAs (drafter) + H200 | 249.62 | 7.14 | 2026-11-09 |
| Qwen3.8-27B | H200, drafter on the GPU | 251.20 | 7.13 | 2026-10-23 |
| Qwen3.8-27B | H200, target model alone | 61.08 | 1 | 2026-10-23 |
Single user (batch 1), HumanEval 80 prompts × 256 new tokens, SGLang, the same H200 in every row. Tokens per step = average tokens accepted per verify step. All other models: TBD (not measured on this system).
| Whole server, idle | 752 W at the wall (BMC reading, 7 FPGAs + H200 powered) measured 2026-11-04 |
|---|---|
| Whole server, serving | 1,011–1,110 W at the wall, Llama-3.2-3B single user, 4-FPGA configuration measured 2026-11-04 |
| Peak at power-on | 1,457 W (two PDU outlets) measured 2026-11-11 |
| Full-load envelope, 7 FPGAs busy | TBD |
SGLang with Eagle's EAGLE4 speculative decoding. The FPGAs draft a token tree, and the H200 verifies it in one pass.
A new FPGA image is written in about 15–20 minutes per card. The same cards have run drafters for both Llama and Qwen models.
Measured: Llama-3.2-3B and Qwen3.8-27B. Broader open-weights coverage: TBD.
| Memory-bandwidth efficiency | TBD |
|---|---|
| Multi-user throughput | TBD |
| Larger models (e.g. 8B, 70B) | TBD |
| Energy per token vs GPU-only | TBD. On 2026-11-04, at tree depth 7, the GPU-only arm used less energy per token than the FPGA arm. |