fpga-archive-reports-process · EAGLE4 FPGA archive
EAGLE4 pair — tested archive (box1, snapshot 2026-09-27 ~02:10)
This folder is the reproducibility archive for every HEAD/TAIL V80 pair image tested on box1 in the 2026-09-21..27 silicon campaign. It holds:
- every flashed PDI (
images/); - the exact weights, with md5 (
weights/), plus the loaders that put them on the cards; - every test script, including the host kits;
- the raw logs and outputs.
Everything here is a copy; the sources stay where they were (see MANIFEST.md). To refresh after new runs, re-run
/sanctuary/lab/tested_archive_build/copy_archive.sh. It is create-only (cp -n), so it adds new files and never overwrites or deletes.
The A9b+F4 ladder was still running at snapshot time; its partial results are in results/lab/2026-09-27/.
Setup of every pair number below:
- System: Llama-3.2-3B-Instruct target on the H200 (SGLang). The drafter is EAGLE4 bitnet_721 2-bit (OSPREY), split across two V80 cards: HEAD 61:00.0 (G4 QKV, G5 attention, G6, G7) and TAIL 41:00.0 (G9, GLM, tree/top-K).
- Benchmark: mt_bench 50 requests × 32 new tokens.
- Settings: fa4 draft attention, K_C 64 shortlist, RACEFIX=1, and the chat date pinned.
- tok/s is the SGLang
Throughputline and accept isAvg Accept Length.
1. Results — best tok/s per image pair (mt_bench 50×32)
| pair (HEAD+TAIL) | best cell | tok/s @ accept | n (rounds) | config | source |
|---|---|---|---|---|---|
| A5+F4 | w2 d2 | 268.01 @ 2.55 (3-round 264.29; fold ON 264.83) | 1 (+3+3) | KVI=1 + FASTGLUE=4 (HOST_GLUE v4.3), SGLv 0.5.16 | results/pair_accuracy_20260921/width_ab_20260926_185929.txt, results/lab/2026-09-26/ADAPTIVE_TREE, A5F4_FOLD_V21 |
| A5+F4 | w1 d2 / w3 d2 | 259.45 @ 2.32 / 257.72 @ 2.61 | 1 | same | width_ab_20260926_193006.txt |
| A9b+F4 | w2 d2 | 241.56 @ 2.55 (241.35 / 244.29 / 239.03) | 3 | same | results/lab/2026-09-27/A9BF4_FLASH/REPORT.md, width_ab_20260927_020147.txt |
| A8+F4 | w2 d2 | 234.62 @ 2.55 (234.56 / 234.71 / 234.60) | 3 | same | results/lab/2026-09-26/HOST_GLUE_V4_FIXED |
| A8+F4 | w2 d2, KVI off | 203.94 (F3 fastglue) / 188.34 (plain) | 3 | FASTGLUE=3 / none, SGLv 0.5.14 | HOST_GLUE_V3, A8F4_FLASH |
| A5+F4 | w2 d2, KVI off | 202.53 @ 1.83 | 3 | protocol v1 | results/lab/2026-09-26/A5F4_PROTOCOL |
| E7+F3 | w2 d2 | 191.52 @ 1.83 | 3 | KVI off, SGLv 0.5.14 | results/lab/2026-09-26/E7F3_FLASH |
| C7+F4 | w3 d2 | 181.13 @ 1.98 | 3 | KVI off | results/lab/2026-09-26/C7F4_FLASH |
| E3+CF | w3 d2 | 180.74 @ 1.97 | 3 | KVI off | results/lab/2026-09-25/E3_FLASH |
| E1+CF | w2 d2 | 178.16 @ 1.83 | 3 | KVI off (the text reference for every later image) | results/lab/2026-09-24/E1_W23_AB |
| C8+CF | w2 d2 | 157.02 @ 1.82 | 1 | FRSPEC=1 | history |
| C9+B0 | w8 d2 | 97.23 @ 2.22 | 4 | KVONLY, older host | history |
| 0x1189+B0 | w8 d2 (1 req) | 57.65 @ 2.00 | 2 | depth-N era | history |
| GPU only (H200) | w3 d5 | 449.68 @ 3.97 (fa3) / 446.58 @ 3.86 (fa4) | 1 | SGLang EAGLE4, same 50×32 | results/gpu_width_depth_grid_20260924, HISTORY |
| GPU only | w3 d2 | 392.23 @ 2.64 (fa4) / 401.39 @ 2.67 (fa3) | 1 | same cell as the pair's w3 d2 | same |
Headline: the best pair is A5+F4 at 268.01 tok/s, which is about 60% of GPU-only (~446–450). At matched w3 d2 the pair's accept (2.61) is 99% of the GPU's (2.64), so what's left is speed per step, not draft quality. Full dense tables:
results/HISTORY/ALL_RESULTS.csv(560 runs) andBEST_BY_CELL.csv(311 cells), regenerated byresults/HISTORY/build_history.py. It covers runs up to 2026-09-26 20:31; later runs are in the lab REPORTs.- Per-image protocol CSVs:
results/lab/*/*_PROTOCOL/*_cells.csv.
Depth/width shape under KVI + FASTGLUE=4 (mt w3)
| d | A8+F4 | A9b+F4 | A5+F4 | accept |
|---|---|---|---|---|
| 2 | 220.13 | 230.88 | 257.72 | 2.61 |
| 3 | 203.88 | 213.51 | 247.16 | 3.20 |
| 5 | 173.75 | 180.68 | 221.21 (fold ON 225.80) | 3.88 |
| 7 | 150.23 | (running) | 198.77 (fold ON 200.25) | 4.35 |
- humaneval w3, A5+F4: d2/5/10/15 = 153.21 / 153.03 / 144.67 / 132.19 (accept 2.89 / 5.14 / 7.62 / 8.99). A8 at the same depths: 137.64 / 128.20 / 116.45 / 101.64.
- A8+F4 KVI grid (w1–7 × d2–7):
results/lab/2026-09-26/A8F4_KVI_GRID/REPORT.md. The sweet spot is w2 d2; narrower and shallower is faster at every depth.
2. Discoveries (each backed by the named result folder under results/lab/)
- KV import (KVI) is the largest single lever.
- The GPU drafter's K/V is imported into the HEAD (Q5.10, one KV_LOAD item per draft), so the FPGA runs zero prime/catch-up launches.
- On A8+F4 the draft tree is byte-identical to card priming, and priming is 9.4× faster (18 vs 170 ms for 639 rows).
- Speed: 1.55–1.74× card-primed tok/s at w3 d2–7.
- Accept is 97–103% of GPU: w3 d2 2.61 vs 2.64 on the GPU.
- Folders:
A8F4_KVI,KVI_PORT.
- The pair accept gap was the unprimed regime, not rounding. Without KVI the pair runs unprimed trees: accept 1.83–1.98 at d2 vs GPU 2.64. KVI closes the gap at no card cost.
- ATTN_PIPE (E3, and inside C7/A5) pays more as trees get deeper.
- On E3, G5 attention drops 2.0× unprimed and 4.0–4.3× primed, with outputs identical.
- Under KVI (every step primed), A5 beats A8 by +17% at d2, +21% at d3, +27% at d5 and +32% at d7. Humaneval shows +11% to +30%.
- Folders:
E3_FLASH,A5F4_FOLD.
- SEAM_WIDE 2 burst export (A5) gives +12–14% mt3 over C7 with KVI off: 202.53 / 202.22 vs 177.84 / 181.13.
- I12 ARRAY_FAST_TP (A9b) cuts HEAD G4 by exactly 9.74% in every census cell (HQ predicted −12.6%).
- Every other phase, the step counts and accept are identical to A8.
- Result: +3–5% over A8 under KVI. That is 3–4× smaller than the ATTN_PIPE gain, so the next HEAD should be A5 + I12, not A9b.
- Folder:
2026-09-27/A9BF4_FLASH.
- G4 embedding fold (A5 + EMBFOLD v2.1) is correct and clean on silicon: host_miss 0, texts 50/50.
- G4 items drop −8% to −35% by cell, but tok/s is neutral at d2: G4 is off the critical path there.
- Deeper KVI trees gain +2.5% at d5 and +1.4% at d7.
- Folders:
A5F4_FOLD,A5F4_FOLD_V21.
- 256-lane GEMV (E7, A8) gains ~+7–9% over the 128-lane E1/E3 with KVI off (E7+F3 191.52 vs E1 178.16). Under KVI, A5's ATTN_PIPE outweighs it.
- Host glue is real time.
- HOST_GLUE v3 (FASTGLUE=3, pipelined RPC) cuts −1.34 ms/step, giving +8.7% / +7.8% (A8 KVI off: 203.94 / 202.37).
- v4 / v4.3 (FASTGLUE=4, composed with KVI) wins 3/3 rounds: +0.8% to +2.2% over KVI alone.
- Folders:
HOST_GLUE_V3,HOST_GLUE_V4_FIXED,HOST_GLUE_V43.
- TAIL F4 (G12X + TREE_P 32) cuts TAIL TREE cycles per step by exactly −33.2% vs F3 (
C7F4_FLASHcensus). - CHILD_ROPE_FIX (C7 and later) raises accept (w3 d2 1.97 → 1.98; primed d3 steps 12 → 11) without changing any text.
- ADAPTIVE_TREE (confidence stop).
- The survivor-mass rule (dmin 1, θ −0.50) gives 246.8 tok/s (+11.5%). The held-out halves are +13.2% / +9.9%.
- The launch rule (≥ +5% held-out) is MET, so HQ is building the F9 TREE_STOP TAIL.
- Folder:
ADAPTIVE_TREE.
- GPU drafter extend inside the pair step costs 0.79–0.97 ms per verify step (6–8% of the step). The earlier "≈12.6 ms/step" was a single 9.5 s warmup call dominating the average. Folder:
GPU_EXTEND_IN_PAIR. - Attention backend: keep the default (target flashinfer + draft fa4). A target-fa3 run is −5%; draft fa3 is a tie. Folder:
A8F4_KVI_LEVERS. - Correctness: every image from E1 to A9b is 50/50 text-identical to E1+CF at mt w2/w3 d2 (fixed chat date), and every KVI arm printed
### kvi: PROOF ok(0 FPGA prime/catch-up launches).
3. What failed or was retracted
- "Pair ≈ 90% of H200": RETRACTED. The matched comparison shows ~37% (Aug) and today ~60% (268 vs ~446).
- The Block 4 "0.139 ms / 4.73× GPU" headline is FALSE. The HLS-era bench read a sticky
ap_done. - HOST_GLUE v2: F2 tied A (187.69 vs 188.34), and F1 lost 3/3; both were dropped. v4's first run was invalid because the runner forced OVERLAP_CATCHUP=0; this was fixed in
pair_width_ab_kvi.sh(commit f536bad6). - EMBFOLD v1: the host hook never attached (host_miss 13,172), because the serving path calls only
setup(). v2.1 fixed it. - KVI canary crashes (
_last_progressAttributeError; theblk==32check vs the wide loader's blk=0) were fixed inkvi_host_shadow/kvi_host.py. - HEAD_PAR=8 (0x1190) accept 1.00 is CONFOUNDED with R2 PIPE_EXP_MULTI. HP8 attention itself is bit-exact (539/539).
- The FA4 tree-draft on the H200 is broken: accept 2.74 vs fa3 3.86 at deep trees. The GPU reference can be wrong, so use matched backends.
- Census tok/s (1 request) is NOT comparable to 50×32 throughput.
4. Reproduce any number
Hard rules:
- Flash
-p 0only, a final FPT PDI only (magic92f7a516), detached. - Never FLR/SBR/device_boot/PCIe remove.
- A cold AC (physical power cycle) is needed after every flash.
- sudo reads a mode-600 password file on stdin (
sudo -p "" … < $PWF). No password is stored anywhere in this archive.
4.1 Flash (per card; HEAD = 61:00.0, TAIL = 41:00.0; both may be flashed in parallel)
Template: job_scripts/flash_a9b.sh. Pick the PDI and md5 from images/IMAGES.md.
PDI=images/oneflip_p2s_head_asgard_0xE4B711A5_66m_1f934249.pdi; MD5=1f934249815c340628f9c099bda288f5; DEV=61:00.0
[ "$(md5sum $PDI | cut -d' ' -f1)" = $MD5 ] && [ "$(od -An -tx4 -N4 $PDI | tr -d ' ')" = 92f7a516 ]
sudo -p "" env PCI_DEV=$DEV PREFLASH_ACK_QUARANTINE=1 bash scripts/R_scripts/preflash_gate.sh $PDI $MD5 < $PWF # rc 0|3 = proceed
sudo -p "" ami_tool cfgmem_program -d $DEV -t primary -p 0 -y -q -i $PDI < $PWF # then COLD AC
Box1 runs every card job under /sanctuary/lab/card_run.sh (flock); a copy is in scripts/card_run.sh.
4.2 Bring-up + arm (loads all weights) after the cold AC
Template: job_scripts/bringup_a9bf4.sh; use bringup_a5f4.sh for the best pair. Set HEAD_ID/TAIL_ID to the flashed build ids.
bash scripts/R_scripts/restore_artifact_links.sh
lspci -d 10ee: # 4 functions
for D in 61:00.0 41:00.0; do sudo -p "" weights/loaders/v80_test_suite/03_setup_qdma.sh 4 $D < $PWF; done # "Roundtrip test: PASS"
sudo -p "" -E scripts/W_scripts/pair_read_state.py < $PWF # 0x100 must be the flashed HEAD id (else the old partition booted: 2nd cold AC)
HEAD_ID=0xe4b711a5 TAIL_ID=0xe4b711f4 PWF=$PWF bash scripts/W_scripts/postcoldac_arm_pair.sh # FRONTIER GATE PASS + VERDICT PASS
HEAD_ID=0xe4b711a5 TAIL_ID=0xe4b711f4 sudo -p "" -E scripts/W_scripts/pair_state_check.py < $PWF # STATE OK tail_width=8
postcoldac_arm_pair.sh runs reports/eagle4_drafter_m7/full_session_m7.py --osprey <boot_streams> --role head|tail --pci-dev … --stage-embed --do-scale
for each card (copy: weights/loaders/eagle4_drafter_m7/). It loads:
- the M1c qweights into URAM and the M9 scales into BRAM;
- the gammas;
- lm_head_full_fp16 and embed_tokens_full_bf16 into HBM (read back and md5-checked:
133c0866…).
Then it prints the [lanes] build-id … LANES= line and the 0x1c8 weight_drops=0 scale_drops=0 line. The arm log for every run is in results/pair_accuracy_20260921/arm_0x<head>_0x<tail>_*.log.
Weights: weights/WEIGHTS.md (paths + md5). The FPGA boot streams are ~/eagle4_artifacts/osprey/boot_streams (small files copied to weights/boot_streams_small/).
The SGLang target is models/Llama-3.2-3B-Instruct; the SGLang draft is ~/fetched_weights/bitnet_721/bf16_ckpt. Both serve from the weight_721/eaglefit vendored SGLang 0.5.14/0.5.16 in conda env eaglefit.
4.3 Tests (run from the worktree with scripts/W_scripts = $W/scripts; PWF = the 600 password file)
export PWF HEADBID=0xe4b711a5 TAILBID=0xe4b711f4 HEAD_ID=0xe4b711a5 TAIL_ID=0xe4b711f4 RACEFIX=1 PAIR=A5F4
R=<cleanup_EAGLE>; W=$R/.claude/worktrees/velvety-puzzling-blossom; FG=kits/HOST_GLUE/v4_3/host_glue_fastglue
BASEP="$W/scripts/tree_readback_racefix:$W/scripts/tree_width_cdt_graph_fix:$R/scripts/pair_sitecustomize_run"
# (a) standard protocol v1 (texts vs E1+CF, mt3, census, grid, primed, humaneval, csv) -- KVI off, fastglue off
bash scripts/W_scripts/pair_image_protocol.sh texts mt3 census grid primed hum csv
# (b) the 268 tok/s cell: KVI + FASTGLUE=4 v4.3, mt w2 d2 (and any W:D list)
KVI=1 ORDER="2:2 2:2 2:2" SGLV=0.5.16 ARM_ENV="PYTHONPATH=$FG:$W/scripts/kvi_host_shadow:$BASEP EAGLE4_PAIR_FASTGLUE=4" \
bash scripts/W_scripts/pair_width_ab_kvi.sh # every arm must print "### kvi: PROOF ok"
# (c) humaneval: add BENCH=~/haochen/eaglefit/inference/dataset/humaneval/question.jsonl ORDER="3:2 3:5 3:10 3:15"
# (d) census (1 request; primed adds EAGLE4_TREE_PRIME_PREFIX=1 EAGLE4_TREE_PRIME_KVONLY=1 EAGLE4_TREE_PRIME_MD=1)
ARM_ENV="PYTHONPATH=$BASEP" W=3 D=2 bash scripts/W_scripts/pair_census.sh
# (e) texts check against the E1+CF reference
python scripts/W_scripts/pair_cmp_perreq_texts.py results/pair_accuracy_20260921/wab20260924_223134_a1_w2d2_20260924_223134.perreq.jsonl <new perreq.jsonl>
Exact per-run commands are in:
job_scripts/:a5f4_ladder.sh,a5_fold_kvi.sh,a5_fold_v21.sh,a5_fold_deep.sh,a8f4_*.sh,v42_*.sh,a9bf4_ladder.shand the other job scripts;- each
width_ab_*.txtheader, which records ORDER, SGLV, RACEFIX, KVI and ARM_ENV.
EMBFOLD (A5 fold ON): job_scripts/a5_fold_v21.sh, plus the kit in kits/EMBFOLD/v2. The 9 GB table is not copied; its path and md5 are in weights/WEIGHTS.md.
5. Layout
| dir | contents |
|---|---|
images/ |
17 flashed PDIs, IMAGES.md (build id, md5, FPT magic, what each image is), inbox build notes |
weights/ |
WEIGHTS.md, boot_streams_small/, model_configs/, loaders/ (arm/session-init/QDMA/relay/role-geometry + worktree overrides), EMBFOLD table md5 |
scripts/ |
R_scripts/ = the shared $R/scripts tree; W_scripts/ = the worktree $W/scripts tree (it wins on PYTHONPATH: KVI shadow, racefix, CDT fix, runners, protocol); card_run.sh |
kits/ |
host kits as run: HOST_GLUE (v3/v4/v4_3), KVI_HOST, KVI_OVERLAP, ADAPTIVE_TREE, EMBFOLD (v1 host + v2/v2.1), HOST_PUSH v1, L256_HOST (v3/v4), E7_HOST |
job_scripts/ |
every box1 driver script from this session (flash, bring-up, ladders, A/Bs) and their console logs |
results/lab/ |
/sanctuary/lab/results/2026-09-24..27/*: every REPORT.md, raw .out, summaries, census, JSON |
results/pair_accuracy_20260921/ |
every raw SGLang arm log, .perreq.jsonl, width_ab_*.txt summary and arm_*.log since 09-21 |
results/gpu_width_depth_grid_20260924/ |
GPU-only width/depth grid raw logs |
results/HISTORY/ |
the dense CSVs + generator |
MANIFEST.md |
every file: md5 + source path |