fpga-archive-demo-talon2-20261110 · EAGLE4 FPGA archive
DEMO_BEST_CONFIG — site chat demo now drafts at TODAY'S BEST CONFIG (2026-07-29)
Goal (user: "with fpga best config hosted please"): the all-in-one site's live chat demo
(hub.html → /demo.html → POST /api/chat backend=fpga) must draft on the real card at the
best measured config — the one that measured accept 1.45 @ 48.7 tok/s
(STATUS_20260729.md 10:2x): whole-tree route, image 0xE4B71124, strict SHIM-4,
EAGLE4_TREE_KC=64, depth 2, 8815-coherent resident session.
What the chat backend is
fpga_chat_server.py (:30018, root) wraps ONE real sgl.Engine (SGLang mode-1,
speculative_exec_mode=1) — not a bespoke loop. The draft route is selected inside sglang by env:
EAGLE4_TREE_FSM=1 → cost_draft_tree_main.fast_model_draft
(sglang-v0.5.6/python/sglang/srt/speculative/cost_draft_tree_main.py:159) →
cost_draft_tree_fpga._draft_fpga_tree → scripts/fpga_tree_marshaller.py. So the upgrade was
almost entirely an env-block change; no marshaller/sglang code was touched.
Change set
~/sglang-eagle4/eagle-project/eagle4/serve_fpga_chat.sh— env block replaced.- DROPPED (stale HLS leg-C, 0xE4B71115 era):
FPGA_READY,EAGLE4_HLS_DRAFT_ENABLE,EAGLE4_PDI6_LAYOUTB,EAGLE4_PDI6_DO_SCALE_BOOTDMA,EAGLE4_PDI6_GOLDEN,EAGLE4_EXPECT_BUILDID=0xE4B71115. - ADDED (==
run_tree_fsm_e2e.shMODE=strict + the KC knob):EAGLE4_TREE_FSM=1,TREE_EXPECT_BUILDID=0xe4b71124,EAGLE4_TREE_KC=64,EAGLE4_TREE_VERBOSE=1,EAGLE4_TREE_KVRES_CLR=1,EAGLE4_TREE_KVRES_STRICT=1,EAGLE4_CLEANUP_DIR=$HWREPO,OMP_NUM_THREADS=4,TREE_DEPTH=2,HOME=~(sudo strips HOME; torch-ext cache). NoEAGLE4_TREE_SHIM4_DEGRADED⇒ strict.
- DROPPED (stale HLS leg-C, 0xE4B71115 era):
~/sglang-eagle4/eagle-project/eagle4/fpga_chat_server.pyTREE_DEPTHdefault 3 → 2 (speculative_num_steps=2; topk=8 / draft_tokens=64 / mem_frac=0.3 unchanged — byte-matches the best-config E2E server_args,~/eagle4_artifacts/tree_fsm_strict_20260729_100926.log:3).BACKEND_LABELnow derives fromTREE_EXPECT_BUILDID+ route (added 0xe4b71124 = "TREE (8815-coherent, lmdown fix)")./healthnow returnsconfig:{route, build_id, kc, depth, strict, backend}.- Startup logs the tree env + the deliberate card-state policy (below).
reports/arch_site/demo_server.pyEXPECT_BUILDID0xE4B71115 → 0xE4B71124; registry entry TREE-8815 (pdi_md5dcd4500f1e99ac554523be92ee7dae57, verified against~/eagle4_artifacts/uni_kvres_0x1124/amd_v80_gen5x8_25.1.pdi); PDI_PATHS entry added._fpga_label()appends the tree config read live from :30018/health.
reports/arch_site/hub.html— Latest-silicon table caption + demo card note that the live demo runs exactly the best-row config.
Card-state policy (deliberate)
The flashed 0xE4B71124 holds the live 8815 session (weights resident from
post_flash_bringup_0xE4B71124.sh). The chat server does NOT redo session-init: the
marshaller verifies build-id at 0x100 on driver build (refuses on mismatch), and pulses
KVRES-CLR (0x154) per new request. No reset paths exist anywhere in this stack (serve script
hard-rules unchanged). This is logged at server startup:
"TREE route: relying on the RESIDENT 8815 session (no session re-init)".
Evidence (live, 2026-07-29 ~10:3x–10:4x)
- Card identity (site gate, read off silicon):
/api/card→{"build_id_hex":"0xE4B71124","match":true,"image":"TREE-8815","kc":64,"state":"VERIFIED"}. - /health (:30018):
{"status":"ok","warm":true,"config":{"route":"tree-fsm","build_id":"0xe4b71124","kc":"64","depth":2,"strict":true,...}} - 3 chat prompts end-to-end through the site's
/api/chat(backend=fpga):- "What is speculative decoding..." → 48 tok,
spec_verify_ct=30, accept_len 1.6, decode 50.8 tok/s, ttft 0.08 s. - haiku → 19 tok, accept_len 1.357, 46.4 tok/s.
- FPGA uses → completed (3rd prompt, same route).
All replies carry backend label
"EAGLE4 FPGA drafter · V80 TREE (8815-coherent, lmdown fix) 0xe4b71124 · whole-tree FSM d2 strict K_C=64 (on silicon)".
- "What is speculative decoding..." → 48 tok,
- Tree-route proof in
fpga_chat_server.log: 55[tree-fsm] draft:lines, e.g.draft: n_nodes=72+root depth=2 ctx_hw=18 committed_ctx=160 launches=78 dma={...}; KVRES-CLR pulses per request:pulse#3 0x154: res_cnt 61 -> 0 ... OK. Numbers are consistent with the measured best config (1.45 @ 48.7 on mt_bench, n=2 prompts).
Incident note (transient, explained)
The FIRST warm attempt (10:22) died with the poison tooth
STALE TREE READBACK: poison word survived (scheduler SIGQUIT). Root cause: card contention —
the FR-Spec cert/campaign session was on the card 10:18–10:30 (AUTONOMOUS_DRIVE_LOG.md 10:5x entry;
both sides invalidated). The relaunch in a clean window warmed and served correctly. The poison
tooth did exactly its job (refused a stale forward instead of consuming it). Standing rule stays:
the card is single-tenant — do not run benches/campaigns concurrently with the live demo.
GPU sharing
Engine holds mem_fraction_static=0.3 (same as run_tree_fsm_e2e.sh), H200 143 GB ⇒ benches can
still run beside the demo. demo_server's idle reaper auto-kills the chat engines after 600 s idle.
2026-07-30 13:2x update (TOPN A/B + amortization arm, TOPN_PRUNE_AB.md)
- prime16 d2 CONFIRMED as the demo config at production N: MAX_NEW=256 strict arm = accept
1.75 @ 46.87 tok/s (log
~/eagle4_artifacts/topn_ab/amort_p16_d2_n256_132406.logmd5 03474875) — the 192 ms prime16 setup amortizes out and accept climbs 1.66→1.75 vs the N=48 number. Full-prefix stays a bench-only arm (permanent per-step cost). - EAGLE4_TREE_TOPN=64 NOT folded into the demo: silicon-proven exactly accept- and speed-neutral (SGLang already applies the same top-64 cumulative cut GPU-side, cost_draft_tree_main.py:285/:505/:3193). Keep unset.
- Demo restarted 13:25 with the unchanged recipe (uvicorn pid 50365).
- ⚠️ OPEN DEFECT (pre-existing): demo_server.py:95
EXPECT_BUILDID=0xE4B71124+ serve_fpga_chat.sh:85TREE_EXPECT_BUILDID=0xe4b71124— the card now serves 0xE4B71127, so a chat request will spawn an engine that REFUSES the card. Needs 0xe4b71127 (+ ABI_UNSHIFT=0); orchestrator decision (touches the live demo stack).