fpga-archive-demo-talon2-20261110 · EAGLE4 FPGA archive

Source reports/results/DEMO_BEST_CONFIG.md · source md5 224890d7af07e0dbb0a97db36c78d05e · archived 2026-11-18

Download PDFOpen PDFRaw markdown

DEMO_BEST_CONFIG — site chat demo now drafts at TODAY'S BEST CONFIG (2026-07-29)

Goal (user: "with fpga best config hosted please"): the all-in-one site's live chat demo (hub.html → /demo.html → POST /api/chat backend=fpga) must draft on the real card at the best measured config — the one that measured accept 1.45 @ 48.7 tok/s (STATUS_20260729.md 10:2x): whole-tree route, image 0xE4B71124, strict SHIM-4, EAGLE4_TREE_KC=64, depth 2, 8815-coherent resident session.

What the chat backend is

fpga_chat_server.py (:30018, root) wraps ONE real sgl.Engine (SGLang mode-1, speculative_exec_mode=1) — not a bespoke loop. The draft route is selected inside sglang by env: EAGLE4_TREE_FSM=1 → cost_draft_tree_main.fast_model_draft (sglang-v0.5.6/python/sglang/srt/speculative/cost_draft_tree_main.py:159) → cost_draft_tree_fpga._draft_fpga_tree → scripts/fpga_tree_marshaller.py. So the upgrade was almost entirely an env-block change; no marshaller/sglang code was touched.

Change set

  1. ~/sglang-eagle4/eagle-project/eagle4/serve_fpga_chat.sh — env block replaced.
    • DROPPED (stale HLS leg-C, 0xE4B71115 era): FPGA_READY, EAGLE4_HLS_DRAFT_ENABLE, EAGLE4_PDI6_LAYOUTB, EAGLE4_PDI6_DO_SCALE_BOOTDMA, EAGLE4_PDI6_GOLDEN, EAGLE4_EXPECT_BUILDID=0xE4B71115.
    • ADDED (== run_tree_fsm_e2e.sh MODE=strict + the KC knob): EAGLE4_TREE_FSM=1, TREE_EXPECT_BUILDID=0xe4b71124, EAGLE4_TREE_KC=64, EAGLE4_TREE_VERBOSE=1, EAGLE4_TREE_KVRES_CLR=1, EAGLE4_TREE_KVRES_STRICT=1, EAGLE4_CLEANUP_DIR=$HWREPO, OMP_NUM_THREADS=4, TREE_DEPTH=2, HOME=~ (sudo strips HOME; torch-ext cache). No EAGLE4_TREE_SHIM4_DEGRADED ⇒ strict.
  2. ~/sglang-eagle4/eagle-project/eagle4/fpga_chat_server.py
    • TREE_DEPTH default 3 → 2 (speculative_num_steps=2; topk=8 / draft_tokens=64 / mem_frac=0.3 unchanged — byte-matches the best-config E2E server_args, ~/eagle4_artifacts/tree_fsm_strict_20260729_100926.log:3).
    • BACKEND_LABEL now derives from TREE_EXPECT_BUILDID + route (added 0xe4b71124 = "TREE (8815-coherent, lmdown fix)").
    • /health now returns config: {route, build_id, kc, depth, strict, backend}.
    • Startup logs the tree env + the deliberate card-state policy (below).
  3. reports/arch_site/demo_server.py
    • EXPECT_BUILDID 0xE4B71115 → 0xE4B71124; registry entry TREE-8815 (pdi_md5 dcd4500f1e99ac554523be92ee7dae57, verified against ~/eagle4_artifacts/uni_kvres_0x1124/amd_v80_gen5x8_25.1.pdi); PDI_PATHS entry added.
    • _fpga_label() appends the tree config read live from :30018 /health.
  4. reports/arch_site/hub.html — Latest-silicon table caption + demo card note that the live demo runs exactly the best-row config.

Card-state policy (deliberate)

The flashed 0xE4B71124 holds the live 8815 session (weights resident from post_flash_bringup_0xE4B71124.sh). The chat server does NOT redo session-init: the marshaller verifies build-id at 0x100 on driver build (refuses on mismatch), and pulses KVRES-CLR (0x154) per new request. No reset paths exist anywhere in this stack (serve script hard-rules unchanged). This is logged at server startup: "TREE route: relying on the RESIDENT 8815 session (no session re-init)".

Evidence (live, 2026-07-29 ~10:3x–10:4x)

Incident note (transient, explained)

The FIRST warm attempt (10:22) died with the poison tooth STALE TREE READBACK: poison word survived (scheduler SIGQUIT). Root cause: card contention — the FR-Spec cert/campaign session was on the card 10:18–10:30 (AUTONOMOUS_DRIVE_LOG.md 10:5x entry; both sides invalidated). The relaunch in a clean window warmed and served correctly. The poison tooth did exactly its job (refused a stale forward instead of consuming it). Standing rule stays: the card is single-tenant — do not run benches/campaigns concurrently with the live demo.

GPU sharing

Engine holds mem_fraction_static=0.3 (same as run_tree_fsm_e2e.sh), H200 143 GB ⇒ benches can still run beside the demo. demo_server's idle reaper auto-kills the chat engines after 600 s idle.

2026-07-30 13:2x update (TOPN A/B + amortization arm, TOPN_PRUNE_AB.md)