# DEMO_BEST_CONFIG — site chat demo now drafts at TODAY'S BEST CONFIG (2026-07-29)

Goal (user: "with fpga best config hosted please"): the all-in-one site's live chat demo
(`hub.html` → `/demo.html` → `POST /api/chat backend=fpga`) must draft on the real card at the
best measured config — the one that measured **accept 1.45 @ 48.7 tok/s**
(STATUS_20260729.md 10:2x): whole-tree route, image **0xE4B71124**, strict SHIM-4,
**EAGLE4_TREE_KC=64**, **depth 2**, 8815-coherent resident session.

## What the chat backend is

`fpga_chat_server.py` (:30018, root) wraps ONE real `sgl.Engine` (SGLang mode-1,
`speculative_exec_mode=1`) — not a bespoke loop. The draft route is selected inside sglang by env:
`EAGLE4_TREE_FSM=1` → `cost_draft_tree_main.fast_model_draft`
(`sglang-v0.5.6/python/sglang/srt/speculative/cost_draft_tree_main.py:159`) →
`cost_draft_tree_fpga._draft_fpga_tree` → `scripts/fpga_tree_marshaller.py`. So the upgrade was
almost entirely an env-block change; no marshaller/sglang code was touched.

## Change set

1. **`~/sglang-eagle4/eagle-project/eagle4/serve_fpga_chat.sh`** — env block replaced.
   - DROPPED (stale HLS leg-C, 0xE4B71115 era): `FPGA_READY`, `EAGLE4_HLS_DRAFT_ENABLE`,
     `EAGLE4_PDI6_LAYOUTB`, `EAGLE4_PDI6_DO_SCALE_BOOTDMA`, `EAGLE4_PDI6_GOLDEN`,
     `EAGLE4_EXPECT_BUILDID=0xE4B71115`.
   - ADDED (== `run_tree_fsm_e2e.sh` MODE=strict + the KC knob): `EAGLE4_TREE_FSM=1`,
     `TREE_EXPECT_BUILDID=0xe4b71124`, `EAGLE4_TREE_KC=64`, `EAGLE4_TREE_VERBOSE=1`,
     `EAGLE4_TREE_KVRES_CLR=1`, `EAGLE4_TREE_KVRES_STRICT=1`, `EAGLE4_CLEANUP_DIR=$HWREPO`,
     `OMP_NUM_THREADS=4`, `TREE_DEPTH=2`, `HOME=~` (sudo strips HOME; torch-ext cache).
     No `EAGLE4_TREE_SHIM4_DEGRADED` ⇒ strict.
2. **`~/sglang-eagle4/eagle-project/eagle4/fpga_chat_server.py`**
   - `TREE_DEPTH` default 3 → **2** (`speculative_num_steps=2`; topk=8 / draft_tokens=64 /
     mem_frac=0.3 unchanged — byte-matches the best-config E2E server_args,
     `~/eagle4_artifacts/tree_fsm_strict_20260729_100926.log:3`).
   - `BACKEND_LABEL` now derives from `TREE_EXPECT_BUILDID` + route
     (added 0xe4b71124 = "TREE (8815-coherent, lmdown fix)").
   - `/health` now returns `config`: `{route, build_id, kc, depth, strict, backend}`.
   - Startup logs the tree env + the deliberate card-state policy (below).
3. **`reports/arch_site/demo_server.py`**
   - `EXPECT_BUILDID` 0xE4B71115 → **0xE4B71124**; registry entry TREE-8815 (pdi_md5
     `dcd4500f1e99ac554523be92ee7dae57`, verified against
     `~/eagle4_artifacts/uni_kvres_0x1124/amd_v80_gen5x8_25.1.pdi`); PDI_PATHS entry added.
   - `_fpga_label()` appends the tree config read live from :30018 `/health`.
4. **`reports/arch_site/hub.html`** — Latest-silicon table caption + demo card note that the live
   demo runs exactly the best-row config.

## Card-state policy (deliberate)

The flashed 0xE4B71124 holds the live 8815 session (weights resident from
`post_flash_bringup_0xE4B71124.sh`). The chat server does **NOT** redo session-init: the
marshaller verifies build-id at 0x100 on driver build (refuses on mismatch), and pulses
KVRES-CLR (0x154) per new request. No reset paths exist anywhere in this stack (serve script
hard-rules unchanged). This is logged at server startup:
`"TREE route: relying on the RESIDENT 8815 session (no session re-init)"`.

## Evidence (live, 2026-07-29 ~10:3x–10:4x)

- **Card identity (site gate, read off silicon):** `/api/card` →
  `{"build_id_hex":"0xE4B71124","match":true,"image":"TREE-8815","kc":64,"state":"VERIFIED"}`.
- **/health (:30018):**
  `{"status":"ok","warm":true,"config":{"route":"tree-fsm","build_id":"0xe4b71124","kc":"64","depth":2,"strict":true,...}}`
- **3 chat prompts end-to-end through the site's `/api/chat` (backend=fpga):**
  - "What is speculative decoding..." → 48 tok, `spec_verify_ct=30`, **accept_len 1.6**,
    decode **50.8 tok/s**, ttft 0.08 s.
  - haiku → 19 tok, accept_len 1.357, 46.4 tok/s.
  - FPGA uses → completed (3rd prompt, same route).
  All replies carry backend label
  `"EAGLE4 FPGA drafter · V80 TREE (8815-coherent, lmdown fix) 0xe4b71124 · whole-tree FSM d2 strict K_C=64 (on silicon)"`.
- **Tree-route proof in `fpga_chat_server.log`:** 55 `[tree-fsm] draft:` lines, e.g.
  `draft: n_nodes=72+root depth=2 ctx_hw=18 committed_ctx=160 launches=78 dma={...}`;
  KVRES-CLR pulses per request: `pulse#3 0x154: res_cnt 61 -> 0 ... OK`.
  Numbers are consistent with the measured best config (1.45 @ 48.7 on mt_bench, n=2 prompts).

## Incident note (transient, explained)

The FIRST warm attempt (10:22) died with the poison tooth
`STALE TREE READBACK: poison word survived` (scheduler SIGQUIT). Root cause: **card contention** —
the FR-Spec cert/campaign session was on the card 10:18–10:30 (AUTONOMOUS_DRIVE_LOG.md 10:5x entry;
both sides invalidated). The relaunch in a clean window warmed and served correctly. The poison
tooth did exactly its job (refused a stale forward instead of consuming it). Standing rule stays:
the card is single-tenant — do not run benches/campaigns concurrently with the live demo.

## GPU sharing

Engine holds `mem_fraction_static=0.3` (same as `run_tree_fsm_e2e.sh`), H200 143 GB ⇒ benches can
still run beside the demo. demo_server's idle reaper auto-kills the chat engines after 600 s idle.

## 2026-07-30 13:2x update (TOPN A/B + amortization arm, TOPN_PRUNE_AB.md)

* **prime16 d2 CONFIRMED as the demo config at production N**: MAX_NEW=256 strict arm = accept
  1.75 @ 46.87 tok/s (log `~/eagle4_artifacts/topn_ab/amort_p16_d2_n256_132406.log` md5 03474875)
  — the 192 ms prime16 setup amortizes out and accept climbs 1.66→1.75 vs the N=48 number.
  Full-prefix stays a bench-only arm (permanent per-step cost).
* **EAGLE4_TREE_TOPN=64 NOT folded into the demo**: silicon-proven exactly accept- and
  speed-neutral (SGLang already applies the same top-64 cumulative cut GPU-side,
  cost_draft_tree_main.py:285/:505/:3193). Keep unset.
* Demo restarted 13:25 with the unchanged recipe (uvicorn pid 50365).
* ⚠️ OPEN DEFECT (pre-existing): demo_server.py:95 `EXPECT_BUILDID=0xE4B71124` +
  serve_fpga_chat.sh:85 `TREE_EXPECT_BUILDID=0xe4b71124` — the card now serves 0xE4B71127, so a
  chat request will spawn an engine that REFUSES the card. Needs 0xe4b71127 (+ ABI_UNSHIFT=0);
  orchestrator decision (touches the live demo stack).
