fpga-archive-demo-talon2-20261110 · EAGLE4 FPGA archive

Source reports/results/DEMO_TALON2_20261110.md · source md5 b7eebd50363e82a18b74f94b055bd289 · archived 2026-11-18

Download PDFOpen PDFRaw markdown

DEMO_TALON2_20261110: box1's demo.html stack copied to talon2 (built and verified, NOT started)

User order 2026-11-10 ~20:40Z: "serve our own https://demo.yshicarrick.com/demo.html for talon2 ... using same code". Program 1 (Llama-3.2-3B target on the H200 + 4-card V80 EAGLE4 drafter). Cards: ATT F3t 0xE4B713F9 on 17/3d, MLPV 0xE4B71342 on 50/63. The box1 demo was not touched. Its front end (pid 3355022) and FPGA engine (pid 3339224) were only read through /proc/<pid>/cmdline.

State

Start / stop / status (from box1)

T2ROOT=/mnt/data/eagle4_scratch/drafter_speed_20261107/evidence/measurements/sibw_talon2_1340/t2root.sh
bash $T2ROOT 'bash /home/carrick/eagle4_talon2_4c1/demo_t2_20261110/start_demo_t2.sh'     # optional: RUN=<run dir> before bash
bash $T2ROOT 'bash /home/carrick/eagle4_talon2_4c1/demo_t2_20261110/status_demo_t2.sh'
bash $T2ROOT 'bash /home/carrick/eagle4_talon2_4c1/demo_t2_20261110/stop_demo_t2.sh'

start_demo_t2.sh runs as root. It refuses to start if any of these fails (file checks only, no card or GPU access):

It then runs, detached: OWNER=demo-t2 bash ~/eagle4_talon2_4c1/lock/card_run.sh bash demo_t2_hold.sh. That card_run.sh waits while the last line of ~/TALON2_CARD_LOCK is someone else's OWNER line, then writes OWNER demo-t2. The demo keeps the lock, the cards and the H200 until stop_demo_t2.sh.

Inside the lock, demo_t2_hold.sh does this:

  1. Starts the FPGA engine on :30020 (demo_server_4c1.py, through engine/demo_4c1_t2.sh serve, record path sudo -E). Waits up to 1200 s for /health warm.
  2. Starts the front end as carrick: uvicorn demo_server:app on 127.0.0.1:8099, plain HTTP.
  3. The front end's own keep-warm then starts the GPU-only engine (:30017, sglang.launch_server) and the GPU + EAGLE engine (:30019, serve_gpuspec_t2.sh), both as carrick.
  4. Every 60 s it runs chown -R carrick on ~/.cache/flashinfer and ~/.triton. The root FPGA engine's JIT files would otherwise lock out the carrick engines (memory feedback_talon2_sglang_as_carrick_not_root). Nothing is deleted.
  5. It exits when both the FPGA engine and the front end are gone. That writes the FREE line to the lock file.

stop_demo_t2.sh stops only what the demo started, in this order:

  1. The front end first (its keep-warm would otherwise relaunch engines). Its pid must match uvicorn demo_server:app --port 8099 with cwd site/.
  2. POST /admin/shutdown on :30019 and :30020.
  3. SIGTERM to the process group of sglang.launch_server --port 30017, only if its cwd is site/ and it runs as carrick.

FPGA engine env = the talon2 F3t record cell

Record cell: runs/cfgsweep_20261109/serve3 d11fa3r2, 883.87 tok/s @ 6.90 on humaneval 80x256, measure-only (golden-depth unproven).

demo_t2.env exports the same knobs the record driver exported, because sudo -E passed them to the engine:

Measured, DRYRUN derive (no card access), command ( . demo_t2.env; bash engine/demo_4c1_t2.sh derive ), output in logs/derive_fpga_check.txt:

Files (talon2 /home/carrick/eagle4_talon2_4c1/demo_t2_20261110/; the same copies are committed in reports/results/DEMO_TALON2_20261110/)

file md5 what changed vs box1
start_demo_t2.sh b7fb746bb32c075c1368bcbec9bcffff new
stop_demo_t2.sh bfc0021ab933ca72764ae4f0122febea new
status_demo_t2.sh 85c6e35860d15657e759f6292a5f0d72 new (read-only)
demo_t2_hold.sh e5c5a42f8cbfbd6ab97f1221a0314474 new: the in-lock supervisor
demo_t2.env a8baecdd3bf14e92cbd1c1449c27a7f1 new: record env + paths
serve_gpuspec_t2.sh 4507ca02013142009b71e466b2f14155 new: GPU + EAGLE launcher (see the note under the table)
engine/demo_4c1_t2.sh e164e2331c46677863f5cd6fc2f83b6f box1 demo_4c1.sh (c2344a68): talon2 paths, kit repo, card_run.sh, DEMO_T2_LOCK_HELD, PP_SHADOW + kit demo dir on PYTHONPATH, DMA hold + NUMA pin
engine/demo_server_4c1.py 31d439fe348da776d81a8f0f12b672b8 box1 (802a1f7d) with only the SCRIPTS default changed to /home/carrick/.... The kit copy lacked --enable-metrics, so box1's version is used
site/demo_server.py 9afd19cc015c8274bb77e1247601b3a1 box1 (65,876 B): DEV 61:00.0 → 17:00.0; flash-record glob → f3twin_20261109/flash_f3t_talon2_*.json; IMAGE_REGISTRY gains 0xE4B713F9 / 0xE4B71342 (data only, so identity reads VERIFIED); GPUSPEC_LAUNCH → serve_gpuspec_t2.sh; retired FPGA_LAUNCH → a path that never exists (keeps the kill allowlist off the card engine)
site/read_card_id.py 463619107e53afe7297f3997fe22df22 BAR 61:00.0 → 17:00.0
site/demo_keepwarm.py 9f50d52e97434f28bb8ee4057fca482a E4 log dir → demo_t2_20261110/logs
site/model_toggle.py, demo_autorun.py, demo.html 442b0d2b…, 78f59ff3…, ca642c58… unchanged (byte copies, with the rest of the static site, 68 entries, 6.8 MB)
site/demo_auth.conf.TEMPLATE 4bd36ce815cc21c9f0104ceebec6800e placeholders only. Box1's values were NOT read or copied

GPU + EAGLE on talon2. Box1's gpuspec_chat_server.py stack is missing on talon2: the sglang-v0.5.6 checkout, the old 2-bit drafter eagle4_2bit_llama3b and /tmp/CostDraftTree are all absent. So :30019 is served by the same demo_server_4c1.py in exec-mode -1 (demo_4c1.sh MODE=gpu, ENGINE_PORT=31000). That is the same bitnet drafter on the H200 at w2 d11 V20 fa3, i.e. the GPU comparator shape. It answers /generate (one-shot, no token streaming in that mode).

Verified (nothing touched the cards or the GPU; talon2 ports 8099/30017/30019/30020 free at 20:5xZ)

Left to do

  1. site/demo_auth.conf: copy it from site/demo_auth.conf.TEMPLATE, set mode 600, owner carrick, and fill AUTH_PW_L0, AUTH_PW_L1, DEMO_PW. SUDO_PW is optional (carrick's talon2 sudo password). Without it the live card-identity read is skipped and the page shows "unreadable".
  2. Public hostname, option A or option B below.
  3. Start with the command above, after the lock frees (junior, about 21:15Z) and junior's restore of the F3t set at depth ≥ 11.
  4. Not enabled: the Qwen switch. MODEL_TOGGLE_TIERS=none, because its card steps take the card lock the demo already holds, which would self-deadlock. The toggle code knows the F3t set; enabling it needs a lock-held path.
  5. Watch the first start: H200 memory, since all three engines run on one H200. FPGA 0.50, GPU-only 0.30, GPU + EAGLE 0.20 by default. The keep-warm footprint numbers were measured on box1's old gpuspec engine.

Public URL, option A: a new Cloudflare tunnel run on talon2

  1. In the Cloudflare dashboard (Zero Trust → Networks → Tunnels), create a tunnel, e.g. demo-talon2.
  2. Add public hostname demo-talon2.yshicarrick.com → service http://localhost:8099.
  3. Paste the tunnel token into /home/carrick/.config/eagle4/cf_demo_t2_tunnel.token (the directory is already 700 carrick; chmod 600 the file).
  4. Run, on talon2, as carrick: setsid nohup /usr/local/bin/cloudflared tunnel --no-autoupdate run --token-file /home/carrick/.config/eagle4/cf_demo_t2_tunnel.token > /home/carrick/eagle4_talon2_4c1/demo_t2_20261110/logs/cloudflared.log 2>&1 &

Public URL, option B: a new hostname on box1's existing tunnel, through an ssh forward

box1 has no ssh alias for talon2. Use the explicit form from the eagle4-talon2-connect skill. ssh config and known_hosts are not edited.

  1. On box1 (port 18100 is free), run: ssh -N -o BatchMode=yes -o ServerAliveInterval=30 -o ExitOnForwardFailure=yes -o UserKnownHostsFile=/mnt/data/eagle4_scratch/talon2_known_hosts -o ProxyCommand="/usr/local/bin/cloudflared access ssh --hostname %h" -L 127.0.0.1:18100:127.0.0.1:8099 [[withheld]]
  2. In the dashboard, on box1's existing token-managed tunnel (cloudflared.service), add public hostname demo-talon2.yshicarrick.com → http://localhost:18100.

Notes