fpga-archive-demo-talon2-20261110 · EAGLE4 FPGA archive
DEMO_TALON2_20261110: box1's demo.html stack copied to talon2 (built and verified, NOT started)
User order 2026-11-10 ~20:40Z: "serve our own https://demo.yshicarrick.com/demo.html for talon2 ... using same code".
Program 1 (Llama-3.2-3B target on the H200 + 4-card V80 EAGLE4 drafter). Cards: ATT F3t 0xE4B713F9 on 17/3d, MLPV 0xE4B71342 on 50/63.
The box1 demo was not touched. Its front end (pid 3355022) and FPGA engine (pid 3339224) were only read through /proc/<pid>/cmdline.
State
- Built on talon2 in
/home/carrick/eagle4_talon2_4c1/demo_t2_20261110/(create-only, owner carrick). NOT started. - Why not started: the card lock belongs to box1-qwen-junior (7-card b3sib tests,
~/TALON2_CARD_LOCK) until about 21:15Z. - The cards must be back on the F3t 4-card set and armed at depth ≥ 11 before starting (junior's restore).
- Still needed from the user:
site/demo_auth.conf(passwords) and one public hostname, see below.
Start / stop / status (from box1)
T2ROOT=/mnt/data/eagle4_scratch/drafter_speed_20261107/evidence/measurements/sibw_talon2_1340/t2root.sh
bash $T2ROOT 'bash /home/carrick/eagle4_talon2_4c1/demo_t2_20261110/start_demo_t2.sh' # optional: RUN=<run dir> before bash
bash $T2ROOT 'bash /home/carrick/eagle4_talon2_4c1/demo_t2_20261110/status_demo_t2.sh'
bash $T2ROOT 'bash /home/carrick/eagle4_talon2_4c1/demo_t2_20261110/stop_demo_t2.sh'
start_demo_t2.sh runs as root. It refuses to start if any of these fails (file checks only, no card or GPU access):
site/demo_auth.confexists and has no__FILL_ME__left;$RUN/discover.jsonis the F3t set (17/3d label 0xe4b713f9, 50/63 label 0xe4b71342);- ports 8099, 30017, 30019 and 30020 are free.
It then runs, detached: OWNER=demo-t2 bash ~/eagle4_talon2_4c1/lock/card_run.sh bash demo_t2_hold.sh.
That card_run.sh waits while the last line of ~/TALON2_CARD_LOCK is someone else's OWNER line, then writes OWNER demo-t2. The demo keeps the lock, the cards and the H200 until stop_demo_t2.sh.
Inside the lock, demo_t2_hold.sh does this:
- Starts the FPGA engine on :30020 (
demo_server_4c1.py, throughengine/demo_4c1_t2.sh serve, record pathsudo -E). Waits up to 1200 s for/healthwarm. - Starts the front end as carrick: uvicorn
demo_server:appon 127.0.0.1:8099, plain HTTP. - The front end's own keep-warm then starts the GPU-only engine (:30017,
sglang.launch_server) and the GPU + EAGLE engine (:30019,serve_gpuspec_t2.sh), both as carrick. - Every 60 s it runs
chown -R carrickon~/.cache/flashinferand~/.triton. The root FPGA engine's JIT files would otherwise lock out the carrick engines (memoryfeedback_talon2_sglang_as_carrick_not_root). Nothing is deleted. - It exits when both the FPGA engine and the front end are gone. That writes the FREE line to the lock file.
stop_demo_t2.sh stops only what the demo started, in this order:
- The front end first (its keep-warm would otherwise relaunch engines). Its pid must match
uvicorn demo_server:app --port 8099with cwdsite/. POST /admin/shutdownon :30019 and :30020.- SIGTERM to the process group of
sglang.launch_server --port 30017, only if its cwd issite/and it runs as carrick.
FPGA engine env = the talon2 F3t record cell
Record cell: runs/cfgsweep_20261109/serve3 d11fa3r2, 883.87 tok/s @ 6.90 on humaneval 80x256, measure-only (golden-depth unproven).
- Shape: w2 d11 V20 fa3, K_C 64, theta -2.5, arm PQC (HP_PREPOISON + HP_PINPAR + HP_CAPHOST + HP_IMGBYTES), PINPAR 0.
- Log:
hp_llama3b_PQC_w2d11_humaneval80x256_n1_183008_MEASUREONLY_goldendepth_unproven_20261109_183008.log, md5e564615f9840c1188541e19b1d50b954. - Driver:
cfgsweep_20261109/serve_cfg_drv.sh(md5 80225fe2…), using kitf3twin_20261109/kit/repo(run_4c1_sglang.shmd5 f20c188e…).
demo_t2.env exports the same knobs the record driver exported, because sudo -E passed them to the engine:
EAGLE4_PAIR_TREE_STOP_THETA=-2.5. It is NOT in the record's### FPGA_ENVline. Without the export the kit default is -1.25.EAGLE4_4C1_BOOT_ROOT,PP_SHADOW,PINPAR=0,EXTRA_FPGA_ENV,MEASURE_ONLY_GOLDEN_DEPTH_UNPROVEN=1,MAX_JOBS=1.RUNdefaults to the record RUNruns/cfgsweep_20261109/l2and can be overridden withRUN=. If junior's restore makes a new run dir, pass it.
Measured, DRYRUN derive (no card access), command ( . demo_t2.env; bash engine/demo_4c1_t2.sh derive ), output in logs/derive_fpga_check.txt:
- The derived
### FPGA_ENVtoken set is IDENTICAL to the record log's### FPGA_ENV(sorted diff empty). - The argv differs from the record only where box1's demo also differs:
demo_server_4c1.pyreplacesengine_test.py, bench-only args (mt_bench 50x32, envstr), and--port 30020 --host 127.0.0.1 --demo-mode fpga --demo-out … --enable-metrics. - DMA-latency hold
on 2 usandnumactl --cpunodebind=0come from the DRYRUN header, as in the record. Lock = held by the caller.
Files (talon2 /home/carrick/eagle4_talon2_4c1/demo_t2_20261110/; the same copies are committed in reports/results/DEMO_TALON2_20261110/)
| file | md5 | what changed vs box1 |
|---|---|---|
| start_demo_t2.sh | b7fb746bb32c075c1368bcbec9bcffff | new |
| stop_demo_t2.sh | bfc0021ab933ca72764ae4f0122febea | new |
| status_demo_t2.sh | 85c6e35860d15657e759f6292a5f0d72 | new (read-only) |
| demo_t2_hold.sh | e5c5a42f8cbfbd6ab97f1221a0314474 | new: the in-lock supervisor |
| demo_t2.env | a8baecdd3bf14e92cbd1c1449c27a7f1 | new: record env + paths |
| serve_gpuspec_t2.sh | 4507ca02013142009b71e466b2f14155 | new: GPU + EAGLE launcher (see the note under the table) |
| engine/demo_4c1_t2.sh | e164e2331c46677863f5cd6fc2f83b6f | box1 demo_4c1.sh (c2344a68): talon2 paths, kit repo, card_run.sh, DEMO_T2_LOCK_HELD, PP_SHADOW + kit demo dir on PYTHONPATH, DMA hold + NUMA pin |
| engine/demo_server_4c1.py | 31d439fe348da776d81a8f0f12b672b8 | box1 (802a1f7d) with only the SCRIPTS default changed to /home/carrick/.... The kit copy lacked --enable-metrics, so box1's version is used |
| site/demo_server.py | 9afd19cc015c8274bb77e1247601b3a1 | box1 (65,876 B): DEV 61:00.0 → 17:00.0; flash-record glob → f3twin_20261109/flash_f3t_talon2_*.json; IMAGE_REGISTRY gains 0xE4B713F9 / 0xE4B71342 (data only, so identity reads VERIFIED); GPUSPEC_LAUNCH → serve_gpuspec_t2.sh; retired FPGA_LAUNCH → a path that never exists (keeps the kill allowlist off the card engine) |
| site/read_card_id.py | 463619107e53afe7297f3997fe22df22 | BAR 61:00.0 → 17:00.0 |
| site/demo_keepwarm.py | 9f50d52e97434f28bb8ee4057fca482a | E4 log dir → demo_t2_20261110/logs |
| site/model_toggle.py, demo_autorun.py, demo.html | 442b0d2b…, 78f59ff3…, ca642c58… | unchanged (byte copies, with the rest of the static site, 68 entries, 6.8 MB) |
| site/demo_auth.conf.TEMPLATE | 4bd36ce815cc21c9f0104ceebec6800e | placeholders only. Box1's values were NOT read or copied |
GPU + EAGLE on talon2. Box1's gpuspec_chat_server.py stack is missing on talon2: the sglang-v0.5.6 checkout, the old 2-bit drafter eagle4_2bit_llama3b and /tmp/CostDraftTree are all absent. So :30019 is served by the same demo_server_4c1.py in exec-mode -1 (demo_4c1.sh MODE=gpu, ENGINE_PORT=31000). That is the same bitnet drafter on the H200 at w2 d11 V20 fa3, i.e. the GPU comparator shape. It answers /generate (one-shot, no token streaming in that mode).
Verified (nothing touched the cards or the GPU; talon2 ports 8099/30017/30019/30020 free at 20:5xZ)
- Python syntax (
ast.parse,venv_eaglefit) OK for the 6 Python files.bash -nOK for all 7 shell scripts. - DRYRUN derive: fpga rc 0 (env identical to the record, above); gpu rc 0.
- Front-end dry run. A copy in
dryrun_site_20261110/with a random throw-away password ran uvicorn on 127.0.0.1:18099 with keep-warm off and the FPGA URL pointed at a dead port. Results:/demo.html200 (69,133 B);/api/caps200;/api/chat/status200, "fpga offline";/api/model200 withallowed:false.- Without auth: 401.
/demo_auth.confand/demo_server.py: 404. - That process was stopped afterwards; it was my own.
- These paths exist on talon2: Llama-3.2-3B-Instruct, bitnet_721 bf16_ckpt, engine_test.py, mt_bench, CostDraftTree (carrick-owned), both local shadows, 4c1_boot_llama, the ami_sudo_pw file, torch_ext_root, the F3t flash record, kit
c4.py/thermal_wrap.sh. venv_eaglefitimports fastapi, uvicorn and sglang./usr/bin/python3has no fastapi.- talon2
cloudflared2026.10.0 at/usr/local/bin/cloudflaredsupportstunnel run --token-file. The existingcloudflared-talon2.serviceserves talon2 SSH.
Left to do
site/demo_auth.conf: copy it fromsite/demo_auth.conf.TEMPLATE, set mode 600, owner carrick, and fillAUTH_PW_L0,AUTH_PW_L1,DEMO_PW.SUDO_PWis optional (carrick's talon2 sudo password). Without it the live card-identity read is skipped and the page shows "unreadable".- Public hostname, option A or option B below.
- Start with the command above, after the lock frees (junior, about 21:15Z) and junior's restore of the F3t set at depth ≥ 11.
- Not enabled: the Qwen switch.
MODEL_TOGGLE_TIERS=none, because its card steps take the card lock the demo already holds, which would self-deadlock. The toggle code knows the F3t set; enabling it needs a lock-held path. - Watch the first start: H200 memory, since all three engines run on one H200. FPGA 0.50, GPU-only 0.30, GPU + EAGLE 0.20 by default. The keep-warm footprint numbers were measured on box1's old gpuspec engine.
Public URL, option A: a new Cloudflare tunnel run on talon2
- In the Cloudflare dashboard (Zero Trust → Networks → Tunnels), create a tunnel, e.g.
demo-talon2. - Add public hostname
demo-talon2.yshicarrick.com→ servicehttp://localhost:8099. - Paste the tunnel token into
/home/carrick/.config/eagle4/cf_demo_t2_tunnel.token(the directory is already 700 carrick;chmod 600the file). - Run, on talon2, as carrick:
setsid nohup /usr/local/bin/cloudflared tunnel --no-autoupdate run --token-file /home/carrick/.config/eagle4/cf_demo_t2_tunnel.token > /home/carrick/eagle4_talon2_4c1/demo_t2_20261110/logs/cloudflared.log 2>&1 &
Public URL, option B: a new hostname on box1's existing tunnel, through an ssh forward
box1 has no ssh alias for talon2. Use the explicit form from the eagle4-talon2-connect skill. ssh config and known_hosts are not edited.
- On box1 (port 18100 is free), run:
ssh -N -o BatchMode=yes -o ServerAliveInterval=30 -o ExitOnForwardFailure=yes -o UserKnownHostsFile=/mnt/data/eagle4_scratch/talon2_known_hosts -o ProxyCommand="/usr/local/bin/cloudflared access ssh --hostname %h" -L 127.0.0.1:18100:127.0.0.1:8099 [[withheld]] - In the dashboard, on box1's existing token-managed tunnel (
cloudflared.service), add public hostnamedemo-talon2.yshicarrick.com→http://localhost:18100.
Notes
- No flashes, resets, card reads or GPU use, no edits to
~/.ssh, and no credential read or printed. No kills except my own dry-run uvicorn. - One
ami_tool sensors -d 17:00.0was issued early as a reachability check. It is read-only, timed out after 60 s and returned nothing. - I removed temp files I had created seconds earlier: talon2
/tmp/{rec_env,dem_env,rec_cmd,dem_cmd,dr}.<pid>, and my scratchpad staging copy ofdemo_4c1.sh. - Left in place:
dryrun_site_20261110/(with its throw-away conf) andlogs/derive_*_check.txt.