fpga-archive-demo-box1-20261110 · EAGLE4 FPGA archive

Source reports/results/DEMO_AUTORUN_BOX1_20261110.md · source md5 c8863bd11587e8be9df520a2df689c39 · archived 2026-11-18

Download PDFOpen PDFRaw markdown

Demo auto-run loop on box1: built, tested, and LIVE (2026-11-10)

The user asked for this verbatim: "can you for demo demo site on box1 build this in with our collectd best prompts?" "This" is Finn's spec, docs/DEMO_AUTORUN_SPEC_FINN_20261106.md. The work was done by a box1-qwen-junior worker.

State: SWAPPED LIVE at 18:41:07Z, 1.2 s downtime. https://demo.yshicarrick.com is served by the new demo_server.py (md5 0c25a7dc…) and demo.html (md5 d020cf31…). The server is pid 2993342 on 0.0.0.0:8099 with TLS. The public URL returned HTTP 200 for /api/autorun/status and /demo.html (demo login), and 404 for /autorun_logs/.... The loop is IDLE. Boot state is Manual.

Nothing else was touched: the engine on :30020, cards 01/42/61/62, cloudflared, eagle4-dashboard.service and the H200 memory. No new GPU engine was started.

Rollback (one command, about 1 s of downtime)

bash scripts/4c1/demo/demo_autorun_swap.sh rollback

The script copies demo_server.py.bak_20261110_pre_autorun (md5 b3e8bfa0…, the pre-swap live file) and demo.html.bak_20261110_pre_autorun (md5 3c6aa0bc…) back over the live files. It then restarts only the uvicorn on :8099. It finds that pid with ss and checks its argv in /proc before stopping it. It relaunches the server exactly as serve_demo.sh does. swap and restart modes also exist.

What was built

File Role
reports/arch_site/demo_autorun.py (md5 98a7a47e) The server-side loop. It provides /api/autorun/{start,pause,resume,stop,status,sets}, with aliases at /autorun/*. It holds MODEL_CFG, the per-model limits with their evidence. Each prompt is a fresh one-turn conversation. The default gap is 1.5 s. The loop wraps at the end of a set; dataset order is the default and shuffle is optional. A prompt fails after 60 s with an error marker and is skipped. After 3 consecutive failures the loop backs off for 10 s. It pauses above 85 °C on card 61:00.0 (ami_tool sensors, polled every 10 s). It keeps the last 5 turns. Status is long-polled with since and hv.
demo_server.py (= tested demo_server_autorun.py) The three chat generators moved to module level unchanged; only their arguments changed. The loop drives the same generate path as manual chat. While the loop is active (running or paused), these answer "busy": /api/chat with 409, /api/chat/stream with an SSE error frame, and POST /api/model, /model/probe, /model/readback and `/drafter/free_run
demo.html (= tested demo_autorun.html) Adds a Manual / Auto-run control beside the mode tabs and a "Max reply 256/512/1024" selector. Manual defaults to 256, as before. The Auto-run bar has a set picker, Shuffle, Start (Apply while running), Pause/Resume and Stop. It also adds the tally strip and a "source · n of N" label above each turn. Replies render with the same gold burst blocks. Model switch and "Measure drafter speed" are disabled while the loop runs. Every viewer follows the one loop, and a refresh reattaches. Switching to Manual stops the loop after a confirm. Clicking a mode tab mid-loop switches modes at the next prompt and resets the tally.
reports/arch_site/prompts/*.jsonl + MANIFEST.json best_llama (15) is the DEFAULT set; best_qwen (10) is used only if Qwen is served. humaneval164 (164) uses the wrapper "Complete the code I provided." + prompt, which matches all 80 measured rows byte for byte. speedbench_qualitative has 880 rows (11 categories × 80, turns[0]). n_tok is counted with the endpoint's chat template; words_in_sentence gives 282, equal to the engine's prompt_tokens 282. Rows over the 512-token prompt budget are dropped at load and the count is shown: 1 of 164 HumanEval rows and 96 of 880 SPEED-Bench rows. test_errskip.jsonl loads only with EAGLE4_AUTORUN_TESTSETS=1 and is hidden on the live server.
scripts/4c1/demo/build_autorun_prompt_sets.py Builds the sets offline. Inputs were fetched once from Hugging Face into /mnt/data/eagle4_scratch/humaneval164_raw_20261110/he_test.parquet (md5 755ae047) and /mnt/data/eagle4_scratch/speedbench_raw_20261110/qualitative_test.parquet (md5 58d4d3ac). SPEED-Bench (nvidia/SPEED-Bench, nvidia-evaluation-dataset-license) was reachable, so all three sets ship.
scripts/4c1/demo/test_demo_autorun.py The acceptance tests below. Credentials are read from demo_auth.conf and never printed.
scripts/4c1/demo/demo_autorun_swap.sh swap / rollback / restart.
reports/arch_site/autorun_logs/autorun_YYYYMMDD.jsonl One line per prompt: set, id, mode, model, cap, tokens, tok/s, accept, verify steps, ttft, wall, ok/err, text. This answers Finn's open question with yes. Not served publicly.

Token cap (max context)

Tests and numbers

Tests ran on the :8098 copy, which shared the same engine on :30020 through the normal chat API. The suite was then re-run on the live :8099 after the swap. Raw logs: /mnt/data/eagle4_scratch/demo_autorun_test_{r1,r2,live}_20261110.log.

Test :8098 copy live :8099
t1: manual FPGA chat while idle PASS, 256 tok, 973.32 tok/s @ 7.529 PASS, 930.22 @ 7.529
t3: Best set, FPGA, cap 256 5 prompts, 0 errors: 1,280 tok, mean 1,049.6 tok/s (token-weighted) @ accept 9.08. Per prompt: words_in_sentence 1083 @ 8.83, even_odd_palindrome 995 @ 8.83, sort_array 1066 @ 8.83, is_sorted 1026 @ 9.14, pluck 1084 @ 9.85 3 prompts: 992.3 @ 8.83 (995 / 1020 / 963)
t4: busy responses during the loop PASS: stream error frame busy=autorun, /api/chat 409, probe 409, model switch 409 PASS
t5: pause, then resume PASS: the reply in flight finished, the count held at 5 for 5 s, and resume continued PASS
t6: refresh reattach PASS: a fresh status call gets the full snapshot (5 turns, v/hv) PASS
t7: stop PASS: stop took 0.09 s, the tally reset, and manual chat answered right after PASS (0.06 s)
t8: error-skip (row 2 with a bad mode) PASS: prompts 4, ok 3, errors 1 (unknown mode 'bogus'), the loop continued and wrapped n/a (test set hidden)
t9: GPU+EAGLE while that engine is down PASS: 409 "engine is not running; the auto-run loop never starts engines" PASS
t10: SPEED-Bench, cap 1024 PASS: 6 prompts, 0 errors, 2,661 tok, 654.3 tok/s @ 5.06 not run
t11: HumanEval 164, cap 512 PASS: 6 prompts, 0 errors, 1,945 tok, 547.1 tok/s @ 5.98 not run
UI (headless Chrome via CDP, :8098) PASS: boot view Manual; Auto-run shows 4 sets; Start gives a running tally ("prompts 4 · tokens 1024 · mean 1027 tok/s · mean accept 8.9 · card 51 °C"); bursts render; model buttons and probe are disabled while running and re-enabled after Stop; page reload reattaches (view auto, 5 turns); Pause/Resume and Stop work; Manual send works afterwards; 0 JS exceptions. Screenshots in the session scratchpad (ui/01_running.png … 04_manual.png) —
Offline mock (demo_autorun.Autorun with fake deps) PASS: thermal pause at 90 °C ("thermal: card device 90 C > 85 C"), back-off after 3 failures, timeout error marker, set switch at the boundary, stop —

Comparison with measured numbers. The 2026-11-09 Best-prompt numbers are 976–1028 tok/s @ 7.31–7.53, measured on F3t/E2b at w2 d7 and cap 256 (DEMO_REQUEST_RECOMMENDATIONS_20261109.md). Through the loop, the live engine (E2b + MLPV 0x1342, w2 d11) measured 992–1050 tok/s @ 8.8–9.1 at cap 256. Accept is higher because the tree is d11 rather than d7. These are MEASUREONLY, goldendepth-unproven runs on Llama, which is how the engine is labelled.

Open items

Addendum 18:45Z: energy card (white paper v5 section 5) — LIVE, demo.html only, no restart

Addendum 18:50-18:55Z: Qwen best-prompt test, then back to Llama d11 (user: "also do the prompt test on qwen")

# prompt tokens tok/s accept measured mean (DEMO_REQUEST_RECOMMENDATIONS, talon2, 10 runs)
1 even_odd_palindrome 256 277.44 7.758 281 @ 7.76
2 maximum 256 275.30 7.758 273 @ 7.76
3 remove_vowels 166 (stop) 285.52 7.905 271 @ 7.90
4 pluck 256 268.29 7.758 271 @ 7.76
5 max_fill 256 234.63 7.758 270 @ 7.76
6 is_sorted 256 269.74 7.758 270 @ 7.76
7 bf 256 272.24 7.758 270 @ 7.76
8 words_in_sentence 256 276.75 7.758 270 @ 7.76
9 make_a_pile 177 (stop) 279.51 7.696 268 @ 7.70
10 fib4 256 269.22 7.529 266 @ 7.53

Addendum 19:50-19:58Z: all three modes kept warm, mode-switch loading bar, no "asleep / warming up" text — LIVE 19:57:37Z (1.18 s)

User, verbatim: "do not show the engine was asleep, warming up… (~1–2 min)!!!! show the mode switching with a loading bar so that a soon as the user type its instant!"

Incident first (19:49–19:55Z, not caused by this change, and it shaped it).

What changed (demo_server.py md5 fe00155b…, demo.html md5 66499db4…, new demo_keepwarm.py, serve_demo.sh):

Engines now (nvidia-smi, 19:58Z):

mode GB
FPGA endpoint (Llama w2 d11) 59.5
GPU + EAGLE (:30019, MEM_FRAC 0.22) 18.5
GPU only (:30017, frac 0.30) 20.5
Total 98.5 of 140.4 GiB

The two GPU engines were already running (started from the page at about 19:48 and 19:50Z); they were not restarted.

Measured on live :8099 after the swap (/api/chat/stream, "Write a Python function that checks whether a string is a palindrome.", 128 tokens, greedy):

mode attach (health + 1 token) client time to first token server ttft decode
FPGA (4 cards) 16 ms 20 ms 0.011 s 1,113.9 tok/s @ 8.53
GPU + EAGLE 22 ms 25 ms 0.017 s 153.4 tok/s @ 2.78
GPU only 29 ms 33 ms 0.027 s 85.2 tok/s

Rollback: bash scripts/4c1/demo/demo_autorun_swap.sh rollback-warm. It restores *.bak_20261110_pre_warm (demo_server.py 0c25a7dc…, demo.html 10e0f757…) and restarts :8099 only. The rollback still runs without the reaper, because the swap script exports EAGLE4_DEMO_NO_REAPER=1. serve_demo.sh.bak_20261110_pre_warm is the pre-change launcher.

Open:

Addendum 20:16-20:18Z: masthead tiles = current facts; design-audit items 2, 5, 6, 8, 10 (demo.html + measured_pair.json, no restart)