# Talon demo: benchmark loop mode and longer replies (spec by Finn, 2026-11-06; pasted by the user 2026-11-10 for the box1 demo)
Condensed copy of the spec the user pasted; the user's instruction: "can you for demo demo site on box1 build this in with our collectd best prompts?"

## Feature 1: benchmark auto-run loop
- UI: Manual / Auto-run segmented control in the chat panel header beside the mode tabs (FPGA (4 cards) / GPU + EAGLE / GPU only). In Auto-run: prompt-set picker + Start / Pause / Stop replacing the message box. Show source + index above each turn ("HumanEval/42 · 43 of 164", "SPEED-Bench · coding · 12 of 80"). Benchmark prompt rendered as a user message, reply streamed with the gold burst blocks as in manual chat. Running tally strip: prompts done, total tokens, mean tok/s, mean accept, elapsed; reset on Stop or set/mode/model change; shows set, mode, model and token cap.
- Behavior: same generate path as manual chat; gap 1.5 s (configurable) after EOS/cap; wrap at end until Stop; dataset order by default, optional shuffle; each prompt a fresh conversation; keep last N=5 turns on screen; Pause finishes current reply then holds; Stop cancels in-flight + clears; typing disabled in Auto-run; switching to Manual stops the loop; error/timeout (60 s) → log, error marker, skip; back off 10 s after 3 consecutive failures.
- Data: HumanEval 164 (same chat template + wrapper as the measured humaneval runs); SPEED-Bench Qualitative split (arXiv 2604.09557; skip the Throughput split). Bundled local JSONL (id, category, prompt) + a build script pulling from Hugging Face; no network at runtime. Drop prompts longer than the prompt budget; log how many.
- Backend: the loop runs on the SERVER (survives refresh; all viewers see one stream; client subscribes to the token stream). Endpoints: POST /autorun/start {set, mode, model}, /autorun/pause, /autorun/stop, GET /autorun/status. While running, a manual request from another browser gets a "demo running benchmark loop" busy response.

## Feature 2: max output 1024
- max_new_tokens default 256 → 1024 in all three modes IF the cards hold the context. Find max context per flashed image; record in model config (not a magic number); max_new = min(1024, card_context − prompt_budget), prompt budget 512. UI selector 256/512/1024 defaulting to the highest supported. Same cap for GPU modes. Confirm no hardcoded 256 in verify path. If Qwen can't do 1024: 512 for Qwen, 1024 for Llama.
- Label the live tally with the cap in use.

## Panel interactions
Energy idle/active badge driven by real activity; sensors unchanged; Last FPGA reply / Last GPU rate update per reply; thermal stop (device > 85 °C pauses the loop) if no board protection; FPGA model switch and "Measure drafter speed" disabled while Auto-run runs (or stop/reload/restart); mode switch mid-loop finishes the reply, resets tally, continues in the new mode.

## Acceptance
Manual unchanged when Auto-run off · both sets in all 3 modes, wrapping · 8 h unattended on FPGA without crash/browser memory growth/stuck request · refresh reattaches · Pause/Stop/Manual within one reply · replies reach the cap without errors and FPGA output == GPU-only under greedy · HumanEval loop at the record cap lands near the headline cards · tally shows set/mode/model/cap.

## Open questions (Finn → Carrick)
Max context per flashed image · kiosk autostart or by hand · shuffle · per-prompt result log to a file · 7-card coverage.
