# Box1 4-card demo: software readiness for E2b (now) and F3t (after a flash) — 2026-11-10

**Scope: code + mock only.** No card, no lock, no GPU, no demo process was touched. Program: the 3-card-class Llama-3.2-3B DRAFTER (4C1) plus its dual-model Qwen-27B profile.
**Result:** the two software blockers that stopped the box1 demo on its current images are fixed and tested offline; the Qwen side needed three kit ports (done) and has a short list of open items (section 5). Nothing here is proven on silicon by this work — section 6 lists what still needs a card.

Cards (from `BOX1_DEMO_TEMPS_HUMANEVAL_20261110.md`): 01/42 = ATT E2b `0xE4B713E2` (md5 `2540c725`, reg100 `0xe4b711e0`, tp_bid `0xe4b713e0` stale by design), 61/62 = MLPV `0xE4B71342` (md5 `3e82ea1d`, reg100 `0xe4b71142`). Flash record `scripts/4c1/flash_records/flash_4c1_box1_e2b_20261107.json`.

## 1. Blocker 1 — `reports/arch_site/model_toggle.py` was hard-wired to the 0x129C pair
Evidence of the blocker: `/mnt/data/eagle4_scratch/box1_temp_20261109/start_demo_llama.out` — `state refused, c4 discover rc 3` (the default flash record has no 0x13E2 entry).

Fix: the accepted image sets are a table, `DUAL_SETS`:

| key | ATT label | ATT reg 0x100 | MLPV label | MLPV reg 0x100 | fold step | golden gate |
|---|---|---|---|---|---|---|
| 129C | 0xE4B7129C | 0xE4B7119C | 0xE4B7128B | not checked (as before) | yes | as before (measure-only) |
| E2b | 0xE4B713E2 | 0xE4B711E0 | 0xE4B71342 | 0xE4B71142 | skipped (G4_EMBFOLD 0) | record identity |
| F3t | 0xE4B713F9 | 0xE4B711F9 | 0xE4B71342 | 0xE4B71142 | skipped | record identity |

- Selection: the set is the one the flash record names (`EAGLE4_TOGGLE_FLASH_RECORD`, default unchanged = the 0x129C Q3 record). `c4 discover` checks the cards against that record; `check_dual` then checks the reg 0x100 read-back against the same row (ATT on every set, MLPV on E2b/F3t).
- Every refusal path is kept. New refusals: a flash record that names an image outside the table ("unknown image"), a record that mixes sets, wrong MLPV on E2b/F3t. When the cards carry another known set, the refusal says which one and names the env to set.
- 0x129C behaviour is unchanged: same constants, same refusal strings (asserted byte for byte), same `/api/model` `requires` dict.
- On E2b/F3t the switch requires the run's golden to be identical to the E2b record (section 2) before the endpoint starts; the Llama endpoint is then served under that gate, without the measure-only stamp. The Qwen endpoint keeps `MEASURE_ONLY=1` (its launcher path), with the same identity requirement enforced by the toggle first.
- The running demo server on :8099 imported the old module; it picks this up only when its owner restarts it.

## 2. Blocker 2 — the SGLang kit refused E2b because the bf16 golden never passes on a LUTG A8 image
Evidence: `4c1_box1_e2b_he80x256_temps_20261109_192011.log` — `golden.json verdict 'FAIL'`. On E2b/F3t the ATT activations are 8-bit, so every tree matches the bf16 golden at L0 (8/8) and differs from L1 down. talon2 served both images on a different proof: the golden output must be identical, tree for tree and level for level, to the E2b record golden (`tools_from_talon2/cmp_golden_levels.py`, `f3t_kit/win_f3t.sh`, the STOP rule in `chain_f3t.sh`), then ran measure-only.

Fix: new module `scripts/4c1/c4_recgate.py` = that proof as a kit gate.
- Applies only when both ATT are `lutg` + `a8q` rows of `c4_regmap` in `{0xE4B713E2, 0xE4B713F9}` and both MLPV are `0xE4B71342`, 4-card topology.
- Records are pinned by md5 (a record whose bytes changed is refused):

| record | model | trees | md5 | file under `TRACK1_RECORD_F3T_20261109/e2b_same_day/` |
|---|---|---|---|---|
| e2b_llama_md15_7_2 | llama3b | 12 (56 levels) | `399b0e06415b601f4301a01803a9036f` | `e2b_record_20261106/golden_llama_e2.json` |
| e2b_llama_md7_2 | llama3b | 9 | `9a1206e9b9b6fca034cf97a7af7fa03a` | `e2bwin/l/golden.json` |
| e2b_qwen_md7_2_kc64 | qwen27b | 9 | `8906b857089b6a8f69b56046a88d2d35` | `e2bwin/q/golden.json` |

- PASS = same tree list (golden, md, ctx, TREE_STOP rule), and per tree the same verdict, depth, compared depth and, per level, the same cand / surv / surv_blocks; same loop mode bits and fold; 0 discarded trees; L0 8/8 everywhere. An empty or shorter list is a FAIL.
- `c4.py golden` writes the result into `golden.json["record_identity"]` (the bf16 `verdict` stays `FAIL`) and returns 0 when identity passes. `c4_recgate.py stamp --run RUN` does the same on an existing golden, file-only, and keeps the file's mtime so a stamp can never make an old golden look fresh.
- `c4_sglang.open_session`: when the golden is not a bf16 PASS, it needs the stamp **and recomputes the comparison** before any tree is timed. It still requires golden.json newer than the arm, the loop-mode and fold match, and a non-discarded TREE_STOP tree when TREE_STOP is on.
- Not an override: `EAGLE4_4C1_ALLOW_UNGATED` is untouched and not needed; `EAGLE4_4C1_RECORD_GATE=0` restores the old behaviour exactly.
- The golden step must run with the record's arguments and the default TREE_STOP theta (-1.25, i.e. do not export `EAGLE4_PAIR_TREE_STOP_THETA` for the golden step); the serve launcher sets -2.5 by itself.

## 3. Qwen side on box1 (checked offline)
Present and consistent:

| item | state |
|---|---|
| Qwen boot root | `~/eagle4_artifacts/4c1_boot_qwen27b/v3_ropestep256_enormk` (the profile default) |
| MANIFEST md5, ATT 0xE4B713E2 rank0 / rank1 | `7def3a2a68106f269249dde62154c1b8` / `f97bf560a8c539476b2c0188a1d78097` |
| MANIFEST md5, ATT 0xE4B713F9 rank0 / rank1 | `dcd72eb3c3e52d1c828d904229aa352a` / `128cf67c6e00ac03cc5fcccb3eb4a1e5` |
| MANIFEST md5, MLPV 0xE4B71342 rank0 / rank1 | `c2b7e343555f0795f635750c1616ae91` / `6694caa3626f47980a87d4c88e514543` |
| Llama boot root `~/eagle4_artifacts/4c1_boot`, 13E2 r0/r1 | `ef71d0a2…` / `8af9f2f9…` (= talon2's) |
| Llama 13F9 r0/r1 | `ddf7653b4105e455c33adfb2354ae375` / `89a855b80ea15f0566040f8f6370df35` |
| Llama 1342 r0/r1 | `1c8e12d7…` / `8378ee10…` |
| Qwen goldens | `~/eagle4_artifacts/4c1_boot_qwen27b/goldens_20261020/c4_golden_q27h200/goldens/md15_golden_q27_{p2_step001_ctx0089,p4_step000_ctx0159,p5_step002_ctx0187}.json` present |
| Qwen switch on box1 E2b | already done once by the card owner: `/mnt/data/eagle4_scratch/4c1_runs/box1_e2b_tempbench_20261110_q2/` (switch.json, golden 0/9 bf16). That golden is 9/9 identical to the Qwen record under the new gate. Its `golden.log` was not teed in the talon2 `.lines` form, so the `7aceae28` lines-md5 was not recomputed here; level identity is the stronger check. |

Missing from the box1 kit, now ported (all additive, only reached by E2b/F3t rows):
1. `c4_bringup.VERIFY_LOG` default list had no E2b/F3t/0x1342 verify logs (the 11-10 bring-up needed an env file). Added the box1 logs: Llama 13E2, Llama 1342, Qwen 13E2 in place; F3t Llama + Qwen and Qwen 1342 under `BOX1_DEMO_READINESS_E2B_F3T_20261110/logs/`.
   - The two F3t logs are new: `f3t_kit/verify_relabel_identity_13F9.py` run on the box1 dirs, `RELABEL_IDENTITY PASS` for both roots (37 and 48 files byte-identical to 13E2, `boot.ops` differs only in the label lines).
   - `verify4c1_qwen27b_mlpv_eaeb_1342_20261105.log` (md5 `ceabce4f…`) is a copy of the box1-twin verify of 11-05; it was only in scratch twins, not in the repo.
2. LUTG zero-bank guard in `load_manifest` (talon2 `patch_13de_host.py`): a rank dir without the 6 zero-bank loads is refused. All eight box1 ATT dirs pass; a dir with one pin removed is refused.
3. `c4_profile` qwen27b `enorm_rescale.images` += 0xE4B713E2, 0xE4B713F9, so the F2 e_norm check runs on them (it was skipped). All four box1 Qwen ATT dirs check exact.

With these, `load_manifest` accepts all 12 (model × card × rank) kits with the default verify list.

## 4. Operator runbook
All commands from the repo root as `y344shi`. `PWF=~/.config/eagle4/ami_sudo_pw`. Nothing below flashes or resets except step F1, which is the owner's `-p 0` flash.

### A. After every boot / cold AC (before anything is timed)
```bash
export PWF=~/.config/eagle4/ami_sudo_pw
bash scripts/restore_artifact_links.sh                       # every line LINKED:
lspci -d 10ee:50b4 | grep -c .                               # 4 (01 42 61 62)
sudo -p "" ami_tool overview < $PWF                       # 4 READY
# SMI / AER corrected-error mask (lost at every boot). Measure, mask all four V80s + their root ports, measure again.
sudo -p "" perf stat -a -e ls_smi_rx sleep 5 < $PWF 2>&1 | grep ls_smi_rx      # storm ~136k / 5 s, clean 0-3k
MST=/sanctuary/lab/smi_aer_mask_state_$(date +%Y%m%d_%H%M%S); mkdir -p $MST
sudo -p "" python3 -I /sanctuary/pdi_inbox/SMI_AER_AB/v1/lib/smi_aer_ab.py apply-persist $MST --scope both --cards "01 42 61 62" < $PWF
sudo -p "" perf stat -a -e ls_smi_rx sleep 5 < $PWF 2>&1 | grep ls_smi_rx      # must be clean; undo = smi_aer_ab.py restore $MST/saved_state.json
for B in 01 42 61 62; do OWNER=box1-4c1-qdma bash /sanctuary/lab/card_run.sh sudo -p "" EAGLE/v80_test_suite/03_setup_qdma.sh 4 $B:00.0 < $PWF | grep "Roundtrip test"; done
```
- The mask must go on **before** the demo endpoint takes the card lock: the tool skips cards named in `CARD_OWNER`.
- Card 01 alone is not enough since 10-18 (`post_coldac_4c1.sh` step 4): mask all four. Use a fresh state dir each boot; the tool refuses to overwrite an undo point.
- `scripts/4c1/post_coldac_4c1.sh` runs the same steps but then starts the default (non-E2b) bring-up; use the lines above for E2b/F3t.

### B. Bring-up, Llama on E2b as flashed
```bash
export RUN=/mnt/data/eagle4_scratch/4c1_runs/box1_e2b_$(date +%Y%m%d_%H%M%S)
export FLASH_RECORD=$PWD/scripts/4c1/flash_records/flash_4c1_box1_e2b_20261107.json
RUN=$RUN PWF=$PWF FOLD=0 DEPTH=15 GOLDEN_MD=15,7,2 STEPS="discover linkfix link boot arm golden" bash scripts/4c1/run_4c1.sh
grep -E "^GOLDEN" $RUN/run.log        # expect: GOLDEN FAIL (0/12 ...) + GOLDEN RECORD-IDENTITY PASS (trees identical 12/12, levels compared 56 ...) + GOLDEN GATE PASS by RECORD IDENTITY
```
- Arm depth 15 once per boot (PERSIST latches the deepest depth at the first run); it covers d7 and d11.
- Cards already armed this boot with an older golden: `python3 -I scripts/4c1/c4_recgate.py stamp --run $RUN` stamps the existing golden.json (file-only, rc 0 = PASS).
- If the identity line says FAIL, stop: that is a correctness failure, not a gate to step around.

### C. Timed runs, Llama (single stream)
```bash
HE=~/haochen/eaglefit/inference/dataset/humaneval/question.jsonl
# humaneval 80 x 256, d7 (VTOK 20 = the record shape) and d11 -- CORRECTED 2026-11-10: run d11 with VTOK 20 (set it explicitly; the
# launcher's full-tree default is 43). VTOK 20 is the measured winner at d11 on talon2 (883.12, F3T_CONFIG_SWEEP) and on box1 (BOX1_E2B_D11_20261110.md)
RUN=$RUN PWF=$PWF BENCH=$HE MAXREQ=80 MAXNEW=256 DEPTH=7  WIDTH=2 VTOK=20 TAG=box1_e2b_he80x256_d7  bash scripts/4c1/run_4c1_sglang.sh
RUN=$RUN PWF=$PWF BENCH=$HE MAXREQ=80 MAXNEW=256 DEPTH=11 WIDTH=2         TAG=box1_e2b_he80x256_d11 bash scripts/4c1/run_4c1_sglang.sh
# the 15-prompt demo set (original humaneval rows, rank order)
python3 -I scripts/4c1/demo/demo_prompts_to_bench.py --model llama --out /mnt/data/eagle4_scratch/demo_prompts_bench_20261110/llama
RUN=$RUN PWF=$PWF BENCH=/mnt/data/eagle4_scratch/demo_prompts_bench_20261110/llama/question.jsonl MAXREQ=15 MAXNEW=256 DEPTH=7 WIDTH=2 VTOK=20 TAG=box1_e2b_demo15_d7 bash scripts/4c1/run_4c1_sglang.sh
```
- Launcher defaults already give FASTGLUE 4, K_C 64, TREE_STOP theta -2.5, KVI on, first draft on GPU, cuda graphs; the log header prints all of them.
- **PQ+C is not available in this kit** (section 5, item 1). Asking for it (`EXTRA_FPGA_ENV="EAGLE4_4C1_HP_PREPOISON=1 ..."`) is now refused with rc 62 instead of being silently ignored. The kit's own older `PREPOISON=1 PINPAR=1` are on by default.
- CORRECTED 2026-11-10: d11 is served with VTOK 20 (record config w2 d11 fa3 V20 K_C 64 theta -2.5), not the launcher's full-tree default 43. Measured on box1 E2b: d11 V20 820.20 (non-PQC) / 836.00 (PQ+C) @ 6.90.
- Each run takes the card lock itself and refuses while the demo endpoint holds it.

### D. Demo endpoint and model switch
```bash
# endpoint by hand (holds the card lock while it serves; stop with POST /admin/shutdown)
RUN=$RUN PWF=$PWF MODEL=llama3b DEPTH=7 WIDTH=2 bash scripts/4c1/demo/demo_4c1.sh serve
```
For the web toggle, the demo server must be started by its owner with:
`EAGLE4_TOGGLE_FLASH_RECORD=<repo>/scripts/4c1/flash_records/flash_4c1_box1_e2b_20261107.json EAGLE4_TOGGLE_LATCHED_DEPTH=15`
(the cards are armed at 15; the toggle's arm step passes `--accept-latched-depth`). The toggle then runs discover → read-back → switch → arm → golden (md 7,2) → identity check → serve.

### E. Qwen on the same cards
```bash
QG=~/eagle4_artifacts/4c1_boot_qwen27b/goldens_20261020/c4_golden_q27h200
export RUNQ=${RUN}_q; mkdir -p $RUNQ
c4() { OWNER=box1-4c1kit bash /sanctuary/lab/card_run.sh sudo -p "" -E env HOME=~ /usr/bin/python3 -I scripts/4c1/c4.py "$@" < $PWF; }
c4 --model qwen27b discover --out $RUNQ/discover.json --flash-record $FLASH_RECORD
c4 --model qwen27b switch --disc $RUNQ/discover.json --model qwen27b
c4 --model qwen27b arm --disc $RUNQ/discover.json --depth 15 --accept-latched-depth 15
c4 --model qwen27b golden --disc $RUNQ/discover.json --md 7,2 --kc 64 --gold-dir $QG \
   --golden md15_golden_q27_p2_step001_ctx0089.json,md15_golden_q27_p4_step000_ctx0159.json,md15_golden_q27_p5_step002_ctx0187.json --out $RUNQ/golden.json
# expect GOLDEN RECORD-IDENTITY PASS (trees identical 9/9 ... record e2b_qwen_md7_2_kc64)
RUN=$RUNQ PWF=$PWF MOCK=0 BENCH=humaneval MAXREQ=80 MAXNEW=256 DEPTH=7 WIDTH=2 VTOK=20 FASTGLUE=4 TAG=box1_e2b_qwen_he80x256_d7 bash scripts/4c1/run_4c1_sglang_qwen.sh
```
- Back to Llama: the same four `c4` lines with `--model llama3b`, `--md 7,2`, no `--kc/--gold-dir/--golden`, into a new run dir; identity is then against `e2b_llama_md7_2` (9/9).
- The H200 must have room for the 27B target (the toggle's guard: about 62.7 GiB / 0.55 = 114 GiB free); the GPU demo engines hold memory.
- talon2 ran the Qwen cell with `MEASURE_ONLY=1`; with the record gate it is no longer needed. The Qwen launcher takes a dataset name in `BENCH` and an explicit file in `BENCH_PATH`: for the 10-prompt Qwen demo set run `demo_prompts_to_bench.py --model qwen --out <dir>` and pass `BENCH_PATH=<dir>/question.jsonl MAXREQ=10`.

### F. The same after an F3t flash of 01/42
1. Flash (owner): the F3t PDI is **not on box1** (`~/eagle4_artifacts/uni_oneflip/` has no `*13F9*`). Copy `oneflip_p2s_head_asgard_0xE4B713F9_66m_97d8445c.pdi` from talon2 `~/eagle4_artifacts/uni_oneflip/`, check md5 `97d8445cc6e64ff95af8894d6f267965` and the FPT magic `16 a5 f7 92`, run `scripts/preflash_gate.sh`, flash 01 and 42 detached with `-p 0`, wait for `FLASH_EXIT=0` on both, then cold AC.
2. Write the flash record (it does not exist yet; a record is written from the flash logs, not ahead of them): copy `flash_4c1_box1_e2b_20261107.json` to `scripts/4c1/flash_records/flash_4c1_box1_f3t_<date>.json`, set both ATT entries to label `0xE4B713F9`, pdi_md5 `97d8445cc6e64ff95af8894d6f267965`, the PDI path, and a `_source` line naming the flash log.
3. Section A, then B–E unchanged with `FLASH_RECORD` (and `EAGLE4_TOGGLE_FLASH_RECORD`) pointing at the new file and run-dir names changed. F3t reads reg100 `0xe4b711f9` and tp_bid `0xe4b713f9` (= label, not stale). The records are the same E2b records: F3t is level-identical to E2b on both models (`f3t_run/` goldens, 12/12, 9/9, 9/9 under the new gate).
4. One F3t-specific rule from the record: the 0x90 / 0xD4 / 0xD8 debug read-outs are cut on this image (TIMING_HARD) and are not evidence.

## 5. Still missing (not fixed here)
1. **Host pre-card levers PQ+C** (`c4_hpre`: HP_PREPOISON, HP_PINPAR, HP_CAPHOST, HP_IMGBYTES). The F3t 831.14 and E2b 816.47 records ran with them; the box1 repo kit has no `c4_hpre.py`. Copies exist at `/mnt/data/eagle4_scratch/hostpre_20261107/kit/new/scripts/4c1/c4_hpre.py` and in the kit copy `/mnt/data/eagle4_scratch/box1_temp_20261109/kit_e2b/4c1/`. Porting touches the card-process path and needs its own gate; until then box1 numbers are the non-PQC arm (talon2 arm B on E2b was 784.06 vs 816.47 with PQC).
2. **Mock endpoint**: `reports/arch_site/mock_4c1_endpoint.py` is referenced by the toggle's mock mode and is not in the repo, so a full mock switch cannot reach "ready". The mock tests cover the card steps and the refusals.
3. **F3t PDI and box1 F3t flash record** (section F).
4. **Verify-log weakness (pre-existing)**: Qwen verify logs also carry un-prefixed `[verify4c1] 0x… rankN: PASS` lines, and the Llama check matches un-prefixed lines, so a Qwen log can satisfy a Llama rank of the same label if it is newer than that MANIFEST. Not changed here; the Llama 1342 log is listed, so the right log is found first only by list order.
5. `demo_4c1.sh` carries someone else's uncommitted `--enable-metrics` edit; left alone, not committed.
6. A stray empty-purpose file `BOX1_DEMO_READINESS_E2B_F3T_20261110/logs/.t1` (timing output of the relabel run) was left in place, uncommitted (no removals).

## 6. What was tested, and what still needs a card
Offline tests (all pass):

| test | count | what |
|---|---|---|
| `scripts/4c1/tests/test_recgate.py` | 33 | records pinned; GREEN on 6 real talon2 goldens (F3t + E2b, both models) and on 3 box1 E2b run dirs (12/12, 9/9, 9/9); 12 planted REDs; scope (0x129C refused as before); stamp / gate / stale / forged stamp; `open_session` with the card layer stubbed; `c4.py golden` exit codes; box1 kits under the default verify list; zero-bank RED |
| `scripts/4c1/tests/test_model_toggle_sets.py` | 15 (278 s) | table vs `c4_regmap`; set per flash record; E2b and F3t identity; RED unknown image; 0x129C strings unchanged; the 0x129C mock switch card steps on `c4_mock`; E2b / F3t identity and two refused jobs on the mock |
| `test_profile_dual.py` | 40/40 | one assertion updated for the two added e_norm labels |
| `test_sglang_seam.py` (eaglefit python, no GPU), `test_tp4_session7.py`, `test_golden_exact.py`, `test_demo_model.py` | pass | regression |
| launcher dry-run | — | d11 gives VTOK 43, K_C 64, FASTGLUE 4; HP lever request refused rc 62 |

Failing before and after, not caused by this work: `test_regmap_cites.py` (4, fork source md5 drift) and `test_kit_units.py::test_att_image_alternates` (duplicate reg100 ids in `c4_regmap`).

Needs a card (not done, not claimed):
- one real `c4.py golden` on box1 writing the stamp, then a served request through `open_session` on the cards (the gate decision ran only on archived goldens and stubs);
- the toggle end to end on E2b (switch → identity → warm endpoint), both models;
- any tok/s or accept number: none was measured here;
- everything in section F.

## Calculations and hypothesis
- "talon2 arm B 784.06 vs PQC 816.47" implies about +4% from PQ+C on E2b; whether box1 shows the same gap is not measured.
- The 114 GiB free-memory figure for Qwen is the toggle guard's own arithmetic ((56,525 + 6,144) MiB / 0.55), not a new measurement.
