# Demo speed audit: per-step budget of the best stack and what can land in 14 h (2026-11-10 07:3xZ)

Program 1: the Llama-3.2-3B DRAFTER on the 4C1 split (2 ATT + 2 MLPV), target Llama-3.2-3B on the talon2 H200.
Read-only audit: no card, GPU or demo access, no new runs. Demo deadline ~21:30Z 11-10 (`USER_DIALOGUE_LOG.md:1995`).
Tags: **[M]** measured (artifact cited) · **[D]** derived by arithmetic from [M] · **[E]** estimate / projection · **[S]** sim only.
Short paths: `T/` = `reports/results/TRACK1_RECORD_F3T_20261109/`, `R/` = `reports/results/DRAFTER_SPEED_RESEARCH_20261107/`, `B/` = `reports/results/BATCH_BUBBLES_20261109/`.

## 0. Answer first
- **The binding term is GPU verify: 3.45 ms = 49 % of the 7.09 ms step.** The card tree is ~2.0 ms (29 %), host glue ~1.2 ms (17 %), per-request cost 0.38 ms (5 %).
- **The best measured numbers need no new image:** F3t + 0x1342 at w2 **d11**, fa3, V20, K_C 64, theta −2.5 = **883 tok/s** on humaneval-80 and **1,059** on the 15 demo prompts (top prompts 1,181–1,193).
- **The record stack is not on any card right now.** talon2 17/3d were re-flashed to Q5120 at 06:3xZ and are being flashed to TP4 B4 since 07:07Z; 50/63 hold the Q5120 MLPV 0x1349 (`USER_DIALOGUE_LOG.md:1990,1994`). box1 holds E2b + 0x1342 (`reports/results/BOX1_DEMO_TEMPS_HUMANEVAL_20261110.md:5-12`). Getting 883 / 1,059 back needs F3t → 17/3d and 0x1342 → 50/63 plus one cold AC, and the talon2 PDU CLI is broken, so that cold AC is manual (`USER_DIALOGUE_LOG.md:1992`). I did not verify card state myself; this is from the logs.
- **Ceilings [D]:** card time → 0 gives 1,023–1,167 tok/s at d7 (1,173–1,330 at d11). Host glue → 0 gives ~1,000 (d7) / ~1,043 (d11). Verify → 0 gives ~1,600. No card or host lever available in 14 h is worth more than the d11 + prompt-choice config already measured.

## 1. Waterfall of one speculative step, F3t 0xE4B713F9 + MLPV 0x1342, Llama w2 V20 K_C 64

### 1.1 Reconciliation against tok/s ÷ accept
| | d7 (record n1) | d11 fa3 (r1) | source |
|---|---|---|---|
| tok/s, accept, steps, time | 831.14, 5.89, 3,329, 23.60 s | 882.38, 6.90, 2,843, 22.23 s | `T/f3t_run/serve/SUMMARY_llama3b.tsv:2`; `R/F3T_CONFIG_SWEEP_20261109.md:80` [M] |
| steps/s = tok/s ÷ accept | 141.1 | 127.9 | [D] |
| **whole-run ms/step** | **7.088** | **7.818** | time ÷ steps [D] (sweep doc `:206` gives 7.116 / 7.812 on the 9-round means) |

### 1.2 Step waterfall (ms, % of whole-run step)
| # | term | d7 ms | d7 % | d11 ms | d11 % | tag | source |
|---|---|---|---|---|---|---|---|
| 1 | **GPU verify forward** (target, 20 tokens, bs 1, cuda events p50) | **3.452** | 48.7 | 3.452 | 44.2 | [M] d7 on F3t same boot; d11 assumed equal (same V20 graph) | `B/INTERLEAVE_BATCHVERIFY_SILICON_20261109.md:76` |
| 2 | **card tree**, doorbell → tree published, run mean | **~2.04** | 28.8 | **~2.63** | 33.6 | [D] full-depth p50 minus tree-stop saving, see 1.4 | `R/F3T_CONFIG_SWEEP_20261109.md:154-156` |
| 3 | **host chain, UpdateResult end → ATT doorbell** | 0.95 | 13.4 | 0.95 | 12.2 | [M] on E2b with PQ+C, same host kit lineage; not re-profiled on F3t | `R/HOST_PRECARD_OPT.md:52,179` |
| 4 | residual: verify end → UpdateResult end (accept / sampling / update) + unmodelled | 0.25 | 3.6 | 0.33 | 4.3 | [D] closes d7 by construction; d11 = 0.25 + 0.08 unexplained | – |
| 5 | per-request fixed cost spread over steps (prefill, first GPU draft, first tree, request boundary, detok) | 0.384 | 5.4 | 0.45 | 5.8 | [D] 15.96 ms/request, see 1.5 | `T/f3t_run/serve/*n1_154551*.perreq.jsonl` |
| | **sum** | **7.088** | 100 | **7.818** | 100 | d7 closes by row 4; d11 predicted 7.736 before the 0.08 | |

Cross-check from the record run's own counters [M]: the host idles **1.330 ms/step** (d7) and **1.937 ms/step** (d11) waiting for the card (`join_wait_ns` 3,991,624,512 / 3,000 and 4,841,374,691 / 2,500), after 0.506 / 0.515 ms of host work that overlaps the card (`host_ahead_ns`). The card is already done at the join on only 5 % of steps (151 / 3,000).
- d7: `T/f3t_run/serve/hp_llama3b_PQC_w2d7_humaneval80x256_n1_154551_MEASUREONLY_goldendepth_unproven_20261109_154551.log:476`.
- d11: `T/config_sweep/serve3/hp_llama3b_PQC_w2d11_humaneval80x256_n1_182607_…_182607.log.gz` line 454 (md5 9c9b32fe).
- Counter meaning: `scripts/4c1/c4_verifypre.py:477-478`.
- d11 − d7 join wait = +0.606 ms, against the +0.59 ms of row 2 and the +0.73 ms whole-run step.

### 1.3 Inside the card tree (µs per tree, chain census, 40 full-depth trees, card-only)
Source: `T/config_sweep/census/chain_llama3b_f3t_w2d7_kc64_185201.analysis.json.gz` (md5 25e30681) and `…w2d11_kc64_185202…` (md5 5b11b7f6), key `occupancy2` [M]. The d7 values match `R/F3T_SILICON_20261109.md:35-44`.

| card | phase | d7 µs | d7 % of p50 | d11 µs |
|---|---|---|---|---|
| ATT | G1 | 83.5 | 3.7 | 136.9 |
| ATT | **G4 QKV** | 390.2 | 17.5 | 630.0 |
| ATT | **G5 attention** | 523.4 | 23.4 | 841.3 |
| ATT | G6 o-proj + TP exchange (the ATT↔ATT QSFP hop sits here) | 125.2 | 5.6 | 195.1 |
| ATT | G7G | 351.1 | 15.7 | 561.3 |
| ATT | G7U | 186.4 | 8.3 | 302.4 |
| ATT | SILU + LVL_EMBG | 88.4 | 4.0 | 144.8 |
| ATT | INGEST + EXPORT (ATT↔MLPV seam) | 73.2 | 3.3 | 144.5 |
| ATT | **busy total** | **1,821.5** | 81.5 | **2,956.5** |
| ATT | PIPE_WAIT (parked on the MLPV) | 255.5 | – | 246.3 |
| MLPV | G9 | 167.8 | – | 268.9 |
| MLPV | G11 + GLM + G12 kick | 138.5 | – | 229.6 |
| MLPV | TREE | 437.7 | – | 701.7 |
| MLPV | G8 exact rescore | 343.2 | – | 551.8 |
| MLPV | INGEST + XCH + LVL_SEL/OUT | 152.9 | – | 243.3 |
| MLPV | **busy total** (mostly hidden under the ATT) | **1,240.0** | – | **1,995.2** |
| joint | both busy / ATT only / **MLPV only (exposed)** / both parked | 1,035.2 / 795.7 / **216.9** / 38.3 | 9.7 exposed | 1,798.1 / 1,178.1 / 210.5 / 35.9 |
| | census window | 2,086.9 | | 3,221.5 |
| | outside the window (issue, SoA read-back / DMA out) | 146.9 | 6.6 | 162.6 |
| | **issue → done p50** | **2,233.8** | 100 | **3,384.1** |

- The tree is ATT-bound: ATT busy is 87 % of the window and the MLPV is exposed for 217 µs.
- Link hops have no separate counter. The visible parts are G6 (TP exchange) and INGEST/EXPORT above; `R/MLPV_AND_STEP_BREAKDOWN_E2b.md:66` says the same for E2b.

### 1.4 Tree-stop correction to the card term [D]
The census runs full-depth trees. In serving, theta −2.5 stops trees early:
- d7: 800 of 3,098 trees stopped, mean stop depth 4.68 (`…154551.log:482`). Saving ≈ 26 % × 2.32 levels × 2 items × ~150 µs ≈ 0.18 ms → mean tree ≈ 2.04 ms.
- d11: 1,400 of 2,526 stopped, mean stop depth 6.40 (`…182607.log.gz` line 457). Saving ≈ 55 % × 4.6 × 2 × ~148 µs ≈ 0.75 ms → mean tree ≈ 2.63 ms.
- Per-item cost 135–152 µs: `R/F3T_CONFIG_SWEEP_20261109.md:160`.

### 1.5 Per-request fixed cost [D]
Per request, service time = difference of consecutive `e2e_latency` values. A least-squares fit of service = a + b × verify steps over requests 2–80 gives:

| run | a ms/request | b ms/step | R² |
|---|---|---|---|
| F3t d7 n1 / n2 | 15.96 / 15.55 | 6.696 / 6.717 | 0.991 / 0.987 |
| F3t d11 fa3 r1 / r2 | 35.71 / 35.35 | 6.807 / 6.798 | 0.969 / 0.970 |
| GPU-only d7 fa3 / d9 / d11 | 7.80 / 8.30 / 8.11 | 7.032 / 7.686 / 8.318 | 0.997 / 0.997 / 0.998 |

- The d11 fit is confounded: high-accept requests have fewer but longer (full-depth) steps, which moves time into the intercept. I use the d7 intercept for both columns.
- The FPGA path costs ~8 ms more per request than GPU-only (16.0 vs 7.8). On humaneval-80 that is 0.65 s = 2.8 % of the run.
- Demo prompts: the TTFT proxy (prefill + first tree + first verify) is 15–26 ms of a 215–286 ms request (`T/PER_PROMPT_BENCH_20261109.md:50-84`).

### 1.6 Inside the 0.95 ms host chain (µs, p50, E2b + PQ profile; C then removed ~54)
Source `R/HOST_PRECARD_OPT.md:38-52,124-132` [M].

| class | items | µs |
|---|---|---|
| DMA in | KV_LOAD in the card process 229 (4 pwrites ≈ 140) + root FASTIN to both ATT 141 | 370 |
| GPU draft-side launch | capture + EXTKV extend launch (113) + prepare_graph | 229 → 210 with C |
| host glue (Python / IPC) | KV gather 44, guards + plan 57, client KV pack/send 121, socket 13, image recv 46 → 34, stage/cfg/arm 19 + 17 + 40, poison 6, doorbell 8 | ~360 |
| root image graph replay (parallel path that ties with the KV path) | 175 | (not additive) |

DMA out is not separately profiled; it sits in the 147 µs outside the census window (1.3).

## 2. Amdahl table
Bases [M]: humaneval-80 d7 828.06 (9-round mean) and d11 fa3 883.12; demo set (15 code prompts, 256 tokens) d7 953.0 and d11 1,059.3 (`T/PER_PROMPT_BENCH_20261109.md:109-111`).
For card levers, Δstep = κ × Δtree. Two values of κ were measured: 1.0 (E2b vs 0x1340, `R/MLPV_AND_STEP_BREAKDOWN_E2b.md:70`) and 0.51 (E2b → F3t, `R/SEVENCARD_PROJECTION_20261109.md:52`). Both are shown where it matters.

| lever | evidence | humaneval-80 | demo set | status | time to land | risk |
|---|---|---|---|---|---|---|
| **depth 11** (with fa3) | `R/F3T_CONFIG_SWEEP_20261109.md:80,83` | **883.12 (+6.65 %)** | **1,059 (+11 %)** | MEASURED | host-only, 0 h; PERSIST must be armed ≥ 11 before the first AP_START | texts differ from d7 on 3/80; fa3 rounds identical |
| depth 8 / 9 / 10 | `:50,52,56` | 850 / 861 / 870 | – | MEASURED | 0 h | – |
| depth 12 / 13 / 15 | `:58-60` | 874.7 / 873.8 / 870.1 (−1.0 to −1.5 % vs d11) | not measured | MEASURED | 0 h | loss |
| width 3 (w4 skipped) | `:73-74` | 767 (−13 % at d11) | – | MEASURED | – | loss |
| verify tokens V16 / V24 / V27 | `:45-47,62-63` | d7 828.6 / 799 / 800; d11 853 / 859 | – | MEASURED | – | loss or level |
| theta −2.0 / −3.0 | `:67-68` | 877 / 868 at d11 | – | MEASURED | – | level / loss |
| K_C 32 (128) | `:75-76`; `R/FAILED_DIRECTIONS_20261109.md:9` | 885.70 (+0.5 %, one round); E2b A/B +0.82 %; K_C 128 −4.9 % | – | MEASURED, inside noise | host-only, 0 h | changes 2–4/80 texts |
| FR-Spec 64K / 32K / 16K | `:141-143` | −1.2 / −9.5 / −20.3 % | – | MEASURED | – | loss |
| FIRST_DRAFT_GPU | already on: `FIRSTGPU=1` at `…154551.log:3` | 0 | 0 | in the base | – | – |
| fa3 draft backend | `:170` | +0.4 % at d11, neutral at d7 | – | MEASURED | 0 h | makes d11 deterministic |
| FASTGLUE variants / HP letters | FASTGLUE 4 + P, Q, C on; R, K, E, W, A rejected (`R/HOST_PRECARD_OPT.md:66-69,126`) | 0 | 0 | MEASURED | – | – |
| PINPAR 1 | `:69-70` | 822 / 876 (no gain) | – | MEASURED | – | – |
| KVONC host fix | +1.52 % on 0x130C + 0x128B at 749.57 (`reports/results/FOURCARD_DESIGN_20261016/4C1_DELIVERY_TRACKER.md:1414-1415`) | not applicable: the F3t regmap row has no `kvoncard` key (`T/f3t_kit/patch_13f9_host.py:46`); needs a KV_ONCARD image | – | MEASURED on another image | a build | not in 14 h |
| CUDA-graph / draft-graph warm | target verify graph already captured (`…154551.log:168-175`); GRAPHWARM refuted, it changes accept (`reports/results/FOURCARD_DESIGN_20261016/STEP_SPLIT_20261019.md:106`) | 0 | 0 | MEASURED | – | – |
| CPU governor | +0.65 %, below the bar (`:171`) | ~+0.6 % | – | MEASURED 11-06 | system-wide setting on a shared host | low gain |
| GPU clocks / thermal | talon2: throttled 0, max 69 °C in every row (`T/f3t_run/serve/SUMMARY_llama3b.tsv:2-3`); box1 H200 throttled at 87 °C on Qwen, SM 1,785 → 360–510 MHz (`USER_DIALOGUE_LOG.md:1989`) | protects the number, adds nothing | same | MEASURED | 0 h | on box1, a hot GPU can cut the rate several-fold |
| **prompt choice** | `T/PER_PROMPT_BENCH_20261109.md:50-84` | – | d7 896–1,042; d11 943–**1,193**; mt_bench prose 404–497 | MEASURED | 0 h | prose prompts halve the number |
| max_new_tokens 512 on code prompts | decode-only rate d11 1,242–1,308 vs e2e 1,111–1,193 (`:70-73`) | – | ≈ +4–5 % [E] (halves the per-request share) | estimate | 0.5 h test | early EOS; accept over a longer output not measured |
| per-request cost (FPGA 16 ms vs GPU 8 ms) | 1.5 above | ≤ +2.8 % [E] | ≤ ~+4 % [E] | estimate | host dev, 3–6 h | untested host change before a demo |
| 2-stream PINGPONG / MPS | 1.55× was the 2-card pair (memory `feedback_batch1_latency_first_pingpong_is_batch2`); on F3t, N2 without MPS = 221 (`B/INTERLEAVE_BATCHVERIFY_SILICON_20261109.md:53`) | below single-stream | – | MEASURED loss without MPS; MPS arm not run | hours | – |
| batched verify BV4 + HBMREPLAY (aggregate) | 730 / 717 vs 665 on the same launcher (`:82`); that launcher lacks the record levers | 730 < 831 today; ~905 [E] if the record levers were ported | – | MEASURED (aggregate only) | host dev, hours | 6–8/80 texts change |
| G5_OVL | in F3t (`R/F3T_SILICON_20261109.md:14`) | 0 more | 0 | in the base | – | – |
| LANES 512 ATT | in F3t (F3t = D9 L512, `R/F3T_SILICON_20261109.md:4`) | 0 more | 0 | in the base | – | – |
| R3_EARLY | −75 µs/tree [calc] (`R/BOTTLENECK_TABLE.md:84`); silicon F0b: no gain shown at equal host config, one unexplained accept-1.04 boot (`R/FAILED_DIRECTIONS_20261109.md:11`) | +0.5 % (κ 0.51) to +1.1 % (κ 1) [E] | same | estimate | a build on the F3t base | not in 14 h |
| **ATT TP4, 6 cards: B4 0x13B4** (GO, flashing now) + MLPV 0x134D | `R/SEVENCARD_PROJECTION_20261109.md:90`; adjudication `R/SEVENCARD_B1_B5_ADJUDICATION_20261110.md:11-13` | d7 869 / d11 948 (κ 1); 848 / 915 (κ 0.51) | not projected | SIM only; no TP4 silicon census exists | flash in progress + manual cold AC + first 6-card bring-up, 4–8 h [E] | high: 0x134D PDI not written at 07:00Z (routed +0.001 / +0.010), new host kit, cables unverified |
| ATT TP4 B5 0x13B5 (SIBW W2) | same doc `:89` | d7 885 / d11 979 (κ 1); 856 / 930 (κ 0.51) | – | SIM only | as B4 | as B4, plus hold exactly at the bar |
| G6_EARLY_ISSUE | root item 11,152 → 9,670 cycles in TP4 xsim (`reports/results/QWEN27B_DRAFTER_20261020/G6_EARLY_ISSUE_20261109.md:19`) | not on any Llama 4C1 image | – | SIM only, Q5120 lineage | a build | not in 14 h |
| MLPV L512 | 0x1351 died at place, 19,133 SLLs vs 18,870 (`…4C1_DELIVERY_TRACKER.md:1502`) | – | – | dead | – | – |
| MLPV TREE_P 64 | 0x135C died at place (`…4C1_DELIVERY_TRACKER.md:1531`) | – | – | dead so far | – | – |
| Q5120 drafter | GPU-only 196.61 @ 3.976 (`…4C1_DELIVERY_TRACKER.md:1516`) vs BitNet Qwen drafter 249.62 @ 7.14 on F3t (`R/F3T_SILICON_20261109.md:22`); Q5120 MLPV boot refused at 07:02Z (`…TRACKER.md:1535`) | other program (Qwen target); slower today | – | MEASURED GPU-only | – | does not raise the Llama number |

## 3. What caps the number
- **Binding term: GPU verify, 3.45 ms of 7.09 ms.** It starts when the card publishes the tree (verify is pre-enqueued behind the card flag, `…154551.log:26`), so verify and card are serial.
- Second: the card tree, ~2.0 ms mean at d7. 87 % of it is ATT busy time, led by G5 (523 µs), G4 (390) and G7G (351).
- Third: host, 0.95 ms chain + 0.25 residual + 0.38 per-request. 0.37 ms of the chain is QDMA writes at ~28 µs fixed cost each (`R/HOST_PRECARD_OPT.md:171`).

| what goes to zero | d7 tok/s | d11 tok/s | how |
|---|---|---|---|
| card wait only (the measured host idle, 1.330 / 1.937 ms) | 1,023 | 1,173 | accept ÷ (step − join wait) [D] |
| whole card tree (2.04 / 2.63 ms) | 1,167 | 1,330 | upper bound; needs the 0.5 ms host prologue hidden as well [D] |
| host chain + residual (1.20 ms) | 1,001 | 1,043 | [D] |
| host chain + residual + per-request | 1,071 | 1,120 | [D] |
| GPU verify | 1,620 | 1,580 | [D] |
| card and host both (verify + per-request left) | 1,535 | – | [D] |

Reference: best GPU-only the same day is 835 (d9 fa3, `R/F3T_CONFIG_SWEEP_20261109.md:120`). Its marginal step is 7.03 ms at d7, so the GPU drafter + host costs 3.58 ms against the cards' 3.25 ms [D from 1.5].

## 4. Top 5 for the next 14 h, ranked by (gain × probability) ÷ hours

### One-flash actions
1. **Put F3t 0xE4B713F9 (md5 97d8445c) back on 17/3d and 0x1342 (3e82ea1d) on 50/63, one cold AC, arm depth 15 before the first AP_START.** This is the only route to the measured 883 / 1,059. Probability ~0.95 (same PDIs, kits and run dirs as 11-09: `R/F3T_SILICON_20261109.md:62-86`). ~2 h including the manual power cycle. Set a hard cutoff for starting it; ~17:30Z leaves 4 h.
2. **Time-box the 6-card TP4 attempt (B4 now flashing; B5 is the faster projection).** Projected gain over F3t at d11: +3.6 % (κ 0.51) to +7.4 % (κ 1) for B4, sim only. It needs MLPV 0x134D on 50/63, which had no PDI at 07:00Z. Probability of a served, golden-checked number before the cutoff: low. If it is not serving by the cutoff, do action 1.

### No-flash, host-only actions (on F3t once it is back)
3. **Serve at w2 d11, fa3, V20, K_C 64, theta −2.5.** +6.65 % on humaneval-80 and +11 % on the demo prompts, measured, 0 h. Exact command: `R/F3T_CONFIG_SWEEP_20261109.md:181-183`.
4. **Use the high-accept code prompts** (`even_odd_palindrome`, `sort_array`, `is_sorted`, `pluck`, `words_in_sentence`): 1,111–1,193 tok/s at d11, measured, 0 h. Avoid prose prompts (404–497).
5. **Test max_new_tokens 512 on those prompts** to dilute the 18–26 ms per-request cost: ≈ +4–5 % [E], 0.5 h, unmeasured. Keep 256 if a prompt stops early or accept falls.

Not worth the hours: width 3, V ≠ 20, FR-Spec, K_C 128, PINPAR, GRAPHWARM, the governor, KVONC (wrong image), R3_EARLY / G6_EARLY_ISSUE (need builds), and the multi-stream modes (below single-stream today).

## 5. Not verified
- Card state on talon2 and box1 is taken from the dialogue log and tracker, not read from registers.
- Row 3 of the waterfall (0.95 ms) is an E2b profile; no STEPPROF run exists on F3t. Row 4 is a residual.
- GPU verify 3.452 ms was timed in the batch-verify B1 arm (665 tok/s launcher), d7 only. A d11 verify timing does not exist.
- The mean tree time in serving (row 2) is an estimate from stop-depth counts; no serving-time card census exists.
- No 6-card TP4 silicon number exists; every TP4 figure is a projection from xsim cycles.
