fpga-archive-reports-process · EAGLE4 FPGA archive
MUST-PASS VERIFICATION LIST: test the built artifact, not the RTL (user-mandated 2026-10-22)
Why this exists. On 2026-10-22, the first Qwen silicon test of 0xE4B7129B failed after about three days of building and routing. The dual-model RTL passed xsim, lint and the Llama back-compat test. The packaged IP had narrowed the control port to 9 bits, so the MODEL_CFG block at 0x200..0x224 was unreachable. Every testbench instantiated eagle4_drafter.sv directly, so none of them could see the packaging layer. Evidence:
component.xml:2510:<spirit:left dependency='("1" ? 9 : 8)'>8.QWEN27B_DRAFTER_20261020/LOG.md, entries dated 2026-10-22.- Memory
feedback_ip_packager_macro_ternary_port_width.
The rule. A feature counts as verified only when it is tested at the layer that ships: the packaged IP, then the block design, then the synthesized netlist, then silicon. RTL xsim is necessary but never sufficient. Each check is report-only: it writes PASS or FAIL into the build's GATE_VERDICTS. It never aborts a build (hard rule 13). A FAIL quarantines the PDI from flash.
Enforcement. Run scripts/must_pass_gate.sh (--stage DIR, then --pkg component.xml --kind ATT|MLPV --dual 0|1 after STEP1). A PostToolUse hook (scripts/hooks/mustpass_after_launch.sh, installed in .claude/settings.local.json) injects this list whenever a Bash command launches a build. The checks run AFTER build_all is launched, alongside it, and never hold a launch (user, 2026-10-22). A FAIL quarantines that PDI and starts a fixed build in parallel at once. Every heartbeat during a build advances a B or C item on that build's artifacts.
When the checks run. Each check is tied to the earliest build step that can run it:
PKG: right after STEP1 packaging, which takes minutes.SYN: post-synth, a few hours in.RTE: post-route.SIL: first boot, before any write.
Any check due at PKG or SYN must have reported on the first build of a new image lineage. Do this before fanning out re-routes or variants. Never wait for a routed PDI to find out.
A. Interface and packaging (the class that cost the 2026-10-22 test)
| # | Check | When | How | Catches |
|---|---|---|---|---|
| A1 | Port-width equality. For every top-level port of eagle4_drafter.sv, the resolved width in component.xml (<spirit:left>/<spirit:right>) equals the width the RTL elaborates to under the staged config. List every port whose <spirit:left> is resolve="dependent". |
PKG | parse component.xml; elaborate the staged top with xelab or synth -rtl and dump the port widths; diff the two |
the 9-bit s_axi_control_araddr (0x129A, 0x129B) |
| A2 | No macro or ternary widths on packaged top ports. grep the staged top for [(`…?…:…):0] on port declarations. Count must be 0. |
stage | grep | the same class, before packaging |
| A3 | The address block covers every host register. The addressBlock range in component.xml, the BD assign_bd_address -range, and the highest offset in scripts/4c1/c4_regmap.py for that card kind satisfy: range > highest offset. |
PKG | parse the three | an aperture too small for a new CSR window |
| A4 | Packaging parameter coverage (existing). Every strap is in the ipx list. | PKG | scripts/check_pkg_param_coverage.sh |
the 0xE4B71118 burn |
B. Packaged-IP and netlist simulation (xsim on what ships)
| # | Check | When | How | Catches |
|---|---|---|---|---|
| B1 | CSR map on the packaged IP. Run an xsim of the IP instantiated through its packaged interface (an export_simulation of a BD holding the packaged IP, or the IP's generated *_0.v wrapper). Use an AXI-lite master with the BD's address width. For every register in c4_regmap.py for that card kind: write a pattern, read it back (or read it if read-only). Check the capability and canary values (MC_CAP 0xE2581080, the 0x4D43 canary). |
PKG+1 h | new gate; see section E | unreachable or aliased registers, wrong widths, a missing block |
| B2 | Alias sweep. For every address bit the host uses (bits 2..N-1), read A and A XOR (1 << bit) on the packaged-IP sim and on silicon. Distinct registers must not mirror each other. | PKG+1 h, SIL | the same harness; on silicon, scripts/4c1/rd_csr_alias_probe.py |
9-bit decode, dropped address bits |
| B3 | No constant-driven feature nets. In the post-synth netlist, the nets that enable a strapped feature are not GND/VCC, for example s_axi_control_araddr[9], awaddr[9], MODEL_CFG write-enables and MODEL_ID mux selects. |
SYN | report_net / get_nets -of [get_pins …] driver check in the post-synth dcp |
features constant-folded away behind a dead select |
| B4 | Feature cell census, dual vs non-dual. Post-synth leaf-cell counts of each strapped block are non-zero and differ from the strap-off parent. On a Qwen image that means SubLN, the rope RAM, the e_norm fix and MODEL_CFG. Instrument: the routed REF_NAME suffix rule. | SYN | cell census per hierarchy vs the parent image's census | logic optimised out (the tail of the 0x129B defect) |
| B5 | Mode-switch test on the netlist. A short funcsim of control_regs plus the MODEL_CFG consumers from the post-synth netlist (write_verilog -mode funcsim on that sub-hierarchy). Run the real c4.py switch write sequence and check that MODEL_STATUS applied=1 and the outputs mcfg_*_q change. |
SYN | sub-hierarchy funcsim | a correct RTL switch that does nothing in the netlist |
| B6 | Host ABI replay. Replay the exact register-write sequence the host kit issues (c4.py discover/boot/switch/arm, captured from the mock) against the packaged-IP sim, and compare it with RTL sim. |
PKG+1 h | capture, then replay | host/RTL ABI drift |
C. Model-mode correctness (each model the image claims)
| # | Check | When | How |
|---|---|---|---|
| C1 | Every model mode the image claims (Llama and Qwen for dual images) passes its xsim golden (L0 8/8, plus the per-level overlap bar) with the mode selected through CSR writes, never by forcing a parameter or a hierarchical signal. | before stage | tb_card4 arms |
| C2 | The same goldens with the CSR writes issued through the packaged-IP interface of B1, or at minimum through a testbench whose AXI-lite address width is taken from component.xml, not hard-coded. |
PKG+1 h | B1 harness |
| C3 | The mock (c4_mockserve) takes its decode width per card from the build's component.xml, not from an assumption. The mock already modelled MLPV's 9-bit aliasing but assumed 10 bits for ATT. |
per image | mock config generated from the PKG artifacts |
D. Silicon first contact (before any write or arm)
| # | Check | How |
|---|---|---|
| D1 | Read the build-id registers on all cards; they must match the flash record (stored is not running). | discover |
| D2 | Alias probe on every new CSR window before the first write to it. Writing an aliased window lands on live control registers. | rd_csr_alias_probe.py |
| D3 | Read-only canary and capability registers of every strapped feature (for example rd_model_cfg.py) pass before the host switches modes. |
per feature |
E. Existing gates that still bind (unchanged)
- Staged sources equal the build's provenance record (
STAGE_SHA_<id>.txt; hard rule 12). The intended change is present in the built netlist (hard rule 13;scripts/build_synth_gate.sh). - FF seams are FDRE; the netlist census matches
RAMB*, never the bareRAMB36E5(check_no_bram_on_act_seams.sh). - Routed setup WNS and hold WHS are both > 0; flash only the final FPT PDI (
preflash_gate.sh). - ctx ≥ 16 with distinct tokens for any Q/K-path claim; md ≥ 3 for FSM-overlap changes; the xsim report carries an ACCEPT number.
- A gate must be able to fail. Each check above needs a RED control. For example, A1 must FAIL on the 0x129B
component.xml, and B2 must FAIL on a 9-bit decode harness. A check without a planted-defect control is not done (feedback_a_zero_is_a_claim_about_the_instrument).
F. Routed-netlist integrity (no timing check can see these)
| # | Check | When | How | Catches |
|---|---|---|---|---|
| F1 | SRPOL tooth 1: reset/set polarity census of the FINAL routed dcp = 0 mixed nets. Every FDRE/FDSE/FDCE/FDPE R/S/CLR/PRE net, grouped by its PARENT (physical) net, carries ONE value of the pin's IS_*_INVERTED on all loads. A mixed net = FAIL, with every minority-polarity load listed. |
RTE: EVERY routed / final dcp before its flash (the dcp write_device_image used), whatever its flow (the M1364 flip entered at place_design, not only in a rescue) |
scripts/must_pass_gate.sh --srpol <dcp> --host <VM> --verdicts <GATE_VERDICTS_id> --quarantine <QUARANTINE_PENDING_GATES_id>; instrument scripts/gates/srpol_census.tcl (md5 pinned as SRPOL_TCL_MD5). vivado -mode batch ON the host holding the dcp; ~16 min, vivado peak 32.8 GB on M1364 (gate default --peak 70); UNVER, never FAIL, if MemAvailable < peak + 25 G or the tcl selftest fails. Appends one PASS/FAIL/UNVER line to the verdicts file and, on FAIL/UNVER, the line + cell list to the quarantine marker. Never stops a build. |
M1364: 12 FDREs u_array/g_grp[0].g_lane[84]...g_wlat1.qw_d_reg[k] with IS_R_INVERTED=0 on active-low rst_n = held in reset in operation, lane 84 weights stuck at 0, wrong trees with every timing gate green. T4_EM: 36 G9 accumulator FDREs on peripheral_aresetn_BUFG[0]. |
| F1b | SRPOL tooth 2: per-cell EFFECTIVE polarity, the build's own impl_1/top_wrapper_opt.dcp (pre-place) vs the final dcp = 0 flips. Python takes the CANDIDATES from two vectorised dumps: a cell whose raw IS_*_INVERTED differs, or a new *_rep*/*_replica* whose base differs. Only candidates are opened again: the pin net is resolved in both dcps through BUFG*/BUF*/LUT1/INV to (root, parity), and effective polarity = inv XOR parity (constants: asserted/inactive). Same root + different effective polarity = FAIL (CHANGED / REP_DIFF). ABSORBED is counted legit: R tied off in final, and the same reset root drives an input of the D-pin LUT (control-set reduction). Any other root change = REVIEW (UNVER, row open). |
RTE, same scope as F1 | must_pass_gate.sh --srpol-optdiff <final dcp> --opt <its top_wrapper_opt.dcp> --host <VM> --verdicts .. --quarantine .. [--reuse <dir> = dumps of an earlier run of the same two dcps]. Default instrument scripts/gates/srpol_optdiff_fast.tcl + srpol_resolve.tcl + srpol_optdiff_fast.py (pinned OPTFAST_TCL/PY_MD5, OPTRES_TCL_MD5). Measured runtime: opt dump ~10-12 min, final dump ~14-15 min, python candidates 3-6 s on 1.1-1.6 M cells, resolve 1-2 min per side plus one re-open of each dcp (skipped at 0 candidates). End-to-end ~55-60 min; --reuse ~30 min. --slow = srpol_optdiff.tcl/.py, per-net leaf queries, measured ~2.6k lines/min (9 h for half of one opt dump): do not use. |
closes tooth 1's blind spot: a flip of EVERY load of a net (or of a whole replicated reset tree) leaves no mixed net, so tooth 1 reads 0. |
| F1c | 4C1 LINK CDC (preflash PDI-8): every clock crossing into/out of hop_xport / tp_l4 / tp_x is bounded (MDDO / false path onto ASYNC_REG) and met in the FINAL routed dcp; every link-xdc selector matches >0 cells (fail-closed). | RTE: every 4C1 ATT/MLPV final dcp before its flash (same dcp as F1) | bash scripts/gates/check_4c1_link_cdc.sh --dcp <final dcp> --clone <AVED clone> --pdi <final FPT pdi> --out <evidence>/<id>.LINK_CDC.txt [--red 'vacuous unconstrained asyncreg'], then LINK_CDC_VERDICT=<file> preflash_gate.sh. ~20-25 min, report-only. MLPV: also run the tcl with -hier "hop_xport hop_xport2 tp_l4 tp_x" and -xdc incl. tp4_link2.xdc (default scope misses hop_xport2). Baseline on the record lineage = FAIL 2 (hop_aurora_cdc.xdc:51/:54, stale rows on pruned st_cdc logic), MLPV full scope = FAIL 4 (+ tp4_link2.xdc:16/:19, same class); any OTHER FAIL row is real. Record images + audit + 0x136F evidence path: reports/results/DRAFTER_SPEED_RESEARCH_20261107/CDC_HOP_TPLINK_TOOTH_4C1_20261116.md |
PDI-7 measures seam_xport only (0 cells on 4C1): before PDI-8 no tooth checked the 4C1 link CDC. |
Instrument facts measured while building F1 (2026-11-14):
- An unset
IS_*_INVERTEDreads''fromget_property; an explicit one reads1'b0/1'b1. Most==0cells read''. A string compare against"0"or1'b0therefore misses most cells: that is how the RCA's interim stage table went vacuous. Both tcls take polarity from Vivado's own filter (prop == 1/prop == 0) or from a regexp on[01]$with''= 0, and their selftest checks it: the two filter sets partition every primitive, parsed counts equal the filter counts, readback contradicts nothing, and >= 1 polarity-1 cell is seen. A selftest miss reports UNVER, never PASS. The first selftest draft failed on exactly this''(3 runs,red_m1364_final,red_1364_routed: UNVER), which is the selftest working. - A raw per-cell inversion diff is not a usable gate. The ECO-clean M1364E (
M1364E_eco.dcp565168bd: 0 mixed nets, lane eq bench EQUAL) has 182 cells whose rawIS_R_INVERTEDwent 1->0 from its own opt netlist 49a621bb (M1364_final: 197 = those 182 + the 15). All 182 are reset ABSORPTION. In opt, R sits on the active-low reset (rootg_rst_pipe.g_rp[0|2].rq_reg[1]_loptx60,usr_0_psr/.../FDRE_PER_Nx122) with inv 1. In final, R is tied to<const0>with inv 0, and the same reset root drives the D-input LUT. QF_PX's 15 candidates are the same transform: 10 SMBus IP cells + 3 escaped-name SMBus cells onpl_psr, plus 2u_attn_full/eng_sel_reg[k]_replica_4onusr_0_psr. So the tooth compares the EFFECTIVE polarity and checks that the root is still present. - Escaped cell names (
reg_ctlr_phy_if\\.reg_wr_done_reg): Tcl list parsing (lrange) strips one backslash, so the lookup misses. The resolver splits lines on spaces and falls back to an exactNAMEcompare (srpol_resolve.tclca4ea7b5 returned MISSING for 3 QF_PX cells; 0a0bd072 resolves them). - The first fast tcl (v1) built the candidate set in a Tcl
dictloop, which ran at one core for more than 1 h on QF_PX's 1.59 M cells. The candidate step moved to python (3-6 s). - The RCA copy of
srpol_census.tcl(080b5542) summed every load once per hierarchical net SEGMENT. On M1364 it printed "INV0 30" for 15 unique cells (295 segments), and on T4_EM "INV0 72" for 36. The gate copy counts unique cells; the mixed-net COUNT is the same in both.
F1 calibration (outputs + verdict files: reports/results/DRAFTER_SPEED_RESEARCH_20261107/lineage1360_root_rca_20261114/srpol_gate/; runs on create-only dcp copies, md5 equal at both ends, buildx01/buildxd04/buildz01:/shared/scratch/srpol_gate_20261114/):
| tooth | dcp | md5 | expected | measured |
|---|---|---|---|---|
| 1 | M1364_final (behind flashed PDI 44b03c14) | 66637728 | RED | FAIL, 1 mixed net (g_rst_pipe.g_rp[0].rq[1]_BUFG), 15 minority loads (INV0 15 / INV1 19,516): lane-84 qw_d_reg[12,13,14,16,17,18,24,25,26,27,28,30] + lane-40 col_offset_d_reg[11..13]; selftest PASS (fin_red_m1364_final_srpol.txt 674e4178; also run on buildz01 red2_m1364_final: same) |
| 1 | 0x1364 _routed |
a09666c9 | RED (the flip entered at place_design) | FAIL, the same 1 net / 15 loads (fin_red_1364_routed_srpol.txt 9fad3e87) |
| 1 | 0x1360 P1360 pr.dcp |
de95589e | GREEN | PASS, 0 mixed nets (FDRE/FDSE/FDCE/FDPE), selftest PASS (fin_green_p1360_srpol.txt 16a93528) |
| 1 | M1364_final with --peak 500 |
66637728 | UNVER (RAM floor) | UNVER: "REFUSED MemAvailable 164G < 525G"; a real refusal (buildz01, 91G < 95G) also came back UNVER |
| 2 | M1364 opt 49a621bb -> M1364_final | 66637728 | RED, the 15 | FAIL, changed=15: the 12 lane-84 qw_d + 3 lane-40 col_offset_d, root g_rst_pipe.g_rp[0].rq_reg[1]_lopt, opt_pol 1 -> final_pol 0. reset_absorbed=182, root_changed=0, rep_diff=0, selftest PASS (buildx03, v3_red_m1364_final/optdiff.txt 15197d66) |
| 2 | M1364 opt 49a621bb -> M1364E_eco | 565168bd | GREEN | PASS, changed=0, reset_absorbed=182, root_changed=0, selftest PASS (buildy02, v3_green_m1364e/optdiff.txt aebb8146) |
| 2 | slow tcl 12d4a3ee / raw diff 423400c1 | (history) | the raw diff FAILed the GREEN with 182, which is why the tooth resolves roots; the slow per-net dump never finished one opt dump in 9 h |
Already censused with tooth 1 (box1-two-v2, RCA copy 080b5542, same mixed-net count): record ATT 0x13BE final 772552f5: 0 mixed; M1361_final e8d4761f: 0 mixed; 0x13DB _placed 0ea99a9b: 0 mixed; M1364E_eco 565168bd: 0 mixed. T4_EM d39f131b: FAIL (1 net, 36 cells, netlist_t4em/t4em_srpol.txt 11e28ec2).
F1 owed before the next flash: tooth 1 AND tooth 2 on every routed/final dcp queued for silicon. Named: T4_RM (re-route of T4_EM; tooth 1 not yet run); 0x1376 / 0x1377; the Q / QF arms (0x1390 / 0x1394 and their rc / px variants; QF_PX tooth 2 = PASS: opt top_wrapper_opt.dcp cfc87335 -> final top_wrapper_postroute_physopt.dcp c4d1a957, changed=0 rep_diff=0 root_changed=0, 15 candidates all reset_absorbed, selftest PASS, buildx01 2026-11-15T06:29Z, v3_qfpx/optdiff.txt 600bfbf2); every rescued / closure ATT image; tooth 2 on 0x13BE and M1361 (tooth 1 clean).
Leftover processes of mine from superseded tooth-2 versions (read-only, not stopped, per the no-kill rule; they hold RAM): buildx01 slow t3_red_m1364_final (tcl 12d4a3ee, since 2026-11-14 18:53Z) and fast-v1 f_red_m1364_final (Tcl candidate loop, since 04:28Z); buildx00 fast-v1 f_qfpx (pid 890775); the box1 waiter run_t3g.sh will start a fast GREEN on buildx01 when the slow one ends.
F1 limit, remaining after tooth 2: tooth 2 compares against the build's own opt netlist, so a polarity defect already present in opt (i.e. introduced by synthesis/opt_design) is invisible to both teeth; that is the RTL/synth layer (xsim + B3/B4). A changed reset ROOT is not proven equal; it is listed and the row stays UNVER until reviewed.
Status of this list (2026-10-22)
- A1, A2, A3: DONE in
scripts/must_pass_gate.sh. RED controls verified: the 0x129Bcomponent.xmlFAILs A1 and A3 (it cannot reach 0x224), and fork @892b286 FAILs A2. The fixed dual, shapert (att, mlpv) and normrt (+b) forks PASS A2. The 0x129C build records the gate output inGATE_PORTW.txt. - B1/B2 (packaged-IP CSR sim): next, with RED = the 0x129B
component.xml. - D2: exists,
scripts/4c1/rd_csr_alias_probe.py. - F1 SRPOL (2026-11-14): DONE in
scripts/must_pass_gate.sh --srpol(tooth 1) +--srpol-optdiff(tooth 2); RED/GREEN controls in section F.--selftestRED-checks the argument parser (a value-less option used to spin the parse loop at 100 % CPU forever: HEAD before this change times out at 5 s, now exits 2). - A2, B3, B4, B5, B6, C2, C3: open. Each gets an owner before the next dual-model image is staged.