fpga-archive-reports-arch · EAGLE4 FPGA archive
ATT LUT-GEMM array on the 132 MHz clock (ATT_ARRAY_FAST_CLK), derived and gated (box1-two-v2 RTL speed worker, 2026-11-17)
Program 1: one Llama-3.2-3B drafter on 7 V80s (ATT TP4 x4 + MLPV x2; H200 verifies). Motto: never deny the idea; engineer toward the fastest flashable PDI. Nothing was built as an image, flashed or killed. No card, talon2, demo or H200 touched. Tags: [M] measured, [R] read from RTL / build record, [P] projection.
Verdict: it works. The ATT array runs on ap_clk_fast (2 x ap_clk) bit-identically to NFA8 in the 4-rank full-drafter sim and cuts the ATT item by 853 cycles (4,979.6 -> 4,126.2, -17.1 %); the array + bridge + quantiser route at 132.0 / 66.0 MHz OOC with WNS +0.118 / WHS +0.010 ns, which is thin.
Calculations and hypothesis (everything that is not a measurement)
- [P] Worth per tree: 853 cyc/item x 21 items (d11) x 15.151 ns = 271 us per tree if the ATT item sets the pace, against the record census of 2,877.7 us issue->done (
SCALING_4C_VS_7C_PHASE_RANK_20261116.mdsection 3). That is up to 9.4 % of the tree; the real gain is smaller wherever the MLPV or a hand-off becomes the pace-setter. No tok/s is claimed before silicon. - [P] Stacks with NFA8 (-581): COMBO3 5,560.8 -> 4,126.2 = -25.8 % per ATT item in this sim.
- [P] Cycle model that the measurement matches: per walked stage, new = walk (np) + 18 + fill-to-launch (np/2 + 3) + tiles x np/2; old = walk + 18 + tiles x np (np = chunk pairs: G4 192, G7g/G7u 96, G6 24). Predicted G4 -490, G7g -140, G7u -190, G6 -55; measured -488, -152, -204, -9.4 (G6 is exchange-bound, see section 3).
- [P] Next levers on this base, not built: (a) remove the np/2 fill exposure by stashing the raw chunks during the dry walk and quantising two pairs per slow cycle (about -99 G4, -51 G7g, -15 G6 per item; costs a second q8 bank + 8 RAMB); (b) drop the bridge's 24.6K-FF result hold (RET_SER's rs_out already holds until the next commit) behind a 2-cycle multicycle; (c) the full-chip risk is ATT SLR0 SLICE (98.12 % on 0x13BE), not the array fmax.
- [P] OOC margin +0.118 is a harness number with a soft per-SLR floorplan; in the full chip the same paths compete with attention for SLR0. A sibling at a lower ratio is not available (the bridge is integer 2:1 only); the fallback if the full chip misses is a derated pair (e.g. 62/124) or the G4-only cache (the strap is per array, so a stage-select would need RTL).
1. Why the lineage refused LUTG + ARRAY_FAST_CLK [R]
Two elaboration guards, both protecting against a silent wrong design, not a physical limit:
- drafter
g_lutg_top_x(eagle4_drafter.sv:1253-1255of NFA8):ERROR_LUTG_REQUIRES_..._NO_FASTCLK. - array
g_lutg_x(gemv_stage_array_scaled_slr.sv:2459-2461):LUTG && LCACHErefused. What they protect: (1) the LUTG clusters take their activation straight from the array'sa_chunkport (src_a,:1481), not fromfwd_a(the cache output), so with LCACHE the cache would be filled and never read while the lanes sampled the 66 MHz seam at 132 MHz; (2) the A8Q quantiser sits inside the array and walks the slow seam throughchunk_idx_outon the array clock, with a combinational q8 (two multipliers) between the seam andq8_ar; at 2:1 that walk reads a seam that answers once per slow cycle, and the q8 path would have to close in 7.5 ns; (3) the drafter'sg_dc_xalso refused TILE_PIPE, and every ATT image is TISEQ 1. The D9 report says the same in one line: "the dual-clock path with the per-die cache is the next step" (QWEN27B_DRAFTER_20261020/LUTGEMM_ATT_INTEGRATION_D9_20261102.md:451).
2. Design: quantise on the slow clock, cache the quantised chunk [R]
seam (66) -> u_a8q_s lutg_a8q (66, UNCHANGED: dry walk, dividers, commit) -> start_out + fill_req
-> gemv_dualclk_bridge (the record MLPV file 0395872d: FAST_TP 1, FAST_RST_SYNC 1, A_WIDTH 8)
fill: re-reads the seam, registers q8(chunk) into fw_*_s (66) -> first_half-qualified write (132)
-> u_array LCACHE, 3 groups x 2 x 128 b x 256 (4 RAMB per die) -> cluster a_r -> lutgemm_lane_w2 (132)
-> RET_SER -> done toggle -> done_seen_s -> res_q hold (66) -> att_g6_tp / body sequencers (unchanged)
- Quantiser clocking.
lutg_a8qstays on ap_clk in front of the bridge (g_array_scaled.g_dc.g_qs.u_a8q_s); one added outputfill_req(commit of a WALKED pass). The seam -> q8 -> register path is the same path on the same clock as today'sq8_ar_k. No quantiser logic runs at 132 MHz. T crosses as a held slow register into the existing per-groupt_r(committed a whole cache fill before the first lane start). - Cache. The array's LCACHE now stores the cluster-width chunk at LUTG (
LC_W= 16 x LUTG_AW = 128 b). The lane's own lookahead index reads it (cache register -> cluster register), the same two-register depth as seam ->a_r; the in-array quantiser and its lead lane are not instantiated (A8Q_INT/A8Q_EXT). - A8Q_OVL reuse. G7u after G7g raises no
fill_req; the bridge keeps the G7g cache (same seam, same M). Any other walked pass refills. - Per-stage handshake. Tile 0: start -> walk -> commit -> bridge start + reload -> launch at the bridge's EARLY_START bound. Tiles 1..N-1: start passes through into the bridge's 1-deep pending slot; the fast issuer launches on the array's own
ready(T-3); the drafter'sarray_readyis the bridge's "slot consumed" pulse, as on the MLPV. - RET_SER is untouched:
rs_outchanges only at the commit =done, which is what the bridge's result capture needs. - CDC. The shipped bridge pattern only: synchronous integer 2:1 from one MMCM, phase toggle
tick_s ^ tick_f, slow->fast pulses qualified byfirst_half, fast->slow by a toggle and one slow register (done_seen_s), reset through the 2-flop fast copyg_frst.frst_q. The MLPV uses no ASYNC_REG and no multicycle on these crossings (clean7c/docs/ARRAY.mdsection 6), and neither does this: every crossing is timed,report_cdcon the routed harness says "All paths are Safely Timed" (section 4d). Adding ASYNC_REG to a timed synchronous crossing would only mis-label it. - Clock. On this lineage
ap_clk_fast= clk_usr_1 is 264 MHz (FREQ_HZ 263,999,763 in the 0x13BE wrapper), the SIBW pump clock, and with SIBW 0 nothing consumes it (the only loads are the array'sclkNxandclkmuxes). The image must change clk_wiz to {66,132,286}; no new port.
3. Cycles per item, 4 ranks, same stimulus [M]
| phase | NFA8 (NA_L100) |
strap 0 (F0_L100) |
strap 1 (F1_L100) |
delta |
|---|---|---|---|---|
| G4 | 1,402.0 | 1,402.0 | 914.0 | -488.0 |
| G6 | 641.6 | 641.6 | 632.2 | -9.4 |
| G7G | 638.0 | 638.0 | 486.0 | -152.0 |
| G7U | 424.0 | 424.0 | 220.0 | -204.0 |
| G1 / G5 / SILU / EXPORT | 457 / 579 / 134 / 704 | same | same | 0 |
| busy (excl. WAIT) | 4,979.6 | 4,979.6 | 4,126.2 | -853.4 (-17.1 %) |
| run end (ps) | 293,350,000 | 293,350,000 | 279,702,000 |
Mean over 4 compute items x 4 ranks; every rank and item reads the same G4 / G7G / G7U. G6 barely moves because its 632 cycles are the TP all-reduce and the re-issue, not the array walk (K = 768, 24 pairs per tile). Bridge latency is inside these numbers: fill-to-launch np/2 + 3 slow cycles per walked pass, about 2 slow cycles per done.
4. Gates [M]
World: buildx00 /shared/scratch/att_fastclk_20261117 (flat sim-baked src/BCPF = the NFA8 BCPN + patch_att_fastclk.py; TB, ops and runner are the NFA8 world's, fc_arm.sh). Reference = the NFA8 image arm combo3_nfa8_20261117/h/runs/NA_L100 (xsim.log 1f5ff665).
| gate | arm | result | xsim.log md5 |
|---|---|---|---|
| (a) strap 0 | F0_L100 |
PASS: 0 taps differ (D9R/D9V/D9Z/D9ZL/KVWR/TAPG4/5/6), 16/16 seams equal, X 0, rdlog 0, cycle-identical end time | 5beed3e4 |
| (a) strap 1 | F1_L100 -d ATT_ARRAY_FAST_CLK_V=1 |
data identical: D9R 320/320, KVWR 5,664/5,664, TAPG4 96/96, TAPG5 16/16, TAPG6 96/96, D9Z 8/8, 16/16 seams byte-equal, rdlog symmetric difference 0 on 4 ranks, exchange scoreboard PASS, G6RI PASS x4, 0 Error/Fatal | 13a16bea |
| (b) cycles | F1_L100 |
table above | |
| (c) RED handshake | RED1_L100 (EARLY_START bound removed) |
caught: 664 taps differ (D9R 304, KVWR 96, TAPG4 80, TAPG6 96), 16/16 seams, 96 cache-chase violations (first: read pair 0, written 0) | d0e1b308 |
| (c) RED CDC phase | RED3_L100 (RED_FILL_PHASE) |
caught: 728 taps differ (D9R 320, KVWR 128, TAPG4 96, TAPG6 96), 16/16 seams | 9e4f5963 |
| (c) RED CDC done | RED2_L100 (RED_DONE_RAW) |
caught as a hang: 24 tile results then no item ever completes (a lost done pulse); see Housekeeping | (running) |
| chase checker | F1_L100 |
8,256 consumed cache reads per group per rank, 0 violations (12 lines); NORMFUSE chase 768 reads, 0 violations |
Two things the stock comparator prints as FAIL on F1, both monitor artefacts and both expected:
- D9ZL 3 -> 2 lines per rank: the missing line is
g_a8q.u_lead, the lead lane that no longer exists. The two real-lane histograms are equal. - D9V vector fields:
d9_arr_monis bound on the array's ports and records(chunk_idx_out, a_chunk); at strap 1 the array fetches nothing through its ports.cmp_fastclk.py --fastclk-d9vcompares D9V on p / st / nc / qz / M / T (all equal, 64/64) and drops X / miss / ncs / conf. The activation tooth at strap 1 is D9R: every lane result of every tile.
Limits (gate shape): context 173, Llama mode, md <= 2, 4 compute items per rank; Qwen DUAL_MODEL forwards not run; inject paths not run; RED4 (late tile index) not run.
4d. OOC place and route at 132 / 66 MHz
buildxd02 /shared/scratch/att_fastclk_20261117/ooc/runs/L512_p7575_loc; harness poc/att_fastclk_ooc_top.sv = slow quantiser + bridge + array (LANES 512, ROLE 1, LUTG, A8 external, RET_SER R 8, PRE_BASE, TILE_PIPE, LCACHE, RST_PIPE 2), one input clock with BUFGCE + BUFGCE_DIV /2 so the 2:1 skew is timed; soft per-SLR pblocks. Synth 45 min / 19 GB, impl 1 h 30 / 24 GB. route.dcp 0809401c.
| fast period | fast / slow MHz | WNS | WHS | fast->fast | slow->fast | fast->slow | slow->slow |
|---|---|---|---|---|---|---|---|
| 7.575 (routed) | 132.0 / 66.0 | +0.118 | +0.010 | +0.118 | +0.221 | +1.111 | +2.990 |
| 7.0 (re-timed) | 142.9 / 71.4 | -0.457 | +0.010 | ||||
| 6.5 (re-timed) | 153.8 / 76.9 | -0.957 | +0.010 |
- 0 routing errors (1,154,166 nets);
report_cdc: All paths are Safely Timed. - Worst setup paths are the PRE_BASE column-base replica -> the zero / weight bank
col_base_acc(SLR2), not the lane, the cache or a crossing. Worst slow->fast isu_a8q_s/t_act_r-> at_rreplica (+0.221), a quasi-static value that could take a multicycle. - Headroom is therefore about 1.5 % at 132 MHz in the harness: it closes, with no room to spare.
- Utilisation (routed): LUT 531,496 (20.65 %), FF 526,762 (10.23 %), BRAM 795 tiles (21.25 %), URAM 512 (26.60 %), DSP 1,638 (15.10 %); SLL 2,032 / 1,175 of 18,870. Added by this lever: bridge 24,929 FF (the result hold), slow quantiser 5,617 LUT / 745 FF / 100 DSP (it replaces the in-array one), cache 12 RAMB (4 per die, 0 read loads in another SLR).
- Netlist teeth: cache RAMB 12, in-array quantiser cells 0, slow quantiser and bridge present.
5. The diff [M]
Fork verilog/forks/att_nfa8_fastclk_20261117 = cp -a of att_combo3_nf_a8qovl_20261117 + scripts/4c1/patch_att_fastclk.py (7ca95bf3, count-asserted, atomic, originals kept as *.pre_fastclk). Diff: att_fastclk_20261117/NFA8_TO_FASTCLK.patch (50336360, 617 lines).
| file | NFA8 -> fork | +/- | what |
|---|---|---|---|
top/eagle4_config.svh |
89565763 -> 6a58f888 | +20 / 0 | [A] ATT_ARRAY_FAST_CLK_V 1'b0, [S] ATT_FASTCLK_RED_V 0 |
top/eagle4_drafter.sv |
92ab14a2 -> 901c191a | +88 / -35 | ATT_AFC, `AFC = ARRAY_FAST_CLK |
top/gemv_stage_array_scaled_slr.sv |
7f754cc3 -> 640a4530 | +60 / -23 | A8Q_INT / A8Q_EXT, cluster source = cache at LCACHE, LC_W, g_a8q_ext, guard LCACHE && !A8Q, sim-only cache chase checker |
primitives/lutg_a8q.sv |
68be50f9 -> 31d6dfdd | +7 / -2 | output fill_req |
top/gemv_array_dualclk.sv |
a44f3e9f -> 0395872d | +34 / -12 | replaced by the record MLPV 0x1361 file (adds FAST_RST_SYNC / f_rst_n) |
The strap is a header localparam (the LUTG / A8Q / A8Q_OVL idiom of this lineage); the top parameter ARRAY_FAST_CLK stays 0. Strap 0 is proven by F0_L100, not by inspection. The fork is not committed in the verilog submodule; it is reproducible from the tool + the MLPV bridge file.
6. Proposed stage recipe (not derived, not launched)
- Base = the NFA8 CE stage
combo3_nfa8_20261117/stages_v2/lutg_d9_tp4stk_CE_stage. Runpatch_att_fastclk.pyon itsforks/head_hxf_cons/src_full(PROOF-REPRO against this fork's 5 md5s), re-pin MD5LIST, CERT_TOP / CERT_CFG / CERT_GSAS andCERT_DCW_Pa44f3e9f -> 0395872d (uni_pipeline_lutg_d9.sh:700). - Bake
ATT_ARRAY_FAST_CLK_V 1'b0 -> 1'b1(count-asserted); keepARRAY_FAST_CLK_V 1'b0andARRAY_FAST_TP_V 1'b0as residency rows. - Clock: clk_wiz
{66,264,286}->{66,132,286}(:2024,:2171),DC_FAST_FREQHZ263999760 -> the {66,132} BD-actual (131999880 on the MLPV; let STEP4b learn it), sanity-synthcreate_clock ap_clk_fastperiodDERATE/4->DERATE/2(:1214). The clk_usr_1 -> ap_clk_fast BD net and the clock bus interface already exist in this pipeline. - Constraints: keep the dual-clock locality XDC import (
:1967-1975); re-root it forg_lcache.g_lc_grp[*]andg_dc.g_qsnext to the seams. No false path, no multicycle, no ASYNC_REG on the bridge crossings. - Report-only rows: XCI / staged-header
ATT_ARRAY_FAST_CLK 1; netlistg_dc.u_dc_bridge,g_dc.g_qs.u_a8q_s,u_array/g_lcache.g_lc_grp[0..2]RAMB (matchRAMB*), ZEROu_array/g_a8q.u_a8q/u_lead,u_arrayclock pin on clk_usr_1; routed WNS / WHS per clock pair (the four of section 4d) +report_cdc. - Build id: pick with
qg_id_check.py(label AND reg100) at derive time; none reserved here. - Fit: SLR0 SLICE is the open question (the bridge adds 24.9K FF near the consumer). Fly it beside an NFA8-only sibling, as the build rules ask.
7. Files
| file | md5 |
|---|---|
scripts/4c1/patch_att_fastclk.py |
7ca95bf3 |
att_fastclk_20261117/NFA8_TO_FASTCLK.patch |
50336360 |
att_fastclk_20261117/sim/ident/{F0,F1,RED1,RED3}_L100_vs_NA.txt |
214ed883, 1851cd71, 6586e1e3, e90716b6 |
att_fastclk_20261117/sim/cmp_fastclk.py, sim/fc_arm.sh, sim/tools/fc_cmp.sh |
(in tree) |
att_fastclk_20261117/ooc/ (retime.txt a95541b8, per_clock_pair, route_cdc, route_util*, lcache_census, netlist_teeth) |
|
att_fastclk_20261117/poc/ (harness + tcl, copies of the fork's poc/att_f*) |
Housekeeping
RED2_L100is still running on buildx00 (my own xsim,xsimkpid 3388037, one core): the planted lost-done hangs the drafter and the TB only ends on its own poll timeout. I did not stop it. It is safe to stop; nothing else depends on it.- Nothing removed or moved.
att_fastclk_20261117/patch_att_fastclk.pyis a byte-equal extra copy of the tool (7ca95bf3). An empty first OOC staging attempt leftflow_L512_loc.out(one error line) on buildxd02. - Scratch: buildx00
/shared/scratch/att_fastclk_20261117(about 3 GB, /shared 95 % full), buildxd02/shared/scratch/att_fastclk_20261117/ooc.