fpga-archive-reports-qwen · EAGLE4 FPGA archive
DESIGN (Fable): K64 on the Qwen GPTQ-2 ATT array, four weight chunks per URAM read cycle at 256 lanes (2026-11-17)
Program: the 7-card drafter (4 ATT + 2 MLPV on V80, H200 verifies) for the Qwen3.8-27B GPTQ-2 W2A16 drafter. Not the 3-card Llama drafter. Motto: a research track never ends by denying the idea; a wall is a measured constraint to engineer around, toward the fastest flashable PDI. Agent: Fable design agent for box1-two-v2. Junior (box1-qwen-junior) owns the Qwen image builds and the array-fast-clock (AFC) wave; this lever is built to compose with it. Rules kept: no PDI build, no flash, no card, no talon2, no demo, no GPU. Create-only files. The only processes stopped were my own superseded OOC and xsim jobs (section 9). Tags: [M] measured (artifact named) · [R] read from RTL or a build record (file:line) · [P] projection or hypothesis. Every [P] is in section 1.
0. Answer first
- It works and it is bit-exact. With
K64_V 1the Qwen ATT array reads four 16-code chunks per lane per cycle from the same 768 URAM. On the 4-rank full-drafter regression the regression's own checker reports 0 differing on every stream for the root item, four level-1 children and four level-2 children (9 compute items x 4 ranks), and on the root item every tap line equals the parent fork's run [M]. - Measured cycle cut per ATT item (root, context 50, identical on 4 ranks) [M]:
| phase | parent fork (the L256 A16 launch gate) | K64_V 0 |
K64_V 1 |
K64_V 2 |
junior's AFC patch alone (measured here, K64_V 0) |
AFC + K64_V 1 |
|---|---|---|---|---|---|---|
| G4 QKV (15 tiles) | 4,825 | 4,825 | 2,586 | 2,586 | 2,584 | 1,464 |
| G6 + 4-way exchange | 903 | 903 | 885 | 885 | 883 | 883 |
| G7g (incl. pre-MLP norm) | 3,937 | 3,937 | 2,738 | 2,738 | 2,736 | 2,136 |
| G7u | 2,585 | 2,585 | 1,386 | 1,306 | 1,384 | 784 |
| G1 / G5 / SiLU / EXPORT | 713 / 612 / 262 / 1,248 | same | same | same | same | same |
| busy | 15,085 | 15,085 | 10,430 (-30.9 %) | 10,350 (-31.4 %) | 10,422 | 8,102 (-46.3 %) |
- G4 and G7u are cut by 46 %; the array part of G7g by 47 % (its other 1,377 cycles are the pre-MLP norm).
- G6 does not move: the G6 hold credit rule refuses 9 of the 19 overlapped issues once tiles arrive every 20 cycles, and the existing re-issue path takes over (section 4.3). No tile is lost.
- K64 and junior's array fast clock stack. Junior's own patch applies unchanged on top of this fork in either order, and the composed source is bit-exact in the same regression: 8,102 cycles per item.
- How 4 chunks per cycle are reached: option (a), same URAM count. The lane's bank is 3 URAM288 holding 24,576 x 32-bit words. Each port already reads a 72-bit row and returns 32 bits of it. Stored as 12,288 x 64-bit entries, each port returns two chunks and the two ports four. Synthesis: 768 URAM288 before and after, 258 DSP58 before and after [M].
- The first tile of a stage still runs at 2 chunks per cycle. The drafter's seams have two read ports. The first tile walks them exactly as shipped while the array records the activations in a per-die BRAM cache (16 RAMB36 per die); every later tile reads 4 chunks per cycle from that cache. No seam, sequencer, loader or result path changes.
- Resource delta of the ATT array (256 lanes, A16) [M, OOC, section 5]: synthesis +357,970 LUT (+1,398 per lane), +206,255 FF, +45 BRAM tiles, URAM +0, DSP58 +0. Routed at 66 MHz with 0 routing errors, WNS +0.689 / WHS +0.010 (shipped lane in the same harness: +1.832 / +0.011). The fast-clock form of the K64 array routes at 132 MHz with WNS +0.263 / WHS +0.010. Placed: +59,748 SLICE (66,402 -> 126,150; +233 per lane), about +20K in each die. The array goes from 20.6 % to 39.2 % of the device's SLICE.
K64_V 0is the parent. Same primitive counts as the parent fork in synthesis (every cell type equal) and a tap-identical, cycle-identical regression run [M].- The host weight packer and the boot stream do not change. The bank assembles each 64-bit entry from the two 32-bit words the loader already sends in ascending order. The one precondition (every lane's weight words arrive as even address then odd address) is checked on the real 0x13A0 and 0x139C boot ops: PASS on all 8 ranks [M].
- Projected effect [P, section 1]: about -1.0 ms per d7 W2 tree alone and about -1.6 ms with the ATT fast clock, against 7.0 ms measured. On the humaneval-80 point (140.15 tok/s) that is about 143 tok/s alone and about 145 with the ATT fast clock. The round is dominated by time outside the array; the card-side number is the claim.
1. Calculations and hypothesis (every projection in this document)
- [P] Cycle law. tile 0 = chunks / 2 cycles, tiles 1..N-1 = chunks / 4. G4 (640 chunks, 15 tiles): 320 + 14 x 160 = 2,560; G7g / G7u (320 chunks, 16 tiles): 160 + 15 x 80 = 1,360; with the G7u reuse 16 x 80 = 1,280. Measured: 2,586 / 1,386 / 1,306 (25 to 26 cycles of fixed overhead per stage, as in the parent).
- Child item (measured, section 4.4, kept here because the tree arithmetic uses it). 15,757 becomes 11,102 at
K64_V 1and 8,774 with the fast clock. - [P] Tree time. Measured d7 W2 card tree 7.0 ms; per item the ATT is 229 to 239 us and the MLPV 235 to 238 us, in series per level (
WHY_SLOWER_LAYERS_PHASES_20261117.mdsection 3). Scaling the tree by (a' + m) / (a + m) with a = 15,757 and m = 15,616 cycles:
| ATT | MLPV | (a' + m) / (a + m) | tree |
|---|---|---|---|
| parent | 0x139D | 1.000 | 7.0 ms [M] |
K64_V 1 |
0x139D | 0.852 | 5.96 ms |
AFC + K64_V 1 |
0x139D | 0.777 | 5.44 ms |
AFC + K64_V 1 |
junior's MLPV AFC (11,900 cycles, STAGING_20261116.md section 24) |
0.659 | 4.61 ms |
- [P] tok/s. humaneval-80: 140.15 tok/s at a 49.5 ms decode round, one tree per round (
DESIGN_FABLE_DRAFT_AHEAD_20261117.md:33). Removing the tree saving from the round: 143.2 tok/s (K64_V 1), 144.7 (with the ATT fast clock), 147.3 (with both fast clocks). The KV-only rows of a round also run G4 on the ATT and are not counted here; each would save about 34 us more. - [P] Full-chip fit. 0x13A0 routed uses 86.84 / 73.26 / 4.68 % SLICE in SLR0 / SLR1 / SLR2 (
BUILDS_20261116.md:187), about 97.7K / 75.1K / 5.0K of 112.5K / 102.5K / 106.7K sites, 177.8K in all. Adding the measured +59.7K gives about 237.6K = 74 % of the device, so it fits device-wide. It does not fit the 0x13A0 floorplan shape: SLR0 and SLR1 together have about 42K free and each K64 lane group needs about +20K more than today. SLR2 has about 102K free. The K64 image therefore needs lane groups moved into SLR2 (two groups in SLR2 and none in SLR0 is the split this arithmetic allows: about 68 / 71 / 83 %). That is a floorplan change with its own timing question (activation and result paths between SLR0 and SLR2), not measured here. - [P] Area relief, not built: the DSP-half form. SLICE is the binding resource and K64 adds about 60K. Slots 2 and 3 of the K64 step (32 products per lane-cycle) could run on the record's packed DSP58 dot (
e3b_packed_dot, 17 DSP58 per 32 products at A16,DESIGN_FABLE_GPTQ2_GEMM_20261116.mdsection 1.8) instead of LUT trees: about 4,352 DSP58 for 256 lanes against roughly half of the added LUT and SLICE. 0x13A0 uses 60.8 / 7.4 / 0.4 % of the DSP58 in SLR0 / SLR1 / SLR2, so the DSP pool is where the lanes would have to move anyway. It needs its own bit-exact gate (unsigned codes on the packed fields) and its own OOC. - [P] After K64 + AFC the ATT item is no longer array-bound: the four GEMV phases are 5,267 of 8,102 cycles, of which about 1,377 are the pre-MLP norm and about 800 the G6 exchange. The next ATT targets are EXPORT (1,248), the pre-MLP norm and the G6 hold credit, not the GEMM.
- [P] Next steps on this lever, not built: (a) tile 0 at 4 chunks per cycle needs four seam read ports or a fill ahead of the stage (worth 160 + 80 + 80 cycles per item); (b) with the fast clock, the bridge reloads at G7u, so
K64_V 2gains nothing there until the bridge skips that reload; (c) a third G6 hold entry would let G6 use the shorter tile (worth up to about 360 cycles in the ideal-link sim, less on real links where G6 is exchange-bound).
2. The URAM geometry and the three options
2.1 What the shipped bank is [R] / [M]
- Fork, from the build record and not from memory:
/mnt/hdd_data/eagle4_scratch/qwen_gptq2_20261116/stage/att_stage_13A0_vm/STAGE_SHA_E4B713A0.txt(md5100da37e):fork=forks/fourcard_4c1_qgptq2_att_20261116,fork_drafter=076c7767= this fork's parent copy. - One bank per lane:
weight_bank_multistage->weight_bank_uram,DATA_W 32,RD_PORTS 2, depthsg_w_depth_role(256, 1)= 24,576 words (gemv_stage_array_scaled_slr.sv:1683-1697,stage_geometry.svh:193-200). Footprint 21,440 words: G4 15 x 640, G6 20 x 80, G7g 16 x 320, G7u 16 x 320. - Memory:
(* ram_style = "ultra", cascade_height = 8 *) reg [31:0] mem [0:24575](weight_bank_uram.sv:76). Port A is a pure read (even chunk); port B is one shared address, boot write or compute read (odd chunk) (:142-151). - Synthesis maps it to 3 URAM288 per lane, 768 for the array [M:
A0b_base_p15/synth.log"24 K x 32 ... 3 ... 3x1"; micro-synthURAM=3]. A URAM288 is 4,096 x 72, so Vivado packs two 32-bit words per row: every port read already fetches 64 stored bits and drops half. - The lane reads 2 chunks (32 codes) per cycle. The array law is cycles = N x K / (LANES x 16 x 2).
2.2 Options
| option | what | 4 chunks per cycle? | URAM | verdict |
|---|---|---|---|---|
| (a) 64-bit entry, same URAM | store {chunk 2e+1, chunk 2e} per entry at half the depth; each port returns a whole entry | yes, at any array clock | 768 -> 768 [M] | chosen |
| (b) URAM at 2x clock | read both ports twice per slow cycle | yes, but it IS the array fast clock: with AFC the whole lane already runs at 2x, so (b) alone buys nothing that AFC does not, and the lane datapath must still double to consume 4 chunks per slow cycle | same | this is junior's AFC lever; (a) multiplies with it (measured: 15,085 -> 8,102) |
| (c) more lanes (L512) or K128 | L512 doubles banks, scale / zero BRAM and the URAM spread over three dies; K128 (a whole 128-input group per cycle) needs two URAM columns read in parallel per lane | L512 yes; K128 yes, as 8 chunks | L512: 1,536; K128: 2 columns x 2 URAM = 1,024 | L512 A16 died at place on the URAM wall (STAGING_20261116.md section 15); K128 doubles the lane datapath again (about +60K SLICE more) for the remaining tiles only; neither pursued |
(a) costs the same lane datapath as L512 (four trees per column-cycle instead of two) but keeps one bank, one scale bank, one zero bank, one scale DSP58 and one result return per lane, and adds no inter-die wire.
3. The design [R]
Fork verilog/forks/qwen_att_k64_20261117/ = a copy of the parent + the edits below. One strap in the single config file.
file (src_full/) |
parent md5 -> fork md5 | what |
|---|---|---|
top/eagle4_config.svh |
6fe55a01 -> 16590515 | [A] K64_V (0 / 1 / 2, default 0), [S] K64_RED_V (sim REDs 1..4) at :789-811 |
top/eagle4_drafter.sv |
076c7767 -> 932905ca | ARR_K64 localparam + elaboration guard g_k64_top_x (:1246-1252), .K64 / .RED_K64 (:7834), .k64_act_new (:7880-7882) |
top/gemv_stage_array_scaled_slr.sv |
592d02ca -> aa7a197c | parameters K64 / RED_K64, port k64_act_new; g_k64 mode flag + cache (:1470-1605); cluster slots 2/3 g_lutcl4 (:1695); cluster source src_a_k (:1648); bank ENTRY64 (:1898); the K64 lane branch (:1933); ready in lane steps (:1105-1114); RET_SER shortest-period check (:1163); guard g_k64_x (:2747) |
primitives/weight_bank_uram.sv |
70c8acfa -> 7fc1460b | ENTRY64: branch g_e64 (:142-204) |
primitives/weight_bank_multistage.sv |
8c7a12cd -> a0358b70 | ENTRY64 pass-through, rd_data_hi / rd_data_b_hi |
systolic/lutgemm_lane_w2q.sv |
new dacfcd99 | the 4-chunk-per-step lane |
systolic/lutgemm_lane_w2.sv |
6edf4036, untouched | the shipped lane |
3.1 Bank (weight_bank_uram ENTRY64)
mem64[e] = {word 2e+1, word 2e}, 12,288 x 64. Port A reads entry (c >> 1), port B entry (c >> 1) + 1; the multistage wrapper still addscol_offset_acc + chunk indexin word units, so PRE_BASE and the stage bases are untouched.- Boot writes keep the 32-bit word interface: an even-address write is held, the following odd-address write commits the entry through one register stage.
- That register stage is required. With the commit enable decoded in the same cycle, Vivado 2025.1 classifies the port READ_FIRST, prints
[Synth 8-12186] ram_style ultra ... ignored because invalid write modeand maps the bank to 24 RAMB36 per lane. Micro-synth of the bank alone [M,k64_evidence/unit/UNIT4_K64_ARRAY_GATE.txt, linesMICRO]: shipped 32-bitURAM=3; ENTRY64 with the same-cycle enableURAM=0 RAMB=24(two variants); ENTRY64 with the registered commitURAM=3 RAMB=0, 16 LUT, 111 FF. - The generate scope keeps the instance path
g_dual.u_bank.u_uram, so the WSNAP snapshot files and the boot kit address map are unchanged; in simulation the snapshot is loaded in its 32-bit form and packed.
3.2 Lane (lutgemm_lane_w2q)
- Same arithmetic as
lutgemm_lane_w2: per chunksum(2 x T[hi] + T[lo]) - zero x S, per-group fold, one scale multiply per group, 48-bit accumulate, the TP_SLIM single bank, the pair-sequence check. A step never straddles a 128-input group (8 chunks per group; step = 2 or 4 chunks from a column start). - A column is latched as pair mode (2 chunks per step) or quad mode (4 chunks per step) when its start is accepted. Every per-step flag (slot valid masks, last-in-group, last-in-column, sequence index) is decided in the step's own cycle and registered, so a chained column of the other mode cannot re-interpret a step in flight.
- One extra register after the trees (the two half sums) keeps the fold path shorter than the shipped lane's. A column's latency is +1 cycle; the tile period is the number of steps.
- The instance path stays
g_dual.g_lutg.u_laneandg_dual.g_lutg.g_zb.u_zero(the alternative branch of the same if-generate carries the same scope name), so snapshots, census scripts and constraints written for the shipped lane still bind.
3.3 Mode and cache (g_k64, per group = per die)
k64_act_new=g4_start | bk_stage_start | g12_start, the expression the dual-clock bridge already uses asact_reload(eagle4_drafter.sv:7686in the parent). AtK64_V 2the G7u entry is left out: G7g and G7u read the same pre-MLP seam (eagle4_drafter.sv:6608-6609in the parent).- The first column accepted after
k64_act_newis a pair-mode column; every later column is a quad-mode column. - Cache: two banks x 160 x 512 bits per group (even pairs, odd pairs), read at the group's own first lane's lookahead index, one read register, then the existing cluster register. That is the register depth of lane -> seam -> cluster in the shipped path.
- Snoop write: in a pair-mode column the port carries chunks (c, c+1) in the cycle the lane counter is c; one per-group pipe register, then the pair is written. A sim-only chase checker counts every quad read of an entry the current fill has not written: 0 violations in every GREEN arm, 5,360 per group in RED 3.
- With
LCACHE 1(array fast clock) the same cache is written through the bridge'slc_*port instead and also feeds the pair-mode column; the bridge and its EARLY_START bound are untouched because tile 0 consumes one pair per fast cycle, as today.
3.4 What does not change
The seams and their two read ports, both body sequencers, the tile-tag FIFO, RET_SER, the G6 exchange, the scale and zero banks, the boot loader, the host stream, the exposed chunk index (the monitors still see a consistent fetch on every tile).
4. Gates [M] (report-only)
4.1 Array-level equivalence (fast gate, final source)
tb/k64/tb_k64_array.sv (aaf3897f) + run_k64_unit.sh (c8d703f7), buildy02 /shared/scratch/qwen_att_k64_20261117/unit4/. Three arrays in the Qwen ATT configuration (ROLE 1, L256 bank geometry, LUTG A16, per-lane zero, TILE_PIPE, RET_SER, PRE_BASE), each with a drafter-like tile sequencer and a registered seam; all booted through the 32-bit word interface in loader order (so the entry assembly is exercised); random activations with extremes, random zeros 0..3. Evidence: k64_evidence/unit/UNIT4_K64_ARRAY_GATE.txt.
| arm | tiles compared | mismatching tiles | verdict |
|---|---|---|---|
K64 1, 3 rounds |
201 | 0 | PASS |
K64 2 (no act_new at G7u), 3 rounds |
201 | 0 | PASS |
K64 1 + LCACHE (junior's AFC array edit applied), 2 rounds |
134 x 2 | 0 / 0 | PASS |
K64 2 + LCACHE, 2 rounds |
134 x 2 | 0 / 0 | PASS |
| RED 1: snooped activation one cycle late | 67 | 63 (exactly the quad tiles) | FAIL as required |
| RED 2: port-B entry halves swapped in the lane | 67 | 63 | FAIL as required |
| RED 3: no pair-mode first tile (stale cache) | 67 | 67, 5,360 chase violations per group | FAIL as required |
| RED 4: boot-write entry halves swapped | 67 | 67 | FAIL as required |
Cycles per stage, shipped lane vs K64: G4 4,815 -> 2,576; G6 815 -> 436; G7g 2,575 -> 1,376; G7u 2,575 -> 1,376 (1,296 with the reuse).
4.2 Full drafter, 4 ranks (the regression's launch gate)
World: buildy01 ~/box1_scratch/qgptq2_xsim_att/ = a copy of the regression world of XSIM_ATT_20261116.md (TB tb_ql256, stimulus stim_root, weights wsnap/l256_green, the stock run_qg2.sh / arm_qg2.sh), DUT = this fork staged with the fork's own q5120/stage_qgptq2_att.py (L256, A16, non-FR). Reference run = the parent's L256_ROOT_GREEN2 (xsim.log md5 9615b3f1). Two comparisons per arm: the regression's checker qg_att_check.py against the 7-card bit-true vectors, and k64_tapcmp.py line by line against the parent's log (D9R every tile result, D9V, TAPG4/5/6, KVWR, TAPR4). Evidence: k64_evidence/checks/.
| arm | source | checker (17 streams x 4 ranks) | taps vs parent (28 classes) | busy | end time |
|---|---|---|---|---|---|
F0_ROOT K64_V 0 |
final | PASS, 0 differing | 28 equal | 15,085 | 395,462,000 = parent |
S0_ROOT K64_V 0 |
s2 | PASS, 0 differing | 28 equal | 15,085 | = parent |
F1_ROOT K64_V 1 |
final | PASS, 0 differing | 28 equal | 10,430 | 376,838,000 |
S1_ROOT K64_V 1 |
s2 | PASS, 0 differing | 28 equal | 10,430 | 376,838,000 |
F2_ROOT / S2_ROOT K64_V 2 |
final / s2 | 16 streams 0 differing; pn(g7u_in) row FAIL (see below) |
24 equal, D9V differs | 10,350 | |
SA0_ROOT AFC, K64_V 0 |
s2 + junior's patch | (checker cannot parse D9V under AFC) | 24 equal, D9V differs | 10,422 | |
FA1_ROOT AFC + K64_V 1 |
final + junior's patch | same | 24 equal, D9V differs | 8,102 | 367,526,000 |
SA1_ROOT AFC + K64_V 1 |
s2 + junior's patch | same | 24 equal, D9V differs | 8,102 | 367,526,000 |
RED SR1_ROOT (K64_RED_V 1) |
s2 | FAIL: G4 3,584 / 3,840 differ on every rank = tiles 1..14, tile 0 exact; first mismatch at element 256 | |||
RED SR3_ROOT (K64_RED_V 3) |
s2 | FAIL: results X (checker aborts on X) | 28 classes differ; 5,360 chase violations per group |
- "final" =
src_k64_s4, "s2" =src_k64_s2. They differ in one file,weight_bank_uram.sv(the registered commit stage of section 3.1), which only acts on boot writes; these sims load weights by snapshot. The fast gate of 4.1 covers boot writes on the final source. - The two expected monitor rows. D9V is the regression's record of what the array fetched through its ports. (1) At
K64_V 2the G7u pass fetches nothing new through the ports by design, so the stock checker'spn(g7u_in)row reads half the vector as missing (miss=160,conf=0);pn(g7g_in)(the same seam) and every G7u result stream are 0 differing. (2) Under AFC the array fetches nothing through its ports at all (junior's lever; the same artefact as inATT_FASTCLK_20261117.mdsection 4). UseK64_V 1if a fully green stock checker is wanted. - Per-rank chase counters at the end of
F1_ROOT: 5,016 quad reads per group per rank, 0 violations.FA1_ROOTandSA1_ROOT: K64 chase 0 violations and junior's LCCHASE 0 violations on all 12 groups; under AFC the G6 credit rule refuses 18 of 19 overlapped issues (hold_full_cycles=6,lost=0). - TPX END 320 beats per card,
lost=0on every GREEN arm.
4.3 G6 under K64
TP5MON at the end of S1_ROOT: g6_overlap_issues=10 refused_windows=9 (parent: 19 / 0), captured=20 lost=0. The TP-5 credit rule (eagle4_drafter.sv:6774-6794 in the parent, G6_HOLD_CAP 2) refuses an overlapped G6 issue while two tiles are in flight; the shipped G6_REISSUE path re-issues them. G6 is therefore 885 cycles instead of the law's 420 + overhead. Results are exact. The hold size is not touched here.
4.4 Child items (md3 split arm: root + level-1 lanes 0..3 + level-2 lanes 0..3)
Stimulus stim_s5 of the regression (10 items: KV_LOAD + 9 compute items), same world. Evidence: k64_evidence/checks/CHK_F1_S5.txt, CHK_S1_S5.txt, CHK_S2_S5.txt, S5_ARMS_SUMMARY.txt.
| arm | items ended on all 4 ranks | checker | chase violations | busy: root / L1 child / L2 child |
|---|---|---|---|---|
F1_S5 K64_V 1, final source (xsim.log 92b76c70) |
10 of 10 | PASS: 0 differing on all 17 streams x 4 ranks (g1_x 0 / 92,160, G4 0 / 34,560, G6 sum 0 / 46,080, G7g / G7u 0 / 36,864, r5 0 / 36,864 per rank) | 0 | 10,430 / 11,102 / 11,108 |
S1_S5 K64_V 1, source s2 (xsim.log de17a6a1) |
10 of 10 | PASS, the same counts; every tap line equal to F1_S5 |
0 | 10,430 / 11,102 / 11,108 |
S2_S5 K64_V 2 (xsim.log 45960f77) |
10 of 10 | 16 streams 0 differing; pn(g7u_in) monitor row 22,892 / 46,080 (the row of section 4.2) |
0 | 10,350 / 11,022 / 11,028 |
SA1_S5 AFC + K64_V 1 (source s2 + junior's patch; xsim.log 4d4a5dca) |
10 of 10 | every data tap line (D9R 603, TAPG4 135, TAPG5 9, TAPG6 180, KVWR 580, TAPR4 4,608 per rank) equal to S1_S5; D9V differs (AFC port monitor) |
0 K64, 0 LCCHASE | 8,102 / 8,774 / 8,780 |
The parent's child items are 15,757 (L1) and 15,763 (L2) (XSIM_ATT_20261116.md section 3): the cut is the same 4,655 cycles on every item (6,983 with the fast clock).
4.5 Limits (gate shape)
- Context 50, md <= 3, Qwen A16 non-FR straps. The FR_A8 strap of 0x13A0 (a requant shift outside the array) was not re-run; inject paths, KV-only items and the CARDSTOP abort were not run.
- The G6 exchange in the TB is the ideal 4-way sum (TP_LAT 40, no stalls).
- The AFC arms use junior's patch as fetched from buildx01
src_l256_afc1at 22:10Z (QG_ATT_AFC_3files.patchmd5 343cc434, bridgegemv_array_dualclk.sv0395872d; copy ink64_evidence/rtl/). If junior's patch changes, the composed arm must be re-run.
5. OOC synthesis, place and route [M]
Harness poc/k64_ooc_top.sv (665fb544): the ATT u_array as the drafter instantiates it (LANES 256, ROLE 1, K 10,240, LUTG A16, per-lane zero bank, RET_SER R 8, PRE_BASE, TILE_PIPE, BOOT_WR_PIPE 2, 3 groups) with a BRAM seam, registered control, boot bus and result consumer; one clock; soft per-SLR pblocks (group g and its cache to SLR g). Part xcv80-lsva4737-2MHP-e-S, Vivado 2025.1, maxThreads 8.
5.1 Synthesis
| run | host | LUT | FF | BRAM tiles | URAM288 | DSP58 | lane type |
|---|---|---|---|---|---|---|---|
parent fork (P0_parent_synth) |
buildz01 | 430,376 | 287,680 | 783 | 768 | 258 | 256 x lutgemm_lane_w2 |
this fork, K64 0 (A0b_base_p15) |
buildy03 | 430,376 | 287,680 | 783 | 768 | 258 | 256 x lutgemm_lane_w2 |
this fork, K64 1 (A1c_k64_p15) |
buildy02 | 788,346 | 493,935 | 828 | 768 | 258 | 256 x lutgemm_lane_w2q, 48 cache RAMB36, 768 URAM in g_e64 |
- Parent and
K64 0: the whole primitive table is equal (md5 of the table rows52c4e56eon both): FDRE 287,234, LUT5 178,454, LUT3 140,791, LUT6 127,975, RAMB36 782, URAM288 768, DSP58 258. K64 1delta: +357,970 LUT (+1,398 per lane), +206,255 FF (+806 per lane), +45 BRAM tiles, +0 URAM, +0 DSP58. 2,816 SRL16E appear (11 per lane: the lane's 4-deep control tag shift registers; not an activation seam).
5.2 Place and route
| run | config | clock | routed WNS / WHS | routing errors | SLICE total | SLICE SLR0 / SLR1 / SLR2 (% of the die) | LUT | FF | BRAM tiles | URAM SLR0 / 1 / 2 | SLL 1-0 / 2-1 | route.dcp |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
A0b_base_p15 |
K64 0 |
15.151 ns | +1.832 / +0.011 | 0 | 66,402 | 21,662 / 23,544 / 21,196 (19.25 / 22.96 / 19.86) | 401,605 | 265,600 | 783 | 258 / 258 / 252 | 2,122 / 2,234 | 203a63f1 |
A1c_k64_p15 |
K64 1 |
15.151 ns | +0.689 / +0.010 | 0 | 126,150 | 41,846 / 43,365 / 40,939 (37.19 / 42.30 / 38.36) | 761,295 | 457,005 | 828 | 258 / 258 / 252 | 1,249 / 1,218 | c17fb2eb |
F1c_k64_lc_p7575 |
K64 1 + LCACHE + RST_PIPE 2 (the fast-clock array; junior's array edit applied) |
7.575 ns | +0.263 / +0.010 | 0 | 128,232 | 41,937 / 45,423 / 40,872 (37.27 / 44.30 / 38.30) | 761,776 | 459,143 | 813 | 258 / 258 / 252 | 1,324 / 1,228 | 3dc6f65a |
- Delta K64 1 vs K64 0, placed and routed: +59,748 SLICE (+233 per lane), +359,690 LUT (+1,405 per lane), +191,405 FF, +45 BRAM tiles, +0 URAM, +0 DSP58. Per die: +20,184 / +19,821 / +19,743 SLICE.
- The K64 cache is 16 RAMB36 per group and each group's cache sits in its own die (
k64_census.txt:RAMB_SLRs={SLR0},{SLR1},{SLR2}). Lane URAM per die is unchanged. - Inter-die wire does not grow (1,249 / 1,218 against 2,122 / 2,234 of 18,870).
- Worst setup paths at 66 MHz.
K64 0: the harness reset to a cluster register (98.7 % route).K64 1: a lanedoneregister to the RET_SER crossing register (6 levels, 95 % route) at +0.689, then the same reset at +1.737. Neither is a K64 arc; the placement of a 15 ns run is relaxed, so these numbers say "closes 66 MHz", not "fmax". Re-timing the same routes to 12.0 ns gives -1.319 (K64 0) and -1.414 (K64 1). - The fast-clock form closes 132.0 MHz (re-timed: 142.9 MHz -0.312, 153.8 MHz -0.813). Its worst setup paths are the per-group reset replica to the cluster S-pipe reset pins (0 logic levels, 98.7 % route), not the lane, the bank or the cache.
- Run times: synthesis 24 min / 15 GB; place and route 49 min / 17.6 GB (
K64 0), 1 h 43 / 22.2 GB (K64 1), 1 h 46 / 22.3 GB (fast-clock form).
6. Composition with the array fast clock (junior's lever)
- Source level [M]. Junior's Qwen ATT AFC patch (3 files + the bridge, fetched from buildx01
src_l256_afc1) applies on top of this fork with every hunk succeeding, and this fork's patch applies on top of junior's; the two orders give byte-identicaleagle4_drafter.svandgemv_stage_array_scaled_slr.sv. The edits touch different lines: junior's cluster source (src_a) and LUTG guard, this fork'ssrc_a_kone line below. - Function [M]. Array level: K64 + LCACHE is bit-exact (section 4.1). Full drafter: AFC +
K64_V 1equals the parent on every data tap, 0 chase violations from both checkers, 8,102 cycles per item (section 4.2). - How it composes. With LCACHE the bridge fills the K64 cache; tile 0 is a pair-mode column on the fast clock under the bridge's unchanged EARLY_START bound; tiles 1..N-1 are quad-mode columns. Junior's
g_lcachememory is left without a reader and is pruned by synthesis. - Timing of the composed array block [M]. The K64 array in its fast-clock configuration (LCACHE write port, per-group reset replicas, junior's array edit) routes at 7.575 ns with WNS +0.263 / WHS +0.010 and 0 routing errors (section 5.2). The harness has no bridge and no slow domain; the slow-to-fast and fast-to-slow crossings are junior's bridge and are not in this number.
- Use
K64_V 1with the fast clock (the bridge reloads at G7u, soK64_V 2adds nothing there).
7. Host weight packer and loader
- No change to the packer, the pack files, the boot ops or the loader RTL. The ATT weights are written by the AXI-Lite trio, per stage and per lane:
W 48 lane,W 4c w_base,BURST 50 tiles*cpc(scripts/4c1/qgptq2/gen_qgptq2_boot_roots.py:319; address auto-incrementcontrol_regs.sv:270-272). The M9 loader walksflat_addr = stage_base + local_col*cpc + chunk(weight_dma_loader.sv:57). Both are ascending per lane from an even base with an even count (bases 0 / 9,600 / 11,200 / 16,320; cpc 640 / 80 / 320 / 320). - The precondition the K64 bank adds: for every lane, weight words arrive as (even address a, then a + 1). A lone odd-address poke, or an even word followed by a different address on the same lane, would commit a wrong entry.
- Gate (implemented, unit-tested):
scripts/4c1/qgptq2/check_k64_boot_pairing.py(983ffbeb) replays a boot.ops through the trio and checks the property for every weight-bank write.selftest: GREEN 1 / 1, RED caught 5 / 5 (odd base, odd count, odd-only poke, even then another address, lane switch mid-pair). On the real kits:bootroots/0xE4B713A0/rank{0..3}/boot.opsand0xE4B7139C/rank{0..3}: PASS x 8, 5,488,640 weight words on 256 lanes each, 0 violations (rank0 boot.ops md5 94d82cd6). - In simulation the bank also raises
$erroron an unpaired odd write. - The xsim snapshot generator (
xsim_att/tools/mk_wsnap_qg.py) is unchanged: the bank loads the 32-bit snapshot and packs it.
8. Image recipe (for the coordinator; not derived, not launched)
- Base = junior's current Qwen ATT stage (the 0x13A0 / 0x139C class, or their ATT AFC stage). Apply
k64_evidence/rtl/QGPTQ2_ATT_TO_K64.patch(62ad924a,patch -p2inside the stagedsrc; it reproduces this fork's six files byte for byte from the parent, and applies before or after junior's AFC patch). Addsystolic/lutgemm_lane_w2q.svto the add_files list and to MD5LIST / the CERT pins ofeagle4_drafter.sv,eagle4_config.svh,gemv_stage_array_scaled_slr.sv,weight_bank_uram.sv,weight_bank_multistage.sv. - Strap: bake
K64_V 0 -> 1(count-asserted, one line ineagle4_config.svh). It is a header localparam like LUTG, so the packaging parameter list does not change. - Floorplan: the K64 array needs about +20K SLICE per lane group (section 5.2) and the 0x13A0 shape has about 42K free in SLR0 + SLR1 together, so lane groups must move into SLR2 (section 1, full-chip fit). Put
u_array/g_k64.g_kc[g].*in the same pblock as lane group g (poc/k64_pblocks.tclis the OOC form). Give the first build a placement probe before calling it a fit. - Report-only rows to add: staged header
K64_V 1; netlistu_array/g_grp[*].g_lane[*].g_dual.g_lutg.u_laneof typelutgemm_lane_w2qx 256 and zerolutgemm_lane_w2;u_array/g_k64.g_kc[0..2]RAMB (matchRAMB*) = 16 per group; URAM288 = 768 ing_e64with zero[Synth 8-12186]lines in the synth log (a non-zero count means the bank fell to BRAM); DSP58 unchanged. - Host: run
check_k64_boot_pairing.pyon the image's boot ops before the first boot. No kit change. - Fly it beside a
K64_V 0sibling, as the build rules ask. Build id: pick withqg_id_check.pyat derive time; none reserved here.
9. Files, evidence, housekeeping
| file | md5 |
|---|---|
verilog/forks/qwen_att_k64_20261117/ (fork; not committed in the verilog submodule) |
six source md5s in section 3; FORK_NOTE_K64.md 1fd5db06 |
.../tb/k64/tb_k64_array.sv, run_k64_unit.sh |
aaf3897f, c8d703f7 |
.../poc/k64_ooc_top.sv, k64_synth.tcl, k64_impl.tcl, k64_pblocks.tcl, run_k64_flow.sh |
665fb544, 564b1927, 688deb2b, 99c1737e, ba359ea1 |
reports/results/QWEN_GPTQ2_20261116/k64_evidence/rtl/QGPTQ2_ATT_TO_K64.patch |
62ad924a |
k64_evidence/unit/UNIT4_K64_ARRAY_GATE.txt |
in tree |
k64_evidence/checks/CHK_*.txt, TAPCMP_*_vs_parent.txt, XSIM_SNAPSHOTS_COMPILE_INFO.txt |
in tree |
k64_evidence/ooc/ |
in tree |
k64_evidence/tools/k64_pull_check.sh, k64_tapcmp.py |
in tree |
k64_evidence/rtl/fork_extras/ (copies of the fork's TB, OOC harness, the new lane and the fork note), FORK_MD5.txt, PARENT_MD5.txt, STAGED_s4_MD5.txt, junior's patch as fetched |
in tree: with the patch they rebuild the fork from the parent without the submodule |
scripts/4c1/qgptq2/check_k64_boot_pairing.py |
983ffbeb |
Scratch: box1 /mnt/hdd_data/eagle4_scratch/qwen_att_k64_20261117/ (staged sources src_k64_s2, src_k64_s4 = final, src_k64_afc2/4, src_parent, run snapshots); buildy01 the xsim world; buildy02 / buildy03 / buildz00 / buildz01 /shared/scratch/qwen_att_k64_20261117/.
Housekeeping:
- I stopped only my own jobs, by process group, after checking each one's working directory: three OOC flows whose source was superseded 9 minutes after launch (a scope rename), two K64 OOC flows whose bank had mapped to BRAM (the finding of section 3.1), and two xsims of the superseded source.
- Stale, unused directories I created and did not remove:
src_k64_s1,src_k64_afc,src_k64_s3,src_k64_afc3(box1 scratch;README_stale_s3.txtsays so), the matching copies on the VMs, theK*arm andqk*snapshot directories on buildy01, and OOC run directoriesA1_*,A0_*,F1_*,A1b_*,F1b_*. - Nothing of mine is running: every xsim arm and every OOC flow has ended (checked by process list on buildy01 / buildy02 / buildy03 / buildz00 / buildz01 at 00:58Z).
- Other sessions started Vivado jobs on buildy01, buildz01 (about 00:04Z), buildy03 and buildz00 (about 00:37Z) while my last arms were still running there; I did not touch them. Free memory stayed at 89 GB or more.
- The K64 scratch on the VMs is 0.1 to 1.8 GB per host and the xsim world on buildy01 is 6.7 GB (2.5 GB of it the embedding image). Nothing was removed.