fpga-archive-reports-qwen · EAGLE4 FPGA archive

Source reports/results/QWEN_GPTQ2_20261116/DESIGN_FABLE_QWEN_ATT_K64_20261117.md · source md5 f3712919b96d1f0ae2db48274b27c146 · archived 2026-11-18

Download PDFOpen PDFRaw markdown

DESIGN (Fable): K64 on the Qwen GPTQ-2 ATT array, four weight chunks per URAM read cycle at 256 lanes (2026-11-17)

Program: the 7-card drafter (4 ATT + 2 MLPV on V80, H200 verifies) for the Qwen3.8-27B GPTQ-2 W2A16 drafter. Not the 3-card Llama drafter. Motto: a research track never ends by denying the idea; a wall is a measured constraint to engineer around, toward the fastest flashable PDI. Agent: Fable design agent for box1-two-v2. Junior (box1-qwen-junior) owns the Qwen image builds and the array-fast-clock (AFC) wave; this lever is built to compose with it. Rules kept: no PDI build, no flash, no card, no talon2, no demo, no GPU. Create-only files. The only processes stopped were my own superseded OOC and xsim jobs (section 9). Tags: [M] measured (artifact named) · [R] read from RTL or a build record (file:line) · [P] projection or hypothesis. Every [P] is in section 1.

0. Answer first

  1. It works and it is bit-exact. With K64_V 1 the Qwen ATT array reads four 16-code chunks per lane per cycle from the same 768 URAM. On the 4-rank full-drafter regression the regression's own checker reports 0 differing on every stream for the root item, four level-1 children and four level-2 children (9 compute items x 4 ranks), and on the root item every tap line equals the parent fork's run [M].
  2. Measured cycle cut per ATT item (root, context 50, identical on 4 ranks) [M]:
phase parent fork (the L256 A16 launch gate) K64_V 0 K64_V 1 K64_V 2 junior's AFC patch alone (measured here, K64_V 0) AFC + K64_V 1
G4 QKV (15 tiles) 4,825 4,825 2,586 2,586 2,584 1,464
G6 + 4-way exchange 903 903 885 885 883 883
G7g (incl. pre-MLP norm) 3,937 3,937 2,738 2,738 2,736 2,136
G7u 2,585 2,585 1,386 1,306 1,384 784
G1 / G5 / SiLU / EXPORT 713 / 612 / 262 / 1,248 same same same same same
busy 15,085 15,085 10,430 (-30.9 %) 10,350 (-31.4 %) 10,422 8,102 (-46.3 %)
  1. How 4 chunks per cycle are reached: option (a), same URAM count. The lane's bank is 3 URAM288 holding 24,576 x 32-bit words. Each port already reads a 72-bit row and returns 32 bits of it. Stored as 12,288 x 64-bit entries, each port returns two chunks and the two ports four. Synthesis: 768 URAM288 before and after, 258 DSP58 before and after [M].
  2. The first tile of a stage still runs at 2 chunks per cycle. The drafter's seams have two read ports. The first tile walks them exactly as shipped while the array records the activations in a per-die BRAM cache (16 RAMB36 per die); every later tile reads 4 chunks per cycle from that cache. No seam, sequencer, loader or result path changes.
  3. Resource delta of the ATT array (256 lanes, A16) [M, OOC, section 5]: synthesis +357,970 LUT (+1,398 per lane), +206,255 FF, +45 BRAM tiles, URAM +0, DSP58 +0. Routed at 66 MHz with 0 routing errors, WNS +0.689 / WHS +0.010 (shipped lane in the same harness: +1.832 / +0.011). The fast-clock form of the K64 array routes at 132 MHz with WNS +0.263 / WHS +0.010. Placed: +59,748 SLICE (66,402 -> 126,150; +233 per lane), about +20K in each die. The array goes from 20.6 % to 39.2 % of the device's SLICE.
  4. K64_V 0 is the parent. Same primitive counts as the parent fork in synthesis (every cell type equal) and a tap-identical, cycle-identical regression run [M].
  5. The host weight packer and the boot stream do not change. The bank assembles each 64-bit entry from the two 32-bit words the loader already sends in ascending order. The one precondition (every lane's weight words arrive as even address then odd address) is checked on the real 0x13A0 and 0x139C boot ops: PASS on all 8 ranks [M].
  6. Projected effect [P, section 1]: about -1.0 ms per d7 W2 tree alone and about -1.6 ms with the ATT fast clock, against 7.0 ms measured. On the humaneval-80 point (140.15 tok/s) that is about 143 tok/s alone and about 145 with the ATT fast clock. The round is dominated by time outside the array; the card-side number is the claim.

1. Calculations and hypothesis (every projection in this document)

ATT MLPV (a' + m) / (a + m) tree
parent 0x139D 1.000 7.0 ms [M]
K64_V 1 0x139D 0.852 5.96 ms
AFC + K64_V 1 0x139D 0.777 5.44 ms
AFC + K64_V 1 junior's MLPV AFC (11,900 cycles, STAGING_20261116.md section 24) 0.659 4.61 ms

2. The URAM geometry and the three options

2.1 What the shipped bank is [R] / [M]

2.2 Options

option what 4 chunks per cycle? URAM verdict
(a) 64-bit entry, same URAM store {chunk 2e+1, chunk 2e} per entry at half the depth; each port returns a whole entry yes, at any array clock 768 -> 768 [M] chosen
(b) URAM at 2x clock read both ports twice per slow cycle yes, but it IS the array fast clock: with AFC the whole lane already runs at 2x, so (b) alone buys nothing that AFC does not, and the lane datapath must still double to consume 4 chunks per slow cycle same this is junior's AFC lever; (a) multiplies with it (measured: 15,085 -> 8,102)
(c) more lanes (L512) or K128 L512 doubles banks, scale / zero BRAM and the URAM spread over three dies; K128 (a whole 128-input group per cycle) needs two URAM columns read in parallel per lane L512 yes; K128 yes, as 8 chunks L512: 1,536; K128: 2 columns x 2 URAM = 1,024 L512 A16 died at place on the URAM wall (STAGING_20261116.md section 15); K128 doubles the lane datapath again (about +60K SLICE more) for the remaining tiles only; neither pursued

(a) costs the same lane datapath as L512 (four trees per column-cycle instead of two) but keeps one bank, one scale bank, one zero bank, one scale DSP58 and one result return per lane, and adds no inter-die wire.

3. The design [R]

Fork verilog/forks/qwen_att_k64_20261117/ = a copy of the parent + the edits below. One strap in the single config file.

file (src_full/) parent md5 -> fork md5 what
top/eagle4_config.svh 6fe55a01 -> 16590515 [A] K64_V (0 / 1 / 2, default 0), [S] K64_RED_V (sim REDs 1..4) at :789-811
top/eagle4_drafter.sv 076c7767 -> 932905ca ARR_K64 localparam + elaboration guard g_k64_top_x (:1246-1252), .K64 / .RED_K64 (:7834), .k64_act_new (:7880-7882)
top/gemv_stage_array_scaled_slr.sv 592d02ca -> aa7a197c parameters K64 / RED_K64, port k64_act_new; g_k64 mode flag + cache (:1470-1605); cluster slots 2/3 g_lutcl4 (:1695); cluster source src_a_k (:1648); bank ENTRY64 (:1898); the K64 lane branch (:1933); ready in lane steps (:1105-1114); RET_SER shortest-period check (:1163); guard g_k64_x (:2747)
primitives/weight_bank_uram.sv 70c8acfa -> 7fc1460b ENTRY64: branch g_e64 (:142-204)
primitives/weight_bank_multistage.sv 8c7a12cd -> a0358b70 ENTRY64 pass-through, rd_data_hi / rd_data_b_hi
systolic/lutgemm_lane_w2q.sv new dacfcd99 the 4-chunk-per-step lane
systolic/lutgemm_lane_w2.sv 6edf4036, untouched the shipped lane

3.1 Bank (weight_bank_uram ENTRY64)

3.2 Lane (lutgemm_lane_w2q)

3.3 Mode and cache (g_k64, per group = per die)

3.4 What does not change

The seams and their two read ports, both body sequencers, the tile-tag FIFO, RET_SER, the G6 exchange, the scale and zero banks, the boot loader, the host stream, the exposed chunk index (the monitors still see a consistent fetch on every tile).

4. Gates [M] (report-only)

4.1 Array-level equivalence (fast gate, final source)

tb/k64/tb_k64_array.sv (aaf3897f) + run_k64_unit.sh (c8d703f7), buildy02 /shared/scratch/qwen_att_k64_20261117/unit4/. Three arrays in the Qwen ATT configuration (ROLE 1, L256 bank geometry, LUTG A16, per-lane zero, TILE_PIPE, RET_SER, PRE_BASE), each with a drafter-like tile sequencer and a registered seam; all booted through the 32-bit word interface in loader order (so the entry assembly is exercised); random activations with extremes, random zeros 0..3. Evidence: k64_evidence/unit/UNIT4_K64_ARRAY_GATE.txt.

arm tiles compared mismatching tiles verdict
K64 1, 3 rounds 201 0 PASS
K64 2 (no act_new at G7u), 3 rounds 201 0 PASS
K64 1 + LCACHE (junior's AFC array edit applied), 2 rounds 134 x 2 0 / 0 PASS
K64 2 + LCACHE, 2 rounds 134 x 2 0 / 0 PASS
RED 1: snooped activation one cycle late 67 63 (exactly the quad tiles) FAIL as required
RED 2: port-B entry halves swapped in the lane 67 63 FAIL as required
RED 3: no pair-mode first tile (stale cache) 67 67, 5,360 chase violations per group FAIL as required
RED 4: boot-write entry halves swapped 67 67 FAIL as required

Cycles per stage, shipped lane vs K64: G4 4,815 -> 2,576; G6 815 -> 436; G7g 2,575 -> 1,376; G7u 2,575 -> 1,376 (1,296 with the reuse).

4.2 Full drafter, 4 ranks (the regression's launch gate)

World: buildy01 ~/box1_scratch/qgptq2_xsim_att/ = a copy of the regression world of XSIM_ATT_20261116.md (TB tb_ql256, stimulus stim_root, weights wsnap/l256_green, the stock run_qg2.sh / arm_qg2.sh), DUT = this fork staged with the fork's own q5120/stage_qgptq2_att.py (L256, A16, non-FR). Reference run = the parent's L256_ROOT_GREEN2 (xsim.log md5 9615b3f1). Two comparisons per arm: the regression's checker qg_att_check.py against the 7-card bit-true vectors, and k64_tapcmp.py line by line against the parent's log (D9R every tile result, D9V, TAPG4/5/6, KVWR, TAPR4). Evidence: k64_evidence/checks/.

arm source checker (17 streams x 4 ranks) taps vs parent (28 classes) busy end time
F0_ROOT K64_V 0 final PASS, 0 differing 28 equal 15,085 395,462,000 = parent
S0_ROOT K64_V 0 s2 PASS, 0 differing 28 equal 15,085 = parent
F1_ROOT K64_V 1 final PASS, 0 differing 28 equal 10,430 376,838,000
S1_ROOT K64_V 1 s2 PASS, 0 differing 28 equal 10,430 376,838,000
F2_ROOT / S2_ROOT K64_V 2 final / s2 16 streams 0 differing; pn(g7u_in) row FAIL (see below) 24 equal, D9V differs 10,350
SA0_ROOT AFC, K64_V 0 s2 + junior's patch (checker cannot parse D9V under AFC) 24 equal, D9V differs 10,422
FA1_ROOT AFC + K64_V 1 final + junior's patch same 24 equal, D9V differs 8,102 367,526,000
SA1_ROOT AFC + K64_V 1 s2 + junior's patch same 24 equal, D9V differs 8,102 367,526,000
RED SR1_ROOT (K64_RED_V 1) s2 FAIL: G4 3,584 / 3,840 differ on every rank = tiles 1..14, tile 0 exact; first mismatch at element 256
RED SR3_ROOT (K64_RED_V 3) s2 FAIL: results X (checker aborts on X) 28 classes differ; 5,360 chase violations per group

4.3 G6 under K64

TP5MON at the end of S1_ROOT: g6_overlap_issues=10 refused_windows=9 (parent: 19 / 0), captured=20 lost=0. The TP-5 credit rule (eagle4_drafter.sv:6774-6794 in the parent, G6_HOLD_CAP 2) refuses an overlapped G6 issue while two tiles are in flight; the shipped G6_REISSUE path re-issues them. G6 is therefore 885 cycles instead of the law's 420 + overhead. Results are exact. The hold size is not touched here.

4.4 Child items (md3 split arm: root + level-1 lanes 0..3 + level-2 lanes 0..3)

Stimulus stim_s5 of the regression (10 items: KV_LOAD + 9 compute items), same world. Evidence: k64_evidence/checks/CHK_F1_S5.txt, CHK_S1_S5.txt, CHK_S2_S5.txt, S5_ARMS_SUMMARY.txt.

arm items ended on all 4 ranks checker chase violations busy: root / L1 child / L2 child
F1_S5 K64_V 1, final source (xsim.log 92b76c70) 10 of 10 PASS: 0 differing on all 17 streams x 4 ranks (g1_x 0 / 92,160, G4 0 / 34,560, G6 sum 0 / 46,080, G7g / G7u 0 / 36,864, r5 0 / 36,864 per rank) 0 10,430 / 11,102 / 11,108
S1_S5 K64_V 1, source s2 (xsim.log de17a6a1) 10 of 10 PASS, the same counts; every tap line equal to F1_S5 0 10,430 / 11,102 / 11,108
S2_S5 K64_V 2 (xsim.log 45960f77) 10 of 10 16 streams 0 differing; pn(g7u_in) monitor row 22,892 / 46,080 (the row of section 4.2) 0 10,350 / 11,022 / 11,028
SA1_S5 AFC + K64_V 1 (source s2 + junior's patch; xsim.log 4d4a5dca) 10 of 10 every data tap line (D9R 603, TAPG4 135, TAPG5 9, TAPG6 180, KVWR 580, TAPR4 4,608 per rank) equal to S1_S5; D9V differs (AFC port monitor) 0 K64, 0 LCCHASE 8,102 / 8,774 / 8,780

The parent's child items are 15,757 (L1) and 15,763 (L2) (XSIM_ATT_20261116.md section 3): the cut is the same 4,655 cycles on every item (6,983 with the fast clock).

4.5 Limits (gate shape)

5. OOC synthesis, place and route [M]

Harness poc/k64_ooc_top.sv (665fb544): the ATT u_array as the drafter instantiates it (LANES 256, ROLE 1, K 10,240, LUTG A16, per-lane zero bank, RET_SER R 8, PRE_BASE, TILE_PIPE, BOOT_WR_PIPE 2, 3 groups) with a BRAM seam, registered control, boot bus and result consumer; one clock; soft per-SLR pblocks (group g and its cache to SLR g). Part xcv80-lsva4737-2MHP-e-S, Vivado 2025.1, maxThreads 8.

5.1 Synthesis

run host LUT FF BRAM tiles URAM288 DSP58 lane type
parent fork (P0_parent_synth) buildz01 430,376 287,680 783 768 258 256 x lutgemm_lane_w2
this fork, K64 0 (A0b_base_p15) buildy03 430,376 287,680 783 768 258 256 x lutgemm_lane_w2
this fork, K64 1 (A1c_k64_p15) buildy02 788,346 493,935 828 768 258 256 x lutgemm_lane_w2q, 48 cache RAMB36, 768 URAM in g_e64

5.2 Place and route

run config clock routed WNS / WHS routing errors SLICE total SLICE SLR0 / SLR1 / SLR2 (% of the die) LUT FF BRAM tiles URAM SLR0 / 1 / 2 SLL 1-0 / 2-1 route.dcp
A0b_base_p15 K64 0 15.151 ns +1.832 / +0.011 0 66,402 21,662 / 23,544 / 21,196 (19.25 / 22.96 / 19.86) 401,605 265,600 783 258 / 258 / 252 2,122 / 2,234 203a63f1
A1c_k64_p15 K64 1 15.151 ns +0.689 / +0.010 0 126,150 41,846 / 43,365 / 40,939 (37.19 / 42.30 / 38.36) 761,295 457,005 828 258 / 258 / 252 1,249 / 1,218 c17fb2eb
F1c_k64_lc_p7575 K64 1 + LCACHE + RST_PIPE 2 (the fast-clock array; junior's array edit applied) 7.575 ns +0.263 / +0.010 0 128,232 41,937 / 45,423 / 40,872 (37.27 / 44.30 / 38.30) 761,776 459,143 813 258 / 258 / 252 1,324 / 1,228 3dc6f65a

6. Composition with the array fast clock (junior's lever)

7. Host weight packer and loader

8. Image recipe (for the coordinator; not derived, not launched)

  1. Base = junior's current Qwen ATT stage (the 0x13A0 / 0x139C class, or their ATT AFC stage). Apply k64_evidence/rtl/QGPTQ2_ATT_TO_K64.patch (62ad924a, patch -p2 inside the staged src; it reproduces this fork's six files byte for byte from the parent, and applies before or after junior's AFC patch). Add systolic/lutgemm_lane_w2q.sv to the add_files list and to MD5LIST / the CERT pins of eagle4_drafter.sv, eagle4_config.svh, gemv_stage_array_scaled_slr.sv, weight_bank_uram.sv, weight_bank_multistage.sv.
  2. Strap: bake K64_V 0 -> 1 (count-asserted, one line in eagle4_config.svh). It is a header localparam like LUTG, so the packaging parameter list does not change.
  3. Floorplan: the K64 array needs about +20K SLICE per lane group (section 5.2) and the 0x13A0 shape has about 42K free in SLR0 + SLR1 together, so lane groups must move into SLR2 (section 1, full-chip fit). Put u_array/g_k64.g_kc[g].* in the same pblock as lane group g (poc/k64_pblocks.tcl is the OOC form). Give the first build a placement probe before calling it a fit.
  4. Report-only rows to add: staged header K64_V 1; netlist u_array/g_grp[*].g_lane[*].g_dual.g_lutg.u_lane of type lutgemm_lane_w2q x 256 and zero lutgemm_lane_w2; u_array/g_k64.g_kc[0..2] RAMB (match RAMB*) = 16 per group; URAM288 = 768 in g_e64 with zero [Synth 8-12186] lines in the synth log (a non-zero count means the bank fell to BRAM); DSP58 unchanged.
  5. Host: run check_k64_boot_pairing.py on the image's boot ops before the first boot. No kit change.
  6. Fly it beside a K64_V 0 sibling, as the build rules ask. Build id: pick with qg_id_check.py at derive time; none reserved here.

9. Files, evidence, housekeeping

file md5
verilog/forks/qwen_att_k64_20261117/ (fork; not committed in the verilog submodule) six source md5s in section 3; FORK_NOTE_K64.md 1fd5db06
.../tb/k64/tb_k64_array.sv, run_k64_unit.sh aaf3897f, c8d703f7
.../poc/k64_ooc_top.sv, k64_synth.tcl, k64_impl.tcl, k64_pblocks.tcl, run_k64_flow.sh 665fb544, 564b1927, 688deb2b, 99c1737e, ba359ea1
reports/results/QWEN_GPTQ2_20261116/k64_evidence/rtl/QGPTQ2_ATT_TO_K64.patch 62ad924a
k64_evidence/unit/UNIT4_K64_ARRAY_GATE.txt in tree
k64_evidence/checks/CHK_*.txt, TAPCMP_*_vs_parent.txt, XSIM_SNAPSHOTS_COMPILE_INFO.txt in tree
k64_evidence/ooc/ in tree
k64_evidence/tools/k64_pull_check.sh, k64_tapcmp.py in tree
k64_evidence/rtl/fork_extras/ (copies of the fork's TB, OOC harness, the new lane and the fork note), FORK_MD5.txt, PARENT_MD5.txt, STAGED_s4_MD5.txt, junior's patch as fetched in tree: with the patch they rebuild the fork from the parent without the submodule
scripts/4c1/qgptq2/check_k64_boot_pairing.py 983ffbeb

Scratch: box1 /mnt/hdd_data/eagle4_scratch/qwen_att_k64_20261117/ (staged sources src_k64_s2, src_k64_s4 = final, src_k64_afc2/4, src_parent, run snapshots); buildy01 the xsim world; buildy02 / buildy03 / buildz00 / buildz01 /shared/scratch/qwen_att_k64_20261117/.

Housekeeping: