# Why the Qwen GPTQ-2 drafter is slower on the 7-card image: layer weights and per-engine cycles (box1-two-v2, 2026-11-17)

User asked (verbatim): "Show work more by layer weights listing comparing the models ..... why slower! Show rpt files per engine time".
Tags: [M] measured (artifact named), [D] arithmetic on [M]/config, [R] read from config/doc.

## 0. What the .rpt files can and cannot say
Vivado `.rpt` files give clock period, slack and area. They contain no per-engine cycle counts
(`reports/results/PER_PHASE_CYCLES_BY_IMAGE_20261110.md:8`). Per-engine time comes from:
- Llama 7-card: the silicon `dbg_observe` census (median of 20 passes): `DRAFTER_SPEED_RESEARCH_20261107/tree4_rca_20261114/m1361_census_md11.json`, tabled in `SCALING_4C_VS_7C_PHASE_RANK_20261116.md:51-71` (us per d11 tree, 21 items). Per item [D] = us/21 x 66.001 MHz.
- Qwen GPTQ-2: **no silicon census exists yet**. Per-item cycles are xsim of the image sources, `D7_TREE_LEVERS_20261117.md:184-198` (sources `XSIM_ATT_20261116.md` §3 and `XSIM_MLPV_20261116.md` §5-6). Real 4-way G6 exchange measured +6,615 cycles at TP_LAT 360 (`XSIM_ATT_20261116.md:9`), not in the per-item table.
- Clocks: Llama ATT COMBO3 66 MHz; MLPV M1361 66 MHz + 132 MHz array (ARRAY_FAST_CLK 1) (`SCALING...:40-45`). Qwen MLPV 0x139D: 66 MHz, ARRAY_FAST_CLK not on (lever L1, `D7_TREE_LEVERS_20261117.md:270`). 0x139D FINAL timing WNS 0.030 / WHS 0.022 (`ADJUDICATION_0xE4B7139D_20261116.md:35`).

## 1. Layer weights (parameters per forward) [R]/[D]
Llama: shipping EAGLE4 3B drafter (CLAUDE.md phase table; G9 8192->6144 on every shipping image).
BitNet Qwen: user drafter step21491, body = Llama shape (memory `project_qwen38_27b_drafter_track`: 24x128, H 3072, MLP 8192), ternary, per-tensor scale; Qwen vocab.
GPTQ-2 Qwen: `~/haochen/models/gptq2bit_lmhead2bit_20261102/W2A16/README.md` (model.safetensors md5 e41d1646).

| layer | Llama 2-bit (in x out) | params | GPTQ-2 Qwen (in x out) | params | ratio |
|---|---|---:|---|---:|---:|
| fc (host/GPU, not card) | 9216 x 3072 | 28.31 M | 15360 x 5120 | 78.64 M | 2.78x |
| QKV (G4) | 6144 x 9216 | 56.62 M | 10240 x 15360 (q,k,v each 10240x5120) | 157.29 M | 2.78x |
| o (G6) | 3072 x 3072 | 9.44 M | 5120 x 5120 | 26.21 M | 2.78x |
| gate (G7g) | 3072 x 8192 | 25.17 M | 5120 x 16384 | 83.89 M | 3.33x |
| up (G7u) | 3072 x 8192 | 25.17 M | 5120 x 16384 | 83.89 M | 3.33x |
| down (G9) | 8192 x 6144 | 50.33 M | 16384 x 5120 | 83.89 M | 1.67x |
| **card body total** | | **166.73 M** | | **435.16 M** | **2.61x** |
| lm_head (G8 exact) | 3072 x 128,256 fp16 (HBM 788 MB) | 394.0 M | 5120 x 248,320 2-bit (340.2 MB) | 1,271.4 M | 3.23x |
| vocab scanned by TREE/G12 | 128,256 | | 248,320 | | 1.94x |
| heads x dim | 24 x 128 | | 40 x 128 | | 1.67x |
| quant extras | per-group scales | | g=128 asym: per-(group,column) zero + bf16 scale (zeros ~50% 1 / ~50% 2, `DESIGN_ARCH_SHIFT.md` §0.3) | | |

BitNet Qwen body = Llama column (H 3072, MLP 8192) but lm_head 3072 x 248,320 = 762.8 M.

## 2. Per-engine cycles per item: Llama record (silicon) vs GPTQ-2 Qwen (xsim)
| card | engine | Llama 7c per item [M]->[D] | Qwen GPTQ-2 per item [M xsim] | ratio |
|---|---|---:|---:|---:|
| ATT | G1 (+R1) | 456 | 713 | 1.56x |
| ATT | G4 QKV | 1,421 | 4,825 | 3.40x |
| ATT | G5 attention | 1,919 (ctx 566) | 612 (ctx 50) / ~990 (ctx 112) | not comparable (ctx) |
| ATT | G6 o-proj + TP exchange | 686 | 903 ideal; +6,615 real exchange at TP_LAT 360 | 1.32x ideal |
| ATT | G7g (+R3, pre-MLP norm) | 1,397 | 3,937 | 2.82x |
| ATT | G7u | 557 | 2,585 | 4.64x |
| ATT | SiLU | 133 | 262 | 1.97x |
| ATT | seam/link (EMBG+EXPORT+INGEST) | 826 | 1,248-1,914 | 1.5-2.3x |
| ATT | **busy** | **7,394** | **15,085 root / 15,757 child** | **2.04-2.13x** |
| MLPV | G9 down | 850 (132 MHz array) | 5,133 (66 MHz array) | 6.04x |
| MLPV | G11 | 466 | 723 | 1.55x |
| MLPV | GLM | 226 | 24 | 0.11x |
| MLPV | TREE (G12 scan + top-K) | 2,224 | 6,816 (13 + 6,803) | 3.06x |
| MLPV | G8 exact rescore | 747 | 1,896-2,109 | 2.5-2.8x |
| MLPV | ingest/XCH/out | 792 | 911-926 | 1.15x |
| MLPV | **busy** | **5,327** | **15,616 mean** | **2.93x** |

Llama per item [D] from `SCALING...:53-70` us/tree: G1 145.09, G4 452.12, G5 610.48, G6 218.14, G7G 444.48, G7U 177.22, SiLU 42.32, seam 262.71, ATT busy 2,352.56; G9 270.45, G11 148.27, GLM 71.91, TREE 707.61, G8 237.71, ingest 252.08, MLPV busy 1,695.03. Cross-check: G9 850 and TREE 2,224 match `PER_PHASE_CYCLES_BY_IMAGE_20261110.md:78,82`.

## 3. Tree and round [M]
| | Llama record (d11, 21 items) | GPTQ-2 Qwen (d7 W2) |
|---|---:|---:|
| card tree issue->done | 2.878 ms (census, 0x13BE+M1361) | 7.0 ms (sweep ctx 45); wall 8.5 ms; golden 9-10 ms |
| per item | 137 us [D] | ATT 229-239 us + MLPV 235-238 us (xsim; in series per level) |
| ms per verify step / round | 7.187 (961.43 tok/s) | 36.8 (110.48 tok/s, levers on) vs GPU-only 21.9 |
| GPU-only tok/s, same prompts | 841.27 | 182.50 (8 prompts) / 296.52 humaneval |

## 4. Reading
1. The card body has 2.61x more weights (QKV/o 2.78x, MLP gate/up 3.33x) and the array is the same width (L256, K_PAR 16, 66 MHz), so G4/G7 cost 2.8-4.6x more cycles per item: ATT busy 2.0-2.1x, MLPV busy 2.9x.
2. The MLPV is worse than the weight ratio because the record's 132 MHz array clock is not on the Qwen MLPV (G9 6.0x for 1.67x weights) and the TREE scans 1.94x the vocabulary (3.06x).
3. Llama's card tree beats the H200's own drafting; Qwen's (7-10 ms per d7 tree + ~5 ms KV-row ingest per round) does not, so the cards add time instead of removing it.
4. Accept is NOT the cause: humaneval accept GPU 7.185 == silicon 7.185 (same 8 prompts, `d7_levers_evidence/serve/*.jsonl`).

## 5. CORRECTION (same day): the Llama record drafter is bitnet_721, not the GPTQ 8815 shape
Record serve log (talon2 `runs/acwin_20261116/d6/serve/acw_hx7s7_kc64_t1_*_142839.log`): `--fast-model-path /home/carrick/fetched_weights/bitnet_721/bf16_ckpt`,
`--bench-data-path .../humaneval/question.jsonl --max-requests-num 80 --max-new-tokens 256`, d11 W2 V20 fa3. Section 1's Llama column used the 8815 GPTQ shapes;
the record drafter has 32 x 96 heads and down_proj 8192 -> 3072. Tensor listings from the safetensors headers (`weight_listing_20261117/hdr.py`):
`weight_listing_20261117/{llama_record_bitnet721_bf16,qwen_bitnet_step21491_bf16ste,qwen_gptq2_W2A16}.tsv`, config md5s in `CONFIG_MD5.txt`.

| tensor (card engine) | Llama record bitnet_721 [out,in] | params | Qwen GPTQ-2 [in/16 packed, out] | params | ratio |
|---|---|---:|---|---:|---:|
| q_proj (G4) | [3072, 6144] | 18.87 M | [640, 5120] = 10240 x 5120 | 52.43 M | 2.78x |
| k_proj (G4) | [3072, 6144] | 18.87 M | [640, 5120] | 52.43 M | 2.78x |
| v_proj (G4) | [3072, 6144] | 18.87 M | [640, 5120] | 52.43 M | 2.78x |
| o_proj (G6) | [3072, 3072] | 9.44 M | [320, 5120] | 26.21 M | 2.78x |
| gate_proj (G7g) | [8192, 3072] | 25.17 M | [320, 16384] | 83.89 M | 3.33x |
| up_proj (G7u) | [8192, 3072] | 25.17 M | [320, 16384] | 83.89 M | 3.33x |
| down_proj (G9) | [3072, 8192] | 25.17 M | [1024, 5120] | 83.89 M | 3.33x |
| **card body** | | **141.56 M** | | **435.16 M** | **3.07x** |
| card body at 2 bit/weight | | 35.4 MB | | 108.8 MB + 6.8 MB scales + zeros | 3.1x |
| lm_head (G8) | [128256, 3072] BF16 | 394.0 M (788 MB) | [320, 248320] 2-bit + scales | 1,271.4 M (337.7 MB) | 3.23x params |
| fc (host) | [3072, 9216] | 28.31 M | [960, 5120] = 15360 x 5120 | 78.64 M | 2.78x |
| embed_tokens (host) | [128256, 3072] | 394.0 M | [248320, 5120] | 1,271.4 M | 3.23x |
| heads x head_dim (G5) | 32 x 96 | | 40 x 128 | | 1.67x |
| file size | 1,915,799,320 B | | 3,030,701,984 B | | |

Qwen BitNet step21491 (the old 250.27 drafter): body identical in shape to the Llama record (q/k/v [3072,6144], o [3072,3072], gate/up [8192,3072],
down [3072,8192]; 24 x 128 heads) plus attn_sub_norm/ffn_sub_norm; lm_head [128256,3072] over a d2t-mapped draft vocab; embed [248320,3072].
Corrected reading: card body work is **3.07x** the record's (not 2.61x); G9 weights are 3.33x (not 1.67x), so G9's 6.0x cycles = 3.33x weights x ~2x from the missing 132 MHz MLPV array clock [D].

## 6. Old Qwen (BitNet step21491, md5 32f20d0d) vs new Qwen (GPTQ-2 W2A16, md5 e41d1646)
Weights: `weight_listing_20261117/qwen_bitnet_step21491_bf16ste.tsv` vs `qwen_gptq2_W2A16.tsv`. GPU accept/tok-s: `QWEN27B_DRAFTER_20261020/OPEN_RELEASE_DRAFTER_EVAL_20261102.md` §0.1/§0.2 (same SGLang build, client, prompts, tree; "OURS" = step21491, `:121`).
- Body: old q/k/v [3072,6144], o [3072,3072], gate/up [8192,3072], down [3072,8192] = 141.56 M; new 435.16 M = 3.07x. Heads 24x128 vs 40x128.
- Head: old dense lm_head [128256,3072] over a d2t-mapped 128,256 draft vocab; new 2-bit lm_head [248320,5120] + low-rank 128 head over the full 248,320 vocab.
- GPU w2d7V20, 411 prompts: accept 3.365 -> 3.990 (+18.6 %); humaneval 7.200 -> 7.384 (+2.6 %); mt_bench 3.499 -> 4.028. Cooled decode 42 prompts 132.15 -> 187.74 tok/s; humaneval 307.36 -> 360.42.
- 7-card: old 250.27 @ 7.14 humaneval 80 (GPU-only 256.67, 0.975x); new 117.78 humaneval 4 prompts pre-levers (GPU-only same prompts 296.52, 0.40x); new d7 card tree 7.0-10 ms vs old ~2.7 ms (junior, msg 17:xxZ, not re-measured here).
