fpga-archive-reports-qwen · EAGLE4 FPGA archive

Source reports/results/QWEN_GPTQ2_20261116/WHY_SLOWER_LAYERS_PHASES_20261117.md · source md5 e1c699020123aaf6716b8f937541a250 · archived 2026-11-18

Download PDFOpen PDFRaw markdown

Why the Qwen GPTQ-2 drafter is slower on the 7-card image: layer weights and per-engine cycles (box1-two-v2, 2026-11-17)

User asked (verbatim): "Show work more by layer weights listing comparing the models ..... why slower! Show rpt files per engine time". Tags: [M] measured (artifact named), [D] arithmetic on [M]/config, [R] read from config/doc.

0. What the .rpt files can and cannot say

Vivado .rpt files give clock period, slack and area. They contain no per-engine cycle counts (reports/results/PER_PHASE_CYCLES_BY_IMAGE_20261110.md:8). Per-engine time comes from:

1. Layer weights (parameters per forward) [R]/[D]

Llama: shipping EAGLE4 3B drafter (CLAUDE.md phase table; G9 8192->6144 on every shipping image). BitNet Qwen: user drafter step21491, body = Llama shape (memory project_qwen38_27b_drafter_track: 24x128, H 3072, MLP 8192), ternary, per-tensor scale; Qwen vocab. GPTQ-2 Qwen: ~/haochen/models/gptq2bit_lmhead2bit_20261102/W2A16/README.md (model.safetensors md5 e41d1646).

layer Llama 2-bit (in x out) params GPTQ-2 Qwen (in x out) params ratio
fc (host/GPU, not card) 9216 x 3072 28.31 M 15360 x 5120 78.64 M 2.78x
QKV (G4) 6144 x 9216 56.62 M 10240 x 15360 (q,k,v each 10240x5120) 157.29 M 2.78x
o (G6) 3072 x 3072 9.44 M 5120 x 5120 26.21 M 2.78x
gate (G7g) 3072 x 8192 25.17 M 5120 x 16384 83.89 M 3.33x
up (G7u) 3072 x 8192 25.17 M 5120 x 16384 83.89 M 3.33x
down (G9) 8192 x 6144 50.33 M 16384 x 5120 83.89 M 1.67x
card body total 166.73 M 435.16 M 2.61x
lm_head (G8 exact) 3072 x 128,256 fp16 (HBM 788 MB) 394.0 M 5120 x 248,320 2-bit (340.2 MB) 1,271.4 M 3.23x
vocab scanned by TREE/G12 128,256 248,320 1.94x
heads x dim 24 x 128 40 x 128 1.67x
quant extras per-group scales g=128 asym: per-(group,column) zero + bf16 scale (zeros ~50% 1 / ~50% 2, DESIGN_ARCH_SHIFT.md §0.3)

BitNet Qwen body = Llama column (H 3072, MLP 8192) but lm_head 3072 x 248,320 = 762.8 M.

2. Per-engine cycles per item: Llama record (silicon) vs GPTQ-2 Qwen (xsim)

card engine Llama 7c per item [M]->[D] Qwen GPTQ-2 per item [M xsim] ratio
ATT G1 (+R1) 456 713 1.56x
ATT G4 QKV 1,421 4,825 3.40x
ATT G5 attention 1,919 (ctx 566) 612 (ctx 50) / ~990 (ctx 112) not comparable (ctx)
ATT G6 o-proj + TP exchange 686 903 ideal; +6,615 real exchange at TP_LAT 360 1.32x ideal
ATT G7g (+R3, pre-MLP norm) 1,397 3,937 2.82x
ATT G7u 557 2,585 4.64x
ATT SiLU 133 262 1.97x
ATT seam/link (EMBG+EXPORT+INGEST) 826 1,248-1,914 1.5-2.3x
ATT busy 7,394 15,085 root / 15,757 child 2.04-2.13x
MLPV G9 down 850 (132 MHz array) 5,133 (66 MHz array) 6.04x
MLPV G11 466 723 1.55x
MLPV GLM 226 24 0.11x
MLPV TREE (G12 scan + top-K) 2,224 6,816 (13 + 6,803) 3.06x
MLPV G8 exact rescore 747 1,896-2,109 2.5-2.8x
MLPV ingest/XCH/out 792 911-926 1.15x
MLPV busy 5,327 15,616 mean 2.93x

Llama per item [D] from SCALING...:53-70 us/tree: G1 145.09, G4 452.12, G5 610.48, G6 218.14, G7G 444.48, G7U 177.22, SiLU 42.32, seam 262.71, ATT busy 2,352.56; G9 270.45, G11 148.27, GLM 71.91, TREE 707.61, G8 237.71, ingest 252.08, MLPV busy 1,695.03. Cross-check: G9 850 and TREE 2,224 match PER_PHASE_CYCLES_BY_IMAGE_20261110.md:78,82.

3. Tree and round [M]

Llama record (d11, 21 items) GPTQ-2 Qwen (d7 W2)
card tree issue->done 2.878 ms (census, 0x13BE+M1361) 7.0 ms (sweep ctx 45); wall 8.5 ms; golden 9-10 ms
per item 137 us [D] ATT 229-239 us + MLPV 235-238 us (xsim; in series per level)
ms per verify step / round 7.187 (961.43 tok/s) 36.8 (110.48 tok/s, levers on) vs GPU-only 21.9
GPU-only tok/s, same prompts 841.27 182.50 (8 prompts) / 296.52 humaneval

4. Reading

  1. The card body has 2.61x more weights (QKV/o 2.78x, MLP gate/up 3.33x) and the array is the same width (L256, K_PAR 16, 66 MHz), so G4/G7 cost 2.8-4.6x more cycles per item: ATT busy 2.0-2.1x, MLPV busy 2.9x.
  2. The MLPV is worse than the weight ratio because the record's 132 MHz array clock is not on the Qwen MLPV (G9 6.0x for 1.67x weights) and the TREE scans 1.94x the vocabulary (3.06x).
  3. Llama's card tree beats the H200's own drafting; Qwen's (7-10 ms per d7 tree + ~5 ms KV-row ingest per round) does not, so the cards add time instead of removing it.
  4. Accept is NOT the cause: humaneval accept GPU 7.185 == silicon 7.185 (same 8 prompts, d7_levers_evidence/serve/*.jsonl).

5. CORRECTION (same day): the Llama record drafter is bitnet_721, not the GPTQ 8815 shape

Record serve log (talon2 runs/acwin_20261116/d6/serve/acw_hx7s7_kc64_t1_*_142839.log): --fast-model-path /home/carrick/fetched_weights/bitnet_721/bf16_ckpt, --bench-data-path .../humaneval/question.jsonl --max-requests-num 80 --max-new-tokens 256, d11 W2 V20 fa3. Section 1's Llama column used the 8815 GPTQ shapes; the record drafter has 32 x 96 heads and down_proj 8192 -> 3072. Tensor listings from the safetensors headers (weight_listing_20261117/hdr.py): weight_listing_20261117/{llama_record_bitnet721_bf16,qwen_bitnet_step21491_bf16ste,qwen_gptq2_W2A16}.tsv, config md5s in CONFIG_MD5.txt.

tensor (card engine) Llama record bitnet_721 [out,in] params Qwen GPTQ-2 [in/16 packed, out] params ratio
q_proj (G4) [3072, 6144] 18.87 M [640, 5120] = 10240 x 5120 52.43 M 2.78x
k_proj (G4) [3072, 6144] 18.87 M [640, 5120] 52.43 M 2.78x
v_proj (G4) [3072, 6144] 18.87 M [640, 5120] 52.43 M 2.78x
o_proj (G6) [3072, 3072] 9.44 M [320, 5120] 26.21 M 2.78x
gate_proj (G7g) [8192, 3072] 25.17 M [320, 16384] 83.89 M 3.33x
up_proj (G7u) [8192, 3072] 25.17 M [320, 16384] 83.89 M 3.33x
down_proj (G9) [3072, 8192] 25.17 M [1024, 5120] 83.89 M 3.33x
card body 141.56 M 435.16 M 3.07x
card body at 2 bit/weight 35.4 MB 108.8 MB + 6.8 MB scales + zeros 3.1x
lm_head (G8) [128256, 3072] BF16 394.0 M (788 MB) [320, 248320] 2-bit + scales 1,271.4 M (337.7 MB) 3.23x params
fc (host) [3072, 9216] 28.31 M [960, 5120] = 15360 x 5120 78.64 M 2.78x
embed_tokens (host) [128256, 3072] 394.0 M [248320, 5120] 1,271.4 M 3.23x
heads x head_dim (G5) 32 x 96 40 x 128 1.67x
file size 1,915,799,320 B 3,030,701,984 B

Qwen BitNet step21491 (the old 250.27 drafter): body identical in shape to the Llama record (q/k/v [3072,6144], o [3072,3072], gate/up [8192,3072], down [3072,8192]; 24 x 128 heads) plus attn_sub_norm/ffn_sub_norm; lm_head [128256,3072] over a d2t-mapped draft vocab; embed [248320,3072]. Corrected reading: card body work is 3.07x the record's (not 2.61x); G9 weights are 3.33x (not 1.67x), so G9's 6.0x cycles = 3.33x weights x ~2x from the missing 132 MHz MLPV array clock [D].

6. Old Qwen (BitNet step21491, md5 32f20d0d) vs new Qwen (GPTQ-2 W2A16, md5 e41d1646)

Weights: weight_listing_20261117/qwen_bitnet_step21491_bf16ste.tsv vs qwen_gptq2_W2A16.tsv. GPU accept/tok-s: QWEN27B_DRAFTER_20261020/OPEN_RELEASE_DRAFTER_EVAL_20261102.md §0.1/§0.2 (same SGLang build, client, prompts, tree; "OURS" = step21491, :121).