Qwen3.8-27B drafters: old vs new weights, per phase

Both lists printed on 2026-11-18 straight from the safetensors headers of the two checkpoint files. Old: Finn's BitNet drafter step21491 (bf16, ternary-trained), file md5 32f20d0d…, 2,692,524,624 B. New: GPTQ 2-bit W2A16 drafter, file md5 e41d1646…, 3,030,701,984 B.
Calculations. "Weights" = logical weight count per phase. A GPTQ qweight is int32 and packs 16 two-bit weights along the input dimension, so its logical count = stored elements × 16; g_idx, qzeros, scales and the vocab maps are metadata and count 0. bf16 tensors count as stored. The N× column = new ÷ old (red = larger, green = smaller). "On-card bytes" = weights × 2 bits ÷ 8 for BOTH drafters (the FPGA holds every body weight as 2-bit), so on-card size grows exactly like the weight count. The "file bytes" columns are what the checkpoint files store: the old file stores its ternary-trained weights as bf16 (16 bits), the new file stores packed 2-bit, so new bytes can be smaller even where the model is larger. On the FPGA both drafters run as 2-bit weights, so on-card weight storage and GEMV time follow the weights column. Config: old hidden 3,072, 24 heads × 128, MLP 8,192; new hidden 5,120, 40 heads × 128, MLP 16,384.

Model size in bytes

Old = Finn's Qwen3.8-27B BitNet drafter, step 21491 (model.safetensors md5 32f20d0d…). New = GPTQ 2-bit W2A16 drafter (model.safetensors md5 e41d1646…). Names are the raw tensor names in each file.

old tensor (Finn BitNet step21491)old file bytesnew tensor(s) (GPTQ-2 W2A16)new file bytesnew ÷ oldold bytes at 2 bit/weightnew bytes at 2 bit/weightnew ÷ old
embed_tokens.weight1,525,678,080embed_tokens.weight2,542,796,8001.67×190,709,760317,849,6001.67×
fc.weight94,371,840fc.g_idx
fc.qweight
fc.qzeros
fc.scales
21,104,6400.22×11,796,48019,660,8001.67×
fc_norm.0.weight10,240fc_norm.0.weight10,2401.00×1,2801,2801.00×
fc_norm.1.weight10,240fc_norm.1.weight10,2401.00×1,2801,2801.00×
fc_norm.2.weight10,240fc_norm.2.weight10,2401.00×1,2801,2801.00×
midlayer.hidden_norm.weight6,144layers.0.hidden_norm.weight10,2401.67×7681,2801.67×
midlayer.input_layernorm.weight6,144layers.0.input_layernorm.weight10,2401.67×7681,2801.67×
midlayer.self_attn.q_proj.weight37,748,736layers.0.self_attn.q_proj.g_idx
layers.0.self_attn.q_proj.qweight
layers.0.self_attn.q_proj.qzeros
layers.0.self_attn.q_proj.scales
14,069,7600.37×4,718,59213,107,2002.78×
midlayer.self_attn.k_proj.weight37,748,736layers.0.self_attn.k_proj.g_idx
layers.0.self_attn.k_proj.qweight
layers.0.self_attn.k_proj.qzeros
layers.0.self_attn.k_proj.scales
14,069,7600.37×4,718,59213,107,2002.78×
midlayer.self_attn.v_proj.weight37,748,736layers.0.self_attn.v_proj.g_idx
layers.0.self_attn.v_proj.qweight
layers.0.self_attn.v_proj.qzeros
layers.0.self_attn.v_proj.scales
14,069,7600.37×4,718,59213,107,2002.78×
midlayer.self_attn.attn_sub_norm.weight6,144(none)0—7680—
midlayer.self_attn.o_proj.weight18,874,368layers.0.self_attn.o_proj.g_idx
layers.0.self_attn.o_proj.qweight
layers.0.self_attn.o_proj.qzeros
layers.0.self_attn.o_proj.scales
7,034,8800.37×2,359,2966,553,6002.78×
midlayer.post_attention_layernorm.weight6,144layers.0.post_attention_layernorm.weight10,2401.67×7681,2801.67×
midlayer.mlp.gate_proj.weight50,331,648layers.0.mlp.gate_proj.g_idx
layers.0.mlp.gate_proj.qweight
layers.0.mlp.gate_proj.qzeros
layers.0.mlp.gate_proj.scales
22,466,5600.45×6,291,45620,971,5203.33×
midlayer.mlp.up_proj.weight50,331,648layers.0.mlp.up_proj.g_idx
layers.0.mlp.up_proj.qweight
layers.0.mlp.up_proj.qzeros
layers.0.mlp.up_proj.scales
22,466,5600.45×6,291,45620,971,5203.33×
midlayer.mlp.ffn_sub_norm.weight16,384(none)0—2,0480—
midlayer.mlp.down_proj.weight50,331,648layers.0.mlp.down_proj.g_idx
layers.0.mlp.down_proj.qweight
layers.0.mlp.down_proj.qzeros
layers.0.mlp.down_proj.scales
22,511,6160.45×6,291,45620,971,5203.33×
norm.weight6,144norm.weight10,2401.67×7681,2801.67×
lm_head.weight788,004,864lm_head.g_idx
lm_head.qweight
lm_head.qzeros
lm_head.scales
340,218,8800.43×98,500,608317,849,6003.23×
(none)0efficient_lm_head.g_idx
efficient_lm_head.qweight
efficient_lm_head.qzeros
efficient_lm_head.scales
efficient_lm_head_down_proj.weight
9,816,192—08,110,080—
d2t1,026,048(none)0—00—
t2d248,320(none)0—00—
TOTAL2,692,522,4963,030,697,0881.13×336,406,016772,268,8002.30×

File bytes = what each safetensors file stores (old: bf16; new: packed 2-bit + scales/qzeros/g_idx). Bytes at 2 bit/weight = weights × 2 ÷ 8 (a GPTQ qweight int32 holds 16 weights), the format the FPGA holds the drafter body in.

Per-phase summary

phaseold weightsnew weightsN×on-card bytes old (2 bit/weight)on-card bytes new (2 bit/weight)N×file bytes oldfile bytes newN× file
Token embedding (host)762,839,0401,271,398,4001.67×190,709,760317,849,6001.67×1,525,678,0802,542,796,8001.67×
fc input norms (3 target hiddens)15,36015,3601.00×3,8403,8401.00×30,72030,7201.00×
fc: fuse 3 target hiddens to H47,185,92078,643,2001.67×11,796,48019,660,8001.67×94,371,84021,104,6400.22×
G1 RMSNorm (input + hidden)6,14410,2401.67×1,5362,5601.67×12,28820,4801.67×
G4 Q/K/V projection (ATT)56,623,104157,286,4002.78×14,155,77639,321,6002.78×113,246,20842,209,2800.37×
G5 attention sub-norm (BitNet only)3,0720removed7680removed6,1440removed
G6 o-projection (ATT)9,437,18426,214,4002.78×2,359,2966,553,6002.78×18,874,3687,034,8800.37×
R3 post-attention norm3,0725,1201.67×7681,2801.67×6,14410,2401.67×
G7 gate + up (ATT)50,331,648167,772,1603.33×12,582,91241,943,0403.33×100,663,29644,933,1200.45×
G7 FFN sub-norm (BitNet only)8,1920removed2,0480removed16,3840removed
G9 down projection (MLPV)25,165,82483,886,0803.33×6,291,45620,971,5203.33×50,331,64822,511,6160.45×
G11 final norm (MLPV)3,0725,1201.67×7681,2801.67×6,14410,2401.67×
G12 cheap low-rank lm-head (MLPV)032,440,320new only08,110,080new only09,816,192new only
lm-head (vocabulary projection)394,002,4321,271,398,4003.23×98,500,608317,849,6003.23×788,004,864340,218,8800.43×
draft-to-target vocab maps00—00—1,274,3680removed
Drafter body on the cards (G4 + G6 + G7 + G9)141,557,760435,159,0403.07×35,389,440108,789,7603.07×

Every tensor, side by side

Old: Finn's BitNet step21491 (bf16)

tensor (raw name)dtypeshapebytes
Token embedding (host)
embed_tokens.weightBF16[248320, 3072]1,525,678,080
fc input norms (3 target hiddens)
fc_norm.0.weightBF16[5120]10,240
fc_norm.1.weightBF16[5120]10,240
fc_norm.2.weightBF16[5120]10,240
fc: fuse 3 target hiddens to H
fc.weightBF16[3072, 15360]94,371,840
G1 RMSNorm (input + hidden)
midlayer.hidden_norm.weightBF16[3072]6,144
midlayer.input_layernorm.weightBF16[3072]6,144
G4 Q/K/V projection (ATT)
midlayer.self_attn.k_proj.weightBF16[3072, 6144]37,748,736
midlayer.self_attn.q_proj.weightBF16[3072, 6144]37,748,736
midlayer.self_attn.v_proj.weightBF16[3072, 6144]37,748,736
G5 attention sub-norm (BitNet only)
midlayer.self_attn.attn_sub_norm.weightBF16[3072]6,144
G6 o-projection (ATT)
midlayer.self_attn.o_proj.weightBF16[3072, 3072]18,874,368
R3 post-attention norm
midlayer.post_attention_layernorm.weightBF16[3072]6,144
G7 gate + up (ATT)
midlayer.mlp.gate_proj.weightBF16[8192, 3072]50,331,648
midlayer.mlp.up_proj.weightBF16[8192, 3072]50,331,648
G7 FFN sub-norm (BitNet only)
midlayer.mlp.ffn_sub_norm.weightBF16[8192]16,384
G9 down projection (MLPV)
midlayer.mlp.down_proj.weightBF16[3072, 8192]50,331,648
G11 final norm (MLPV)
norm.weightBF16[3072]6,144
G12 cheap low-rank lm-head (MLPV)
(no tensor)
lm-head (vocabulary projection)
lm_head.weightBF16[128256, 3072]788,004,864
draft-to-target vocab maps
d2tI64[128256]1,026,048
t2dBOOL[248320]248,320
file total (tensor bytes)2,692,522,496

New: GPTQ-2 W2A16

tensor (raw name)dtypeshapebytes
Token embedding (host)
embed_tokens.weightBF16[248320, 5120]2,542,796,800
fc input norms (3 target hiddens)
fc_norm.0.weightBF16[5120]10,240
fc_norm.1.weightBF16[5120]10,240
fc_norm.2.weightBF16[5120]10,240
fc: fuse 3 target hiddens to H
fc.g_idxI32[15360]61,440
fc.qweightI32[960, 5120]19,660,800
fc.qzerosI32[120, 320]153,600
fc.scalesBF16[120, 5120]1,228,800
G1 RMSNorm (input + hidden)
layers.0.hidden_norm.weightBF16[5120]10,240
layers.0.input_layernorm.weightBF16[5120]10,240
G4 Q/K/V projection (ATT)
layers.0.self_attn.k_proj.g_idxI32[10240]40,960
layers.0.self_attn.k_proj.qweightI32[640, 5120]13,107,200
layers.0.self_attn.k_proj.qzerosI32[80, 320]102,400
layers.0.self_attn.k_proj.scalesBF16[80, 5120]819,200
layers.0.self_attn.q_proj.g_idxI32[10240]40,960
layers.0.self_attn.q_proj.qweightI32[640, 5120]13,107,200
layers.0.self_attn.q_proj.qzerosI32[80, 320]102,400
layers.0.self_attn.q_proj.scalesBF16[80, 5120]819,200
layers.0.self_attn.v_proj.g_idxI32[10240]40,960
layers.0.self_attn.v_proj.qweightI32[640, 5120]13,107,200
layers.0.self_attn.v_proj.qzerosI32[80, 320]102,400
layers.0.self_attn.v_proj.scalesBF16[80, 5120]819,200
G5 attention sub-norm (BitNet only)
(no tensor)
G6 o-projection (ATT)
layers.0.self_attn.o_proj.g_idxI32[5120]20,480
layers.0.self_attn.o_proj.qweightI32[320, 5120]6,553,600
layers.0.self_attn.o_proj.qzerosI32[40, 320]51,200
layers.0.self_attn.o_proj.scalesBF16[40, 5120]409,600
R3 post-attention norm
layers.0.post_attention_layernorm.weightBF16[5120]10,240
G7 gate + up (ATT)
layers.0.mlp.gate_proj.g_idxI32[5120]20,480
layers.0.mlp.gate_proj.qweightI32[320, 16384]20,971,520
layers.0.mlp.gate_proj.qzerosI32[40, 1024]163,840
layers.0.mlp.gate_proj.scalesBF16[40, 16384]1,310,720
layers.0.mlp.up_proj.g_idxI32[5120]20,480
layers.0.mlp.up_proj.qweightI32[320, 16384]20,971,520
layers.0.mlp.up_proj.qzerosI32[40, 1024]163,840
layers.0.mlp.up_proj.scalesBF16[40, 16384]1,310,720
G7 FFN sub-norm (BitNet only)
(no tensor)
G9 down projection (MLPV)
layers.0.mlp.down_proj.g_idxI32[16384]65,536
layers.0.mlp.down_proj.qweightI32[1024, 5120]20,971,520
layers.0.mlp.down_proj.qzerosI32[128, 320]163,840
layers.0.mlp.down_proj.scalesBF16[128, 5120]1,310,720
G11 final norm (MLPV)
norm.weightBF16[5120]10,240
G12 cheap low-rank lm-head (MLPV)
efficient_lm_head.g_idxI32[128]512
efficient_lm_head.qweightI32[8, 248320]7,946,240
efficient_lm_head.qzerosI32[1, 15520]62,080
efficient_lm_head.scalesBF16[1, 248320]496,640
efficient_lm_head_down_proj.weightBF16[128, 5120]1,310,720
lm-head (vocabulary projection)
lm_head.g_idxI32[5120]20,480
lm_head.qweightI32[320, 248320]317,849,600
lm_head.qzerosI32[40, 15520]2,483,200
lm_head.scalesBF16[40, 248320]19,865,600
draft-to-target vocab maps
(no tensor)
file total (tensor bytes)3,030,697,088

Source files: old_step21491_bf16ste.tsv and new_gptq2_W2A16.tsv printed by hdr.py (safetensors header reader). Downloads: old_step21491_bf16ste.tsv · new_gptq2_W2A16.tsv · hdr.py.