Architecture rethink model output
Source reports/results/DRAFTER_SPEED_RESEARCH_20261107/arch_rethink_20261117/arch_model.out · source md5 8294e164da4a642528a2bc502958ef62 · EAGLE4 FPGA archive, 2026-11-18
=== LLAMA (today COMBO3 [M]: item 8,240 cyc = 124.8 us; level (2 items) 238.3 us; d11 tree 2,622 us)
T1 16,384 (L512@66 or L256@132) N=6 A batch 8140 cyc level 16280 cyc = 246.7 us arr/sib 1440 tree d11 2590 us step 7.170 tok/s 963.6
T1 16,384 (L512@66 or L256@132) N=6 A+C batch 11441 cyc level 11441 cyc = 173.3 us arr/sib 1440 tree d11 1857 us step 6.792 tok/s 1017.3
T1 16,384 (L512@66 or L256@132) N=6 A+C+fuse+TREE4 batch 9798 cyc level 9798 cyc = 148.4 us arr/sib 1440 tree d11 1588 us step 6.653 tok/s 1038.5
T1 16,384 (L512@66 or L256@132) N=7 A batch 7791 cyc level 15582 cyc = 236.1 us arr/sib 1234 tree d11 2479 us step 7.113 tok/s 971.4
T1 16,384 (L512@66 or L256@132) N=7 A+C batch 10743 cyc level 10743 cyc = 162.8 us arr/sib 1234 tree d11 1746 us step 6.735 tok/s 1026.0
T1 16,384 (L512@66 or L256@132) N=7 A+C+fuse+TREE4 batch 9196 cyc level 9196 cyc = 139.3 us arr/sib 1234 tree d11 1492 us step 6.604 tok/s 1046.3
T2 32,768 (L512@132) N=6 A batch 7420 cyc level 14840 cyc = 224.8 us arr/sib 720 tree d11 2361 us step 7.052 tok/s 979.8
T2 32,768 (L512@132) N=6 A+C batch 10000 cyc level 10000 cyc = 151.5 us arr/sib 720 tree d11 1628 us step 6.674 tok/s 1035.3
T2 32,768 (L512@132) N=6 A+C+fuse+TREE4 batch 8357 cyc level 8357 cyc = 126.6 us arr/sib 720 tree d11 1359 us step 6.535 tok/s 1057.3
T2 32,768 (L512@132) N=7 A batch 7174 cyc level 14348 cyc = 217.4 us arr/sib 617 tree d11 2283 us step 7.012 tok/s 985.4
T2 32,768 (L512@132) N=7 A+C batch 9508 cyc level 9508 cyc = 144.1 us arr/sib 617 tree d11 1549 us step 6.633 tok/s 1041.7
T2 32,768 (L512@132) N=7 A+C+fuse+TREE4 batch 7961 cyc level 7961 cyc = 120.6 us arr/sib 617 tree d11 1296 us step 6.503 tok/s 1062.6
T2.5 41,300 (L512@166) N=6 A batch 7271 cyc level 14543 cyc = 220.3 us arr/sib 571 tree d11 2314 us step 7.028 tok/s 983.2
T2.5 41,300 (L512@166) N=6 A+C batch 9703 cyc level 9703 cyc = 147.0 us arr/sib 571 tree d11 1580 us step 6.649 tok/s 1039.2
T2.5 41,300 (L512@166) N=6 A+C+fuse+TREE4 batch 8060 cyc level 8060 cyc = 122.1 us arr/sib 571 tree d11 1312 us step 6.511 tok/s 1061.3
T2.5 41,300 (L512@166) N=7 A batch 7046 cyc level 14093 cyc = 213.5 us arr/sib 490 tree d11 2242 us step 6.991 tok/s 988.4
T2.5 41,300 (L512@166) N=7 A+C batch 9253 cyc level 9253 cyc = 140.2 us arr/sib 490 tree d11 1509 us step 6.613 tok/s 1045.0
T2.5 41,300 (L512@166) N=7 A+C+fuse+TREE4 batch 7706 cyc level 7706 cyc = 116.8 us arr/sib 490 tree d11 1255 us step 6.482 tok/s 1066.0
sensitivity of the all-reduce term (N=7, T2, A+C+fuse+TREE4):
AR 150 cyc -> level 7163 cyc AR share 9.5 % tree 1163 us tok/s 1073.9
AR 300 cyc -> level 7463 cyc AR share 13.2 % tree 1213 us tok/s 1069.7
AR 549 cyc -> level 7961 cyc AR share 18.6 % tree 1296 us tok/s 1062.6
AR 800 cyc -> level 8463 cyc AR share 23.4 % tree 1379 us tok/s 1055.6
AR 1200 cyc -> level 9263 cyc AR share 30.1 % tree 1513 us tok/s 1044.6
(D) draft-ahead, Llama: tok/s vs hit rate, for today's tree and for a 1,400 us tree
hit 0.0: today 961.4 fast tree 1053.9
hit 0.2: today 1003.3 fast tree 1082.6
hit 0.3: today 1025.7 fast tree 1097.6
hit 0.4: today 1049.1 fast tree 1113.0
hit 0.5: today 1073.5 fast tree 1128.8
hit 0.6: today 1099.2 fast tree 1145.1
hit 0.8: today 1154.3 fast tree 1179.1
hit 1.0: today 1215.2 fast tree 1215.2
=== QWEN (today xsim [M]: ATT 15,757 + MLPV 15,616 per item; d7 W2 = 13 items; serve tree wait 5.82 ms, tree RPC 8.18 ms)
T0 8,192 (L256@66, today) N=6 A batch 18111 cyc level 548.8 us arr/sib 8853 d7 tree 3.57 ms
T0 8,192 (L256@66, today) N=6 A+C batch 31009 cyc level 469.8 us arr/sib 8853 d7 tree 3.09 ms
T0 8,192 (L256@66, today) N=6 A+C+fuse+floor batch 27841 cyc level 421.8 us arr/sib 8853 d7 tree 2.77 ms
T0 8,192 (L256@66, today) N=7 A batch 16404 cyc level 497.1 us arr/sib 7589 d7 tree 3.23 ms
T0 8,192 (L256@66, today) N=7 A+C batch 27597 cyc level 418.1 us arr/sib 7589 d7 tree 2.76 ms
T0 8,192 (L256@66, today) N=7 A+C+fuse+floor batch 24751 cyc level 375.0 us arr/sib 7589 d7 tree 2.47 ms
T1 16,384 (L256@132) N=6 A batch 13684 cyc level 414.7 us arr/sib 4427 d7 tree 2.70 ms
T1 16,384 (L256@132) N=6 A+C batch 22156 cyc level 335.7 us arr/sib 4427 d7 tree 2.22 ms
T1 16,384 (L256@132) N=6 A+C+fuse+floor batch 18988 cyc level 287.7 us arr/sib 4427 d7 tree 1.90 ms
T1 16,384 (L256@132) N=7 A batch 12610 cyc level 382.1 us arr/sib 3794 d7 tree 2.48 ms
T1 16,384 (L256@132) N=7 A+C batch 20008 cyc level 303.1 us arr/sib 3794 d7 tree 2.01 ms
T1 16,384 (L256@132) N=7 A+C+fuse+floor batch 17162 cyc level 260.0 us arr/sib 3794 d7 tree 1.72 ms
T2 32,768 (L512@132 / +DSP) N=6 A batch 11471 cyc level 347.6 us arr/sib 2213 d7 tree 2.26 ms
T2 32,768 (L512@132 / +DSP) N=6 A+C batch 17729 cyc level 268.6 us arr/sib 2213 d7 tree 1.79 ms
T2 32,768 (L512@132 / +DSP) N=6 A+C+fuse+floor batch 14561 cyc level 220.6 us arr/sib 2213 d7 tree 1.47 ms
T2 32,768 (L512@132 / +DSP) N=7 A batch 10713 cyc level 324.6 us arr/sib 1897 d7 tree 2.11 ms
T2 32,768 (L512@132 / +DSP) N=7 A+C batch 16214 cyc level 245.7 us arr/sib 1897 d7 tree 1.64 ms
T2 32,768 (L512@132 / +DSP) N=7 A+C+fuse+floor batch 13368 cyc level 202.5 us arr/sib 1897 d7 tree 1.35 ms
=== today's role split, pipeline law tree = F + 21*max(a,(a+m+H)/2), calibrated x0.957 to the COMBO3 census
COMBO3 + M1361 [M inputs] a 6533 m 5327 item 7270 tree 2622 us tok/s 961.4
+ NORMFUSE/G1FUSE/A8Q_OVL (-971 a) a 5562 m 5327 item 6784 tree 2474 us tok/s 971.8
+ ATT array 132 MHz (-853 a, xsim) a 4709 m 5327 item 6358 tree 2344 us tok/s 981.0
+ TREE4 (-1,100 m) a 4709 m 4227 item 5808 tree 2177 us tok/s 993.2
+ H halved (1,340) a 4709 m 4227 item 5138 tree 1973 us tok/s 1008.4
breakdown, Llama N=7 T1, W2 batch, fuse+TREE4:
level 9196: G5 1893 (21%) all-reduce x2 1482 (16%) array 2x1234 (27%) tree scan 2x288 G8 2x211 other once 2208
breakdown, Qwen N=7 T1, W2 batch, fuse+floor: level 17162: array 2x3794 (44%) G5 1507 (9%) AR 1994 (12%) tree 2x973 G8 2x571
Llama survival S(depth) from the hazards: d1:0.939 d2:0.870 d3:0.771 d4:0.683 d5:0.593 d6:0.516 d7:0.452 d8:0.400 d9:0.340 d10:0.292 d11:0.244
full-accept d11 (product) 0.244 vs logged 0.244; x(1-0.164) next-token = hit 0.204
Qwen humaneval:80 7.4166: fitted per-level miss 2.2 % S(7) 0.86 hit 0.84 accept at d11 10.66 d15 13.64
Qwen he4+mt4 mix 4.0674: fitted per-level miss 20.8 % S(7) 0.20 hit 0.16 accept at d11 4.52 d15 4.70
Qwen round (he4+mt4, w3b [M] 36.50 ms; card RPC 12.54, IPC-inject 1.14): hit h removes 13.68 ms
h 0.00: round 36.5 ms (1.00x of today) vs GPU-only 21.9 ms
h 0.23: round 33.4 ms (1.09x of today) vs GPU-only 21.9 ms
h 0.50: round 29.7 ms (1.23x of today) vs GPU-only 21.9 ms
h 0.84: round 25.0 ms (1.46x of today) vs GPU-only 21.9 ms
h 1.00: round 22.8 ms (1.60x of today) vs GPU-only 21.9 ms