fpga-archive-reports-record · EAGLE4 FPGA archive

Source reports/results/DRAFTER_SPEED_RESEARCH_20261107/SCALING_4C_VS_7C_PHASE_RANK_20261116.md · source md5 909ba03dd537a39e16119a3feab168ce · archived 2026-11-18

Download PDFOpen PDFRaw markdown

Why 7 cards gave +8.9 % and not more: per-phase rank, 4-card F3t vs 7-card record (2026-11-16)

Program 1: ONE Llama-3.2-3B 2-bit EAGLE4 drafter. 4-card = ATT TP2 F3t 0xE4B713F9 x2 + MLPV 0x1342 x2; 7-card = ATT TP4 x4 + MLPV M1361 x2 (talon2, H200 verify). Motto: a wall is a measured constraint to engineer around, toward the fastest flashable PDI. Read-only research worker for box1-two-v2. No card, lock, build, flash or kill. talon2 was read as carrick (one tar of four COMBO3 serve logs into box1 scratch). User question (verbatim): "Any reason why that the increase in card did not bring as much improv as expected., compare per phase timing in .rpt files for a detailed which got faster and which got slower..... in terms of ranked form" Tags: [M] measured (artifact + md5 named), [D] derived by arithmetic from [M], [P] projection / hypothesis (also collected in the bar below). Short paths: R/ = reports/results/DRAFTER_SPEED_RESEARCH_20261107/, T/ = reports/results/TRACK1_RECORD_F3T_20261109/.

Calculations and hypothesis (everything that is not a measurement)

0. Answer first

  1. .rpt files have no cycle counts. A Vivado report gives period, slack, path and area only (R/../PER_PHASE_CYCLES_BY_IMAGE_20261110.md:8). What the .rpt files DO say: the drafter clock is 66.001 MHz / 15.151 ns on both sets, so fmax contributes 0 to the 4c->7c delta (section 2). Per-phase timing below is from the on-card counters (7c) and the FSM-sampling chain census (4c).
  2. Drafter speed (card us per full d11 tree, 21 items) went 3,384.05 -> 2,877.70 us = -15.0 % (4c F3t+0x1342 vs 7c 0x13BE x4 + M1361 x2) [M]. Doubling the ATT ranks cut ATT busy by only 20.4 % (2,956.5 -> 2,352.6 us), not 50 %, and 84 % of that reached the tree (-506 of -604); the other +98 us is new waiting on the MLPV side.
  3. Served step 7.812 -> 7.187 ms (-0.625 ms, -8.0 %) = 883.12 -> 961.43 tok/s (+8.9 %) [M]. Of the -0.625 ms, -0.445 ms is the card tree path and -0.181 ms is host-side "A". The card owns only the tree path, which was 31.4 % of the 4-card step; GPU verify (3.45 ms [M d7]) + host (~1.9 ms [D]) are the other 68.6 % and do not change with the card count.
  4. The bare 4->7 card change was SLOWER: B4 x4 + 0x134D = 859.06 vs 883.12 (-2.7 %) [M]. The +8.9 % came afterwards, roughly half from host levers that remove per-card MMIO (hx7, HX_O7, HX_S7) and half from new card images (G6_REISSUE, G8_OWN, HCUT, COMBO3) (section 5).
  5. Why the ATT did not halve (ranked): G5 is context sweeps (3 per group at 1 row/cycle), not heads: -27 %; G4 pads 768-lane Q/K/V to 512-lane tiles + a fixed 253-cycle prologue: -28 %; G7G keeps the serial R3 + pre-MLP norm (844 cyc/item): -21 %; G1 walks the full vector on every rank: +6 %; G6 got longer (+12 %) because the 3,072-lane all-reduce crosses two dependent QSFP hops; ATT seam/link time +26 %. The MLPV is still 2 cards: TREE 708 us/tree and G9 270 us/tree are unchanged; only G8 was split (G8_OWN, -314 us).

1. Sources [M]

what file md5
4c card census d11 (F3t 0x13F9 x2 + 0x1342 x2, md 11, w2, K_C 64, 40 trees) T/config_sweep/census/chain_llama3b_f3t_w2d11_kc64_185202.analysis.json.gz, key occupancy2 5b11b7f6
7c card census d11 (ATT 0x13BE x4 + MLPV M1361 x2, md 11, w2, K_C 64, ctx 566, 100 trees, 20 census passes, items_per_tree_counted 21/21) R/tree4_rca_20261114/m1361_census_md11.json (pairing named at R/TREE4_SILICON_RCA_20261114.md:69-71) 3230661e
7c census md 7 (0x13BC x4 + 0x1360), cross-check R/tree4_rca_20261114/bc_1360_census_md7.json 6cbab262
4c serve d11 fa3 (the two rounds behind 883.12: 882.38 / 883.87, 2,843 steps, @6.90) T/config_sweep/serve3/hp_llama3b_PQC_w2d11_humaneval80x256_n1_182607_…log.gz :454,486-490 and …_183008_…log.gz :454,486-490 9c9b32fe / f11c5ff1
7c serve d11 fa3, record 0x13BE + M1361 (2 rounds, lfc + FRESH) R/W4_SERVE_BOUND_20261115.md table row "rec W2" (STEP_DECOMP.tsv e3943d74) –
7c serve d11 fa3, COMBO3 0x13D6 + M1361 (t1/t2/t3 + KV1H2C-off off1, boot 79cc9d1b) talon2 runs/acwin_20261116/d6/serve/acw_hx7s7_kc64_{t1,t2,t3,off1}_…MEASUREONLY_….log, decomposed with R/w4bound_20261115/w4_step_decomp.py (e4c895a6); box1 scratch copy only t1 adee0bf6, t2 0e06f0cc, t3 c92c1f57, off1 a0319bb5
COMBO3 tok/s rows R/ATT_COMBO_SILICON_20261116.md (COMBO3 table, t1-t3 = 960.23 / 960.97 / 963.10, mean 961.43) –

2. Clock and timing from the .rpt files [M]

image role drafter clock FINAL_TRUE WNS / WHS source
F3t 0xE4B713F9 4c ATT clock_reset_clk_usr_0 15.151 ns / 66.001 MHz +0.040 / +0.011 T/timing_gates/impl_1/route_report_timing_summary_0.rpt.gz:197 (period), :237 (usr_0 0.000 / +0.006 with margin); R/F3T_SILICON_20261109.md:4
0x1342 4c MLPV 66 MHz drafter + 132 MHz array (ARRAY_FAST_CLK 1) – R/MLPV_AND_STEP_BREAKDOWN_E2b.md:15
COMBO3 0xE4B713D6 7c ATT 66 MHz (PDI …_0xE4B713D6_66m_d2e737d7.pdi) +0.044 / +0.010, 0/0 failing R/ADJUDICATION_0x13D6_0x13DB_20261116.md:14,57
M1361 (0x1361 margin re-route) 7c MLPV 66 MHz drafter + 2x array +0.041 / +0.016 reports/results/FOURCARD_DESIGN_20261016/4C1_DELIVERY_TRACKER.md:1682
Both censuses count 15.151 ns cycles. No clock moved between the two sets; the scaling question is entirely cycles and waits.

3. Card per-phase table, one full d11 tree (21 items), us per tree

4c = F3t x2 + 0x1342 x2 (chain census) · 7c = 0x13BE x4 + M1361 x2 (counters, ATT0@17 = the slowest-G6 rank, MLPV0@50). "share" = delta / 7,812 us (the 4c served step).

# card phase 4c us [M] 7c us [M] delta 7c-4c delta % of phase share of 4c step
1 MLPV G8 exact rescore (proper) 551.83 237.71 -314.12 -56.9 % -4.02 %
2 ATT G5 attention 841.34 610.48 -230.86 -27.4 % -2.96 %
3 ATT G4 QKV 630.03 452.12 -177.91 -28.2 % -2.28 %
4 ATT G7U 302.43 177.22 -125.21 -41.4 % -1.60 %
5 ATT G7G (+ serial R3 + pre-MLP norm) 561.34 444.48 -116.86 -20.8 % -1.50 %
6 ATT G7P / SILU 80.74 42.32 -38.42 -47.6 % -0.49 %
7 MLPV GLM 73.49 71.91 -1.58 -2.1 % -0.02 %
8 MLPV G12 kick 8.01 7.00 -1.01 – -0.01 %
9 MLPV G11 148.05 148.27 +0.22 0 % 0.00 %
10 MLPV G9 down 268.89 270.45 +1.56 +0.6 % +0.02 %
11 MLPV TREE (G12 scan + top-K merge) 701.70 707.61 +5.91 +0.8 % +0.08 %
12 ATT G1 (+R1) 136.94 145.09 +8.15 +5.9 % +0.10 %
13 MLPV INGEST + XCH + LVL_SEL/OUT (7c: PIPE_IO 170.54 + uncounted 81.54) 243.25 252.08 +8.83 +3.6 % +0.11 %
14 ATT G6 o-proj + TP all-reduce (QSFP hops inside) 195.07 218.14 (ranks 218.1 / 215.7 / 210.4 / 206.3) +23.07 +11.8 % +0.30 %
15 ATT seam / link: EMBG + EXPORT + INGEST (7c: PIPE_IO 191.64 + uncounted 71.07) 208.60 262.71 +54.11 +25.9 % +0.69 %
16 tree new waiting = tree - ATT busy (MLPV-only, both parked, issue / read-back) [D] 427.56 525.14 +97.58 +22.8 % +1.25 %
ATT ATT busy 2,956.49 2,352.56 -603.93 -20.4 % -7.73 %
MLPV MLPV busy (mostly hidden under the ATT) 1,995.22 1,695.03 -300.19 -15.0 % –
tree issue -> done p50 (the drafter speed) 3,384.05 2,877.70 -506.35 -15.0 % -6.48 %

Per item [D]: ATT 140.8 -> 112.0 us; MLPV 95.0 -> 80.7 us. 4c joint occupancy [M]: both busy 1,798.1, ATT only 1,178.1, MLPV only (exposed) 210.5, both parked 35.9, outside the window 162.6 (occupancy2.joint_us_per_tree, window 3,221.48). No 7c joint split exists (section 7). Instrument check: the two instruments agree to <0.3 % on unchanged MLPV phases at d7 (G9 852 vs 850, TREE 2,222 vs 2,224 cyc/item, R/../PER_PHASE_CYCLES_BY_IMAGE_20261110.md:78,82); here G9 / G11 / TREE again agree to <1 %.

COMBO3 0x13D6 on top of row "7c" [P, no census]: G5 -157.5 us, G7G -112.3 us (R3_EARLY) => ATT busy ~2,083 us/tree, ATT item ~99 us vs MLPV item 80.7 us + hand-offs. KV1H2C measured on the serve only: tree path 2.112 (off1) -> 2.010 ms (t1-t3) [M, n = 1 vs 3].

4. Step budget per verify step, d11 fa3 w2, humaneval 80x256 (ms)

term 4c F3t + 0x1342 (883.12) 7c 0x13BE + M1361 (938.44) 7c COMBO3 + M1361 (961.43) delta 4c -> COMBO3 tag
ms / step 7.812 7.364 7.187 -0.625 [M]
tree path = host_ahead + join_wait 2.455 (31.4 %) 2.194 (29.8 %) 2.010 (28.0 %) -0.445 [M]
- host_ahead (host work under the card) 0.512 0.511 0.510 -0.002 [M]
- join_wait (host blocked on the card reply) 1.943 1.683 1.501 -0.442 [M]
A = step - tree path 5.358 (68.6 %) 5.170 5.176 (72.0 %) -0.181 [M]
- GPU verify forward 3.452 (44.2 %) 3.452 3.452 (48.0 %) 0 [M] d7 F3t only; [D] elsewhere
- host rest of A (UpdateResult, accept, KV import, arm, root image, doorbell, per-request) 1.906 (24.4 %) 1.718 1.724 (24.0 %) -0.181 [D]
card full tree (census, for reference; serve trees stop early) 3.384 2.878 NOT-MEASURED -0.506 (to 0x13BE) [M]
Steps 2,843 (4c) vs 2,839 (7c), accept 6.90 vs 6.91, tokens 19,615 both: the tok/s gain (x1.0887) equals the step gain (x1.0870) to 0.2 %.

5. The 4->7 ladder: where the +8.9 % actually came from (d11 fa3, same workload) [M]

set tok/s delta lever class source
4c F3t x2 + 0x1342 x2 883.12 – – T/config_sweep/serve3/*182607*, *183008*
7c B4 x4 + 0x134D x2 (no 7c host levers) 859.06 -24.06 (-2.7 %) more cards only R/C2_6CARD_TP4_20261110.md:14,62
+ hx7 (host: fewer pre-doorbell MMIO) 879.57 (A/B vs 859.47) +20.1 host reports/results/SESSION_CONTEXT_BACKUP_20261111.md:10
0x13BC (G6_REISSUE) + 0x1360 890.30 (4-run mean) +10.7 card R/BA_0x13BA_SILICON_20261112.md:38
0x1361 MLPV G8_OWN 902.77 +12.5 card R/MLPV_G8_OWN_SILICON_20261112.md:24
+ HX_O7 (host: trims post-tree reads) 927.65 +24.9 host R/DESIGN_FABLE_START_HALF_O_20261112.md:3
+ HX_S7 v 932.71 +5.1 host R/ATT_13C3_SILICON_20261114.md:7
0x13BE HCUT ATT + M1361 938.94 +6.2 (+7.1 vs same-boot parent BELL 931.82) card R/ADJUDICATION_0x13D5_20261113.md:129
COMBO3 0x13D6 (G5_P2W 8 + R3_EARLY + KV1H2C) + M1361 961.43 +22.5 card R/ATT_COMBO_SILICON_20261116.md COMBO3 table
Cross-day rows; steps between rows are indicative, not paired A/Bs, except where named. Sum: host levers ~+50, card images ~+52, the card count itself -24.
The 7c host cost per card is measured: 54 register reads per poll pass vs 28 on 4c, 1.75-2.15 us per posted write + read (R/DESIGN_FABLE_OUTSIDE_WINDOW_O_20261112.md:41-46), and KV_LOAD (427.8 us) + stage_cfg..doorbell (348.1 us) scale with ATT ranks (same :53-58). So the host levers partly repay what the extra cards cost.

6. Why not more, and the top 3 levers

Facts [M]:

Top 3 levers by modelled card us per d11 tree [P]:

  1. MLPV TREE (largest unchanged phase, 707.6 us/tree; the MLPV is now co-critical).
    • TREE4 measured 2,224 -> 1,124 cyc/item on talon2 silicon (R/TREE4_SILICON_RCA_20261114.md:74) = -350 us of MLPV time per d11 tree.
    • The tree gain is capped by the new waiting (525 us/tree) and grows once COMBO3's ATT cuts land; modelled -100..-200 us/tree.
    • Blocker: the T4 image's wrong trees are an implementation defect upstream of G8 (that doc, sec. "Answer first" 5), not a TREE4 defect.
  2. ATT serial prologues: NORMFUSE (-387 cyc/item) + G1FUSE (-390) + A8Q_OVL (-194) (R/DESIGN_FABLE_ATT_ITEM_20261112.md:51-53).
    • Together -971 cyc/item x 21 = -309 us of ATT busy per tree; at the measured 84 % pass-through, about -260 us/tree.
    • NORMFUSE needs R3_EARLY (now in COMBO3). G1FUSE costs +256 DSP and has an open auditor HOLD on its own lineage.
  3. G6 all-reduce link time. B4's 472 cyc/item over the ideal link x 21 = up to -150 us/tree; G6 + seam/link grew +77 us/tree here.
    • Needs the per-rank first-beat timestamp to split hop latency, skew and retransmit. Then a TP_BEAT / one-hop reduce on TP4.

Not card levers, stated as facts: GPU verify 3.452 ms and the host rest of A ~1.72 ms are 72 % of the COMBO3 step. The card -> 0 bound is ~1,215 tok/s.

7. NOT-MEASURED, and how to capture each on talon2

  1. COMBO3 0x13D6 per-phase census (needed to replace the [P] row in section 3). ac_steps.sh (4b69ec38) has no census phase. Use the census block of R/m136Bwin_20261116/ship/m4_steps.sh:156-168 (29092fd3), as root, under the window lock.
    cd $KD6 && OWNER=<window>-census bash ~/eagle4_talon2_4c1/lock/card_run.sh env \
      EAGLE4_4C1_TOPOLOGY=7card EAGLE4_4C1_CTRLFSM=0 EAGLE4_4C1_CUDAPACK=0 EAGLE4_7C_RING_CHECK=0 \
      EAGLE4_4C1_CARD_BDFS=$BDFS EAGLE4_4C1_BOOT_ROOT=$BOOT_ROOT EAGLE4_4C1_VERIFY_LOG=$EAGLE4_4C1_VERIFY_LOG \
      EAGLE4_PAIR_EAGER=1 EAGLE4_PAIR_EARLY_A=1 EAGLE4_PAIR_SPEC0=0 EAGLE4_PAIR_TREE_STOP=0 \
      /usr/bin/python3 -I $KD6/scripts/4c1/c4.py --model llama3b bench --disc $RUN/discover.json \
      --md 11 --width 2 --kc 64 --trees 100 --census 20 --out $RUN/census/bench_P11w2_md11_w2.json
    
    • Do it after the clean golden of the boot. A second arm --md 7 matches bc_1360_census_md7.json.
    • Two bench calls of 100 trees each are a few minutes, inside the ~10-min window.
  2. 7-card joint occupancy (ATT parked vs MLPV exposed, i.e. the split of the +97.6 us). c4_chain --set fsm cannot run on 4 ATT ranks (R/C2_6CARD_TP4_20261110.md:96). Extending it is tooling work, not a window job.
  3. GPU verify forward on the 7-card set. One serve round with VTIMING=1 (the cuda-event instrument behind the 3.452 ms, INTERLEAVE_BATCHVERIFY_SILICON_20261109.md:69-76). It is instrumented and slower, so run it as a separate row.
  4. G6 hop split (latency / rank skew / retransmit). This needs the per-rank first-beat timestamp in RTL (PER_PHASE_CYCLES_BY_IMAGE_20261110.md:206).

8. HYPOTHESIS: one drafter on all 15 cards (talon2 7 + talon1 8), single stream [D/P, not a measurement]

User (verbatim, via box1-two-v2): "If it helps, we might be able to use all 15 cards as a drafter?" Scope: ONE drafter, never 2 drafters. Everything in this section is a projection. The inputs are the measured per-phase numbers above plus each phase's RTL closed form. Nothing here was run.

8.1 Topology assumed

8.2 Which phases shrink at TP8 (per ATT item, cycles; 0x13BE TP4 base = measured)

phase TP4 measured measured 2->4 ratio naive (ratio again) mechanism TP8 [D from RTL closed form] why
G1 456 1.06 483 456 (fixed) full-vector norm on every rank
G4 1,421 0.72 1,020 829 384-lane Q/K/V = 1 tile each: 3 x 192 + 253 prologue (2 of 512 lanes idle per 4)
G5 1,919 0.73 1,393 1,919 (fixed) 4 heads on 8 engines is still ONE group = 3 context sweeps; fewer heads buys nothing below 8/rank
G6 + all-reduce 686 1.12 767 884 (longer) array 144 -> 72; exchange 542 over 2 hops -> ~813 over 3 hops
G7G 1,397 0.79 1,106 1,193 array halves; the 844 serial R3 + norm stays
G7U 557 0.59 327 349 array halves; the 157 prologue stays
G7P 133 0.52 70 67 halves
seam / link (PIPE_IO + uncounted) 826 1.26 1,040 826 (best) / 1,040 (realistic) grew 26 % per doubling, measured
ATT item 7,394 (112.0 us) 6,204 (94.0 us) 6,523 (98.8 us), -11.8 % the second doubling buys about half of the first doubling's -20 %
With COMBO3's levers on top (G5_P2W 8 -495, R3_EARLY -353 [P]), the ATT item is 99.2 us at TP4 and 86.0 us at TP8 (89.2 us with the seam growth).

8.3 What stays fixed

8.4 Projection (d11 w2, per verify step; base = COMBO3 7c modelled tree 2,743 us, measured step 7.187 ms)

Card-tree deltas reach the serve at the measured 0.516 (serve tree path / census tree, 0x13BE vs F3t).

case ATT item a MLPV item m H tree item full tree step tok/s
COMBO3 7c (model base) 99.2 80.7 40.6 110.2 (loop-bound) 2,743 7.187 [M] 961.43 [M]
Best: TP8 + TREE / G8 / G9 split ideally over 7 MLPVs (new RTL) + 0 host cost 86.0 39.4 40.6 86.0 (ATT-bound) 2,233 6.924 ~998
Best + context-split G5 (2 engines per head, new RTL) 76.0 39.4 40.6 78.0 2,065 6.838 ~1,010
Realistic: TP8 with seam growth, only G8 split further, 2 x 3.6 us cross-chassis hop, host +0.1 / +0.2 / +0.3 ms 89.2 72.6 47.8 104.8 (loop-bound) 2,628 7.23 / 7.33 / 7.43 ~956 / 943 / 930
Read it as follows:

8.5 Break-even link latency