fpga-archive-reports-arch · EAGLE4 FPGA archive

Source reports/results/DRAFTER_SPEED_RESEARCH_20261107/IDEAL_DATAPATH_D2_20261118.md · source md5 395ea89bb3c46be1c749f925293c977e · archived 2026-11-18

Download PDFOpen PDFRaw markdown

Ideal drafter datapath "D2" — die-local residual stream, one fast clock, fused norms (box1-two-v2, 2026-11-18)

Author: box1-two-v2 (orchestrator-authored design; user 11-18: "feel free to do your own design for ideal datapath"). Program: the 7-card DRAFTER (Llama-3.2-3B record 961.43 tok/s @ 6.91 on COMBO3 0x13D6 x4 + M1361 x2; Qwen3.8-27B GPTQ-2 drafter, best humaneval 181.79 @ 12.14 d15). Motto: never end by denying the idea; engineer toward the fastest flashable PDI. Tags: [M] measured (artifact), [R] read from RTL/report, [P] projection.

0. Answer

The current ATT card is a single-card design grown by levers: a 66 MHz core with 132 MHz islands, a full-width HUB in SLR0 that every GEMV pass returns to, and attention KV filling the same die. SLR0 is 98.8 % SLICE (A8OV 0x13D3 placed) [M] while SLR1/SLR2 sit at 69 % / 78 % [M]. D2 changes three structural things, each provable on its own:

  1. Die-local residual stream (no hub). Each die owns a slice of the output columns of EVERY GEMV and the residual-stream slice for those columns. Attention heads live on the die whose lanes produce their Q/K/V columns, with their KV local. Only small vectors cross dies: the int8 activation all-gather (3,072 B per GEMV input for Llama), the 768-element attention output, and scalar partial sums for RMSNorm.
  2. Fused norms by scale invariance. RMSNorm followed by per-vector absmax A8 quantisation is invariant to the scalar 1/rms: quant(x∘g/rms) = quant(x∘g) with the dequant scale carrying 1/rms. So G1 (input norm), the pre-MLP norm (done: NFA8 NORMFUSE) and G11 never hold a GEMV: the GEMV starts on x∘g at once and rms (a scalar reduced in parallel) is applied at the output dequant.
  3. One fast clock for the whole datapath (target 133 MHz, = 2 x 66.0, integer clk_wiz ratio). Removes every 66<->132 bridge (AFC8 bridge = +24.9K FF) and halves every non-array phase too (G5, norms, hand-offs, FSM). The die-local layout is what makes this closable: the measured 100 MHz wall on the old single-card HEAD was WNS -4.850 ns [M, WHOLE_DRAFTER_FMAX_CENSUS_20260923.md running log 10:40], dominated by long full-width paths that D2 deletes.

Plus two scheduling changes that need no new datapath: sibling interleave (the attention engine works on sibling A while the array runs sibling B's GEMV) and one host message per step (batched KV rows, junior's c5 KV_MROW for Qwen).

1. Why (measured)

fact value source
Llama ATT item on silicon 7,394 cyc @ 66 MHz = 112 us (G1 456, G4 1,421, G5 1,919, G6 686, G7g 1,397, G7u 557, rest 958) SCALING_4C_VS_7C_PHASE_RANK_20261116.md census
weight GEMV share of a Llama node 24 % (2,832 of 11,860) ARCH_RETHINK_7CARD_20261117.md
ATT item after NFA8 + AFC8 (xsim) 4,126 cyc @ 66-equiv (G4 914, G6 632, G7g 486, G7u 220) ATT_FASTCLK_20261117.md
SLR SLICE, A8OV placed 111,216 / 70,765 / 83,010 = 98.83 / 69.02 / 77.78 % /var/tmp/box1two_nfa8_20261117/watch_D3/STATUS 02:03Z
HUB contents (SLR0) RET_SER landing/commit/per-lane shadow 512x48 b, g7g/g7u latches, R4/SiLU, u_tp (136,279 prims; 27,195 with TP_BEAT), q/k/v_full_buf WHY_CROSS_DIE_RTL_READ_20261108.md, F_IMAGE_CONGESTION_DOSSIER_20261108.md
hub moved to SLR1 24,212 SLL > 18,870 available same
cards' share of a Llama step 2.0 ms of 7.19 ms ARCH_RETHINK_7CARD_20261117.md
Qwen humaneval round 49.5 ms; card RPC 25.7 ms of which KV rows 16.7 ms DESIGN_FABLE_DRAFT_AHEAD_20261117.md
Qwen ATT item 15,085 -> 8,102 with K64 + AFC (xsim) DESIGN_FABLE_QWEN_ATT_K64_20261117.md
placement constraints pblock_dc_grp0/1/2 NCELLS = 0 (vacuous) on junior's AFC images; 0x13CE pblockgate 22 dead filters junior 11-18; GATE_VERDICTS s3LAtp4CE 00:46:55

2. D2 ATT card (TP4 rank), per die

            broadcast x∘g (int8, all-gather)            all-gather attn out (768)          all-gather h (int8)
 die d:  [lanes N-slice d] --G4 Q/K/V cols of heads(d)--> [G5 heads(d), KV(d) local] --> [lanes N-slice d: G6] --> resid slice d
                                                                                                     |-> partial Σx² (scalar) -> rms
                                                         [lanes N-slice d: G7g/G7u] -> SiLU·up slice d -> to MLPV (via link die)
 cross-card: G6 partial sums all-reduced per column slice over the TP4 link (SLR1 GTM), each die sends only its slice.

3. Projection [P] (to be replaced by each stage's measurement)

stage ATT item (us) basis
today (silicon) 112 [M]
NFA8 + AFC8 (building) ~62 xsim-equiv (silicon will be higher by link waits) [M] xsim 4,126 cyc
+ G1 fold ~56 removes the 456-cycle G1 at 66 MHz
+ one 133 MHz clock ~36-44 non-array phases halve; array already fast
+ sibling interleave ~30-36 G5 hidden under the other sibling's GEMV
Llama: cards 2.0 -> ~0.8 ms per step => ~6.0 ms step => ~1,150 tok/s (+20 %) if the MLPV gets the same treatment. Qwen gains more (card-dominant round).

4. Stages (each a unit gate first, then an image; one lever per image for attribution)

# stage proof before image kill / confirm
S0 fix vacuous pblocks (dc_grp, dead filters) NCELLS > 0 on a placed copy SLR0 SLICE and WNS on re-place (SLR0_DECONGEST agent)
S1 G1 fold (input norm by scale invariance) on the AFC8 fork xsim bit-exact vs AFC8 on 4 ranks + RED; numerics proof of the scale identity incl. fp rounding of the scale G1 cycles -> ~0
S2 133 MHz fmax census on the routed NFA8 dcp (period override, every failing endpoint bucketed) census tool exists (whole_drafter_fmax_census_20260923/census.tcl) list of path classes to pipeline; if > ~2,000 distinct failing paths outside the hub/RET_SER/u_tp, D2's clock target drops to 100 MHz
S3 die-local residual stream RTL (fork from AFC8) xsim bit-exact vs AFC8 on 4 ranks; SLL estimate from an OOC place of the 3-die array top SLR0 SLICE < 85 % and SLL < 15,000 on the OOC place
S4 single 133 MHz clock on the S3 fork OOC route of the S3 top at 7.5 ns WNS >= 0 OOC, then full image
S5 sibling interleave (attention vs array across W2 siblings) xsim cycle drop, bit-exact per-pair item time
Silicon order: NFA8/AFC8 (in flight) -> S1 image -> S3+S4 image (the D2 card) -> S5.

5. Ownership

Orchestrator authors this spec and re-examines every critical RTL diff (S1, S3, S4); agents implement in their own forks with paired read-only auditors. Qwen images stay junior's; D2 applies to both models (dual-model ATT) and the recipe goes to junior once S3 is green.