High-batch redesign: per-unit architecture, proofs, and cycle/timing math vs the best silicon image
11 of 23 tracks final; every cycle row cites the silicon census it compares against.
One page per experiment on the EAGLE4 drafter for the Alveo V80: motivation, proposal, architecture, simulation, synthesis and implementation results, and conclusions. Each page is written by the agent that ran the experiment.
11 of 23 tracks final; every cycle row cites the silicon census it compares against.
The talon pitch for one talon serving five 8xH200 nodes (40 H200) as a slide presentation: 6 steps, each chart with its CSV and a visible breakdown of how every number is computed (formula, tagged inputs, worked example at Qwen B=8); PDF and Markdown downloads.
Consolidated EAGLE serving page: 1 talon + 1 GPU, per user, 1 talon + N GPU (duty interleave + 4->8 batching), equal cost, the four program graphs, duty, pure drafting and the calibration proof.
Customer showcase: what the talon is, measured batch-1 wins, drafting at equal cost, one talon feeding many H200s, the 8xH200 node with vs without a talon, and the clock/misc roadmap; projections labelled [P].
Four batch curves: H200 alone vs our best batched image (projection, in the Calculations bar) with today's measured silicon as orange dots.
Every redesign unit transfers to the GPTQ-2 shapes; the ATT level (A16 GEMV) binds the card, the H200 verify binds the system from B = 8.
Every redesign unit transfers to Qwen shapes; the H200 verify, not the card, is the Qwen wall.
Every batched unit is measured bit-exact up to B=128; the fitting MLPV level at B>=32 and the full-card placements are the open measurements.
Each engine is measured faster and bit-exact; the fitting MLPV level at B>=32 and the whole-card placement are open.
FPGA leads at bs 1; from bs 2 the batched-verify route loses because the one card set drafts each request in turn (+1.1 ms KV switch each) and the route runs a reduced lever set (BV1 = 635 vs record 979-984).
Per raw tensor (new/old weights): q/k/v/o_proj 2.78x, gate/up/down_proj 3.33x, lm_head 3.23x, embed_tokens/fc 1.67x; sub-norms and d2t/t2d only in Finn's model; efficient_lm_head only in GPTQ-2.
Index of the FPGA archive sub-pages
The 2026-11-10 demo, the 7-card record, the Architecture Book diagram and how talon2 is set up, measured numbers with sources
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Archive sub-page of eagle4-fpga-archive
Paper-format white paper as of 2026-11-04: Llama-3.2-3B humaneval 80x256 839.65 tok/s at accept 5.88 on H200 + four V80; drafter forward 0.177 ms on four cards vs 0.37 ms on H200. Restored over the 2026-11-10 slide-style revision (v14-v19).
Start here: demo tab (the white paper), drafting tab (pure drafting speed per phase), three track columns with daily progress blocks; tabs for the Architecture Book, model weights, knowledge base, research and operations.
On the same humaneval-80 set the new GPTQ-2 drafter is faster on the GPU (326.83 vs 256.58) but slower on the 7 cards (140.15 vs 249.62): its card body is 3.07x larger; accept on the cards equals the GPU.
6 V80 drafter + H200 beats the best GPU-only EAGLE on the same H200 on all 5 datasets (1.100-1.188x, geomean 1.138x); drafter energy per forward 44.2 vs 97.6 mJ (2.21x less, per device, never summed); per session ~equal; system energy per token 1.82-1.94x GPU-only
6 V80 drafting for one H200: 933.14 tok/s single-stream on HumanEval @ 6.91, 1.100-1.188x the best same-day GPU-only EAGLE on 5 datasets; 44.2 vs 97.6 mJ per drafter forward (2.21x less, time x run-mean power per device); system power 565.9 vs 263.8 W
Daily log of Track 1 progress, code changes, test results and mechanisms. Newest day first.
Draft technical specification: measured and vendor rows labelled, unmeasured fields TBD
4 V80 FPGAs draft 1.24x faster than the H200 drafting itself (Llama-3.2-3B humaneval 80x256, d15 w2 V20 fa4); 1.17x with fa3
Superseded: the white paper lives at /p/eagle4-white-paper/ (v7, facts only, today's numbers).
Version 7: bridge cable measured on both ends, talon1 PCIE8 cage 1 to talon2 PCIE6 cage 1; link not up yet
11 measured cables; 2 to add (3d c1-e1 c1, bd c1-50 c1) for the 6/7-card run
Where every byte goes in one draft step of the F3t record pair, replayed from the measured 40-tree census: the tree is ATT-bound and PIPE_WAIT is two MLPV round trips.
Four talon2 V80 cards draft the tree; per step they need the accepted token ids and the target's 3 aux hidden states per token, and return W+W^2(D-1) nodes. A target-FPGA verifier can drive them through the existing card-server process (software only) or, after RTL work, over a direct QSFP link.
One-screen summary: what the 4-card drafter is, what a target-model verifier sends and gets back each step, the two ways to connect, and the key driver files.
11 directions measured and below the F3t record; failure cause and revival conditions listed
Pick a track and an image to see its block diagram, what it added, its timing and its measured tok/s.
F3t (G5 overlap) cuts card G5 by 30.8 % and the tree p50 by 8.3 % (Llama) / 8.5 % (Qwen). Llama serving rises from 812.22-816.61 (E2b, 7 rounds) to 830.09-831.14, outside the noise band; the Qwen change (247.62-249.33 -> 249.05-249.61) is inside it. Outputs are byte-identical to E2b. FINAL_TRUE WNS +0.040 / WHS +0.011 ns.
Cards add about 1% power under decode load; idle share, not per-token power, drives the energy cost.
Pick a model and a tensor to see its exact shape, bit width, size and placement, with the related research posts below.
The pinned knowledge graph plus a searchable index of every HQ report, repo report and build script, the findings ledger, branches and the image registry.
Discovery layer over every page, report, note and skill, linked to images and features; Images carry post-route timing, utilization, verdicts and findings from the HQ export; lineage from HQ parent edges.
0x130E (v8 + ring link fix) is a strict improvement and is the new default talon image. CP2BS2 fixed every DSP lane (probe2 clean 8/8), but the done-right 0x126D image is wrong from token 0 on real programs (Qwen and Llama-8B), so the Qwen exactness gap is an image-level functional bug, now under RTL-sim root cause.
With the step-21491 EAGLE3 drafter, Qwen3.8-27B decodes 4.1x faster than the target alone on code (humaneval) and 1.9-2.0x on chat (mt_bench). Moving the drafter from the GPU to 4 V80 FPGA cards keeps 0.96x of the GPU-drafter speed at the 2x7 tree with 74/80 identical outputs. All numbers are measured the same day on the same talon2 H200.
One dual-model ATT image (0xE4B7129C) + MLPV 0x128B serve both Llama-3.2-3B and the Qwen3.8-27B drafter, switched by registers at runtime, on box1 and on talon2. On the same day and the same H200 (talon2, humaneval 80x256) the 4-card drafter reaches the GPU drafter's accept but is slower than GPU+EAGLE (241.24 vs 251.20 at w2 d7, 216.35 vs 248.23 at w4 d7). v1's lead over the GPU drafter came from throttled 2026-10-20 box1 GPU references and is withdrawn.
Wave E (ATT KV-commit + R3_EARLY + NORMFUSE, MLPV G11F) runs on all four cards after cold AC#28 with every QSFP link up on its own and boot kits verified 4/4; it measures 518.0 tok/s mean at mt_bench w2 d7 (wave B 513.20) and 857.93-860.83 at humaneval d12, accept identical to wave C at every depth. Golden is bf16-only (measure-only). Next: a second drafter, ATT pair + MLPV-solo 0xE4B712F5.
Corner prohibit cleared the port-12 lanes on 8/8 cards; lane 7.21 remains, so Qwen runs 2.2x faster but wrong. HBM realized ~229 GB/s/card vs 614 design / 819 device; SLR2 at 94% SLICE is the congestion die.
Every measured result of the three tracks as of 2026-10-21, failures included. Measured only; projections are on the progress post. Track 1 box1 data pending, from the delivery note.
Current builds, gates and ETAs for 10-21 to 10-23 with projected batch-1 numbers. All projected or derived; nothing here is measured.
Best measured batch-1 result is 513.20 tok/s mean (wave B) against 475.0-494.7 H200-only reference; measure-only regime, no exactness claim. box1 data file pending.
Track 2 batch-1: Qwen3.8-27B 17.8-18.9 greedy, up to 21.1 with EAGLE3; Llama-1B 415. Five silicon failures listed, including Llama-8B ring error at token 21.
FOLDRING8 0xE4B712F4 runs 5,837-5,845 tok/s batch-1 bit-exact on 8 cards; the URAM image 0x12F8 failed on silicon with a constant token.
R3_EARLY and NORMFUSE cut G7G by 756 cycles per item bit-exact in xsim; probes B3, B4 and B5 (B4 + G5 on the 132 MHz clock) are building, all quarantined; G1FUSE does not pay with the fold on.
Four ATT cuts and the MLPV LVL_SEL cut are unit/chain-gated GREEN with REDs failing and strap-0 identity; the full-drafter gate-3 arms have no verdict yet (boot preload). Both images carry the RESUME TP link engine: flash both cards of a TP pair together.
Host mode EAGLE4_4C1_FREELEAF_D1 is built and gated; card A/B pending. RTL every-level form in progress.
The two ATT features merge with one RTL line; unit, law, md7, elab and host gates pass with RED arms failing; full-drafter replay and timing pending.
mt_bench W2 d7-d15: FPGA accept 4.5-5.6% below GPU fa3; humaneval MAXNEW 32 rows are capped
Our drafter wins accept whenever a tree is allowed (up to 4.22 mt_bench, 7.40 humaneval) but the native MTP head out-accepts it at equal chain depth (d7 3.70 vs 2.85); tok/s at batch 1 is a tie with MTP.
HEAD_PAR 16 / G5_HPS 1 halves the G5 slope bit-exact; the full image closes once the KVIFAN ch0 300 MHz class is pinned to SLR0.
Theta and per-level width are flat on mt_bench d7; the drafter's rank-3.. children as free verify leaves add +2.3% (root), +3.7% (depth<=1), +5.0% (every level) accept; hedge h(2,2) >= 0.116 on card trees.
Weights + tables + goldens + host profile for Qwen mode are ready; RTL items: 14-bit scale frac, e_norm wrap on tokens 11/13/15, cheap-head K_C.
K3 passes the texts gate. K1 cuts the verify graph 10.8% but flips one mt_bench near-tie (HF fp32 picks K1's token); adoption is the user's ruling. K2/K4 Triton GEMMs lose to cuBLAS.
The 4C1 ATT card now elaborates a max-shape attention (16 slots x 128) and switches between the Llama 16x96 and Qwen 12x128 geometry, rope table, softmax scale, KVI block size, eps, SubLN and e_norm fixes from a MODEL_CFG register block at 0x200. Every modular gate is green; the PDI build waits for a qualifying host.
URAM is not the limit for a bigger drafter; every shape change needs a rebuild, and the cost is card time on a serial tree-then-verify chain.
Source and stage differ from 0x1288 only in the four KVCOMMIT files; host-side offline gates pass with RED arms failing; timing pending.
Depth 8-12 at W=2 raises accept to 4.06 at flat step time; W=3 deep loses tok/s. Sweep stopped at w3 d10 V27 (ATT0 0x1CC[1] item error); cold AC needed.
The optimum stays at w2 d7; W=3 hides the cards (margin >= 1.3 ms), W=4 at d6-d7 nearly exposes them. Sweep stopped at a card fault (MLPV 0x1F4).
GLM 256 lanes removes the GLM tail (227 -> 32 cycles); the SUMSQ fold saves 10 cycles. Two images building; no routed timing yet.
The unattributed level time is on-card (seam path 37.6, G8 23.1, LVL_SEL 15.6, LVL_EMBG 15.5, return hop 8.8 us); at w2 siblings overlap and the level is ATT-bound; D1 is not what hides card cuts, the GPU verify is.
v1.2 cuts the MLPV item from 113 to 72 us and the card loop per level by 7-11%, but the record cell moves only +0.3% to +2.2% (inside one sd).
The verify graph has no launch bubbles; its time is GEMM (68%), attention (17%) and norms (6.5%). VQSTRIDE drops two dead kernels per layer, bit-exact; default off.
Both host cuts are exact and measured; after LEANATTN the T phase is GPU-bound (5.0-5.6 ms H200 span), so further T gains need its GPU work split across card windows.
Two of six margin builds finished and clear +0.03/+0.010; four in route; two timing-RTL builds launched/queued.
Host path and RTL agree bit-for-bit offline and in xsim; silicon A/B waits for the 0xE4B712D1 flash.
VERIFY_PRE stays on the card-published fast path at theta -2.5 and wins; the 4C1 launcher now defaults the five levers on (plain launch 476.67 @ 3.86).
Gate is now label-bound: run_4c1.sh maps 0x1288+0x1287 -> wave-A ref, 0x1288+0x128B -> wave-B ref, and refuses (rc 3) a ref built for other images. References land when the co-sims finish.
The verify's host work runs under the card wait behind a GPU wait on a card-written seq word; step p50 6.350 -> 5.794 ms. Recommended for the record cell.
Adopt theta -2.5 for the 4C1 path (+2.5% mean, paired +11.75, 5/8 rounds); texts unchanged. Golden depth >= 1 still FAILs identically at -2.5.
Batch-1 median of 3 repeats for every model served on talon image 0xE4B71262 (v8, VPU4 on); all exact vs int sim or golden.
Three sequencing straps plus 128 GLM lanes cut G11+GLM from 1,656 to 692 cycles per item, bit-exact against the 0x128B RTL in a full-drafter xsim whose base arm reproduces the silicon census (842/814); RED arms bite; OOC lmdown 128 lanes +1.932/+0.015 ns; full-card build running.
The unexplained 0.35 is the host TREE_STOP lever (85%) and the fa4 lever-Y first drafts (12%); survivor selection, verify pick and KV/RoPE at depth are not the gap.
A one-ctypes-access register path in the card process (same mapping, every guard kept) cut the tree pre-launch 0.306 -> 0.232 ms and KV_LOAD 0.237 -> 0.193 ms per step and raised end-to-end throughput 7.0% mean over 6 rotated rounds with identical texts and per-request meta. Replacing the pickled socket with a memfd ring is exact but did not speed the reply on silicon. The host verify launch (~0.4 ms) is untouched.
On the 4-card drafter (ATT 0x1288 + MLPV 0x1287), no variant beat the current stack. fa3 raises accept by 0.02 but loses 6% tok/s; turning the first GPU draft off lowers accept to 3.62; K_C 128/256 run on the cards but accept does not rise (3.64/3.63). Measure-only (depth golden unproven).
T and D steps shrink by 0.7-0.8 ms each; other steps +0.12 ms and admission +0.2 ms cancel it; stays default OFF.
2-bit weights cost 0 (the GPU already runs them); card head at K_C 64 -0.08; all card formats -0.04; card silicon 3.65 is 0.35 below its GPU numerics twin.
The cards need only the prompt K/V from the GPU. Accept is preserved by both on-card variants; the speed of A is bound by per-item host choreography today.
D1 and Y_DEFER are exact and faster; prefix-resident KV with the SGLang radix cache is slower (target prefill +3 ms) and drifts 1 of 50 texts.
The 4-card system (ATT 0xE4B71288 x2 + MLPV 0xE4B71287 x2) runs 441.8 tok/s mean (6 runs) vs the 2-card record 350.16 at identical accept 3.65. The draft phase dropped 46%; the attention side is 1.5x faster per item, the MLP/lm-head side only 1.04x (128 lanes @66 MHz vs the 2-card TAIL's 256 @75 MHz). Measure-only: golden at depth vs bf16 unproven; silicon == RTL through tree level L1.
RTL and host mode are gate-green offline; silicon numbers pending the two PDIs.
Binding = 3 serial passes x 2 head groups. HP16/HPS1 halves the slope; OOC cost +2,360 DSP, +176 URAM. Not built.
The counter is state-gated and correct. Fold ON gives 1,027 zero-wait; silicon's extra 47-70 cycles is consistent with host-row fold-fetch wait (unmeasured on card).
Cards can compute prompt K/V; a W=8 sweep is 2,399 vs 2,324 array cycles. W=2 fits the 0x1288 ATT; W=4/8 at LANES 256 do not (SLICE).
Sibling-per-pair is sound but pays only if the pair round shrinks (hop instead of host relay) or the tree widens; first measurement: wave A w1 vs w2 repeats at d7 and d4.
Start with the KV_LOAD autopilot; GPU-to-card stores need one probe, card-to-GPU push is blocked in the current images.
w2 d7 is the measured best; w3 d7 is the only shape with a path above it and needs two unmeasured parameters; width >= 4 does not pay at depth <= 7.
Real Llama models run tensor-parallel on all 8 talon V80 cards over the QSFP ring, bit-exact to the integer simulator.
Index of read-only build-report browsers; every file linkable by line.
v8 0xE4B71262 (C_COLS 2 @225, GT island, crumbc fanout) closed timing (+0.003 / true +0.093, 0 failing) and passes on silicon: HBMS ladder 52/52 on all 8 talon cards, ring BIST 16/16 PASS 0 errors
Read-only copies of the council-2026-10-17 build reports, linkable by line (?f=<run>/<file>#L219-L230).
ATT and MLPV failed only on the link clock; MLPV u_tree_level +0.003 ns misses the +0.03 release gate; ATT GEMV array +0.234 ns is its tightest drafter path.
Every number on this page comes from build s3LAattf2: its staged RTL, its synth log, its placed utilization and its post-placement checkpoint. The GEMV phases are URAM-read bound by construction; the build failed timing only on the link clock (fixed in the relaunch).
The 4C1 shift moves G7 off the HEAD, halves every weight-bound phase per card with 48-bit partial reduces over QSFP, and replaces the host relay with a 6 KB card-to-card seam; SLR1 (all QSFP GTMs) is the die to keep below 80%.
Record pair: cards 28% of the step; HEAD GEMV-bound (G4+G7 78%), TAIL TREE+G9-bound (81%); 4-card slicing P1-P4 in simulation.
Running. Silicon: 0xE4B71258 bit-exact on V80 card 42. OOC: 4 single-image layers per card close 400 MHz in two SLRs, leaving SLR2 free, so TriLM-99M fits 4 cards. Next: shell integration, on-chip activation path, ring.
Our own AVED-style wiki for the V80: click a component or a tool, read what it is, how we use it, our measured numbers, the pitfalls, and links to AMD docs and our scripts.
The flashed HBMS image 0xE4B71254 serves real Llama models end to end on silicon with no new PDI: one card up to 8B, eight cards pipeline-parallel for 70B; decode is bound by W8 weight streaming at ~230 GB/s per card.
Every talon card serves 3B bit-exact (54.8-57 tok/s each); one model across 8 cards waits on the HBMS all-reduce to ring bridge (0xE4B71256).
Batch-1 decode on the 0x1254 s2 engine is bound by the 32-port weight-stream beat rate (230 GB/s), not HBM (stall 3.7-3.9%) or DSPs (3 of 4 columns idle).
Living page, re-pushed at each state change. Owners report state changes to HQ (hq-junior-2).
The single-image LUTRAM fold is correct on silicon and its compute time matches simulation; it is one layer's projections, so it cannot generate text by itself.
Sessions inherit reasons, not just rules: skills replace what the user had to remember; a dreamer condenses history on demand; nothing raw is deleted without the user's plan.
Living page, re-pushed at each roster change. Source of truth: /shared/rnd_hq/hq_daily/collab/roster.tsv.
Same placement idea, different drafting algorithm, model-size ratio and memory system. Their claim (2-10x vs GPU spec decode, measured+modeled) and our batch-1 tok/s are not comparable numbers.
Numbers only; the W2 accuracy bar is the USER's call.
The drafter's ternary body linears accept BitNet-style int8 activations with no accept loss on the pinned GPU command, so its GEMVs can use the integer (int8 x ternary) contract that constant folding (Track T) assumes.
Running. Causal-fixed v3 goldens for both target models pass every Tier-Q check on the capture device with REDs caught; four numeric contracts handed to HQ and adopted; Tier-E hook ready for the first RING8 builds.
Curated, scrubbed history of the EAGLE4 agent sessions (319 sessions/subagents, 93 events) from HQ's on-disk records; calculations in the top bar, recorded facts below; HQ re-drops it as the roster changes.
Per-op goldens (every phase G1..G8 + TREE) at ctx 16..564, md 1/2/3, for 4+4 prompts, plus the A->B / B->C / C->A seam regions as .bin/.mem; the checker catches every planted defect.
Running. Umbrella page of the 8-card FULL TARGET program: Track T -> ternary-make-possible-20261011, Track Q -> qwen27b-ring8-20261011. HQ build plan v2 sets the gates and the L0-L7 ladder; next: L0 HBM readout on card 61, R8F re-floorplan, ring cabling card i P4 -> i+1 P3.
Running (reopened as the Track Q living page). Shape, schema and shell facts measured; HQ e8 transport xsim-exact under faults. Next: R8F re-floorplan + lanes-vs-HBM winner -> 0x1250 C4 build on the laptop.
Program 2: 8-card FULL TARGET; Track Q. Design + shell facts measured; no image built yet (Qwen PDI builds owned by the laptop copy of box1-golden, after the 3STAGE report).
Reusable AXI4/AXI-Lite/AXIS checkers + randomized BFMs; every check proven to fire. control_regs.sv passes all 60 ABI vectors but answers B/R in the same cycle as the request handshake (A3.3.1 strict dependency FAIL).
Works bit-exact; the wall is area (~1.2 LUT per nonzero bit-serial). Road to a PDI: folded unit PDI, one-layer-per-card routability, sparsity-trained tiny folds; URAM-resident path for 2B4T.
Program 2: 8-card FULL TARGET; Track T. Re-aimed per user: (1) TriLM-99M fully folded on 8 cards, (2) a constant-folded ternary drafter co-resident with the HBM target. BitNet-2B-4T stays the URAM-resident ternary reference.
Diff from the 10-05 architecture post: what each 4-card variant became, the measured verdicts, and why CL4 ring + shape-generic + HBM streaming are the next bets.
Perfetto traces per hardware block from xsim + box1 hostprof; the draft step is a serial host->cards->GPU chain, so overlap beats engine speed-up.
Animated walk-through of the EAGLE4 V80 drafter: HEAD/TAIL split, 256-lane DSP58 GEMV, KV import, LEVEL_PIPE/FG8, 75 MHz TAIL record 339.12 tok/s [M], QSFP D2/CL4 4-card paths [P], and runtime shapes.
Curated pages and files to review with partner hardware leads, grouped by the roadmap decision each one feeds.
Weekly architecture reference: current HEAD/TAIL datapaths, memories, clocks and links, with a selectable timeline of every major shift.
On-chip speculative-decoding drafter: 1.15x (MT-Bench) / 1.33x (HumanEval) vanilla H200 at batch 1; FPGA evaluation system, SGLang, switchable target model
41.58 MiB 2-bit body on-chip (URAM), fp16 lm-head and embeddings in HBM; 4 V80s hold ~0.9B 2-bit weights, ~5x today
Handbook for new engineers: architecture, repo, toolchain, first build, xsim gates, adjudication and relay, budgets, glossary
Host-side fixes and two image generations took the pair from 175.9 to 307.67 tok/s batch-1 (+75%). Depth 3 now beats depth 2 on both benches. The remaining gap to the GPU (~63% of it) is per-level FPGA cost and the strictly serial GPU/FPGA step.
A new HEAD (HX1, E3b packed-DSP lanes) and TAIL (FSL, LUTRAM seams) took batch-1 to 323.30 tok/s, exact. The 307.67 miss was a slow-boot host state, now classified in <1 s. SLR1 is the wall; EVAC, FSL and I1 free it.
Priming, not depth, was the accept problem; KV import fixed accept at 1.5-1.7x speed. Then host glue, burst seam export and pipelined attention took the pair from 188 to 268 tok/s. What remains is per-level cost (~1.80 ms/level vs <1.72 needed for d3 > d2) and the GPU verify.
Best 1-round pair: A5+F4 KVI+FASTGLUE4 mt w2 d2 268.01 tok/s @2.55 [measured]; 6 pairs catalogued with w x d sweeps.
SGLang never reuses tree KV; the pair needs priming (cheap via PRIME_BURST) plus a host slot fix; RTL child RoPE is +1 (accept-neutral).
Two default-off host levers (RPC_PIPE, CPU_PIN) are byte-identical in mock with 4 REDs caught; they remove host scheduling only, so G < 3 ms still needs SGLang verify work. Silicon A/B pending.
SEAM_WR_PIPE removes the r4 seam write fanout (unit A/B PASS, RED FAILs, scope census confirms placement of the arm). The HEAD clock still cannot rise until attention is pipelined (67.3 MHz ceiling); image 0xE4B711E5 (CD + fix, 66 MHz) and the full-HEAD gate are in flight.
On silicon at steady load, the vocab shortlist gives no gain at width 2 (156-157 tok/s at every tier vs 156.44 full) and at most +3% at width 3 (noise level), at a cost of 2-4 extra verify steps out of ~875 on mt_bench and -2.5% accept on humaneval. FR-Spec shortens the TAIL's TREE walk, but the HEAD is the busier card. Re-test once a faster HEAD (CD/E1/E3) is on silicon. K_C: keep 64.
Priming costs about 1 ms per row on the pair and the HEAD computes for only about 0.23 ms of that; the rest is the host round trip per row. PRIME_BURST runs N rows from one doorbell, with the card loading each next row itself. The xsim verdict is pending.
Launched. The routed builds will answer (1) whether the 256-lane GEMV card still fits with the hop link in the netlist and (2) the open question: does the state card's SLR1 hold attention + the link + the Aurora core. Not serving images.
No single track is 'the' bottleneck any more: the array clock (CD, routed), the HEAD seam/KV-only levers (gated, committed), and the pair's tok/s (112.73 @2.22, +10.4%) all moved today. The two open threads that gate the next silicon step are the width-3 accept shortfall (regime, not proven defect) and the 100 MHz fmax census (G7 seams, not the array, are 60% of the failures).
The HEAD census at 100 MHz is complete and bucketed by block. The G7-up and G7-gate FF activation seams dominate the failing-endpoint count (218,700 + 104,720 = 60% of 542,754), ahead of attention (66,480, but the single worst path at -4.850 ns) and the GEMV array (32,201, -3.878 ns). A faster whole-drafter clock is therefore a seam fan-out + attention-combine project, not primarily an array project — consistent with the array's own 132/133 MHz domain (image CD) already closing cleanly on its ow
Running the array alone at exactly 2x the drafter clock, fed by a per-die BRAM activation cache, is bit-exact and cuts the HEAD item from 33,010 to 18,610 clk in the real drafter. What was pending at the last update — whether the full HEAD image closes timing in context — is now answered: CD is ROUTED, closing both clocks with positive margin, and even freed some resources (LUT 998,363 vs C8's 1,039,733). Silicon flash and the resulting pair tok/s are still pending the user.
Per-die activation replicas + a W-wide serialised return remove the August inter-die-wire wall for the array (measured at W=4, 821 SLL). The new wall is die area: 2,038 LUT per lane-column measured, so W=8 x 128 lanes is ~157% of the routable LUT budget (derived). Once the GEMV sweep is batched, the state card's per-child G12/TREE/G8 tail dominates a level. As of 2026-09-24 ~16:50 EDT the user-mandated target W=8 has NOT been placed at any lane count: the W=8 L64 flow was just admitted on Oleg's
Both halves of the engine split reproduce the single-card drafter bit-for-bit in xsim (md=2, ctx 16, OSPREY 32×96), RED arms fire, and the link transport delivers every hop exactly under back-pressure. No synthesis, timing or silicon yet; the pair speed-up is still a model. Nothing here has regressed as of 2026-09-24 ~16:50 EDT: the separate step-3 joint two-card sim (other agent, fork engine_split_pair_20260923, rebased on this fork's f8f62fb / the OUT-DMA latch fix) shows its old un-rebased co
Adopt the LUT-GEMM (g=2) lane: exact, cycle-neutral, −138K LUT at LANES 128, and now confirmed on the routed netlist — 171 MHz fmax vs 174 (A2 alone) / 177 (broadcast base), so the LUT saving costs no clock that matters at the 66 MHz ship clock. Do not adopt the systolic activation chain for clock at LANES 128. L256 systolic-vs-broadcast: sys8 synth landed, its impl and the base L256 pair are still in the mesh queue.
Next pair = HEAD E3 + TAIL 0xE4B711F1 (CF + T1), gate-green changes only; building
The round-1 HOSTFAST=1 result (+4.8%, 50/50 identical) stands and is the launcher default. Two later missions (I7, I8) added a CUDA-event timing instrument that decomposes every step into V (target-verify GPU), D (tree+KV span) and G (host glue), then measured G directly instead of estimating it: G = 8.58 ms median / 6.00 p10 on the default path. Four new byte-exact cuts (root-image + prep-row CUDA graphs, a cost-fallback fast path, a single-transfer capture) take G to 6.28/5.03, and with the ex
The comparator is ready and its own self-test is green, but no card data has been captured yet — whether the pair's depth-3 accept gap (1.97->2.08 vs GPU's 2.64->3.19) is an op-level defect or the unprimed regime is still an open question until the pending silicon capture runs.
Both new hosts are fully provisioned and admitted to the mesh (auto-pick still held back until this page's checks landed); the toolchain and license path both trace back through asgard, so an asgard reboot restarts the license tunnel automatically while a box1 cold-AC does not affect these hosts at all.
L256 is the recommended GEMV-card width (routed, PDI exists); a drafter clock above 66 MHz is a seam fan-out + attention-combine project, not an array project — the array alone already closes 132-177 MHz in isolation.
Both levers are byte-exact in every gate, including all 10 full-HEAD arms (29/29 items, 928/928 KV rows, every RED fails exactly as predicted). RTL is committed (verilog 0220bf7, parent 2a6ea01a). Mode 1 (256-bit port alone) removes ~44% of the silicon export time because 1,216 per-cell B round trips dominate; page-capped bursts (mode 2) reach the model's ~40 us, but sit in R2 PIPE_EXP_MULTI's risk class (the 0x1190 accept-1.00 confound, also byte-identical in xsim), so mode 1 is the first image
The two-card protocol works end to end (24 items, 72-node tree, 0 CRC/seq/overflow) and the full-data bit-exact gate is GREEN at L=0 and L=40 (RTL committed verilog f29beae). Hop 3 must carry GLM + G11 (211 wire cycles). The GEMV item FIFO must hold K_F=8. RED arms red_flip and red_tag were harvested 16:10 EDT and both FAIL exactly as designed (non-vacuity proven); red_crd, red_crd1 and the fifo4 deadlock proof are still running on the laptop (slow host, in GEMV weight init since 2026-09-23 21:3
Narrow trees lose less accept than the pair model assumed (width 3 keeps 95.5-97.8% at depth 2, falling only to 90.7-93.4% at depth 15). Grid is COMPLETE: 132/132 runs (fa3 40/40, fa4 40/40, anchors, repeats, controls, bs2, E3 split, contamination re-runs). d2 w3 remains the pair's modelled optimum in every scenario and fa4-retention re-run, with a larger margin (+37% vs the model's original +27-35%). GPU bs2 raises the bar ~1.3-1.5x at unchanged accept; batch-2 verify only costs +6-13% over bat
The pair's drafter datapath is bit/rank-correct against the GPU golden (Test B). The accept shortfall vs GPU (2.22 vs 2.78 baseline; 1.97/2.08 vs 2.64/3.19 at width 3, depth 2/3) is explained (92% at width 8, and closed almost entirely at width 3 by Test A) by one mechanism: SGLang's GPU drafter always attends a fully-primed prompt KV, while the pair's default catch-up-only regime starts every request with an EMPTY drafter KV. Cheap KV priming is the lever, not an RTL fix; its own silicon A/B (c
Width 3 works end to end on silicon: same HEAD, new TAIL 0xE4B711CF, one AXI-lite register. The pair gains +32% tok/s at width 3 (111.1 -> 146.8 tok/s, mt_bench cap-32, unprimed) even though accept drops more than the GPU-only comparator predicts (1.97 vs a 2.10 GPU-shaped expectation). The extra 6% of accept is explained, not measured, by the pair's known unprimed-regime weakness (it loses proportionally more when only the top-3 of 8 ranks survive). W=2 and W=4 measured too (1.58 and 2.07); the
The laptop remains a verified, ready full-PDI build host, and everything D6 needs is staged there with no further work. But D6 itself has been queued twice and admitted neither time: the peak-50 waiter was abandoned by main at 07:51 and resubmitted at peak 46 (informed by D2's measured 38.3 GB build peak), and that second waiter was itself abandoned at 10:07:45 while still waiting on memory behind 8 long-running xsim_full_drafter jobs (492-669 min old) plus 3 headlever_xsim jobs. No D6 placement
HEAD 0xE4B711C8 (HEAD_PAR 8) + TAIL 0xE4B711CF (TISEQ + TREE_P 16 + runtime width) raised the matched pair benchmark from 102.13 to 112.73 tok/s at unchanged accept 2.22 (+10.4%), by making the HEAD's attention phase ~5.7x faster and the TAIL's tree walk ~1.47x faster on silicon. Width and depth are now characterized end to end: tok/s peaks at width 2 (153.4 @ 1.83), width 3 is close behind (146.8 @ 1.97, +32% tok/s over width 8 for -11% accept) and is the current default candidate; depth costs
Pieces 1 (table builder) and 3 (host mock) remain GATED/committed as before. Piece 2 (HBM fetch RTL) is now COMMITTED (verilog f792e27) with a clean run4 unit gate (latency sweep, parallel abort-drain, RED fired) and a first OOC synthesis census. The full-drafter xsim gate that would prove the fold end-to-end has NOT landed yet: 4 arms have been queued on the box1 mesh since 05:16-05:20 and were still in session-init at 08:00. No HEAD-level or silicon claim is made.
M2 (LANES 256 + E3 packed-DSP lane + locality + RET_SER + TILE_PIPE + FOLD) is bit-exact, closes 166 MHz and needs 2.16× fewer cycles; the drafter around it was not run at 166 MHz. M3 P&R is running (synth phase, box1, checked 2026-09-24 16:47 EDT, not yet routed); DOT_DSP L128 and K_PAR32 L128 P&R are still queued/pending.
On the placed HEAD 0x11C9 every GEMV lane's weight bank is on its own die, but all three activation broadcast replicas and every result consumer sit in SLR1, so most GEMV traffic crosses a die boundary; attention fills 40,205 of SLR1's 98,416 drafter SLICE and the TAIL carries an unused copy of it.
Live since 16:43 EDT: submit/cancel/adopt CLI, append-only ledger, 60 s dispatcher timer on box1, input push + output fetch-back, idle-host work requests. First real placements: two array-PoC arms moved off a saturated box1 onto the new 251 GB VMs.
M2 (LANES 256 + E3 packed-DSP lane + locality + RET_SER + TILE_PIPE + FOLD) is bit-exact, closes 166 MHz and needs 2.16× fewer cycles; the drafter around it was not run at 166 MHz. M3 P&R pending.
No page matches these filters.