EAGLE4 hardware research · author box1-qwen-junior · 2026-11-09 · source reports/results/DRAFTER_INTERFACE_GUIDE_4C1_20261109.md at repo commit 5ea8c7c7e · file references are repo-relative paths (path:line).

4C1 drafter interface guide: access, host driver and the per-step protocol (talon2, 2026-11-09)

For the user and Igor. Program 1 (the 4-card "4C1" EAGLE4 drafter for Llama-3.2-3B / Qwen3.8-27B, verified today by an H200) seen from the side of Program 2 (the target model running entirely on FPGAs), which wants to drive this drafter instead of the GPU. Written from the code: every claim carries a repo-relative path:line; files that live only on talon2 are marked talon2:~/... and were read read-only. Short version: [DRAFTER_INTERFACE_ONEPAGER_4C1_20261109.md](DRAFTER_INTERFACE_ONEPAGER_4C1_20261109.md). Nothing was flashed, run on a card, locked or removed to write this.

0. Answer first

  1. The drafter is four V80 cards on talon2 driven by one Python host process over PCIe (BAR0 registers + QDMA). Two ATT cards (PCI 17, 3d) hold half the attention heads each; two MLPV cards (63, 50) hold the MLP, vocab head and tree search. The cards talk to each other over QSFP; the host never relays card-to-card data during a tree.
  2. Per step the drafter needs, per accepted token: the token id and the target model's 3 auxiliary hidden states (3 x 3,072 for Llama-3.2-3B, 3 x 5,120 for Qwen3.8-27B). From those the host builds one 12,288-byte row per token (embedding + fc projection, fixed-point int16). It returns a tree of W + W^2(D-1) nodes (42 at width 2, depth 11): token id, first-child / next-sibling links, and a log2 score.
  3. Today the GPU is also used for three things beyond verify: the fc projection, the drafter prompt prefill whose K/V is imported into the ATT cards ("KVI"), and the first draft of each request ("lever Y"). All three have card-only replacements in the kit; the card-only K/V path is measured on silicon with unchanged accept and a −13.7 % / −24.6 % tok/s cost (section 5.3).
  4. Option (a), the target FPGAs talk to the drafter host process, is software-only: the card side already runs as a separate, torch-free card-server process with a request/reply API (scripts/4c1/c4_cardproc.py:191-212).
  5. Option (b), a direct QSFP link from the target cards, is not built. The link layer has free cages and one idle engine slot per ATT card, but no "verifier -> drafter" frame exists in RTL, and the per-item cross-card protocol currently needs host register traffic.

1. System picture

1.1 Cards and roles (talon2, 2026-11-09)

slotPCI (PF0)imagereg 0x100tp_link4 build idrolesource
ATT00000:17:00.0F3t 0xE4B713F9 (md5 97d8445c…)0xe4b711f90xe4b713f9attention, heads 0-15, rank 0serve log header CARDS: in reports/results/TRACK1_RECORD_F3T_20261109/config_sweep/serve3/*183008*.log.gz line 12
ATT10000:3d:00.0F3t 0xE4B713F90xe4b711f90xe4b713f9attention, heads 16-31, rank 1same
MLPV00000:63:00.00xE4B71342 (md5 3e82ea1d…)0xe4b711420xe4b71342MLP + vocab + tree, rank 0same; TRACK1_RECORD_F3T_20261109/README.md:3
MLPV10000:50:00.00xE4B713420xe4b711420xe4b71342MLP + vocab + tree, rank 1same
(other)99, bd, e1Program 2 / other owners––not part of the drafter.claude/skills/eagle4-talon2-connect/SKILL.md:37

1.2 QSFP links (the "K4" map in use)

linkcable (card + cage)link enginecarriessource
ATT TP pair17c0 – 3dc3e1 (tp_xchg)the G6 partial-sum exchange between the two attention halvestalon2:…/c4_bringup.py:32
MLPV TP pair63c3 – 50c0e1 (tp_xchg)G9 partial sums + vocab-shard merge:33
stage rank 017c1 – 63c2e0 (hop_xport)ATT0 -> MLPV0 per-item "BELL" (22,528 B) and MLPV0 -> ATT0 next item (6,144 B):34
stage rank 13dc0 – 50c3e0 (hop_xport)ATT1 <-> MLPV1, same:35
idle diagonal17c2 – 50c2, 3dc2 – 63c1–cabled, unused:36-37
empty cages17c3, 3dc1, 50c1; 63c0 has a stray cable (link up, no beacon)–freescripts/4c1/talon2/patch_talon2_k4_link_20261023.py:6

PORTMAP words written at bring-up: 17 = 0x45, 3d = 0x74, 63 = 0x76, 50 = 0x47 (talon2:…/c4_bringup.py:38). The link layer is tp_link4 (4 cages, Aurora 64B66B x4 per cage, beacon discovery, run-time engine-to-cage binding; verilog/forks/fourcard_4c1_att_dual_sibw_b_20261027/link4/x1_snapshot_2fcaa3d_rtofix/tp_link4.sv:1-24).

flowchart LR
  subgraph HOST["talon2 host"]
    V["verifier today: SGLang on the H200"]
    S["scheduler-side client: root row (embed + fc), KV rows, tree -> token/parent/score"]
    P["card-server process c4_cardproc.py (no torch): registers + QDMA"]
    V --> S --> P
  end
  P -- "PCIe: BAR0 regs + QDMA H2C/C2H" --> A0["ATT0 card 17 (heads 0-15)"]
  P -- PCIe --> A1["ATT1 card 3d (heads 16-31)"]
  P -- PCIe --> M0["MLPV0 card 63"]
  P -- PCIe --> M1["MLPV1 card 50"]
  A0 <-- "QSFP 17c0-3dc3 TP (G6 sums)" --> A1
  M0 <-- "QSFP 63c3-50c0 TP (G9 sums, vocab merge)" --> M1
  A0 <-- "QSFP 17c1-63c2 stage (BELL / next item)" --> M0
  A1 <-- "QSFP 3dc0-50c3 stage" --> M1
  M0 -. "tree SoA read by host (C2H)" .-> P

1.3 What runs where

on the host (talon2 CPU / GPU)on the cards
target model + verify (SGLang, H200)the whole drafter layer for every tree item: G1 norm, QKV, attention over the drafter KV cache (ATT); o-proj, MLP, final norm, lm-head, top-K and tree search (MLPV)
root row: embed_tokens(token) and fc(target 3 x hidden), quantised to int16 (scripts/fpga_tree_marshaller.py:1414-1467)item-to-item hand-off of the tree, level by level, over QSFP with no host involvement (loop_seq_d2, scripts/4c1/c4_loop.py:5-17)
drafter K/V import for the prompt and accepted tokens (KVI) unless the card-only path is enabled (scripts/4c1/c4_kvi.py:1-3)KV-only items: the cards compute K/V rows from host rows (c4_loop.py:225-253)
the first draft of each request on the GPU when lever Y is on (EAGLE4_4C1_FIRST_GPU=1 in the serving env)–
SoA -> func_info (tokens, parents, scores) for SGLang (fpga_tree_marshaller.py:586-660)the tree result written by MLPV's OUT-DMA into card HBM (verilog/forks/fourcard_4c1_mlpv_g11f_lsi_eaearly_20261104/src_full/top/tree_dma_writer.sv:7-35)

2. Access

2.1 Reaching talon2

2.2 The card lock

2.3 QDMA and device files

2.4 Identity checks (before every run)

2.5 PERSIST: arm the deepest depth before the first AP_START

3. Host driver stack, layer by layer

Kit root scripts/4c1/ (stdlib only, runs under python3 -I; the talon2 kit is a copy at talon2:~/eagle4_talon2_4c1/f3twin_20261109/kit/repo plus the image patch TRACK1_RECORD_F3T_20261109/f3t_kit/patch_13f9_host.py).

layerfilepurposekey functions (line)
L0 accessc4_bar.pyBAR0 mmap with write windows + QDMA pread/pwriteSiliconBar, QdmaDev :165, pwrite :186, pread :197
L0 mapc4_regmap.pythe ONE register table, each row citing the RTL decode line; image table; ring bases_r(...) rows :338-450, image rows :81-267 (F3t :192), ring bases :841-856, BELL :858-859, loop CSRs LC :809, tp_link4 CSRs TP :796
L1 cardc4_card.pyone card = drafter / loop / tp_link4 accessors + item helpersprogram_inputs :109, issue_item :157, wait_out_seq :172, wait_tree_done :183, poison :194, read_soa :205, census :271
L1 bring-upc4_bringup.py, c4.pydiscover / link / boot weights / arm / golden / census CLIc4.py subcommands and rc table :4-22
L2 treec4_tree.pyheader word, pool geometry, SoA decodep2_hdr :27, n_nodes_of :39, unpack_soa :51, link_problems :61
L2 sessionc4_loop.pyone request / KV-only row / whole tree on four cardsbegin_request :204, kvonly_row :225, enable_links :255, program_cfg :291, run_tree :346, _run_tree_wait :434
L2 chain/censusc4_chain.pyback-to-back trees with per-phase dwell (--set fsm)used for the census in section 4.7
L3 KVc4_kvi.py, c4_kvi_hooks.pyimport the GPU drafter's K/V into both ATT ranks (16 heads each)c4_kvi.py:1-45 contract; hooks c4_kvi_hooks.py:1-12
L3 KV (card-only)c4_kvpo.py, c4_kvx.pyaccepted-token (and prompt) K/V computed on the cards from host rowsc4_kvpo.py:1-20, c4_kvx.py (send, run_batch)
L3 KV commitc4_kvcommit.pykeep drafted K/V of accepted nodes on the card (COMMIT_PATH), images with KV_COMMITheader :1-60; off in today's serving env
L4 SGLang backendc4_sglang.pyFourC1Fpga behind the stock FpgaTreeDrafter: captures register writes, runs run_treeFourC1Fpga :48, launch :126, open_session :225, build :415, arm :472
L4 card processc4_cardproc.pyFASTGLUE=4: all card ops in a separate torch-free process, pickled RPC over a socketpairop table :191-212, client marshal_inputs :459, remote_stage_cfg_launch :478
L4 tree mathscripts/fpga_tree_marshaller.pyroot row build, SoA -> func_info, structural checksq412 :305, qhfc :424, marshal_inputs :1374, root_image :1414, soa_to_func_info :586, validate_func_info :783
L5 SGLang hookscripts/4c1/sglang_site/sitecustomize.pyput on PYTHONPATH by the launcher (run_4c1_sglang.sh:96), so it runs in every interpreter incl. SGLang's scheduler: arms the eaglefit FPGA hook, points its driver factory at c4_sglang.build, arms the KVI hooks; edits nothing inside SGLang:1-40
L5 SGLang patchesscripts/cost_draft_tree_fpga_eaglefit.py, scripts/cost_draft_tree_fpga.py (in this repo, no other copies)import-time patches of SGLang 0.5.16's CostDraftTree: draft_model_decode -> one whole-tree FPGA launch _draft_fpga_tree_v14 (:219-221, :638), prefill capture of the target 3H hidden (:223-225), verify-extend capture _capture_extend_feed (cost_draft_tree_fpga.py:394-440), marshal_inputs + launch_and_read (:852-853)
L5 launchersrun_4c1.sh, run_4c1_sglang.sh, run_4c1_sglang_qwen.sh, run_4c1_nstream.sh, demo/demo_4c1.shbring-up chain; serving env + engine commandrun_4c1_sglang.sh:60-101
extrasc4_tp4.py, c4_nstream.py, c4_pingpong.py, c4_hbmreplay.py, c4_batchverify.py, native/7-card TP4 map; N-stream / ping-pong serving; HBM replay; batched verify; C fast path (native/c4h.c) and a dma-buf probe (native/qdma_dmabuf)not on the 883 tok/s path

Serving levers of the 883 tok/s config, "PQ+C": P = EAGLE4_4C1_HP_PREPOISON, Q = EAGLE4_4C1_HP_PINPAR, C = EAGLE4_4C1_HP_CAPHOST + EAGLE4_4C1_HP_IMGBYTES (mapped in TRACK1_RECORD_F3T_20261109/f3t_kit/serve_f3twin.sh:11,41-44; implemented in c4_hpre.py, which exists only in the talon2 kit, repo copy reports/results/SEVENCARD_BOARD_20261026/sibw_talon2/hostpre_20261107/kit/c4_hpre.py:14-21). run_4c1_sglang.sh refuses W > 2 unless EAGLE4_4C1_ALLOW_WIDE=1 (:48-51), takes the card lock itself unless CARD_LOCK_HELD=1 (:108-115), and runs a link check before SGLang starts (:128-138). Discover/link/boot/arm/golden are done before it by run_4c1.sh (:4,38).

Call order in a serving run (in-process view; with FASTGLUE=4 the card half runs in c4_cardproc):

  1. sitecustomize -> cost_draft_tree_fpga_eaglefit.arm() -> c4_sglang.arm() replaces _build_drv_v14 with c4_sglang.build (sitecustomize.py:13-35, c4_sglang.py:472-475).
  2. build(cdt) -> c4_cardproc.build_remote (FASTGLUE=4) or build_local -> open_session (identity, arm state, golden record) -> FourC1Fpga + the drafter class + c4_kvi.attach (c4_sglang.py:415-469).
  3. Per request: begin_request pulses 0x154 on both ATT ranks (c4_loop.py:204-213); KVI imports the prompt K/V.
  4. Per step (scripts/cost_draft_tree_fpga_eaglefit.py:638-859): after the previous verify the extend capture stores {pos: (token, target 3H row)}, the GPU drafter extend runs and its K/V rows are gathered (c4_kvi_hooks.py:65-109) -> the next draft joins pending card work, pulses 0x154 on a new request (eaglefit.py:736-762) -> picks the root input (the captured 3H row at seq_len-1, :764-781) -> KV catch-up c4_kvi.catchup loads the K/V rows into both ATT ranks (:784) -> marshal_inputs (root row + config capture, :852) -> launch_and_read (:853) -> run_tree on four cards -> SoA -> rank cross-check -> soa_to_func_info -> func_info_to_torch -> SGLang verify.

4. The per-step protocol

4.1 Registers the host writes (drafter CSR block, offsets from 0x0201_0001_0000)

A/ = verilog/forks/lutgemm_att_d9_e0_20261104/src_full/top/, M/ = verilog/forks/fourcard_4c1_mlpv_g11f_lsi_eaearly_20261104/src_full/top/. (The _r rows in c4_regmap.py cite the older v0/v1 forks; the lines below are for the two images on the cards.)

offnamewritten whenmeaningATT decodeMLPV decode
0x000AP_CTRLonce per boot (first item)W[0] startA/control_regs.sv:862-865M/control_regs.sv:811
0x010ctx_lenevery tree / KV rowsequence position (md-aware clamp by host, c4_loop.py:59):872:821
0x018K_Fevery treefrontier 8 (c4_loop.py:304)––
0x020max_deptharm + every treetree depth D (<= armed depth):900:834
0x028-0x03Cout0/1/2 baseevery tree (MLPV)OUT-DMA bases 0x41_4000_0000 / +0x1000 / +0x2000:901-906:836-841
0x054-0x074boot-DMAper input rowHBM -> input buffers walk (emb 0x54[2], hfc 0x54[3]), status 0x74:918-925–
0x100BUILD_IDread onlyidentity:1185:1141
0x14Cn_tilesevery tree (MLPV)vocab shortlist tiles, 0 = full vocab:991:920
0x150K_Cevery tree (MLPV)exact-rescore shortlist (64 in serving):994:923
0x154KV resetevery new request (ATT)W[0] pulse, R {0x4B56, cnt} must read cnt 0:998:927
0x158NORM_SELarm0x2F (reads 0x4E53002F):1003:932
0x15CACCFIXper KV-only item (ATT)bit3 = the next item is KV-only; R {0xACF1, cfg}:1005:934
0x180IN_SEQper itemdoorbell k+1(c4_regmap.py:362)
0x184OUT_SEQreaditems finishedA/control_regs.sv:1078M/control_regs.sv:1034
0x198HDRper item{new_tree[31], level[27:24], branch[23:21], parent[20:18], token[17:0]} (c4_tree.py:4,27-31)|
0x1C0CONSUMEDreaditems started; must equal k after the header write (c4_card.py:160-163)|
0x1CCrank / widtharm (ATT rank), every tree (MLPV width)ATT: rank + sticky item error; MLPV: tree width W:885 / R :1097:835 / R :1052
0x004, 0x00CCOMMIT_LANES / CFGKV commit onlyATT KV_COMMIT (not used in today's serving):883-884absent

Loop sequencer CSRs (0x0201_0104_F000 + off, c4_regmap.py:809-813): per tree the host writes TREE id, KH0/KT0 (item counters), DW = (W << 4) | md, WANT, ROOT header, THETA (TREE_STOP threshold, Q12.20), DMIN, the four ring bases, and CTL = EN | mode bits | ARM, on the MLPV first and then its ATT (c4_loop.py:380-392).

4.2 Card HBM buffers (addresses as the host sees them through QDMA)

addresswhatwritten bysource
0x42_0040_0000FASTIN: the 12,288 B root/KV row, staged per item on both ATT cardshost H2Cc4_regmap.py:847, c4_card.py:109-128
0x42_0041_0000 + t x 12,288batched KV-only rows (KVX)host H2CKVONLY_EXTEND_PHASE0_20261108.md:15-17
0x41_4010_0000ATT OUT ring (MLPV reads it over the link)ATTc4_regmap.py:841
0x42_B000_0000MLPV / ATT IN ring (slot stride 0x14000, item offset 0x11800)linkc4_regmap.py:842-846
0x41_4000_0000tree result SoA (20,480 B), MLPV onlyMLPV OUT-DMAc4_regmap.py:851, c4_tree.py:18-21
0x41_8000_0000, 0x41_C000_0000exact fp16 lm-head + duplicatebootc4_regmap.py:849-850
KVI base (0x1F0/0x1F4)imported K, V at +128 MiBhost KV_LOADc4_kvi.py:12-14

4.3 One tree, step by step (scripts/4c1/c4_loop.py:346-405)

  1. Poison the SoA on both MLPV (0xEEEEEEEE at region0[0] and the last score; c4_card.py:194-196) so a stale read cannot pass.
  2. Program config on all four cards (ctx, K_F, md) and on MLPV (K_C, n_tiles, out bases, width with read-back) (c4_loop.py:291-315).
  3. Enable the link DMA once per card and prove it in st_misc (c4_loop.py:255-276).
  4. Arm the loop sequencer of each pair, TAIL (MLPV) first then HEAD (ATT), with identical tree/depth/width/root (:380-392).
  5. Stage the root row on both ATT cards (FASTIN H2C + two boot-DMA walks; c4_card.py:109-128).
  6. Issue only the root item on both ATT ranks: header 0x198, audit 0x1C0, doorbell 0x180 (c4_card.py:157-170, c4_loop.py:403-404). From here the cards run the whole tree themselves: ATT -> MLPV BELL, MLPV -> ATT next item, level by level.
  7. Poll the four loop STATUS words; TREE_STOP verdicts (early stop when the level's probability mass is below theta) are taken by the host only when both MLPV ask at the same level with identical scores (c4_loop.py:317-344).
  8. Done = four DONE, links drained, loop DIGEST of HEAD == TAIL per pair and pair 0 == pair 1 (:484-497).
  9. Read the SoA from BOTH MLPV, require byte-identical regions, identity retrieve-index, per-item status clean (:510-536). Any failure discards the tree (never returned).

4.4 BELL / frame contract ATT -> MLPV (card to card, no host)

messagebytes per item per ranklayoutsource
BELL, ATT -> MLPV22,528r5 product half 8 KB @ +0, unwritten gap @ +0x2000, h1 6,144 B @ +0x4000c4_regmap.py:858-859; A/pipe_seam_bridge.sv:5,20-24
ITEM, MLPV -> ATT6,144next item's hidden half, TAIL OUT slot(parent)+0x11800 -> HEAD IN slot k+0x11800M/eagle4_drafter.sv:2297; loop_seq_d2.sv:58,333
control frame256-bit[51:48] type (BELL 1, ITEM 2, SPEC 3, COMMIT 4, CANCEL 5, STOP 6), [63:52] tree, [95:64] header, [127:96] kverilog/forks/fourcard_4c1_mlpv_g11f_lsi_eaearly_20261104/common_overlay_strap1/rtl/loop_seq_d2.sv:7-13,101

4.5 How the tree comes back (OUT-DMA SoA, written by MLPV only)

Writer M/tree_dma_writer.sv:7-35,66-69; host decode scripts/4c1/c4_tree.py:51-54.

regionbyte offsetint32 per node
0+0x0000draft token id (18-bit)
1+0x1000retrieve index = identity (integrity check)
2+0x2000first child (pool index, −1 none)
3+0x3000next sibling (−1 none)
4+0x4000cumulative log2 probability, Q12.20

4.6 KV after verify (what the verifier must send back)

4.7 Measured timing

whatvaluesource
card tree issue -> done, p50, w2 K_C 642,234 µs at d7; 2,774 µs at d9; 3,384 µs at d11 (about 135-152 µs per extra item)reports/results/DRAFTER_SPEED_RESEARCH_20261107/F3T_CONFIG_SWEEP_20261109.md:154-157
ATT G5 attention per item~40 µssame table
GPU verify-only time, batch 13.45 msreports/results/BATCH_BUBBLES_20261109/INTERLEAVE_BATCHVERIFY_SILICON_20261109.md:76
end-to-end, d11 w2 fa3883.87 tok/s @ accept 6.90 (19,615 tokens, 2,843 verify steps, 22.19 s)serve log config_sweep/serve3/*183008*.log.gz (md5 f11c5ff1)
card-only KV path (E2b image, w2 d7)KVX 696.24 tok/s (−13.7 %), KVX+PROMPT 607.83 (−24.6 %), accept 5.88-5.89 unchanged; step p50 6.94 -> 8.24 msKVONLY_EXTEND_PHASE0_20261108.md:152-166
QSFP link12.48 GB/s per direction; echo round trip 1.20 µs at 6 KiBreports/results/MULTIFPGA_INTERCONNECT_CENSUS_20260915.md:13,71
QDMA host relay (card -> host -> card)1.85-1.90 GB/s at 1 MiB; 28.7-29.3 µs one-way at 6 KiBMULTIFPGA_INTERCONNECT_CENSUS_20260915.md:14,78

5. Verifier interface (for Program 2)

5.1 What a non-GPU verifier must provide and consume, per step and per stream

diritemLlama-3.2-3BQwen3.8-27Bdtype / notes
inaccepted token ids (this step), in order1 .. D+1 idssametarget vocab ids (128,256 / 248,320)
intarget aux hidden of every accepted position3 x 3,072 per token (18,432 B fp16)3 x 5,120 per token, layers 3/31/59 (30,720 B bf16)c4_profile.py:47,69-70
inprompt (once per request): ids + aux hidden of every prompt positionas above per tokenas abovefor the prompt K/V (KVX_PROMPT) or for a GPU/CPU drafter prefill (KVI)
inper-step configctx_len, depth D (<= armed), width W, K_Csamethe host writes the CSRs
host-internalroot / KV row12,288 B = emb Q4.12 int16 x 3,072 + fc-out Q5.11 int16 x 3,072same 12,288 B (fc 15,360 -> 3,072, with fc_norm)fpga_tree_marshaller.py:305-314,424-428; EAGLE4_TREE_HFC_FRAC=11 in the serving env
outtreeW + W^2(D-1) nodes + root (43 at W2 D11)same; tokens mapped draft -> targettokens int64, scores float32, prev/next/side int
outverify candidatesthe verifier prunes to VTOK nodes (20 in the 883 config)sameSGLang's own prune today

Needed on the drafter side regardless of who verifies: the drafter's embed_tokens table and fc weights (Llama fc 9,216 -> 3,072; Qwen fc_norm x3 + fc 15,360 -> 3,072) to build rows, and for Qwen the d2t table (c4_profile.py:13-18,74-75).

5.2 Latency budget (measured parts)

The cards need 3.38 ms per d11 tree (2.23 ms at d7). Today's whole step at d11 is 7.81 ms (derived in the bar below) with a 3.45 ms GPU verify. A target-FPGA verifier is inside today's budget if its verify + the two hand-offs stay at or below what the H200 path uses now.

5.3 Option (a): target FPGAs talk to the drafter host process; the host keeps driving the cards

exists todaymust be built
card-server process with a request/reply API, torch-free: ops stage_cfg, stage_cfg_launch(emb, hfc, ctx, md, vid) -> func_info, launch_and_read, kvonly_rows(imgs, ctxs, toks), kv_residency_clear, kvi_load, stats (c4_cardproc.py:132-212)a network endpoint in front of it (today: pickle over a local socketpair, c4_cardproc.py:21)
row builder root_image / prep_rows (embed + fc + quantise)a CPU (or target-FPGA) implementation of embed + fc, today done on the GPU in the client (c4_cardproc.py:449-457)
card-only K/V: KV-only items, KVX batching, KVX_PROMPT for the prompt; accept unchanged on siliconrun with EAGLE4_4C1_FIRST_GPU=0 (first draft on the cards) and no GPU drafter prefill; not measured together end to end
per-tree checks (rank cross-check, digests, poison) inside the card processthe verifier-side tree prune and accept logic (SGLang does it today)

5.4 Option (b): direct QSFP from target cards to drafter cards

exists todaymust be built
tp_link4: 4 QSFP cages per card, Aurora 64B66B, beacon discovery, run-time PORTMAP binding (tp_link4.sv:1-24)a "verifier -> drafter" frame (accepted ids + hidden rows, or ready-made 12,288 B rows) and an RTL consumer that writes FASTIN and rings the doorbell without the host
free cages: 17c3, 3dc1, 50c1 empty; 63c0 cabled (stray); per the 7-card plan 17c3 may already run to card bd and 63c0 to e1 (ATT_TP4_20261108.md:44,49), unverified by beaconcables / beacon proof to the target cards
one idle engine slot per ATT card: tp_link4 has 3 engine slots and at TP2 the ATT's e2 is tied idle (ATT_TP4_20261108.md:32; tp_link4_bd.v:99-108)a third engine (e.g. hop_xport2) bound to that slot on the ATT image
precedents: KV-PREFILL design fed the ATT through a free cage/engine (DESIGN_KV_PREFILL_CARD_20261022.md:15,17,77); MLPV-SOLO and TP4 "ring B" (second ingest ring at IN + 0x140000 with hop_xport2) (MLPV_SOLO_20261022.md:27-28, SEVENCARD_MLPV_0x134D_20261109.md:27-47)the fc (and Qwen fc_norm) on the target or drafter card, today host-side
tree result already in MLPV HBM at 0x41_4000_0000a "drafter -> verifier" return path over QSFP; and moving today's host-side per-tree work (config writes, loop arming, TREE_STOP verdicts, SoA cross-check) onto the cards

6. How to run it today (talon2, from the committed docs)

All card steps as root inside one card lock; SGLang as carrick.

  1. Bring-up after a cold AC (TRACK1_RECORD_F3T_20261109/f3t_kit/win_f3t.sh:18-31): QDMA queues on 17/3d/50/63 (03_setup_qdma.sh 4 <bdf>), then c4.py --model llama3b discover, link, boot, arm --depth 15.
  2. Golden / identity check (win_f3t.sh:32-33):
    c4.py --model llama3b golden --disc RUN/discover.json --md 15,7,2 --out RUN/golden.json, then python3 -I cmp_golden_levels.py <reference golden.json> RUN/golden.json (rc 0 = level-identical). The F3t window passed 12/12 trees vs the E2b record (TRACK1_RECORD_F3T_20261109/EVALUATION.md:81).
  3. Serving, the 883 tok/s config (F3T_CONFIG_SWEEP_20261109.md:178-191; as root inside a lock, the driver starts SGLang and chowns flashinfer back to carrick after each round):
    OWNER=<you> bash ~/eagle4_talon2_4c1/lock/card_run.sh env MODEL=llama3b ORDER=PQC \
    RUNDIR=/home/carrick/eagle4_talon2_4c1/runs/cfgsweep_20261109/l2 OUTD=<out> \
    DEPTH=11 WIDTH=2 VTOK=20 KC=64 BACKEND=fa3 EAGLE4_PAIR_TREE_STOP_THETA=-2.5 ARM_PINPAR=0 EAGLE4_4C1_ALLOW_WIDE=1 \
    bash /home/carrick/eagle4_talon2_4c1/cfgsweep_20261109/serve_cfg_drv.sh

    The engine command it produces: engine_test.py … --draft-tree-depth 11 --draft-tree-width 2 --verify-tokens-num 20 --max-new-tokens 256 --max-requests-num 80 --speculative-exec-mode 1 --speculative-draft-attention-backend fa3 --sglang-version 0.5.16 --enable-cuda-graph --enable-piecewise-cuda-graph with the FPGA env in the log header (serve3/*183008*.log.gz lines 5, 8).
  4. Census (card µs per tree, per phase): c4_chain.py --set fsm, 40 trees (F3T_CONFIG_SWEEP_20261109.md:151; scripts config_sweep/scripts/census.sh, chain_b*.sh).
  5. Offline (no card): python3 -I scripts/4c1/c4_selftest.py | tail -1 runs the whole flow against the mock four-card world (4C1_SILICON_RUNBOOK.md:113,134).

7. Traps

  1. PERSIST latch. The first AP_START after boot fixes the OUT-DMA ABI; arm the deepest depth first (section 2.5). Recovery needs a cold AC.
  2. Build-id guards. TREE_EXPECT_BUILDID (scripts/fpga_tree_marshaller.py:193-197) is read only by the single-card driver builder (scripts/cost_draft_tree_fpga_eaglefit.py:428-433), which the 4C1 path replaces, so setting it does nothing for 4C1. The 4C1 kit keys on its regmap image rows (c4_regmap.py:81-267) plus the flash-record md5 (:36-41), re-checked when the serving process opens the cards (c4_sglang.py:238). A new image needs a row (talon2 uses patch_13f9_host.py); adding ids to the old pair allowlists would misdrive the cards (4C1_SILICON_RUNBOOK.md:11 blocker B3, :85-96).
  3. Root-owned lock files. card.lock, CARD_OWNER and card_runs.log are root-owned in a carrick directory, so the lock only works as root; give outputs back to carrick after root jobs (scripts/talon2/post_boot_talon2.sh:24,51).
  4. Chat-date drift. The Llama-3.2 chat template inserts today's date; text-identity gates across midnight change with no defect. Pin EAGLE4_CHAT_DATE ('26 Jul 2024', scripts/4c1/run_4c1_sglang.sh:87; memory feedback_chat_template_date_drifts_texts_across_days).
  5. SGLang as carrick, not root. A root run leaves root-owned flashinfer JIT locks in ~carrick/.cache/flashinfer that crash later carrick runs; the serve driver chowns them back after every round (serve_cfg_drv.sh:57; memory feedback_talon2_sglang_as_carrick_not_root).
  6. fa4 tree-draft defect. FA4 paged-KV non-causal attention is wrong/nondeterministic on the H200; use --speculative-draft-attention-backend fa3 (makes d11 deterministic, F3T_CONFIG_SWEEP_20261109.md:21-25; memory project_gpu_fa4_tree_draft_defect).
  7. 1-token requests crash the harness. engine_test.py:286 reads meta_info["spec_verify_ct"], absent when no verify ran (TRACK1_RECORD_F3T_20261109/PER_PROMPT_BENCH_20261109.md:39). Use max_tokens >= 2.
  8. Hardcoded REPO paths. run_4c1_nstream.sh hardcoded the talon2 kit copy sibw/repo and refused F3t; now REPO=${EAGLE4_HP_REPO:-…} (scripts/4c1/run_4c1_nstream.sh:19, commit 1753f9c37). The box1 copies of run_4c1_sglang.sh:28, run_4c1_sglang_qwen.sh:20 and demo/demo_4c1.sh:32 still hardcode box1's repo path; the talon2 kit copies of the two serving launchers read EAGLE4_HP_REPO. Set EAGLE4_HP_REPO to the kit you mean to run.
  9. MEASURE-ONLY stamp. The serving launcher runs with MEASURE_ONLY_GOLDEN_DEPTH_UNPROVEN=1 and stamps every line (run_4c1_sglang.sh:79-85): the bf16 GPU golden at depth >= 1 prints FAIL (2-bit vs bf16); correctness is carried by level identity vs the previous image and text agreement with GPU-only (75/80 at d11 fa3) (EVALUATION.md:21-26, F3T_CONFIG_SWEEP_20261109.md:21-25).
  10. **Wide trees need EAGLE4_4C1_ALLOW_WIDE=1 in the launcher's own env**, or it refuses with rc 61 (F3T_CONFIG_SWEEP_20261109.md:87-91; run_4c1_sglang.sh:48-51).

Calculations and hypothesis