EAGLE4 · hardware research · 2026-10-06

EAGLE4 Trace: what every block is doing during a draft step

EAGLE4 Trace converts xsim cycle logs, box1 host-side profiles and (once added) on-card counters into Chrome Trace JSON for ui.perfetto.dev. Each card, the GPU and the host is a process. Each hardware block of the architecture diagram is a track. Slices are stages, items and levels; arrows are hand-offs (seam HEAD→TAIL, host relay, verify). analyze.py marks every idle gap as a bubble, infers its cause, and summarises utilisation per engine. Every number is labelled M measured, D derived or P projected.

Swimlanes

Use the wheel to zoom and drag to pan. Hover a slice for its args. Grey hatched slices are bubbles, labelled with their inferred cause. Each row is one diagram block: label [block id · clock]. The block chips below link to the architecture report.

Utilisation per engine

busy (blue) vs bubble (grey) D

The what-if column P is the projected step if that engine's top bubble were removed: span − min(top-cause bubble, the time the causing engines run alone on the critical path). It is an upper bound on the gain. Removing one bubble can expose the next dependency.

What it shows

Collecting a trace

All file:line citations are for the files in this directory unless a path is given.

1. Trace format contract (schema eagle4-trace/1)

The output is one JSON object: {"traceEvents": [...], "displayTimeUnit": "ns", "otherData": {...}}, written by Trace.write (eagle4trace.py:134-141).

FieldType / unitMeaning in EAGLE4 TraceEmitted at
otherData.schemastringFormat version, "eagle4-trace/1" (SCHEMA_VERSION, bumped on any field change).eagle4trace.py:58, :136
otherData.title, .tool, .labelsstringTrace title, "EAGLE4 Trace v1", and the label legend. Converters can add keys (e.g. summary_us).eagle4trace.py:136-137
displayTimeUnit"ns"Display hint only. It does not change the unit of ts/dur.eagle4trace.py:135
phstringEvent type. Emitted: X complete slice, i instant, C counter, s/f flow start/end, M metadata. Not emitted: B/E, flow step t, bind_id.see the rows below
namestringSlice name: stage, item, level, or bubble: <cause>. A counter name carries its label: "<name> [M|D|P]".eagle4trace.py:95, :105
catstringFor X: the diagram block id, "bubble" for analyzer bubbles, "flow" for flows.eagle4trace.py:96, :111; analyze.py:120
tsfloat, µsStart time from the trace origin, rounded to 1 ns.eagle4trace.py:95
durfloat, µsX only; clamped to ≥ 0.eagle4trace.py:95
pidintOne per process: HOST 1, GPU 2, HEAD 10, TAIL 20, HEAD2 30, TAIL2 40, LINK 50.eagle4trace.py:20
tidintInteger index of the track in TRACKS (1-based, in dict order), not the block-id string. The block id is in args.block and in the thread name; block_track_map.tsv lists track → block.eagle4trace.py:23-57
idint (flows) / string (util counters)Flow pairing id (s and f share it). On util counters it is the analysis window name.eagle4trace.py:111-112; analyze.py:112
s"t"Instant scope = thread.eagle4trace.py:101
bp"e"Flow end binds to the enclosing slice.eagle4trace.py:112
args.label"M"/"D"/"P"Fidelity of the timing (on every X and i; on C it is in name).eagle4trace.py:91, :100
args.srcstringSource description (log run / hostprof file). xsim slices also carry args.src_line = <log file>:<line> of the monitor line used.eagle4trace.py:91; xsim2trace.py:98-99
args.blockstringDiagram block id of the track.eagle4trace.py:91
args.clockstringClock-domain label of the track, e.g. "usr_1 132 MHz (2:1)".eagle4trace.py:91
args.clock_mhzfloatNumeric MHz of the track's clock, set when the label has N MHz (not on CPU/GPU/link/HBM tracks).eagle4trace.py:61-65, :92-93
args.cycles, args.cycle_clock_mhzint, floatxsim only: TB-clock cycles of the slice and the silicon clock used to convert them (dur = cycles / cycle_clock_mhz). HEAD stage cycles are counted in the top-FSM domain, so h_array slices have cycle_clock_mhz 66 while their track clock_mhz is 132.xsim2trace.py:98-99, :133-134
args.item, args.levelintxsim HEAD: ITEMMON done_seq and lvl_max. TAIL: level (0 for first-level stages, idx on t_lvl slices).xsim2trace.py:98, :104-106, :115-117, :143
args.step, args.raw, args.kindint, stringhostprof: decode-step index, original hostprof label, card interval kind (item/kvonly/commit).hostprof2trace.py:44-62
args.bubblebooltrue on slices that are waiting, not work: xsim WAIT, host envelopes (draft_fpga_tree_v14, cf._card_join, sglang.target_model_verify), analyzer bubbles. Busy-time accounting skips them.xsim2trace.py:104-106; hostprof2trace.py:20, :47; eagle4trace.py:153-154
args.causestringAnalyzer bubbles only: the inferred cause.analyze.py:121
args.resolution_usfloatcardctr level slices: the poll period (± timing).cardctr2trace.py:90-91

Counters (ph: C) live on the process (no tid). Each key in args is one numeric series.

NameSeries (unit)LabelEmitted at
KV rows written [M]kv_rows (KVWR monitor lines per item)Mxsim2trace.py:108
TREE serializer FIFO occupancy [M]occupancy, capacity, survivors (entries)Mxsim2trace.py:146
util <track> [D]busy_pct, bubble_pct (% of the window; two points, window start and end)Danalyze.py:109-113
pipe_lvl_count [M]lvl (CSR 0x19c value)Mcardctr2trace.py:84
boot_drop / tp_drop [M]boot_drop, tp_drop (counts)Mcardctr2trace.py:85
kvi_status [M]kvi_status (raw CSR 0x1e8)Mcardctr2trace.py:86
stage busy % (perf window) [D]one series per perf-window stage (% of cycles in the poll interval)Dcardctr2trace.py:103-105

Counters do not carry args.src. The label is in the name (eagle4trace.py:103-106).

Flows (s + f with the same id, cat: "flow"):

Process/thread metadata (ph: M, eagle4trace.py:114-132):

Minimal examples (taken from the shipped traces):

{"ph":"X","name":"G4 qkv GEMV","ts":6.924,"dur":54.091,"pid":10,"tid":5,"cat":"h_array","args":{"label":"D","src":"xsim FH_REF3 (full-HEAD BOOTSKIP, HX1 BD RTL) ITEMMON","block":"h_array","clock":"usr_1 132 MHz (2:1)","clock_mhz":132.0,"item":0,"level":0,"cycles":3570,"cycle_clock_mhz":66.0,"src_line":"xsim_FH_REF3_head.log:363"}}
{"ph":"X","name":"bubble: dependency stall (other HEAD stage busy)","ts":125.394,"dur":176.197,"pid":10,"tid":2,"cat":"bubble","args":{"label":"D","cause":"dependency stall (other HEAD stage busy)","block":"h_emb","bubble":true}}
{"ph":"i","s":"t","name":"step 52 start","ts":0.0,"pid":1,"tid":24,"args":{"label":"M","src":"box1 hostprof hostprof_P1HP_950437.json","step":52}}
{"ph":"C","name":"TREE serializer FIFO occupancy [M]","ts":722.411,"pid":20,"args":{"occupancy":1689,"capacity":16384,"survivors":2579}}
{"ph":"s","id":1,"name":"seam HEAD->TAIL","cat":"flow","ts":452.485,"pid":10,"tid":10}
{"ph":"f","bp":"e","id":1,"name":"seam HEAD->TAIL","cat":"flow","ts":488.424,"pid":20,"tid":12}
{"ph":"M","name":"process_name","pid":10,"args":{"name":"V80 HEAD card"}}
{"ph":"M","name":"thread_name","pid":10,"tid":2,"args":{"name":"G1 embedding gather  [h_emb · usr_0 66 MHz]"}}

2. Collection methods

(a) xsim (cycle-level, any host with Vivado; no card)

All paths are under /shared/scratch/shape_generic/.

(b) Silicon host side (box1 only; HQ reads copies)

The relay process ships its card stamps back when EAGLE4_HOSTPROF=1 (pair_card_server.py:129-131, :167). Card intervals are built in _card_busy (pair_hostprof.py:372).

Cited paths are the box1 reference copy under /shared/scratch/consolidated_hxf/cons_host/box1ref/R_scripts/. On box1 they live in the repo scripts/.

(c) On-card counters (cardctr2trace.py, box1 only; HQ has not run it)

CardOffsetRegister
HEAD0x244G1
HEAD0x248G4
HEAD0x24cG5
HEAD0x250G6
HEAD0x254G7G
HEAD0x258G7U
HEAD0x25cEXPORT
HEAD0x260WAIT
HEAD0x264KVIFAN
HEAD0x268 / 0x26cHBM_BYTES lo / hi
HEAD0x270URAM_RD
HEAD0x274DSP_BUSY
TAIL0x244INGEST
TAIL0x248G7P
TAIL0x24cG9
TAIL0x250G11
TAIL0x254GLM
TAIL0x258G12
TAIL0x25cTREE
TAIL0x260G8
TAIL0x264WAIT

3. Analysis step (analyze.py)

4. Producer API (eagle4trace.py)

from eagle4trace import Trace, TRACKS, PID, CLK_MHZ, SIM_CLK_NS, bubbles, attribute, overlap, clock_mhz
tr = Trace(title)                                                   # :75
tr.slice(track, name, ts_us, dur_us, label, src, proc=None, **args) # :89  ph X; args become args.*
tr.instant(track, name, ts_us, label, src, proc=None, **args)       # :98  ph i (thread scope)
tr.counter(proc, name, ts_us, values: dict, label, src)             # :103 ph C on process `proc` ("HEAD", ...)
tr.flow(track_a, ts_a, track_b, ts_b, name, proc_a=None, proc_b=None)  # :108 ph s + f, auto id
doc = tr.write(path, extra=None)                                    # :134 adds M metadata + otherData; returns the dict
bubbles(doc, window=None, cause_fn=None) -> {t0, t1, span_us, tracks:{(pid,tid,block): {busy_us, bubble_us, busy_pct, bubble_pct, gaps, segments, causes}}}  # :144
attribute(g0, g1, evs, cands=[(cause, pred)], default="dependency / idle") -> [(a, b, cause)]  # :188

On-card counters

Today's bitstreams have no per-stage counters. Only level count, status and drop registers exist (0x19c, 0x1c4, 0x1c8, 0x1e8, OBS ring 0x160/0x164). cardctr2trace.py polls them read-only on box1. It is BDF-guarded, refuses 41:00.0, and only HQ's box1 session may run it. DESIGN.md proposes a perf window at 0x240–0x2BF: per-stage BUSY counters, HBM bytes, URAM reads, DSP-busy, and a 256-entry timestamp ring. Estimated cost per card is about 1.5–3k LUT, about 1k FF and 1 BRAM18 P. With it, trace (a) can come from silicon.

Use it

  1. Pick a trace above and click download JSON for Perfetto. Or take traces/*.json from /shared/rnd_hq/2026-10-06/tracks/eagle4_trace/.
  2. Open ui.perfetto.dev, then Open trace file (or drag the file in). The trace stays in your browser.
  3. To make a new trace: xsim2trace.py, hostprof2trace.py or cardctr2trace.py, then analyze.py. run_all.sh rebuilds everything.

Collection details: Collecting a trace. Sources: xsim FH_REF3 and TGL_CL4REF2 logs; box1 hostprof P1HP_950437 (10-03); box1 SIBXE_4CARD AC4 totals (10-06). Block ids come from the week-40 architecture diagram.