The wall we chose to hit. Lane count, reduction width and head replication are all sized to fit underneath it.
Weight-stationary inference silicon
Every accelerator in production streams billions of parameters out of DRAM on every single token. We don't move them at all. The entire transformer body sits in on-chip SRAM permanently and stays there while it computes.
Every URAM block on the device, drawn to scale. The design is packed to 89.51 / 88.46 / 88.46 % across SLR0 / 1 / 2. URAM is the binding resource — deliberately.
The wall
Autoregressive decoding produces one token at a time. To produce that token, the machine must read every weight in the model — and then, for the next token, read every weight again. Nothing is reused between steps. When you are serving a single user's stream there is no batch dimension to amortise the traffic across.
So the ceiling is not arithmetic. It is memory bandwidth. A modern accelerator's multipliers spend most of a decode step idle, waiting on a memory controller to deliver parameters they will use exactly once and discard. Adding FLOPS does not move that ceiling. It is why serving cost per token is stubborn, why utilisation charts look the way they do, and why the industry's answer so far has been to buy more bandwidth rather than to need less of it.
The structural fix is to stop moving the weights. If the parameters never leave the die, there is nothing to be bandwidth-bound on. That is the entire premise of this architecture — and it is now running in fabricated silicon.
The approach
Five decisions taken together. Each one only works because of the other four.
The whole transformer body — quantised to 2 bits, 41.58 MiB — is written into on-chip URAM once at boot and stays resident. During a forward pass there is zero HBM traffic for weights. Not reduced: none. Every matrix-vector product reads its parameters from SRAM sitting beside the multiplier that consumes them.
The intermediate tensors between stages live in registers — a counter-addressed flip-flop register file, not block RAM. That removes the last memory round-trip from the datapath, and it removes an entire class of silicon failure we learned about the expensive way: a registered block-RAM read whose timing has to cross a die boundary.
All fifteen phases of a forward pass — norm, QKV projection, 24-head attention, output projection, gated MLP, final norm, low-rank projection, vocabulary logits, tree select, exact rescore — are sequenced by an on-chip FSM. There is no host round-trip per step. The CPU hands over a request and receives a result.
Tree-structured speculative decoding runs entirely in hardware: a 512-candidate tree, an on-chip top-8 frontier, survivor selection and re-parenting all in the fabric. The branching structure that makes speculative decoding fast is exactly the structure that makes it chatty with a host — so we never talk to one.
A single tensor does not fit: the exact fp16 language-model head, 788 MB (788,004,864 bytes). It lives in HBM, and the final rescoring stage gathers 512 rows from it per forward. We name it rather than bury it — it is the one place the design still touches off-chip memory, it is the current throughput bottleneck, and on an ASIC it disappears entirely.
Measured, not estimated
| Phase | Cycles | ms | Share of draft |
|---|---|---|---|
| Level-0: root body pass + tree select + serialize | ≈44,000 | 0.67 | 11% |
| Level-1: 8 child body passes (sequential, 45,900 each) | 367,000 | 5.56 | 89% |
| Whole on-chip draft | 411,281 | 6.23 | 100% |
| Component | ms | Share | Note |
|---|---|---|---|
| On-chip draft (all 15 phases × 9 passes + tree) | ≈6 | ≈20% | the fabric |
| Exact-rescore gather (off-chip, K_C=64) | ≈1 | ≈3% | was ≈9 ms at K_C=512 — shortlist shrink bought +30% throughput, accept-free |
| Host marshal + target verify + DMA | ≈23 | ≈77% | the machine is ~3× faster than the software around it — the current optimization frontier, and precisely the term that vanishes in a dedicated implementation |
Instruments: launch/stage timestamps from bit-true simulation of the shipping source (memory-resident phases exact; off-chip gather idealized in sim, cross-checked against the silicon K_C sweep), and wall-clock step budgets from live strict-mode serving runs. The eight child passes run sequentially today — overlapping them through the existing register-file seams is the pipelining headroom the dedicated-silicon section describes.
Inside the machine
Filling 88.83 % of a device's SRAM is not an accident of synthesis. It is the constraint the entire microarchitecture was designed against.
The wall we chose to hit. Lane count, reduction width and head replication are all sized to fit underneath it.
128 output columns computed simultaneously, each consuming 16 weights per URAM access, spread across 3 super-logic regions.
Six attention heads evaluated concurrently against a shared key/value stack — strapped and netlist-verified in the shipped image.
The unusual part
At 2 bits, a DSP is the wrong instrument — a hardened wide multiplier is far coarser than the operation needs, and we measured that rather than assumed it. So the body's matrix-vector products are multiplier-free: shift-add MACs built in pure logic, reduced through a Wallace 3:2 carry-save adder tree. Only the very end touches hard silicon — one DSP58 for the final 48-bit carry-propagate add, and one DSP58 per lane for the Q1.15 dequantisation scale.
Attention is a different precision problem, so it gets different hardware: a dedicated INT16 × INT16 DSP58 engine with its own carry-save reduction tree, serving 24 heads six at a time. Two arithmetic styles coexisting on one die, each matched to its own operand width, is precisely why the compute fits in the area the SRAM leaves behind.
Physical residency
| Parameter | Value | Note |
|---|---|---|
| Hidden dimension | 3,072 | Llama-3.2-3B family drafter |
| Attention heads | 24 Q / 24 KV | True multi-head attention — not grouped-query |
| Head dimension | 128 | INT16 × INT16 DSP58 engine, 6-way replicated |
| MLP intermediate | 8,192 | Gate and up projections, SiLU-gated |
| Vocabulary | 128,256 | Cheap head on-chip; exact head in HBM |
| Candidate tree | 512 | Selected and rescored entirely in fabric |
| Frontier width | 8 | Top-8 survivors carried to the next tree level |
| Body weight precision | 2-bit | Group-quantised, Q3.13 dequantisation scales |
Fabricated, flashed, measured
Not an emulator, not a projection, not a synthesis estimate. A programmed AMD Alveo V80 with a host reading results back over PCIe.
Median of three consecutive on-card probes (1,960 / 1,962 / 1,960 µs). Derived rate: ≈510 drafter forwards / s.
Both endpoints measured on the same card. The gain came from a staging fix, dual-port SRAM reads, 6-way head replication and a wider gather.
Zero failing endpoints out of 3,562,588. Hold clean. A design this saturated with SRAM closes tight — but it closes.
| Resource | Used | Available | Utilisation | Role in the design |
|---|---|---|---|---|
| URAM | 1,710 | 1,925 | 88.83 % | Binding — resident weights + key cache |
| Block RAM tile | 3,017 | 3,741 | 80.65 % | Value cache, norm gains, dequant scales |
| CLB LUT | 1,419,499 | 2,574,208 | 55.14 % | Multiplier-free 2-bit MACs + carry-save trees |
| CLB register | 1,497,181 | 5,148,416 | 29.08 % | All inter-stage activation staging |
| DSP58 | 2,497 | 10,848 | 23.02 % | Attention engine, final CPA, dequant scale |
What we are claiming, precisely. The figure above is the latency of one drafter forward pass on the card. It is not an end-to-end tokens-per-second number and we do not present it as one. End-to-end throughput is currently limited by an accept-length defect in the tree stage that is under active repair, and we will not publish an end-to-end comparison against a GPU until that is fixed. What the silicon proves today is the architecture: that a full transformer body can be held stationary in on-chip SRAM, sequenced entirely by hardware, and made to close timing across three dies at 88.83 % memory occupancy.
The next substrate
An FPGA is a de-risking vehicle. It let us prove a weight-stationary transformer in fabricated silicon before anyone commits to a mask set. The same architecture on a custom ASIC changes by three multipliers — and loses its last off-chip tensor.
Projection — not measured
A hard SRAM macro is roughly an order of magnitude denser per unit area than a reconfigurable URAM block. At that density the entire model fits on-chip, including the 788 MB exact head. Zero off-chip memory. At all.
Our 2-bit MACs are currently assembled from general-purpose lookup tables. As standard cells they occupy a fraction of the area — so far more lanes fit into whatever space the on-chip memory leaves over.
We close timing at 66 MHz because that is what a fully-packed reconfigurable fabric with three die crossings permits. A custom implementation of the same datapath is not bound by that constraint.
Stacked, the same architecture plausibly reaches ~50–200 µs per forward.
This range is an engineering projection derived from the three density
and frequency arguments above. It is not a measurement, there is no fabricated ASIC
behind it, and it should be read as a target rather than a result. The only measured
latency on this page is the 1,962 µs FPGA figure.
The port is not a rewrite. Everything expensive we learned on the FPGA transfers directly: the tree-decode state machine, the accept and re-parenting machinery, the register-staging discipline that keeps activations out of memory, the two-arithmetic split between the 2-bit body and the attention engine, the boot-time weight-residency protocol. Those are architecture, not device features. What changes on an ASIC is that they become faster, denser, and finally free of the one remaining off-chip table.
The strategic point is the sequencing. Most accelerator programmes ask for a mask set on the strength of a simulator. We have a fully packed, timing-closed, measured implementation of the architecture running on real hardware first — which means the risk remaining on an ASIC is a physical-design risk, not an architectural one.
Diligence
This programme retracted a headline performance claim once. The standing rule since: a latency, utilisation or frequency figure appears only if it was pulled from an artifact — a silicon run, a synthesis report, or a place-and-route report.
[depth0] forwards us=['1960','1962','1960'] — ~/eagle4_artifacts/angle_d/iva_both_8815_depth0_post.log:2. Corroborated at reports/results/SILICON_0xE4B71120_E2E.md:20.
kernel_us 3167.1524 — reports/SPEEDWAVE_0xE4B71111_SILICON_REPORT.md:100.
reports/results/SILICON_0xE4B71120_E2E.md:20.
impl_1/place_report_utilization_0.rpt:100. Per-SLR 606 / 552 / 552 = 89.51 / 88.46 / 88.46 % at :360. Block RAM tile 3,017 at :95; LUT 1,419,499 at :43; register 1,497,181 at :77; DSP58 2,497 at :113.
impl_1/route_report_timing_summary_0.rpt — clock table gives 66.002 MHz / 15.151 ns period; WNS 0.017, 0 failing of 3,562,588 endpoints, worst hold slack +0.010.
reports/AUTONOMOUS_DRIVE_LOG.md:1888.
8c5fd75a… — reports/DEPTHN_RUNGC_SIM_DOSSIER.md:463.
reports/perf/intent/INTENT_0xE4B71120.txt and netlist-verified across all 18 straps before flash. SLR count from the per-region utilisation table.
Deliberately absent. No power figure, no energy per token, no tokens-per-watt, and no end-to-end throughput comparison against a GPU. None of those has an artifact behind it today. A number without a measurement is left off this page rather than estimated.