betaFeedback

WorksComputer Science

SPMW: A Grid-Based Programming Model for Scalable Accelerator Design

Abstract

Modern accelerators specialize recurring computation and replicate it across regular hardware structures to achieve high performance. Sustaining this parallelism requires equally deliberate data delivery. Yet design tools often express these structures as sequential programs or flatten them into separate hardware modules and connections. This obscures shared patterns and leads to repeated compiler work. We introduce Single Program, Multiple Connected Work Units (SPMW), a grid-based programming model for scalable accelerator design that makes grid regularity a first-class abstraction. SPMW decouples local work-unit behavior from grid connectivity and data delivery. To our knowledge, SPMW is the first programming model to retain regularity throughout the compilation process, combining behavioral HLS with structural RTL generation for both fixed-function and domain-specific instruction-programmable accelerators. We implement SPMW with a Python frontend and an MLIR-based compiler that synthesizes each distinct kernel configuration once for reuse across independent hardware instances. Preserving regularity also enables parallel compilation and functional simulation to accelerate iterative design feedback. We use SPMW to generate a 32x32 systolic array that reduces GEMM latency and parallel kernel-to-RTL compilation time by 4.36x and 619x relative to AutoSA. Our FFT achieves up to 2.65x lower latency than HP-FFT. Our mini-TPU and our SPMW implementation of the FEATHER accelerator demonstrate complex composition and outperform their RTL baselines. Parallel functional simulation provides up to 18.4x speedup over Vitis HLS C simulation on the mini-TPU workloads. SPMW is also agent-friendly: given the same design specification, all five coding agents produce SPMW designs that satisfy the correctness and latency requirements, with 7.51x lower token usage than for SystemVerilog.

Introduction

Modern accelerators rely on structural regularity in computation and communication to achieve high performance within limited area and power budgets. They specialize recurring computations within an application or domain and spatially replicate the resulting processing elements (PEs) for high throughput. For example, TPUs replicate multiply-accumulate units in systolic arrays [39, 40, 41]. Cerebras’s Wafer-Scale Engine connects replicated programmable PEs through a two-dimensional mesh with configurable routes [47]. Replication scales parallel execution, while regular communication paths distribute operands and carry results between PEs.

Regularity in computation and communication is fundamental to accelerator design. Yet, preserving this regularity throughout the design flow remains challenging. Hardware description languages (HDLs) describe circuits at register-transfer level (RTL). A generate loop compactly specifies repeated modules, but compilation expands it into instances and wires. Module hierarchy may survive synthesis without retaining the parameterized grid and connection rule that generated those instances. Full flattening removes the hierarchy itself. High-level synthesis (HLS) raises the abstraction to sequential loop-based programs with scheduling directives [32, 78] and is mostly used to generate application-specific accelerators [14, 18, 93]. These function-level abstractions leave architectural regularity to the designer. In both cases, compact source can become a bloated intermediate representation (IR) enumerating replicated components and connections. Recovering shared PE behavior, repeated subgraphs, or placement patterns after architectural changes requires additional analysis, delaying

Scalable accelerator design requires exploiting regularity throughout the toolflow. Compiler tools must exploit repeated structures to avoid redundant frontend and backend work, but current toolflows face obstacles at both stages.

Challenge 1: Replication and connectivity lack a shared architectural specification. Replication alone does not describe a regular accelerator architecture. Single Program, Multiple Data (SPMD) models such as CUDA and Triton organize instances of one program over a logical grid [57, 69]. Connectivity and memory supply are expressed separately through communication and memory-access mechanisms, leaving the overall architecture implicit.

This separation also appears in accelerator frontends. The C++ HLS design connects PE instances through indexed streams while loaders separately encode array boundaries and memory accesses, as in Figure 1(a). Resizing the array or partitioning it into independently supplied groups requires coordinated changes across these specifications, even when local arithmetic is unchanged. TAPA [17, 33] replicates tasks through parameterized invocations and connects them through typed streams, but lacks a first-class representation of grid regularity and connection rules. Changes to the interconnection require coordinated edits to stream wiring and memory-access tasks. Even with the help of coding agents, connection indices and loader accesses must be traced across these scattered specifications [23, 90]. This indirection increases an agent’s context and editing effort, potentially increasing token use and introducing mismatched connections or memory accesses.

Simplified systolic GEMM construction in C++ HLS and SPMW

Figure 1. (a) C++ HLS with indexed streams [78]. (b) SPMW design fragment. Simplified systolic GEMM construction in C++ HLS and SPMW. Colors map the three limitations in § 2.3 to the HLS constructs and their SPMW counterparts. Python reserves from, so the binding uses from_.

Challenge 2: Flattening regular PE arrays hides compilation reuse opportunities. On the backend side, tools may repeatedly schedule and synthesize equivalent behavior when they process a regular array as independent kernels. Flattened graphs also leave placement and simulation tools to recover relationships among instances. RapidStream [35] partitions and pipelines dataflow designs for parallel placement and routing. This exploits parallelism across partitions, but does not retain a shared PE program and its replication rule to identify equivalent instances for reuse. Boundaries, constants, and timing contexts can make instances differ, so reuse requires both the original regularity and explicit specialization conditions. Although agents can operate compiler tools [12, 82, 86], long simulation and synthesis runs delay feedback. Generated hardware still needs functional validation [61, 88]. An error discovered late triggers another edit-and-build cycle, delaying design correction and validation.

To address these challenges, we propose Single Program, Multiple Connected Work Units (SPMW), a programming model that makes regularity a first-class abstraction for scalable accelerator design. Its central construct is a grid of connected work units: a shared component definition is replicated over a logical index space to form processing elements (PEs), and typed connection rules specify how these PEs exchange data. A PE can be a simple processing block or a hierarchy of lower-level components. The grid and connection rules are explicit program objects, available for composition and compiler analysis. Memory bindings connect the resulting boundaries to storage and transfer sequences, making data delivery part of the same description. SPMW extends the SPMD principle to explicitly connected PEs, providing a shared architectural specification of replication, connectivity, and data delivery.

The model decouples three design concerns. Work-unit programs define behavior through named, typed ports. Grids and topology define replication and connectivity. Surrounding fabrics define memory organization, boundary supply, and composition. Changing topology can reuse the work-unit program, while changing computation can preserve a compatible interface. A fabric may compose grids with different programs, and units may retain state or interpret instructions. “Single program” applies to each replicated family, which allows heterogeneous modules. Grids are logical index spaces and can express nonlocal links and hierarchy.

We implement SPMW with a Python frontend and an MLIR flow combining HLS for work-unit behavior and instruction control with structural RTL for the fabric. To our knowledge, SPMW is the first programming model to preserve grid regularity across the compilation for both fixed-function and domain-specific instruction-programmable accelerators. Its language-independent architectural contract guides boundary derivation and structural checks before specializing kernels for port usage and coordinate-dependent constants. Each distinct kernel configuration is synthesized once and instantiated as independent hardware modules. The retained structure also enables parallel simulation and physical implementation (§ 7).

This paper makes the following contributions:

  • We propose SPMW, a programming model2 that makes regularity a first-class abstraction and decouples PE behavior from grid connectivity to construct both fixed-function and domain-specific instruction-programmable accelerators.

  • We implement an MLIR-based compilation flow that retains this regularity to reuse specialized implementations across the grid. It combines behavioral HLS with RTL generation for better structural composition of both accelerator classes, with parallel kernel compilation and functional simulation support to shorten design feedback loops.

  • We achieve 4.36×4.36\times lower GEMM latency and 619×619\times faster parallel kernel-to-RTL compilation than AutoSA with a 32×3232 \times32 systolic array, and up to 2.65×2.65\times lower FFT latency than HP-FFT.

Our mini-TPU and our SPMW implementation of FEATHER reduce latency by up to 2.32× and 7.98× versus Gemmini and optimized RTL, respectively. Functional simulation is up to 18.4× faster than Vitis HLS C simulation (csim). All five coding agents’ designs pass simulated correctness and latency checks with 7.51× lower median token cost than SystemVerilog.

Background and Motivation

In this section, we review accelerator architectures and design tools, then use a GEMM example to explain the obstacles to architectural changes.

Modern Accelerator Architectures

Accelerators combine repeated execution resources with memory and communication hierarchies. GPU lanes share instruction control within warps, and replicated streaming multiprocessors draw operands from registers, shared memory, and caches [6, 57], while general-purpose GPU architectures are beyond this paper’s focus on domain-specific accelerators. TPU and AWS Trainium pair systolic matrix engines with on-chip operand storage [28, 39, 41]. Figure 2 shows TPU-style operand reuse through PE forwarding, with surrounding accumulation and instruction control.

Figure 2. Systolic datapath. Control is omitted.

Programmable-core networks provide another organization. Cerebras connects cores and local memories through an on-chip fabric [47]. OpenAI’s Jalapeño pairs replicated programmable cores with local memory views and a dedicated collective network [36]. Regular compute and data-delivery structures also appear in Tenstorrent’s Blackhole [68], AMD’s AI Engine NPUs [64], d-Matrix’s Corsair [20], and HyperAccel’s LPU [50].

More PEs improve sustained throughput only if operands arrive and results leave fast enough. Memory capacity limits resident tiles, while bank and channel bandwidth constrain concurrent accesses. Hiding off-chip latency requires buffering and scheduled transfers. Grid dimensions, local reuse, and memory organization must therefore be designed together.

Hardware Design Flow and Programming Models

Accelerator designers specify execution resources, replication, connections, storage, and control. HDLs such as Verilog describe registers, combinational logic, and their connections at RTL. Chisel [8] generates Verilog from reusable, parameterized hardware components written in Scala.

HLS compiles behavior and scheduling directives to RTL, then logic synthesis and physical implementation map it to FPGA resources [78] or ASIC standard cells [65]. Vitis HLS streams provide typed FIFO channels for passing values between functions [78]. SYCL’s oneAPI FPGA implementation connects kernels through pipes [37, 42]. TAPA [33] expresses FPGA accelerators as hierarchies of concurrent tasks connected through typed streams. Spatial [43] expresses nested parallel loops and reductions and automatically banks on-chip memories to serve concurrent accesses. Allo [15] decouples algorithms from hardware customization and composes customized kernels through schedules. These systems organize design around modules, tasks, or computation transformations. We discuss additional DSL and compiler approaches in § 8.

A Motivating Example

Figure 1 compares output-stationary GEMM descriptions in C++ HLS and our Python realization of SPMW. In the shown array, each PE column computes a different column of C. Values from each row of A enter at the west boundary and pass through all PEs in that row, where they are multiplied by different columns of B. Suppose the designer divides the PE columns into two groups to shorten the horizontal forwarding paths. The PE program and PE count stay unchanged, but each row gains a new input for the second group and an output to drain from the first. Both groups must receive the same A values. This split requires coordinated edits to channel storage for the replicated PEs, indexed PE connections, and boundary loaders and sinks. We summarize three limitations.

Limitation 1: replication spans multiple constructs. Array shapes and partition directives (L16–L19) define storage for the channels, while unrolled loops replicate PE calls (L21–L28). Turning off PE inlining (L5) does not prevent repeated scheduling. Shared constants parameterize the dimensions, but the extra boundary channels must still match the PE instances created by these loops. The grid is implicit across declarations and control flow. Flattening can obscure the common PE program and grid, leaving downstream compilation to rediscover opportunities for reuse.

Limitation 2: topology is implicit in indexed wiring. The PE has reusable stream ports (L3–L4), while the PE call encodes neighbors through array subscripts and argument order (L25–L26). At the split, the second group’s first PE must read the new boundary stream. A stale index can leave a PE attached to its former neighbor or without an input source, causing incorrect results or a stalled stream.

Limitation 3: rewiring PEs leaves operand delivery unchanged. Loaders feed the west and north boundaries (L20), while sinks and a drain consume outputs (L29). The loader must supply the same A values to both groups with sufficient bandwidth, and a sink must drain the first group’s new output. Editing PE calls alone does not update these boundary operations.

SPMW connects these design choices through the declarations in Figure 1(b). The designer instantiates a shared PE program (L14–L21) on a grid whose shape parameterizes the topology (L26). The resulting instance group P names these logical instances and their ports. Physical placement assigns their chip locations later. The compiler derives the PE instances and channels together, retaining their shared structure for compilation and addressing Limitation 1. The PE uses named ports from the interface contract MacIO (L3–L6), while a separate connection rule specifies their neighbors (L8–L12). This makes topology explicit and allows it to change without rewriting the PE program (Limitation 2). The instance group also exposes the boundary bundles used by input bindings and result collection (L27–L30), linking data delivery to the declared topology (Limitation 3). To form the two column groups, the designer only needs to remove the horizontal links between them and bind both groups’ west inputs to the same rows of A, while retaining the PE program.

C++ generators in Vitis HLS [78], Catapult HLS [65], and SYCL [42] can automate these edits, but must specify how topology changes update channels, PE connections, and loader accesses. For example, splitting a chain creates a new input that must be supplied. SPMW retains grids, connections, and boundary bindings so the compiler can derive instances and exposed ports for structural checks and kernel reuse. § 4.3 develops a detailed splitting example for attention.

The SPMW Programming Model

SPMW decouples what each work unit executes from how units are assembled and supplied with data. A unit defines local computation. A fabric defines its hardware organization. Elaboration evaluates this structure at compile time to construct instances, connections, memories, and bindings. Static parameters, such as the GEMM dimensions MM, NN, and KK in Figure 1(b), are fixed per build but may change between builds. Array contents and instruction streams arrive at run time. Figure 3 shows the abstract syntax of SPMW and distinguishes run-time unit statements ss from elaboration-time fabric statements aa.

Core SPMW abstract syntax

Figure 3. Core SPMW abstract syntax. Overbars denote sequences, ⊕\oplus denotes binary operators, and RR is a compile-time iteration domain.

Interfaces and Units

An interface II is a finite set of named ports. Each port declares its protocol, access direction, payload type, and shape. The protocol distinguishes ordered FIFO transfers from indexed memory access, independently of direction. In the frontend, a spmw.Interface class defines this contract. spmw.In(T) and spmw.Out(T) declare stream inputs and outputs with payload type TT, which may be a scalar or a statically shaped array. spmw.MemIn(T) and spmw.MemOut(T) declare memory inputs and outputs with the same type notation. p.get() removes and returns the next input value, blocking on an empty FIFO. p.put(e) sends ee through the output port, blocking until the FIFO has capacity. Memory ports obey their declared access permissions. Undeclared ports, wrong-direction accesses, and stream operations on memory ports are invalid even when payload types match. Port types specify no transfer rate, latency, or initiation interval. depth specifies FIFO capacity (§ 3.3).

A unit is a leaf component U=(I,σ,B;κ)U = (I, \sigma, B; \kappa), with interface II, private state σ\sigma, imperative body BB, and static parameters κ\kappa, such as the GEMM kernel dimension KK. Private state may include local memory and registers. The spmw.unit decorator declares the component, whose interface parameter exposes exactly the ports in II, as shown in Figure 4(a). In Figure 1(b), pe(io: MacIO) reads and forwards two streams, accumulates their products, and writes the result (L14–L21).

Typed composition of units and fabrics

Figure 4. Typed composition of units and fabrics. (a) A unit declares its local behavior and ports. (b) A fabric instantiates it on a topology with a matching interface and rejects an incompatible component. (c) tile_gemm exposes memory ports to tiled_gemm while encapsulating its PE grid and storage. Figure 5 gives the code.

A unit must be self-contained. Run-time inputs and externally visible effects pass through its ports, and other mutable storage belongs to the instance. Local variables, control flow, and helpers obey this rule. Types and immutable elaboration-time constants, such as int32 and KK, accompany specialization. Capturing fabric arrays or instance-group handles to bypass the interface, accessing another instance’s ports, or sharing mutable Python globals violates the contract.

Each instance owns its state, including the GEMM accumulator acc, initialized on entry and updated across KK iterations. Instances may respond independently to run-time data or instructions. Unit bodies cannot execute structural operations such as spmw.place (§ 3.3), declare fabric memories, or change connections at run time.

M, N, K = 512, 512, 512
Rt, Ct = 16, 16
TM, TN = M // Rt, N // Ct
class TileIO(spmw.Interface):
    a = spmw.MemIn(int8[Rt, K])
    b = spmw.MemIn(int8[K, Ct])
    c = spmw.MemOut(int32[Rt, Ct])
@spmw.fabric
def tile_gemm(io: TileIO):
    P = spmw.place(pe, on=mesh(MacIO, (Rt, Ct)))
    a_stage = spmw.mem(int8[Rt, K],
        layout=spmw.banked(on="row"))
    b_stage = spmw.mem(int8[K, Ct],
        layout=spmw.banked(on="col"))
    spmw.copy(io.a, into=a_stage)
    spmw.copy(io.b, into=b_stage)
    t = spmw.steps(K)
    spmw.stream_in(a_stage, into=P.west,
        index=(P.rows, t))
    spmw.stream_in(b_stage, into=P.north,
        index=(t, P.cols))
    spmw.gather(io.c, from_=P.c)
@spmw.fabric
def tiled_gemm(A: int8[M,K],
               B: int8[K,N],
               C: int32[M,N]):
    T = spmw.place(tile_gemm,
        on=spmw.Topology(TileIO, grid=(TM, TN)))
    spmw.shard(A, into=T.a, dim=0)
    spmw.shard(B, into=T.b, dim=1)
    spmw.shard(C, from_=T.c)

Fabrics and Structural Elaboration

A @spmw.fabric body runs during elaboration to construct hardware from component ports and array references. It uses static arithmetic to calculate shapes, coordinates, and binding maps, but does not execute unit bodies or compute on run-time values.

Given a fabric FF and static design parameters κ\kappa whose values are chosen for this build, elaboration produces a finite hardware graph

Elab⁡(F;κ)=(V,E,M,β),\operatorname{Elab}(F;\kappa) = (\mathcal{V}, \mathcal{E}, \mathcal{M}, \beta),

where V\mathcal{V} denotes logical unit instances, E\mathcal{E} contains their connections, M\mathcal{M} describes memories, and β\beta records external and memory bindings. Standalone storage in M\mathcal{M}, such as the A/B banks in Figure 4(c), is declared with spmw.mem (§ 3.4.2) and need not belong to a PE. A memory binding associates a component port with a tensor view specifying which elements it may read or write. The compiler uses this association in β\beta to generate access paths and address mappings.

This graph describes hardware semantics. The IR retains each regular family as an instance group referencing one shared unit function, without copying its body at every coordinate. Retained connection rules let the compiler recompute links and exposed ports after architectural changes.

A fabric loop generates repeated structure. A unit loop describes repeated work on the generated hardware, subject to scheduling. Grid sizes, component choices, and connection rules must therefore be determined without run-time stream or array values.

A fabric can expose a typed interface and become a component in a parent fabric. In Figure 5, tiled_gemm replicates tile_gemm on a TM×TNT_M \times T_N grid and binds its TileIO ports to tensor views (L23–L31). Each child contains an Rt×CtR_t \times C_t PE mesh with local buffers and stream bindings (L8–L22). The parent uses only the exposed interface, so the child can change its banking or connections independently. This composition rule applies at further levels.

Hierarchical GEMM for Fig. 4(c)

Figure 5. Hierarchical GEMM for Figure 4(c). TileIO exposes child memory ports. Dashed bindings share AiA_i by row and BjB_j by column. Each tile banks its operands and owns CijC_{ij}.

Grids and Typed Connectivity

A logical dd-dimensional grid with nan_a coordinates on axis a=0,…,d−1a = 0,\ldots,d-1 is the Cartesian product

G=∏a=0d−1{0,…,na−1},d≥1, na>0.(1)\mathcal{G} = \prod_{a=0}^{d-1} \{0,\ldots,n_a-1\}, \qquad d \ge1,\ n_a > 0. \tag*{(1)}

For example, an M×NM \times N mesh has d=2d = 2, n0=Mn_0 = M, and n1=Nn_1 = N, with coordinates (i,j)(i,j).

A topology T=(I,G,λ)T = (I,\mathcal{G},\lambda) combines an interface with this grid and a connection rule. For the coordinate form used in the paper, λ\lambda is a partial function from a source coordinate and stream output to a destination coordinate and stream input: λ:G×Out⁡(I)⇀Zd×In⁡(I)\lambda: \mathcal{G} \times\operatorname{Out}(I) \rightharpoonup\mathbb{Z}^d \times\operatorname{In}(I). In Figure 1(b), the link function returns these output-to-destination pairs using spmw.to (L11–L12). It depends only on coordinates and static parameters. An in-grid destination creates a channel. An omitted mapping or an off-grid destination leaves the output exposed at the boundary.

Every edge must name ports owned by the topology’s interface, connect an output to an input, and preserve payload type and shape. Each destination input has at most one producer. Connecting two outputs to the same input is invalid because the rule supplies no implicit arbitration or reduction. A coordinate link is point-to-point. Broadcast or reduction requires calling an explicit communication operation (§ 3.4.1). Fabric bindings associate memory ports with tensor views through spmw.shard, or collect memory outputs into a destination tensor through spmw.gather (§ 3.4). Coordinate rules create FIFO channels only between stream ports.

FIFO capacity is chosen during elaboration. A stream port may request a capacity with depth. A link uses the largest explicit request at its endpoints, or two values if neither specifies one. Reads block on an empty FIFO and writes block on a full FIFO, propagating backpressure between units. The designer can increase capacity to accommodate pipeline overlap or cyclic dependencies. SPMW does not infer a minimum deadlock-free depth.

The operation spmw.place(C, on=T) instantiates the explicitly supplied component CC on topology TT. The component may be a unit or a fabric with an exposed interface. The frontend checks that CC declares TT’s named interface II, then creates one independent instance per grid coordinate. Interface matching validates CC. Distinct interface declarations remain incompatible even with identical port names. Figure 4(a,Figure 4b) shows an accepted match and a rejected mismatch. Thus, io.east in the unit and iface.east in the link rule refer to the same port declaration. In Figure 1(b), the mesh connects east to west and south to north (L8–L12).

Logical grid coordinates identify instances without prescribing nearest-neighbor connectivity or a physical floorplan. Rules can describe meshes, stage permutations, and nonlocal links. Several instance groups may use different unit programs and connect through compatible boundaries, allowing heterogeneous accelerators. Elaboration resolves connectivity before physical placement.

Data Delivery

Data delivery connects exposed ports to operand sources and result destinations. A boundary bundle collects the exposed instances of one declared port within an instance group, indexed by grid coordinates. For example, P.west collects the west input at the start of each mesh row. It groups endpoints of one port, rather than whole interfaces or all same-typed ports in a fabric. Both stream and tensor-memory bindings use these bundles.

Formally, let ET\mathcal{E}_T be the in-grid edges derived from λ\lambda. An endpoint (x,p)(x,p) pairs a coordinate xx with a port pp, in the same order used by λ\lambda. For an input port pp, the instance group exposes the bundle

Bnd⁡p(T)={(x,p)∣x∈G,(x,p)∉dst⁡(ET)}.\operatorname{Bnd}_p(T) = \{(x,p) \mid x \in\mathcal{G}, (x,p) \notin\operatorname{dst}(\mathcal{E}_T)\}.

Here, dst⁡(ET)\operatorname{dst}(\mathcal{E}_T) contains all destination endpoints in ET\mathcal{E}_T, so exposed inputs have no internal producer. Output bundles analogously contain outputs without an internal consumer. Cutting a horizontal link adds its destination’s west input to P.west, even if that coordinate lies inside the grid. Memory ports likewise form bundles indexed by grid coordinates and are bound to tensor views.

Every consumed input must have exactly one source, supplied by an internal channel or fabric binding. Residual inputs cannot be left unspecified or silently treated as zero. Unused forwarding outputs may be discarded. The elaboration-time structural checker complements computation type checking by validating port accesses and topology constraints. However, well-formed connectivity alone proves neither deadlock freedom nor numerical correctness. In the following, we introduce two kinds of bindings.

3.4.1 Stream Bindings. Stream bindings specify how values enter and leave a grid over time, independently of unit computation. The spmw.stream_in primitive supplies an input bundle from a memory array, a constant, or a compatible output bundle. For a memory source, an index map selects elements by endpoint coordinates and transfer step. The primitive spmw.steps(K) names KK ordered transfers. Generated loaders advance when their output streams can accept values.

For whole-bundle transfers, SPMW provides spmw.scatter to distribute tensor elements to instances and spmw.gather to collect their outputs into a tensor. Both support stream and memory ports. A coordinate map assigns each output’s destination in gather, while a temporal index maps successive stream values.

In Figure 1(b), tt = spmw.steps(K) defines t=0,…,K−1t = 0, \ldots, K - 1 (L27). The binding spmw.stream_in(A, into=P.west, index=(P.rows,t)) sends A[r,t]A[r,t] to the boundary input in row rr (L28). The north binding sends B[t,c]B[t,c] to column cc (L29). Finally, spmw.gather(C, from_=P.c) collects each PE’s scalar memory output into the corresponding element of CC (L30). For a stream source, spmw.stream_in pairs the source output bundle with the destination input bundle in coordinate order. Their counts, payload types, and shapes must match. This binding preserves value order and requires no memory loader. The same unit can therefore receive operands from memory loaders or other units without changing its arithmetic.

3.4.2 Tensor-Memory Bindings. Tensor sharding assigns tensor views to component memory ports. The compiler records spmw.shard mappings during elaboration to generate access paths from storage to component instances. Views must match port types and shapes and stay within tensor bounds. Read-only views may overlap, while output views must be disjoint and cover the result with one writer per element. SPMW applies tensor partitioning and replication over logical grids from Alpa [94] and other tensor-sharding work [1, 13, 31, 80] to hardware memory ports.

Banking physically partitions a buffer for concurrent access. The banked layout of spmw.mem creates independent banks, while spmw.shard maps components to tensor elements. Both construct hardware at compile time. Explicit copies or generated data movers load values at run time, allowing a banked weight buffer to receive new tiles without changing its structure. Bank conflicts occur when simultaneous accesses exceed a bank’s ports, while sharding supplies no extra memory ports or run-time arbiter between independent components, which require separate banks or replicas. Our structural checker rejects multiple writers or concurrent read/write to one declared memory.

In Figure 5, spmw.shard shares each row block of AA within a tile row and each column block of BB within a tile column (L29–L30). Each tile owns a distinct output block in CC (L31). These bindings connect parent tensors to child memory ports. Each child declares row-banked a_stage and column-banked b_stage, populated by explicit copies (L15–L16). The stream_in bindings supply the west and north boundaries concurrently (L18–L21). Staging can be bypassed if the source provides sufficient bandwidth, but sharing a view does not create physical read ports.

Case Studies

We use three cases to demonstrate SPMW’s capabilities. (1) Generality. We show how SPMW supports different systolic-array dataflows by changing PE computation and connection patterns. (2) Programmability. We then prototype a mini-TPU that executes run-time programs by adding instruction control to the systolic array as its matrix unit. (3) Flexibility. We further prototype a grouped attention array by changing topology and data delivery while reusing its arithmetic units, as an example of rapid hardware exploration.

class WsIO(spmw.Interface):
    a_in = spmw.In(int8); a_out = spmw.Out(int8)
    p_in = spmw.In(int32); p_out = spmw.Out(int32)
    w = spmw.MemIn(int8) # resident weight
class ActIO(spmw.Interface):
    z_in = spmw.In(int32); y_out = spmw.Out(int8)

def mxu_links(i, j):
    return {
        WsIO.a_out: spmw.to((i, j+1), WsIO.a_in),
        WsIO.p_out: spmw.to((i+1, j), WsIO.p_in)}

@spmw.unit
def mac(io: WsIO):
    for m in range(Mt):
        a = io.a_in.get(); p = io.p_in.get()
        io.p_out.put(p + a * io.w); io.a_out.put(a)
@spmw.fabric
def tpu_matmul(A: int8[Mt,Kt], W: int8[Kt,Nt],
               Y: int8[Mt,Nt]):
    T = spmw.Topology(WsIO, grid=(Kt, Nt),
                      link=mxu_links)
    P = spmw.place(mac, on=T)
    # act(io: ActIO) omitted for simplicity
    Ta = spmw.Topology(ActIO, grid=(Nt,))
    Pact = spmw.place(act, on=Ta)
    spmw.shard(W, into=P.w) # W[k,n]
    m = spmw.steps(Mt)
    spmw.stream_in(A, into=P.a_in,
                   index=(m, P.rows))
    spmw.stream_in(0, into=P.p_in)
    spmw.stream_in(P.p_out, into=Pact.z_in)
    (lane,) = Pact.axes
    spmw.gather(Y, from_=Pact.y_out, index=(m, lane))

Weight-Stationary Systolic Array

First, we change a fixed-function array’s dataflow with changes to PE computation and communication. SPMW decouples boundary handling, so interior and edge PEs keep the same program.

Figure 6 transforms the output-stationary GEMM in Figure 1(b) into a weight-stationary array. The original PE accumulates a result locally across KK products and forwards both operands. The new mac holds one weight, adds its product to an incoming partial sum, and forwards the activation. Accordingly, WsIO declares int8 activation streams, int32 partial-sum streams, and a scalar memory input for the weight. The connection rule sends activations horizontally and partial sums vertically. A separate act unit applies an activation function and fixed scaling to the result.

Weight-stationary matrix unit and activation row

Figure 6. Weight-stationary matrix unit and activation row. Each MAC holds one weight, forwards activations, and adds to a partial sum. The activation body, constants, and imports are omitted.

The fabric assigns W[k,n]W[k,n] to PE (k,n)(k,n), supplies activations, seeds each column with zero, and connects the result to an activation lane. Derived boundaries let edge and interior PEs execute the same mac, receiving inputs from loaders or neighbors. This decouples dataflow changes from boundary handling and lets the activation unit evolve independently.

Instruction-Programmable Mini-TPU

General neural network modules require reductions and elementwise operations beyond fixed matrix-and-activation computation. To support these operations without rebuilding hardware for each module, we add instruction control to the matrix and vector units. These units decode instructions locally, while the fabric supplies programs through the existing connections.

We port the matrix and vector operations of VTA’s instruction set architecture (ISA) [52] into @spmw.unit, adapting instruction decoding and execution to our stream interfaces. Imperative while loops and if/else branches let a unit interpret run-time programs and update private state. Typed ports carry instructions and operands, while the fabric determines their delivery.

Programmable vector lanes replace the fixed activation row. Each receives int32 matrix results and a separate instruction stream. Its register-based program supports accumulation, arithmetic, shifts, maxima, and stores. The fabric binds matrix and vector programs independently and connects matrix outputs to vector inputs using the stream binding in § 3.4.1. Vector processing can change independently of the matrix array’s connections. Array dimensions and storage capacities remain build parameters, while programs select operations without rebuilding hardware. § 6.1.2 evaluates LLaMA and DeepSeek feed-forward network (FFN) projections using this mini-TPU.

Grouped Attention-PV Array

Beyond prototyping existing designs, hardware engineers need to explore new architectures for emerging workloads. Instruction control cannot reconnect a fixed TPU-style datapath to match operand shapes. For example, during large language model (LLM) prefill [62], a head with sequence length SS and head dimension dhd_h has QQ, KK, and VV of shape [S,dh][S,d_h]. Scaling, masking, and softmax of QK⊤QK^{\top} produce probabilities P[S,S]P[S,S]. Attention-PV then computes O[S,dh]=P[S,S]V[S,dh]O[S,d_h] = P[S,S]V[S,d_h]. Modern TPUs like TPUv7 have 256×256256 \times256 systolic matrix units (MXUs) that hold and reuse the right-hand operand [38], while models like LLaMA 3 8B have head dimension dh=128d_h = 128 [24]. This limits the reduction dimension in QK⊤QK^{\top} and the output width in PVPV to 128. Both attention matmuls therefore achieve at most 50% MXU utilization under this per-head mapping [38].

One architectural change to increase utilization is to use idle columns for the same head’s PVPV reduction, reusing the units in Figure 6. For an R×CR \times C array with dh<Cd_h < C dividing CC, we split the columns into G=C/dhG = C/d_h groups, forming a G×R×dhG \times R \times d_h grid indexed by group gg, row kk, and lane ee. In each GRGR-position reduction tile, group gg holds VV rows gRgR through (g+1)R−1(g+1)R-1 and receives matching PP columns. Activations stay within groups, while partial sums descend a column and continue at the next group’s top, forming dhd_h chains of depth GRGR. In Figure 7(b), the original 2×22 \times2 mapping uses two PEs (50%) in two passes for dh=1d_h = 1. Grouping uses all four PEs (100%) in one pass. With R=C=256R = C = 256 and dh=128d_h = 128, two groups process 512 sequence positions per pass.

Grouped attention connections and fabric and dataflow comparison

Figure 7. Grouped attention reuses the weight-stationary units in Figure 6. (a) Connections and fabric on a (g,k,e)(g,k,e) grid. (b) Two 2×22 \times2 arrays compute P[4,4]V[4,1]P[4,4]V[4,1]. Grouping joins both columns into one reduction chain.

Figure 7(a) implements these connections and bindings on tiles with Mt query positions and Lt =GR= GR reduction positions. The connection rule joins groups, while the bindings map select tile elements V[gR+k,e]V[gR+k,e] and P[m,gR+k]P[m,gR+k]. Derived boundaries expose the new activation inputs, seed only the first group, and drain only the last. Adding a group requires no per-PE edits or separate enumeration of loaders and drains. Partial sums accumulate across reduction tiles before activation.

SPMW enables rapid hardware–software co-design by decoupling arithmetic units from topology and data delivery. Designers can express new connections and workload mappings through shared declarations while reusing unit implementations. § 6.3 evaluates the resulting utilization and pipeline-depth tradeoffs at equal PE counts.

Compilation Flow

Flattening a regular accelerator into independent kernels can duplicate scheduling work and obscure the relationship between PE behavior and array connectivity. SPMW preserves this relationship so the compiler can specialize shared PE code for reuse in a fabric generated from the same architectural description. In Figure 8, our Python frontend lowers SPMW directly to MLIR. The @spmw.unit bodies become HLS C++ for scheduling and pipelining. Elaborating @spmw.fabric determines how these kernels connect and receive data in the assembled RTL.

Compilation and simulation flow

Figure 8. Compilation and simulation flow. Unit bodies and memory movers pass through HLS (blue), while fabric structure drives RTL assembly (green). Dashed paths show feedback for iterative design refinement.

Structural Elaboration and Functional Simulation

We design an SPMW MLIR dialect to preserve regularity through compilation. Each instance group retains a logical grid and shared unit symbol, with rules for communication and boundary supply. The frontend parses each unit’s Python abstract syntax tree (AST), resolves port accesses and grid coordinates against the fabric, and performs type inference to emit MLIR functions with arithmetic, control flow, and memory operations.

Elaborating connection rules derives internal channels and exposed boundaries, which the compiler checks against interfaces and fabric bindings. Port usage and static parameters determine kernel specialization. Shared symbols keep instances associated with their unit function, so topology changes update communication and specialization while compatible instances reuse the function. The compiler therefore works directly from the generating description.

Within an instance group, all instances share a unit definition, static design parameters, and synthesis settings. For each coordinate xx in the logical grid, the compiler constructs the following specialization key

χ(x)=(sx,rx,cx),x∼y⟺χ(x)=χ(y).\chi(x) = (s_x, r_x, c_x), \qquad x \sim y \Longleftrightarrow\chi(x) = \chi(y).

Here, sxs_x is the bound-port signature selecting the unit body, rxr_x records how its port accesses are routed in generated code, and cxc_x contains statically specialized coordinate values. The compiler resolves these keys from the topology and bindings, partitions the grid by key equality, and lowers one representative per class. It specializes the body and interface and synthesizes them once. RTL assembly instantiates the module at each class member with independent state and connections.

Furthermore, we implement a parallel functional simulator that reuses unit functions across the grid before hardware synthesis. Unit instances and memory movers become MLIR calls in omp.section blocks within an omp.parallel region. One thread per section keeps producers and consumers scheduled when a task blocks. Each typed channel connects one producer’s stream output to one consumer’s input through a bounded FIFO with synchronized access. Capacity uses the largest endpoint depth request, or two values by default. The simulator uses at least one slot per channel and otherwise preserves these capacities by default. The compiler lowers the MLIR to LLVM for host execution with OpenMP. Comparing outputs with software references provides quick correctness feedback, but we currently do not support performance-oriented simulation (§ 7).

HLS Kernel Generation

HLS compiles unit computation, including instruction decoding, and generated memory movers. The flow also supports traditional loop-based generation by expressing an entire loop nest within a single spmw.unit for HLS synthesis. Instances share an implementation only when their specialized kernel definitions and synthesis settings match. Specialization uses bindings to remove unused forwarding outputs. Coordinate-dependent constants and port usage can require different configurations, so sharing a unit symbol is insufficient. The compiler generates one HLS C++ kernel per distinct configuration and instantiates its IP with independent state for each PE.

Scheduling each distinct kernel once avoids repeating expensive HLS optimization across identical instances. For example, enlarging a systolic grid adds interior PEs without necessarily adding kernel configurations. Edge PEs can require separate configurations when their forwarding ports differ. HLS work therefore depends on the distinct specialized kernels, while the RTL retains the full spatial array. Independent kernel projects can also run concurrently within the worker budget.

RTL Integration

From the elaborated fabric, the RTL generator instantiates specialized kernel IP, materializes topology links as FIFO channels, and exposes residual ports as boundary bundles. Repeated PEs share a module definition but retain independent state and connections.

Vitis HLS exports each kernel IP [4]. Generated wrappers map its signal names to fabric ports. For each channel, structural RTL connects the producer and consumer through payload signals, read/write enables, and empty/full feedback [5]. Boundary bindings supply constant sources or connect external streams and supported memory movers. For memory-connected kernels, the generator assembles loaders, drains, and command control around the PE array. This path realizes data delivery directly from the fabric description without a monolithic HLS schedule for the array.

Vivado synthesizes the assembled RTL, and selected designs go through placement, routing, and bitstream generation. RTL assembly, logic synthesis, and routing still process the instantiated design. As kernel synthesis is amortized, these stages can dominate total build time.

Evaluation

In this section, we evaluate SPMW’s performance, resource usage, compilation and simulation time, and design effort. We use systolic GEMM to test replication with nearest-neighbor communication and FFT to test butterfly connections that change across stages. To evaluate composed accelerators, we implement an instruction-programmable mini-TPU and a research accelerator FEATHER with programmable reduction and reordering [70], and compare them with baseline implementations.

AI Use. We build on top of Allo [15] and reuse the compiler skeleton that lowers Python ASTs to standard MLIR dialects and emits C++ HLS. After finalizing the SPMW specification in §3, we use Claude Code with Opus 5 to add frontend extensions, MLIR lowering, and RTL glue logic.

Experimental setup. We target the AMD Alveo U280 FPGA [3] at a default 300 MHz and leave ASIC evaluation to future work. In the following, we use Vitis HLS and Vivado 2023.2 [78, 79]. All experiments run on a server with two Intel Xeon Silver 4214 processors, totaling 24 physical cores and 48 hardware threads. We measure RTL cosimulation latency in cycles and run placement and routing (P&R) to obtain resource counts and timing.

Performance and Resource Usage

Compute and Connectivity

We first evaluate GEMM and FFT performance and resources across different communication topologies.

Output-stationary systolic GEMM. We compare int8 multiplication with int32 accumulation in SPMW against two accelerator design frameworks. AutoSA [74] is a state-of-the-art polyhedral systolic-array generator that derives PE computation and I/O networks from loop nests, with default Vitis HLS and TAPA [33] backends. We use Allo’s systolic-array library [15], which unfolds PE loops and uses communication primitives to stream data between PEs.

Figure 9(a) measures 64364^3–5123512^3 GEMMs on a 16×1616 \times16 array. SPMW’s boundary bindings generate separate A and B feed chains that connect 512-bit memory interfaces directly to edge FIFOs, overlapping delivery and computation. It drains sums while PEs compute the next tile, eliminating boundary stalls and reaching 91.76%–99.98% of peak. AutoSA also overlaps input delivery through double buffering, but adds distribution and aggregation stages. Its serialized result chain delays short workloads. It reaches 99.2% cycle efficiency at 5123512^3, but routing from FIFO control to accumulator BRAM limits its clock to 248 MHz, versus 300 MHz for SPMW. Allo’s library implementation copies all of A and then B into local arrays, executes the tiles, and finally writes C. Its generated C++ has no outer HLS dataflow region across these stages, so PE-to-PE streaming does not overlap memory transfers with computation. Its 32-bit ports further increase transfer time, but widening them alone would not remove this sequential schedule. Overlap would require restructuring its load, compute, and store stages and their buffers. SPMW combines high utilization with workload flexibility: all points use one hardware build, with the reduction length and tile count sent before the operands on each row’s input stream. Boundary units use these values to mark tile ends for PEs. The AutoSA and Allo designs fix dimensions and generation time and require a separate build per workload.

Figure 9. (a) 16 × 16 GEMM throughput (M=N=K=DM = N = K = D), with theoretical peak 153.6 GOP/s dashed. (b) GEMM latency (M=N=K=DM = N = K = D) on D×DD \times D arrays. (c,d) FFT latency (cycles) versus sustained throughput for N=256,1024N = 256, 1024. SPMW point labels give samples/cycle. Routed frequencies are capped at 300 MHz.

To better understand SPMW’s scalability, we measure complete M=N=K=DM = N = K = D workloads on D×DD \times D arrays, ranging from 4×44 \times4 to 32×3232 \times32 in Figure 9(b). Latency spans the first input-memory transfer through the final result write, excluding host communication and control. SPMW reduces latency by 1.59–4.36×\times over AutoSA and 2.51–12.17×\times over Allo, with the largest gains at 32×3232 \times32. All three use one DSP per PE and a one-cycle reduction-loop initiation interval (II), so SPMW’s gains come from shorter delivery and drain paths and overlapped transfers. Table 1 reports 32×3232 \times32 performance and resources. SPMW uses fewer LUTs and FFs than AutoSA with similar BRAM usage. Both meet 300 MHz. Allo misses because its result-store control reaches an array buffer through a high-fanout net dominated by routing delay.

DesignConfig.Latency (cycles)Throughput (GOP/s or MS/s)LUTFFDSPBRAM18Freq. (MHz)
GEMM (int8/int32, M=N=K=32M = N = K = 32)
SPMW32232^228070.22147,551193,9531,02446330
AutoSA32232^21,22216.09186,369320,7721,02448309
Allo32232^23,4084.82151,663109,2511,0243251
FFT (complex FP32, 1,024 points)
SPMWw=16w = 164104,800.00220,111257,72967896306
HP-FFTw=16w = 169373,308.90263,862177,909666404239
Spiralw=16w = 165514,800.00172,301255,005440336327

Table 1. U280 performance and routed resources at f=min⁡(300,frouted)f = \min(300, f_{\mathrm{routed}}) MHz. GEMM throughput is 2D3f/(1000L)2D^3f/(1000L) GOP/s for dimension D=M=N=KD = M = N = K and latency LL cycles. For NN-point FFTs, throughput is Nf/IFFTNf/I_{\mathrm{FFT}} MS/s, with steady interval IFFTI_{\mathrm{FFT}} cycles and width ww samples/cycle.

Radix-2 FFT. We compare complex FP32 transforms with HP-FFT [72], a state-of-the-art manually optimized HLS implementation, and Spiral [49], an RTL FFT generator. Figure 9(c,Figure 9d) plots latency against sustained streaming throughput for 256- and 1,024-point designs at w=2,4,8,16w = 2, 4, 8, 16 samples/cycle. HP-FFT uses UF=w/2U_F = w/2 butterfly lanes per stage. Spiral uses fully streaming radix-2 cores with natural input/output order. Latency spans the first accepted input through the final output of one transform.

Relative to HP-FFT, SPMW reduces latency by 1.35–2.37× at 256 points and 2.29–2.65× at 1,024 points. HP-FFT’s transform-sized ping-pong buffers delay each consumer until its producer completes the transform. SPMW’s explicit stage connections lower to scalar FIFOs, letting stages overlap within a transform as operands arrive. HP-FFT’s inner loops achieve II=1, but its stage processes drain and restart between transforms in our Vitis 2023.2 toolchain, adding roughly 10–13 cycles beyond the ideal N/wN/w interval. Both SPMW and Spiral achieve N/wN/w at every measured width, so their latency difference comes from pipeline startup. SPMW’s coordinate-based rules specialize butterfly connections by stage and lane. Permutation units buffer partners arriving on different beats, and a final reorder buffers one transform for natural output order. These fill delays grow with N/wN/w, leaving SPMW slower than Spiral at 1,024 points and w=2,4w = 2, 4. Wider lanes shorten the delays and let more stages connect simultaneous partners directly, without permutation buffers. This reduces SPMW’s startup enough to outperform Spiral at w=8,16w = 8, 16. At 256 points, the shorter buffers give SPMW lower latency at every width.

Table 1 reports the widest 1,024-point FFT configurations. HP-FFT’s partitioned ping-pong arrays duplicate many small BRAM banks. SPMW uses short FIFOs and local register delays, reserving BRAM for twiddle tables. Its lower BRAM usage comes with more FFs than HP-FFT. All measured SPMW and Spiral configurations meet 300 MHz. HP-FFT misses at several wide configurations. In the failing 1,024-point configurations, the critical paths pass through FIFO and control signals in the bit-reversal stage, with about 85% of their delay from routing. Spiral uses fewer LUTs and DSPs and achieves higher routed frequency, but matches SPMW’s throughput under the 300 MHz cap. Moreover, Spiral rounds toward zero and uses two DSPs per multiplier, while the HLS designs use Vitis floating-point units with three. This contributes to SPMW’s higher DSP count.

Composability

We use mini-TPU and FEATHER to demonstrate how SPMW composes different hardware components into complete accelerators.

Mini-TPU. We compare SPMW with Gemmini [29], a systolic DNN accelerator generator, and VTA [52], a programmable tensor accelerator integrated with TVM [16]. These engines combine matrix computation, runtime control, and vector processing. SPMW composes its systolic array with instruction dispatch, vector lanes, and memory movers through typed interfaces. One GEMM instruction selects tensor addresses, tile dimensions, reduction length, and the epilogue.

We measure gate/up projection slices from LLaMA 3.2 1B [48] and a DeepSeek-V4-Pro expert [22]. These cover dense and expert FFN dimensions for the paired projections used in gated feed-forward layers, a structure shared by the LLaMA family [71] and the dense and MoE Qwen3 models [63]. Each slice uses 64 tokens and 64 columns per projection, giving (M,K,N)=(64,2048,128)(M, K, N) = (64, 2048, 128) and (64,7168,128)(64, 7168, 128). We use int8 inputs and int32 accumulation, a common integer configuration supported by all three engines, to compare their data delivery and control under the same arithmetic. All engines use identical random inputs and shift-and-clip requantization.

Gemmini uses its weight-stationary, matmul-only configuration. We replace floating-point scaling with integer shift for the common epilogue and reorganize accumulator banks into 64-bit columns to enable FPGA RAM inference. VTA uses its Chisel Core with batch size one and shipped storage depths. Both baselines execute their native instruction sequences.

Figure 10(a,Figure 10b) compares whole-engine cycle latency and effective throughput for n×nn \times n arrays with n=4,8,16n = 4, 8, 16. Effective throughput is 2MKNf/C2MKNf/C, counting two operations per MAC, where CC covers instruction delivery through result writes and ff is the best reported routed frequency capped at 300 MHz. At 16×1616 \times16, SPMW’s LLaMA slice uses 14.6% and 11.6% fewer cycles than Gemmini and VTA, respectively. SPMW stays within 1.1% of the arithmetic cycle bound MKN/n2MKN/n^2 by reusing weight blocks across input rows and overlapping operand fetching through fabric bindings. Its throughput is 2.012.01–2.32×2.32\times Gemmini’s and 1.391.39–1.51×1.51\times VTA’s, primarily because it reaches 300 MHz. SPMW’s critical path stays within a registered multiply-add cell and partial-sum link, whereas the baselines are limited by memory-access control and arithmetic. Gemmini’s weight-tile loading and VTA’s separate epilogue passes account for the cycle differences.

Programmable engines on 64-token FFN gate/up slices

Figure 10. Programmable engines on 64-token FFN gate/up slices. (a,b) Cycle latency versus effective throughput at routed frequency capped at 300 MHz. Labels give array side length. (c) 16 × 16 routed resources.

Figure 10(c) reports resource usage for all three engines. At 16×1616 \times16, SPMW uses 46% fewer LUTs than Gemmini. Its narrower GEMM instruction and blocked access pattern reduce control and storage costs. Against VTA, SPMW uses similar LUTs and fewer RAM blocks but more FFs because it retains weights and array state in registers.

FEATHER. FEATHER [70] combines the NEST compute array with BIRRD, a programmable reduction and reordering network. The NEST array has height AHA_H and width AWA_W. Figure 11 shows BIRRD at AW=8A_W = 8, with six stages of four switches. The topology reverses selected lane-index bits between stages and connects paired-PE column sums to the first stage. SPMW decouples weight delivery from these connections, using parallel weight streams with the same reduction network.

BIRRD topology and switch composition

Figure 11. BIRRD at AW=8AW = 8. (a, b) Six stages of four two-input, two-output switches. (c) The topology reverses low lane-index bits and binds NEST to BIRRD. Unit bodies, interfaces, and reverse_bits are omitted.

We evaluate 1283128^3 GEMM and 3×33 \times3 convolution at spatial size 16×1616 \times16 with 64 input/output channels on n×nn \times n arrays (n=4,8,16n = 4, 8, 16). The released RTL [27] has a weight-buffer controller error, which we correct in both baseline variants. In Figure 12, RTL loads one PE weight per cycle, taking n3n^3 cycles despite its nn-byte port. Our row-wise loader in RTL (fixed) uses all nn bytes but still serializes nn rows with n2n^2 weight slots per PE, taking n2n^2 cycles. Private weight files also require repeated delivery of weights shared across columns. SPMW separates this delivery from PE behavior and reduction connectivity. Grid-derived boundary bundles expose each pair’s weight port, which we bind to an independent stream to fill all pairs’ 2n2n slots concurrently in 2n2n cycles. Matching this parallelism requires redesigning the RTL distribution interconnect and control beyond the row-loader fix. This n/2n/2 loading advantage yields 2.002.00–7.84×7.84\times GEMM speedups over RTL (fixed), while all three designs compute at an nn-cycle interval with resident weights. Convolution pads nine taps to 12/16/16 and similarly achieves 2.002.00–7.98×7.98\times speedup.

FEATHER latency and resource results

Figure 12. FEATHER latency for (a) 1283128^3 GEMM and (b) 3×33 \times3 convolution at spatial size 16×1616 \times16 with 64 input/output channels. (c) Resident-weight compute-port resources with LUT-based multipliers.

Figure 12(c) compares resident-weight compute ports with LUT-based multiplication and no DSPs, BRAM, or URAM. At 16×1616 \times16, SPMW uses 5.3% fewer LUTs and 20.5% fewer FFs, but 4×44 \times4 and 8×88 \times8 require 52–53% more LUTs and 20–27% more FFs. SPMW’s HLS control and FIFOs add overhead at small sizes, while RTL’s register-based weight files and selection logic grow with nn.

Compilation and Simulation Time

We measure compilation time for the systolic array experiment in § 6.1.1. Figure 13(a) breaks down HLS through routing for one memory-connected build per size, using eight HLS workers and excluding frontend time. At 32 × 32, HLS takes 110.6 s, only 2.1% of the 89.3-minute total. Preserving regularity lets the compiler reuse specialized PE kernels, keeping HLS small, while synthesis and P&R still process the instantiated hardware.

Figure 13. Compilation and simulation time. (a) GEMM systolic array build time. (b) Compilation time from kernel to RTL. (c) LLaMA and DeepSeek simulation on a 16×1616 \times16 mini-TPU.

Figure 13(b) measures time from the source kernel to complete RTL, including the frontend, HLS, and RTL assembly but excluding IP packaging and P&R. We additionally measure the TAPA [33] backend of AutoSA, which also uses parallel compilation for its tasks. We use eight workers for SPMW and AutoSA (TAPA), and measure serial SPMW with one worker. AutoSA (Vitis HLS) and Allo combine separately measured frontend and single-process HLS times, excluding HLS tool startup. We also add SYCL/oneAPI [37] for comparison with Altera’s FPGA flow, using DPC++ 2025.0 for Agilex 7 at 300 MHz and timing the kernel frontend through RTL generation. As we do not have an Agilex 7 device, this experiment compares compilation time only.

At 32 × 32, SPMW takes 56.6 s with eight workers, giving 619×, 107×, and 340× speedups over AutoSA (Vitis HLS), Allo, and SYCL. Its measured 319.2 s serial run gives 110×, 19×, and 60× speedups. AutoSA (Vitis HLS) and Allo process the full array in one HLS project. AutoSA (TAPA) reuses distinct synthesized tasks, but its RTL assembly grows to 39.2 s at 32 × 32, while SPMW generates fabric RTL during frontend elaboration. SPMW is still 1.84× faster with eight workers. Parallel SPMW compilation stays below 57 s, although AutoSA and SYCL are faster at 4 × 4.

Figure 13(c) compares SPMW’s OpenMP simulator, which executes grid instances concurrently, with Vitis HLS csim and xsim on the 16 × 16 mini-TPU LLaMA and DeepSeek setup in § 6.1.2. We time execution after compilation. The csim runs use the default debug executable. The xsim timer excludes HLS, testbench generation, and RTL compilation and elaboration. For LLaMA and DeepSeek, SPMW takes 1.93 s and 6.94 s, giving 18.4× and 18.0× speedups over Vitis HLS csim and 47.7× and 47.5× over cycle-accurate xsim. This faster functional feedback lets designers identify computation and communication errors before running lengthy RTL simulations, shortening each correction and validation iteration.

Architectural Exploration

We implement the architectural exploration in § 4.3 and measure the latency benefit of reconnecting unused columns. Table 2 compares complete P[S,S]V[S,dh]P[S,S]V[S,d_h] products for dh=8d_h = 8 and S=128S = 128–1,0241,024 on two 16 × 16 arrays with 256 PEs each. Inputs are int8 and results int32. RTL timing spans reset release through the final output, including loading VV and accumulating all key tiles.

SSOriginalGroupedSpeedup
1281,2228251.48×
2564,3022,2941.88×
51216,6068,4461.97×
1,02465,79033,0381.99×

Table 2. Latency (cycles) for complete P[S,S]V[S,8]P[S,S]V[S,8] products.

The original mapping holds 16 keys per tile in eight columns, with zeros in the other eight. Grouping joins the two column groups into eight reduction chains of depth 32, uses all 256 PEs, and halves the tile count from S/16S/16 to S/32S/32. Both designs preload all tiles’ VV entries into each PE. After each tile, feedback FIFOs retain partial sums for all SS queries and return them to the chain inputs for the next tile. The final tile emits the complete product. Both designs drain pending results through flushable HLS pipelines during feedback stalls.

Grouping provides 1.48–1.99× speedup, limited by pipeline fill and feedback at S=128S = 128. At S≥256S \ge256, both sustain one query per cycle per tile, so halving tiles approaches 2×. Adapting the RTL requires extensive changes to inter-group wiring and input delivery, while SPMW only needs several lines of changes to fabric connections and boundary bindings without editing the MAC program.

Productivity

We assess productivity through source-size comparisons and a controlled coding-agent study.

Code Size

Code Size. Table 3 counts design source for systolic GEMM, FFT, mini-TPU, and FEATHER, excluding configuration, tests, software references, comments, and docstrings through language-aware filtering. FFT RTL is Spiral’s 256-point, two-samples/cycle design [49]. Mini-TPU compares the SPMW stage engine with VTA HLS and Chisel, counting Chisel as RTL [52].

DesignHLSRTLSPMWRatio
Systolic array94–243.92×
FFT2995,2841062.82×
Mini-TPU6733,2261544.37×
FEATHER–1,8029718.58×

Table 3. Design-only source lines. Ratios use HLS/SPMW, except FEATHER (RTL/SPMW).

For systolic GEMM and FFT, SPMW uses up to 3.92× fewer source lines than HLS. A shared PE program replaces repeated arithmetic definitions, while grid and connection rules replace per-instance wiring. Boundary bindings also express operand delivery without spelling out each buffer and channel. FEATHER’s reduction network needs 97 SPMW lines versus 1,802 RTL lines using one switch program and connection rule.

Agentic Design

Agentic Design. We evaluate five coding models accessed through OpenRouter [59], Opus 5 [7], GPT-5.6 Sol [58], Kimi K3 [51], GLM-5.3 [89], and DeepSeek-V4-Pro [21], using SPMW, C++ HLS, and SystemVerilog. Each model implements the same 8×8 output-stationary int8 systolic array with int32 accumulation, giving fifteen trials. The specification fixes 64 PEs, nearest-neighbor operand movement, and row-wise result draining. All implementations expose the same scalar stream ports. Every trial receives an interface stub, the same functional specification and testbench, and a language reference. The references have approximately equal token counts. Trials use temperature zero and nominal budgets of 1M billed tokens, eight builds, three routing attempts, and four hours. Each agent works in an isolated sandbox without access to the SPMW source repository. It can edit designs, read supplied files, build, and route only through our provided APIs. The reported latency criterion requires exact outputs and first-product latency between 22 and 64 cycles, measured from the first input to the last output. The lower bound reflects nearest-neighbor propagation, while the upper bound rules out serial execution of all 512 MACs.

The left panel of Figure 14 compares total token cost with latency. All five SPMW designs, three HLS designs, and all five SystemVerilog designs meet the latency criterion. SPMW latencies are 33–53 cycles, compared with 30–34 for SystemVerilog and 32–74 for the four buildable HLS designs. GPT-5.6 Sol’s HLS artifact fails to build, and Kimi’s HLS design takes 74 cycles. The right panel of Figure 14 reports median total tokens, builds, and elapsed time across all five trials per framework, including target misses and build failures. Elapsed time includes model calls and tool execution through trial stop, excluding subsequent evaluation. SPMW reduces median token use by factors of 4.48×4.48\times and 7.51×7.51\times relative to HLS and SystemVerilog, respectively. It also reduces median elapsed time by factors of 1.81×1.81\times and 1.29×1.29\times, respectively.

Agent study for an 8 × 8 array
FrameworkMeets latencyTokens (K)BuildsTime (min)
SPMW5/567.1220.5
C++ HLS3/5300.4837.1
SystemVerilog5/5503.8426.6

Figure 14. Agent study for an 8 × 8 array. Colors identify frameworks and symbols identify models. Filled markers meet the 64-cycle limit.

Post-trial routing meets the 300 MHz target for all fourteen buildable artifacts. Every SPMW design uses exactly 64 DSPs. SPMW needs a median of two builds. This suggests that decoupling PE behavior from replication and connectivity makes the design task easier for agents to express. SystemVerilog also yields five working designs, but explicit registers, wiring, and handshakes require more source construction. C++ HLS additionally requires coordinating stream types, array partitioning, and scheduling pragmas, which makes high-performance designs more challenging.

Discussion and Limitations

Simulation and verification. SPMW supports concurrent functional simulation, but performance exploration requires kernel synthesis and assembled-RTL simulation. Future work will reuse unit timing models across grid sizes and boundary schedules to provide fast performance simulation at the programming-model level. Moreover, structural checks and tests do not establish functional correctness or deadlock freedom for all inputs, which requires a formal verifier.

Physical implementation. Logic synthesis and P&R still process each full RTL configuration and dominate the largest array’s build time (§ 6.2). Our preliminary floorplanner improves frequency by mapping grid tiles and input FIFOs to neighboring FPGA clock regions and pipelining inter-die links, following the ideas of AutoBridge [34] and RapidStream [35]. On a 32×3232 \times32 matrix–vector engine on U280, it raises post-P&R frequency by 32%, from 346 to 457 MHz. Future work will jointly optimize partitioning and pipelined placement across accelerators and reuse placed-and-routed tiles to reduce rebuild time.

Programming accelerators. The evaluated backend constructs FPGA hardware. Future work will map SPMW programs onto AMD AI Engines [64, 73, 76, 77] or Cerebras [11]. We also plan to generate application compilers from SPMW hardware specifications. Work-unit semantics and hardware contracts would guide instruction selection, tiling, and scheduling as hardware evolves.

Related Work

SPMD programming models. CUDA [57], SYCL [42], and Triton [69] organize communicating parallel programs over indexed execution instances. Cypress [81] and Prism [9] improve GPU program composition through task/tensor mappings and typed execution perspectives, respectively. SPMW makes replication, interconnect, and boundary supply explicit for hardware construction.

Hardware DSLs and compilers. TAPA [17, 33] connects C++ tasks through typed streams, supporting hierarchy, replication, kernel reuse, and parallel HLS compilation. Other flows optimize dataflow and data placement [10, 75, 83, 84, 85, 95], support dynamic streaming [30, 66], or derive dataflow hardware from algorithms and control flow [46, 91]. Hardware construction languages provide parameterized generators [2, 8, 60], explicit control [56], resource-aware types [53], and timing abstractions [25, 54, 55, 87]. HLS tools such as XLS [32] and accelerator DSLs derive hardware from computations and mappings [19, 26, 43, 44, 45, 67, 74, 78, 92], while Allo [15] decouples algorithms from hardware customization. SPMW provides a shared work unit's connect and receive data for structural checks and kernel specialization before RTL assembly. Topology changes recompute boundaries and port usage while reusing compatible HLS implementations.

Coding agents for hardware. SWE-agent [82] studies tool interfaces. VerilogEval [61] and Spec2RTL-Agent [88] study hardware generation. CAKE [86] co-designs a GPU schedule IR and agent workflow with compiler verification and localized diagnostics. For accelerator construction, SPMW lets agents update all instances through shared declarations. Before HLS, the compiler detects inconsistent connections and missing boundary bindings.

Conclusion

SPMW decouples work-unit behavior from grid connectivity and data delivery. We demonstrate its generality with fixed-function and domain-specific programmable accelerators. Shared contracts reduce repetitive edits for engineers and coding agents. Explicit connections enable early structural checks, supporting faster exploration and composition as accelerators become more heterogeneous.

References

References

  1. [1]Sami Alabed, Daniel Belov, Bart Chrzasczez, Juliana Franco, Dominik Grewe, Dougal Maclaurin, James Molloy, Tom Natan, Tamara Norman, Xiaoyue Pan, Adam Paszko, Norman A. Rink, Michael Schaarschmidt, Timur Sitdikov, Agnieszka Swietlik, Dimitrios Vytiniotis, and Joel Wee. 2025. PartIR: Composing SPMD Partitioning Strategies for Machine Learning. In Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1 (Rotterdam, Netherlands) (ASPLOS ’25). Association for Computing Machinery, New York, NY, USA, 794–810. doi:10.1145/3669940.3707284DOI
  2. [2]Amaranth HDL contributors. 2024. Amaranth HDL: A Modern Hardware Definition Language. https://github.com/amaranth-lang/amaranth.
  3. [3]AMD. 2023. Alveo U280 Data Center Accelerator Card Data Sheet (DS963). https://docs.amd.com/r/en-US/ds963-u280. Revision 1.7.
  4. [4]AMD. 2023. Output of RTL Export. https://docs.amd.com/r/2023.2-English/ug1399-vitis-hls/Output-of-RTL-Export. Vitis High-Level Synthesis User Guide (UG1399), version 2023.2.
  5. [5]AMD. 2023. Port-Level I/O: Memory Interface Protocol. https://docs.amd.com/r/2023.2-English/ug1399-vitis-hls/Port-Level-I/O-Memory-Interface-Protocol. Vitis High-Level Synthesis User Guide (UG1399), version 2023.2.
  6. [6]Michael Andersch, Greg Palmer, Ronny Krashinsky, Nick Stam, Vishal Mehta, Gonzalo Brito, and Sridhar Ramaswamy. 2022. NVIDIA Hopper Architecture In-Depth. https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/.
  7. [7]Anthropic. 2026. Introducing Claude Opus 5. https://www.anthropic.com/news/claude-opus-5. Accessed September 8, 2026.
  8. [8]Jonathan Bachrach, Huy Vo, Brian Richards, Yunsup Lee, Andrew Waterman, Rimas Avižienis, John Wawrzynek, and Krste Asanović. 2012. Chisel: Constructing Hardware in a Scala Embedded Language. In Proceedings of the 49th Annual Design Automation Conference. Association for Computing Machinery, New York, NY, USA, 1216–1225. doi:10.1145/2228360.2228584DOI
  9. [9]Manya Bansal, Daniel Sainati, Joseph W. Cutler, Saman Amarasinghe, and Jonathan Ragan-Kelley. 2026. Modular GPU Programming with Typed Perspectives. Proc. ACM Program. Lang. 10, PLDI (2026), 1077–1101. doi:10.1145/3808290DOI
  10. [10]Suhail Basalama and Jason Cong. 2025. Stream-HLS: Towards Automatic Dataflow Acceleration. In Proceedings of the 2025 ACM/SIGDA International Symposium on Field Programmable Gate Arrays. ACM, New York, NY, USA, 103–114. doi:10.1145/3766628.3708878DOI
  11. [11]Cerebras. 2024. The Future of AI is Wafer Scale. https://www.cerebras.ai/chip.
  12. [12]Hongzheng Chen, Alexander Novikov, Ngũ Phan Nũ, Hanna Alam, Zhiru Zhang, Aiden Grossman, Mircea Trofin, and Amir Yazdanbakhsh. 2026. Magellan: Autonomous Discovery of Novel Compiler Optimization Heuristics with AlphaEvolve. doi:10.48550/arXiv.2601.21096 arXiv:2601.21096 [cs.AI]DOI
  13. [13]Hongzheng Chen, Cody Hao Yu, Shuaijie Zheng, Zhen Zhang, Zhiru Zhang, and Yida Wang. 2024. Slapo: A Schedule Language for Progressive Optimization of Large Deep Learning Model Training. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (La Jolla, CA, USA) (ASPLOS’24). Association for Computing Machinery, New York, NY, USA, 1095–1111. doi:10.1145/3620665.3640399DOI
  14. [14]Hongzheng Chen, Jiahao Zhang, Yixiao Du, Shaojie Xiang, Zichao Yue, Niansong Zhang, Yaohui Cai, and Zhiru Zhang. 2024. Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference. ACM Transactions on Reconfigurable Technology and Systems 18, 1 (2024), 1–29. doi:10.1145/3656177DOI
  15. [15]Hongzheng Chen, Niansong Zhang, Shaojie Xiang, Zhichen Zeng, Mengjia Dai, and Zhiru Zhang. 2024. Allo: A programming model for composable accelerator design. Proc. ACM Program. Lang. 8, PLDI (2024), 593–620. doi:10.1145/3656401DOI
  16. [16]Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning. In Proceedings of the 13th USENIX Conference on Operating Systems Design and Implementation (Carlsbad, CA, USA) (OSDI’18). USENIX Association, USA, 579–594.arxiv.org/abs/1802.04799
  17. [17]Yuze Chi, Licheng Guo, Jason Lau, Young-kyu Choi, Jie Wang, and Jason Cong. 2021. Extending High-Level Synthesis for Task-Parallel Programs. In 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, Piscataway, NJ, USA, 204–213. doi:10.1109/FCCM51124.2021.00032DOI
  18. [18]Jason Cong, Bin Liu, Stephen Neuendorffer, Juanjo Noguera, Kees Vissers, and Zhiru Zhang. 2011. High-Level Synthesis for FPGAs: From Prototyping to Deployment. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 30, 4 (2011), 473–491. doi:10.1109/TCAD.2011.21110592DOI
  19. [19]Jason Cong and Jie Wang. 2018. PolySA: Polyhedral-Based Systolic Array Auto-Compilation. In 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). Association for Computing Machinery, New York, NY, USA, 1–8. doi:10.1145/3240765.3240838DOI
  20. [20]d-Matrix. 2024. d-Matrix Unveils Corsair, the World’s Most Efficient AI Computing Platform for Inference in Datacenters. https://www.d-matrix.ai/announcements/d-matrix-unveils-corsair-the-worlds-most-efficient-ai-computing-platform-for-inference-in-datacenters/
  21. [21]DeepSeek. 2026. DeepSeek-V4-Pro GA Release. https://api-docs.deepseek.com/news/news260813/. Accessed September 8, 2026.
  22. [22]DeepSeek. 2026. DeepSeek-V4-Pro Model Configuration. https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/config.json. Accessed October 3, 2026.
  23. [23]Zijian Ding, Yang Zou, Yizhou Sun, and Jason Cong. 2026. Can Agents Design Better Chips with a Higher Level Abstraction? doi:10.48550/arXiv.2609.21157 arXiv:2609.21157 [cs.AI]DOI
  24. [24]Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783
  25. [25]David Durst, Matthew Feldman, Dillon Huff, David Akeley, Ross Daly, Gilbert Louis Bernstein, Marco Patrignani, Kayvon Fatahalian, and Pat Hanrahan. 2020. Type-Directed Scheduling of Streaming Accelerators. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (London, UK) (PLDI 2020). Association for Computing Machinery, New York, NY, USA, 408–422. doi:10.1145/3385412.3385893DOI
  26. [26]Shihan Fang, Hongzheng Chen, Niansong Zhang, Jiajie Li, Han Meng, and Zhiru Zhang. 2025. Data: A Task-Based Programming Model for Dataflow Accelerators. doi:10.48550/arXiv.2509.06794 arXiv:2509.06794 [cs.PL]
  27. [27]FEATHER Project. 2024. FEATHER Hardware Implementation. GitHub repository. https://github.com/maeri-project/FEATHER Accessed October 11, 2026.
  28. [28]Xinwei Fu, Zhen Zhang, Haozheng Fan, Guangtai Huang, Mohammad El-Shabani, Randy Huang, Rahul Solanki, Fei Wu, Ron Diamant, and Yida Wang. 2024. Distributed Training of Large Language Models on AWS Trainium. In Proceedings of the 2024 ACM Symposium on Cloud Computing (Redmond, WA, USA) (SoCC ’24). Association for Computing Machinery, New York, NY, USA, 961–976. doi:10.1145/3698038.3698535DOI
  29. [29]Hasan Genc, Saechin Kam, Alon Amid, Ameer Haj-Ali, Vighnesh Iyer, et al. 2021. Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration. In 2021 58th ACM/IEEE Design Automation Conference (DAC) (San Francisco, CA, USA). IEEE Press, San Francisco, CA, USA, 769–774. doi:10.1109/DAC18072.2021.9586216arxiv.org/abs/1911.09925
  30. [30]Souradip Ghosh, Yufei Shi, Brandon Lucia, and Nathan Beckmann. 2025. Ripple: Asynchronous Programming for Spatial Dataflow Architectures. Proc. ACM Program. Lang. 9, PLDI, Article 157 (June 2025), 28 pages. doi:10.1145/3729256DOI
  31. [31]Google. 2025. OpenXLA Shardy. https://github.com/openxla/shardy.
  32. [32]Google. 2026. XLS: Accelerated HW Synthesis. https://google.github.io/xls/. Accessed November 3, 2026.
  33. [33]Licheng Guo, Yuze Chi, Jason Lau, Linghao Song, Xingyu Tian, Moazin Khatti, Weikang Qiao, Jie Wang, Ecenuр Ustun, Zhenman Fang, Zhiru Zhang, and Jason Cong. 2023. TAPA: A Scalable Task-Parallel Dataflow Programming Framework for Modern FPGAs with Co-Optimization of HLS and Physical Design. ACM Trans. Reconfigurable Technol. Syst. 16, 4 (sep 2023), 1–31. doi:10.1145/3609335DOI
  34. [34]Licheng Guo, Yuze Chi, Jie Wang, Jason Lau, Weikang Qiao, Ecenuр Ustun, Zhiru Zhang, and Jason Cong. 2021. AutoBridge: Coupling Coarse-Grained Floorplanning and Pipelining for High-Frequency HLS Design on Multi-Die FPGAs. In The 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (Virtual Event, USA) (FPGA’21). Association for Computing Machinery, New York, NY, USA, 81–92. doi:10.1145/3431920.3439289DOI
  35. [35]Licheng Guo, Pongstorn Maidee, Yun Zhou, Chris Lavin, Jie Wang, Yuze Chi, Weikang Qiao, Alireza Kaviani, Zhiru Zhang, and Jason Cong. 2022. RapidStream: Parallel Physical Implementation of FPGA HLS Designs. In Proceedings of the 2022 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA ’22). ACM, New York, NY, USA, 1–12. doi:10.1145/3490422.3502361DOI
  36. [36]Richard Ho, Ravi Narayanaswami, and Chris Leary. 2026. You Can Just Build Chips. Hot Chips 2026, AI 2 session. https://hc2026.hotchips.org/program/ OpenAI presentation.
  37. [37]Intel Corporation. 2024. Pipes Extension: Intel oneAPI DPC++/C++ Compiler Handbook for FPGAs. https://www.intel.com/content/www/us/en/docs/oneapi-fpga-add-on/developer-guide/2024-0/pipes-extension.html.
  38. [38]Jevin Jiang, Ying Chen, Blake A. Hechtman, Fenghui Zhang, and Yarong Mu. 2026. Ragged Paged Attention: A High-Performance and Flexible LLM Inference Kernel for TPU. doi:10.48550/arXiv.2604.15464 arXiv:2604.15464 [cs.PF].DOI
  39. [39]Norm Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagraraj et al. 2023. TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings. In Proceedings of the 50th Annual International Symposium on Computer Architecture (Orlando, FL, USA) (ISCA ’23). Association for Computing Machinery, New York, NY, USA, Article 82, 14 pages. doi:10.1145/3579371.3589350
  40. [40]Norman P. Jouppi and Sridhar Lakshmanamurthy. 2026. The Eighth Generation TPU Family: Two Chips Optimized for Training and Serving in the Agentic Era. Hot Chips 38, Google presentation. https://hc2026.hotchips.org/program/
  41. [41]Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, et al. 2017. In-Datacenter Performance Analysis of a Tensor Processing Unit. In Proceedings of the 44th Annual International Symposium on Computer Architecture (Toronto, ON, Canada) (ISCA ’17). Association for Computing Machinery, New York, NY, USA, 1–12. doi:10.1145/3079856.3080246DOI
  42. [42]Khronos Group. 2026. SYCL 2020 Specification, Revision 12. https://registry.khronos.org/SYCL/specs/sycl-2020/pdf/sycl-2020.pdf.
  43. [43]David Koeplinger, Matthew Feldman, Raghu Prabhakar, Yaqi Zhang, Stefan Hadjis, Ruben Fiszel, Tian Zhao, Luigi Nardi, Ardavan Pedram, Christos Kozyrakis, and Kunle Olukotun. 2018. Spatial: A Language and Compiler for Application Accelerators. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (Philadelphia, PA, USA) (PLDI 2018). Association for Computing Machinery, New York, NY, USA, 296–311. doi:10.1145/3192366.3192379DOI
  44. [44]Yi-Hsiang Lai, Yuze Chi, Yuwei Hu, Jie Wang, Cody Hao Yu, Yuan Zhou, Jason Cong, and Zhiru Zhang. 2019. HeteroCL: A Multi-Paradigm Programming Infrastructure for Software-Defined Reconfigurable Computing. In Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (Seaside, CA, USA) (FPGA’19). Association for Computing Machinery, New York, NY, USA, 242–251. doi:10.1145/3289602.3293910DOI
  45. [45]Yi-Hsiang Lai, Hongbo Rong, Size Zheng, Weihao Zhang, Xiuping Cui, Yunshan Jia, et al. 2020. SuSy: A Programming Model for Productive Construction of High-Performance Systolic Arrays on FPGAs. In Proceedings of the 39th International Conference on Computer-Aided Design (Virtual Event, USA) (ICCAD ’20). Association for Computing Machinery, New York, NY, USA, Article 73, 9 pages. doi:10.1145/3400302.3415644DOI
  46. [46]Shangkun Li, Jinming Ge, Diyuan Tao, Zeyu Li, Jiawei Jiang, Jinfeng Du, Jiang Xu, Wei Zhang, and Cheng Tan. 2026. NEURA: A Unified and Retargetable Compilation Framework for Coarse-Grained Reconfigurable Architectures. Proc. ACM Program. Lang. 10, PLDI (2026), 953–979. doi:10.1145/3808285DOI
  47. [47]Sean Lei. 2023. Cerebras Architecture Deep Dive: First Look Inside the HW/SW Co-Design for Deep Learning. Cerebras technical report. https://www.cerebras.ai/blog/cerebras-architecture-deep-dive-first-look-inside-the-hw-sw-co-design-for-deep-learning
  48. [48]Meta. 2024. Llama 3.2: Revolutionizing Edge AI and Vision with Open, Customizable Models. https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/
  49. [49]Peter Milder, Franz Franchetti, James C. Hoe, and Markus Püschel. 2012. Computer Generation of Hardware for Linear Digital Signal Processing Transforms. ACM Trans. Des. Autom. Electron. Syst. 17, 2, Article 15 (apr 2012), 33 pages. doi:10.1145/2159542.2159547DOI
  50. [50]Seungjae Moon, Jung-Hoon Kim, Junsoo Kim, Seongmin Hong, Junseo Cha, Minsu Kim, Sukbin Lim, Gyubin Choi, Dongjin Seo, Jongho Kim, Hunjong Lee, Hyunjun Park, Ryeowook Ko, Songyu Choi, Jongse Park, Jinwon Lee, and Joo-Young Kim. 2024. LPU: A Latency-Optimized and Highly Scalable Processor for Large Language Model Inference. doi:10.48550/arXiv.2408.07326 arXiv:2408.07326 [cs.AR]arxiv.org/abs/2408.07326
  51. [51]Moonshot AI. 2026. Kimi K3. https://platform.kimi.ai/docs/guide/kimi-k3-quickstart. Accessed September 8, 2026.
  52. [52]Thierry Moreau, Tianqi Chen, Luis Vega, Jared Roesch, Eddie Yan, Lianmin Zheng, Josh Fromm, Ziheng Jiang, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. A Hardware-Software Blueprint for Flexible Deep Learning Specialization. arXiv:1807.04188 https://arxiv.org/abs/1807.04188
  53. [53]Rachit Nigam, Sachille Atapattu, Samuel Thomas, Zhijing Li, Theodore Bauer, Yuwei Ye, Apurva Koti, Adrian Sampson, and Zhiru Zhang. 2020. Predictable Accelerator Design with Time-Sensitive Affine Types. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (London, UK) (PLDI 2020). Association for Computing Machinery, New York, NY, USA, 393–407. doi:10.1145/3385412.3385974DOI
  54. [54]Rachit Nigam, Pedro Henrique Azevedo de Amorim, and Adrian Sampson. 2023. Modular Hardware Design with Timeline Types. Proc. ACM Program. Lang. 7, PLDI, Article 120 (jun 2023), 25 pages. doi:10.1145/3591234DOI
  55. [55]Rachit Nigam, Ethan Gabizon, Edmund Lam, Carolyn Zech, Jonathan Balkind, and Adrian Sampson. 2026. Parameterized Hardware Design with Latency-Abstract Interfaces. In Proceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. ACM, New York, NY, USA, 14 pages. doi:10.1145/3779212.3790199DOI
  56. [56]Rachit Nigam, Samuel Thomas, Zhijing Li, and Adrian Sampson. 2021. A Compiler Infrastructure for Accelerator Generators. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (Virtual, USA) (ASPLOS ’21). Association for Computing Machinery, New York, NY, USA, 804–817. doi:10.1145/3445814.3446712
  57. [57]NVIDIA. 2026. CUDA Programming Guide: Programming Model. https://docs.nvidia.com/cuda/cuda-programming-guide/01-introduction/programming-model.html.
  58. [58]OpenAI. 2026. GPT-5.6 Sol Model. https://developers.openai.com/api/docs/models/gpt-5.6-sol. Accessed September 8, 2026.
  59. [59]OpenRouter. 2026. OpenRouter. https://openrouter.ai/. Accessed November 4, 2026.
  60. [60]Charles Papon. 2024. SpinalHDL: A High-Level Hardware Description Language. https://github.com/SpinalHDL/SpinalHDL.
  61. [61]Nathaniel Pinckney, Christopher Batten, Mingjie Liu, Haoxing Ren, and Brucek Khailany. 2024. Revisiting VerilogEval: A Year of Improvements in Large-Language Models for Hardware Code Generation. arXiv preprint. arXiv:2408.11053 [cs.AR] https://arxiv.org/abs/2408.11053
  62. [62]Reiner P. Reiner, Thomas Douglas, Akanksha Chowdhery, Jacob Devlin, James Bradbury, Anselm Levskaya, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. 2023. Efficiently Scaling Transformer Inference. In Proceedings of Machine Learning and Systems, Vol. 5. MLSys, USA, 606–624. https://proceedings.mlsys.org/paper_files/paper/2023/file/c4eb71ab8d24cdfbf45e3d06dfbca2780-Paper-mlsys2023.pdf
  63. [63]Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 https://arxiv.org/abs/2505.09388
  64. [64]Alejandro Rico, Satyaprakash Pareek, Javier Cabezas, David Clarke, Baris Ozgul, Francisco Barat, Yao Fu, Stephan Münz, Dylan Stuart, Patrick Schlangen, Pedro Duarte, Sneha Date, Indrani Paul, Jian Weng, Sonal Santan, Vinod Kathail, Ashish Sirasao, and Juanjo Noguera. 2024. AMD DXDNA NPU in Ryzen AI Processors. IEEE Micro 44, 6 (2024), 73–82. doi:10.1109/MM.2024.3423692DOI
  65. [65]Siemens EDA. 2026. Catapult C++/SystemC Synthesis Tool. Product documentation. https://www.siemens.com/en-us/products/ic/catapult-high-level-synthesis/hls/c-cplus/
  66. [66]Gina Sohn, Genghan Zhang, Konstantin Hossfeld, Jungwoo Kim, Nathan Sobotka, Nathan Zhang, Olivia Hsu, and Kunle Olukotun. 2026. Streaming Tensor Programs: A Streaming Abstraction for Dynamic Parallelism. In Proceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. ACM, New York, NY, USA, 1912–1932. doi:10.1145/3779212.3790229DOI
  67. [67]Nitish Srivastava, Hongbo Rong, Prithayan Barua, Guanyu Feng, Huangqi Cao, Zhiru Zhang, David Albonesi, Vivek Sarkar, Wenguang Chen, Paul Petersen, Geoff Lowney, Adam Herr, Christopher Hughes, Timothy Mattson, and Pradeep Dubey. 2019. T2S-Tensor: Productively Generating High-Performance Spatial Hardware for Dense Tensor Computations. In 2019 IEEE 27th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, Piscataway, NJ, USA, 181–189. doi:10.1109/FCCM.2019.00033DOI
  68. [68]Tensorstorrent. 2024. Blackhole. https://tenstorrent.com/en/hardware/blackhole.
  69. [69]Philippe Tillet, H. T. Kung, and David Cox. 2019. Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages. ACM, New York, NY, USA, 10–19. doi:10.1145/3315508.3329973DOI
  70. [70]Jianming Tong, Anirudh Itagi, Prasanth Chatarasi, and Tushar Krishna. 2024. Feather: A reconfigurable accelerator with data reordering support for low-cost on-chip dataflow switching. In 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA). IEEE, IEEE, New York, NY, USA, 198–214. doi:10.1109/ISCA59077.2024.00024DOI
  71. [71]Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. doi:10.48550/arXiv.2307.09288 arXiv:2307.09288 [cs.CL]
  72. [72]Chengyue Wang, Jiahao Zhang, Yingquan Wu, and Jason Cong. 2025. HP-FFT: A General High-Performance FFT Generator Using High-Level Synthesis. In Proc. IEEE FCCM. IEEE, Piscataway, NJ, USA, 19–23. doi:10.1109/FCCM62733.2025.00026DOI
  73. [73]Erwei Wang, Samuel Bayliss, Andra Bisca, Zachary Blair, Sangeeta Chowdhary, Kristof Denolf, Jeff Fifield, Brandon Freyberger, Erika Hunhoff, Phil James-Roxby, Jack Lo, Joseph Melber, Stephen Neuendorffer, Eddie Richter, Andre Rosti, Javier Setoain, Kagandeep Singh, Endri Taka, Pranathi Vasireddy, Zhewen Yu, Niansong Zhang, and Jimning Zhuag. 2026. From Loop Nests to Silicon: Mapping AI Workloads onto AMD NPUs with MLIR-AIR. ACM Trans. Reconfigurable Technol. Syst. 19, 2, Article 28 (2026), 36 pages. doi:10.1145/3785670DOI
  74. [74]Jie Wang, Licheng Guo, and Jason Cong. 2021. AutoSA: A Polyhedral Compiler for High-Performance Systolic Arrays on FPGA. In The 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (Virtual Event, USA) (FPGA’21). Association for Computing Machinery, New York, NY, USA, 93–104. doi:10.1145/3431920.3439292DOI
  75. [75]ShaoJie Xiang, Yi-Hsiang Lai, Yuan Zhou, Hongzheng Chen, Niansong Zhang, Debjit Pal, and Zhìru Zhang. 2022. HeteroFlow: An Accelerator Programming Model with Decoupled Data Placement for Software-Defined FPGAs. In Proceedings of the 2022 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (Virtual Event, USA) (FPGA ’22). Association for Computing Machinery, New York, NY, USA, 78–88. doi:10.1145/3490422.3502369DOI
  76. [76]Xilinx. 2025. MLIR-AIR Dialect. https://github.com/Xilinx/mlir-air.
  77. [77]AMD Xilinx. 2022. AI Engines and Their Applications. https://www.xilinx.com/content/dam/xilinx/support/documents/white_papers/wp506-ai-engine.pdf
  78. [78]AMD Xilinx. 2023. Vitis HLS v2023.2. https://www.xilinx.com/products/design-tools/vitis/vitis-platform.html.
  79. [79]AMD Xilinx. 2023. Vivado v2023.2. https://www.amd.com/en/products/software/adaptive-socs-and-fpgas/vivado.html.
  80. [80]Yuanzhong Xu, HyoukJoong Lee, Dehao Chen, Blake Hechtman, Yanping Huang, Rahul Joshi, Maxim Krikun, Dmitry Lepikhin, Andy Ly, Marcello Maggioni, Ruoming Pang, Noam Shazeer, Shibo Wang, Tao Wang, Yonghui Wu, and Zhifeng Chen. 2021. GSPMD: General and Scalable Parallelization for ML Computation Graphs. arXiv:2105.04663 [cs.DC] https://arxiv.org/abs/2105.04663
  81. [81]Rohan Yadav, Michael Garland, Alex Aiken, and Michael Bauer. 2025. Task-Based Tensor Computations on Modern GPUs. Proc. ACM Program. Lang. 9, PLDI, Article 163 (June 2025), 25 pages. doi:10.1145/3729262DOI
  82. [82]John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. arXiv preprint. arXiv:2405.15793 [cs.SE] https://arxiv.org/abs/2405.15793
  83. [83]Hanchen Ye and Deming Chen. 2025. StreamTensor: Make Tensors Stream in Dataflow Accelerators for LLMs. In Proceedings of the 58th IEEE/ACM International Symposium on Microarchitecture (MICRO ’25). Association for Computing Machinery, New York, NY, USA, 201–216. doi:10.1145/3725843.3762817DOI
  84. [84]Hanchen Ye, Cong Hao, Jianyi Cheng, Hyunmin Jeong, Jack Huang, Stephen Neuendorffer, and Deming Chen. 2022. ScaleHLS: A New Scalable High-Level Synthesis Framework on Multi-Level Intermediate Representation. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE Computer Society, Los Alamitos, CA, USA, 741–755. doi:10.1109/HPCA53966.2022.00060DOI
  85. [85]Hanchen Ye, Hyegang Jun, and Deming Chen. 2024. HIDA: A Hierarchical Dataflow Compiler for High-Level Synthesis. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1. ACM, New York, NY, USA, 215–230. doi:10.1145/3617232.3624850DOI
  86. [86]Zihao Ye, Yingyi Huang, Hongyi Jin, Bohan Hou, Junru Shao, Zhongming Yu, Jinqi Chen, Meghan Cowan, Shiyi Cao, Shanli Xing, Hanfeng Chen, Vinod Grover, Tianqi Chen, and Luis Ceze. 2026. CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution. doi:10.48550/arXiv.2608.12629 arXiv:2608.12629 [cs.LG]DOI
  87. [87]Jason Zhijingcheng Yu, Aditya Ranjan Jha, Umang Mathur, Trevor E. Carlson, and Prateek Saxena. 2026. Anvil: A General-Purpose Timing-Safe Hardware Description Language. In Proceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. ACM, New York, NY, USA, 110–136. doi:10.1145/3779212.3790125
  88. [88]Zhongzhi Yu, Mingjie Liu, Michael Zimmer, Yingyan Celine Lin, Yong Liu, and Haoxing Ren. 2025. Spec2RTL-Agent: Automated Hardware Code Generation from Complex Specifications Using LLM Agent Systems. arXiv preprint. arXiv:2506.13905 [cs.AR] https://arxiv.org/abs/2506.13905
  89. [89]Z.ai. 2026. GLM-5.3 Model Card. https://huggingface.co/zai-org/GLM-5.3. Accessed September 8, 2026.
  90. [90]Niansong Zhang, Sunwoo Kim, Andrew Butt, Shreesh Srinath, and Zhìru Zhang. 2026. From Pragmas to Partners: A Symbiotic Evolution of Agentic High-Level Synthesis. (Nov. 2026). https://www.cl.cam.ac.uk/~zhiruz/publications.html Invited paper, IEEE/ACM International Conference on Computer-Aided Design (ICCAD). To appear.arxiv.org/abs/2602.01401
  91. [91]Weichuang Zhang, Yiquan Wang, Xinzhuo Zhang, Chi Zhang, Yu Feng, Xiaofeng Hou, Chao Li, Jieru Zhao, and Minyi Guo. 2026. An Automated Compiler for Comprehensive Dataflow Optimization. In 2026 ACM/IEEE 53rd Annual International Symposium on Computer Architecture (ISCA). IEEE, Piscataway, NJ, USA, 1–17. doi:10.1109/ISCA66397.2026.00018DOI
  92. [92]Weichuang Zhang, Jieru Zhao, Guan Shen, Quan Chen, Chen Chen, and Minyi Guo. 2024. An Optimizing Framework on MLIR for Efficient FPGA-based Accelerator Generation. In 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, Los Alamitos, CA, USA, 75–90. doi:10.1109/HPCA57654.2024.00017DOI
  93. [93]Yichi Zhang, Junhao Pan, Xinsheng Liu, Hongzheng Chen, Deming Chen, and Zhìru Zhang. 2021. FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations. In Proceedings of the 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. Association for Computing Machinery, New York, NY, USA, 171–182. doi:10.1145/3431920.3439296DOI
  94. [94]Lianmin Zheng, Zhuohan Li, Hao Zhang, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Eric P. Xing, Joseph E. Gonzalez, and Ion Stoica. 2022. Alpha: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). USENIX Association, Carlsbad, CA, 559–578. https://www.usenix.org/conference/osdi22/presentation/zheng-liamminarxiv.org/abs/2201.12023
  95. [95]Jinming Zhuang, Shaojie Xiang, Hongzheng Chen, Niansong Zhang, Zhuoping Yang, Tony Mao, Zhiru Zhang, and Peipei Zhou. 2025. ARIES: An Agile MLIR-Based Compilation Flow for Reconfigurable Devices with AI Engines. In Proceedings of the 2025 ACM/SIGDA International Symposium on Field Programmable Gate Arrays. ACM, New York, NY, USA, 92–102. doi:10.1145/3706628.3708870DOI

Paper details

vlnc:W_qh7qunwk
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 12

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents