RoadBlock: Rethinking GPU Tensor Core Microarchitecture for Emerging Microscaling Format Support

CCS Concepts

• Computer systems organization → Parallel architectures.

Introduction

Emerging Microscaling (MX) formats such as 4over6 [2], IF4 [3], and RaZeR [1] are typically evaluated as standalone MAC-unit prototypes intended for systolic-array accelerators, silently ignoring the system-level integration overheads in NVIDIA Blackwell and AMD CDNA4-like GPU Tensor Cores that production ML systems stacks actually use. To bridge this gap, we propose RoadBlock¹, an open-source framework for end-to-end custom MX format modeling and Perplexity-PPA trade-off analysis, built on the programmable RISC-V Vortex GPGPU [5]. It exposes data size, scale-factor size, and block size as configurable parameters for MX format design-space exploration, and automatically derives per-thread scale-factor metadata tiling and register requirements for arbitrary formats.

RoadBlock Microarchitecture

MX-enhanced Fused Dot Product (FEDP). The dot product for two MX-compliant vectors $A$ and $B$ of length $k$ is given by:

where $X^{(A)}$, $X^{(B)}$ are block scale factors and $P_i^{(A)}$, $P_i^{(B)}$ are their corresponding elements. RoadBlock adopts and enhances the 4-cycle Ten-Four [4] mixed-precision FEDP microarchitecture as illustrated in Figure 1. Since Ten-Four performs early accumulation of the “C” addend alongside the vector products, the required scale-factor computations and shifts need to be applied to each lane’s significand multiplication result in the first MULTIPLY stage itself.

Vector (PDF)

MX Scale Factor Metadata Tiling. Vortex’s RISC-V-extended SIMT programming model provides a 32 registers/thread budget that is divided between matrices A, B, and C/D (8 each). Given that contiguous low-precision elements are packed along K into 32-bit bundles, Figure 3 derives the required per-thread MX metadata for an arbitrarily designed format. Assuming NVFP4 (d=4, w=8, k=16) as the finest granularity, these formulae indicate a maximum requirement of 64 bits/thread (two extra registers) for MX metadata.

MX Metadata Delivery and Storage. Since Vortex TCU WMMA instructions only expose three source operands, MX metadata cannot be supplied as an additional explicit source. RoadBlock instead duplicates registers f8, f9 into a lightweight TCU-SRAM through a MUX activated when the WMMA’s format field indicates an MX format. The scoreboard treats these as “hidden source registers,” to preserve dependency tracking. Additional metadata required by unorthodox format proposals can also be stored in this TCU-SRAM.

RoadBlock Host-Kernel ABI

RoadBlock extends the Vortex WMMA ABI with pointers to the A/B scale-factor buffers produced during host-side quantization, as shown in Figure 4. For two-level formats such as NVFP4, block-wise E4M3 scaling is handled within the TCU, while tensor-wise FP32 dequantization is executed as WMMA epilogue on the FPU lanes.

Vector (PDF)

Experiments & Evaluation

To evaluate the framework’s effectiveness, we modeled MXFP8, MXFP4, NVFP4, 4over6, IF4, and RaZeR on RoadBlock, and synthesized format-specific FEDP units using ASAP7 LVT/TT in Yosys under a 1.5 GHz timing constraint. Figure 2 characterizes Llama3.1-8B and Qwen3-32B perplexity against the formats’ hardware efficiency across standard GPU Tensor Core scaling points ($N \in\{2, 4, 8, 16\}$).

Vector (PDF)

We find that, while MXFP4 has the worst perplexity among the group, its simple single-level E8M0 scale-factor and 32-element block size still make it the most efficient datatype when accounting for both accuracy and hardware cost together. On the other hand, while IF4 preserves model quality quite well, its extra range-alignment correction multiplier makes it an impractical addition to existing Tensor Core FEDP datapaths. As a middle-ground, RaZeR delivers the strongest Perplexity-PPA tradeoff among its NVFP4 counterparts, becoming especially attractive at larger FEDP lengths, exactly where modern GPU Tensor Core designs are headed.

Future Work & Conclusion

Next, we plan on prototyping more unconventional emerging MX formats on RoadBlock and investigating warp-specialized kernels for multi-level block-scaled formats. In conclusion, RoadBlock provides a flexible, open-source, hardware-grounded means to evaluate whether a custom format proposal’s algorithmic advantage survives integration into a realistic, programmable GPU Tensor Core setting.

References