Read, Allocate, Route: Role-Coupled Tokenization for Pixel-Space Generative Transformers

Project Page: https://yangjiongfan.site/specslot/

GitHub: https://github.com/JiongfanYang/SpecSlot

Hugging Face: https://huggingface.co/Gokottaw434/SpecSlot

Introduction

Pixel-space generative models learn directly from noise to pixels, eliminating the separately trained autoencoder and its compression bottleneck. This simplicity, however, comes with a computational cost: the full-resolution image must be represented as Transformer tokens. Most pixel-space Transformers use one token per spatial patch, so the patch grid simultaneously determines what the model reads, what each token predicts, and how many tokens participate in global attention. As a result, finer image detail directly increases attention cost: on a $256 \times256$ image, reducing the patch size from $16 \times16$ to $8 \times8$ increases the token count from 256 to 1,024, making the quadratic term of dense attention $16\times$ larger.

Existing methods address this cost from different directions, but largely retain the coupling between output detail and token communication. JiT routes every patch token through the global sequence [1]. Multiscale systems move detail to extra stages or detailer heads [2, 3, 4], and efficient-attention methods shorten the sequence with generic patterns—windows, selected global tokens, or learned sparsity [5, 6, 7, 8]. Spectral Forcing brings in frequency structure, but only to filter the noisy input; its tokens remain spatial patches [9]. Across the pixel-space generators discussed above, output detail remains tied to a spatial patch grid, even when communication is compressed or restricted.

Our key observation is that finer detail does not necessarily require proportionally more global communication: in the frequency domain, low-frequency structure carries broad image organization and benefits from long-range exchange, while fine detail is largely local. A token carrying broad structure therefore stands to gain from exchanging with the whole image, while a token carrying fine detail mostly does not. Yet in existing designs, what a token communicates is fixed by where it sits, not by what it produces. Can a token’s output role also determine how it should communicate?

SpecSlot builds on this idea by making output roles explicit and using them to control communication. It organizes computation around three operations: Read, Allocate, and Route. Read forms a grid of spatial anchors that collect input evidence. Allocate expands each anchor into $K$ indexed slots, with each slot assigned a disjoint radial group of DCT coefficients. Route then uses these spectral roles to decide communication: only the slot assigned the lowest-frequency group from each anchor enters the global stage, while all slots interact locally within the anchor. This separates fine-grained output representation from the length of the global sequence. A spatial head further reuses the same slot states to add a local correction to the spectral prediction, without introducing additional tokens. Figure 2 illustrates this design.

This representation exposes two separate allocation variables. $W$ controls the density of spatial anchors, while $K$ controls the spectral subdivision at each anchor. The total number of slots grows with $WK$, but the global image sequence contains only $W$ carriers. A fixed token budget can therefore be allocated either to denser spatial anchors or to finer spectral roles without forcing both to increase together. This turns tokenization from a single patch-size choice into an explicit quality–compute allocation problem.

Our contributions are:

• We introduce an indexed $W \times K$ representation in which spatial anchors collect input evidence and slots receive disjoint spectral coefficient groups. The split exposes anchor density and coefficient subdivision as separate allocation variables, and the same slot states also decode a local pixel correction.

• We define carrier-routed global–global–local–local (GGLL) attention from the slot index: one lowest-radial carrier per anchor enters global blocks, and all slots enter anchor-local blocks, so the global sequence contains $W$ carriers rather than $WK$ slots.

• On ImageNet-256 and ImageNet-512, SpecSlot provides two measured quality–compute operating points between JiT’s low-cost and high-cost settings. At ImageNet-512, W256/K4 lowers FID by 2.84 over our FLOPs-matched JiT control (JiT-CM) with 3.9% fewer GFLOPs, and trails JiT-B/16 by 0.69 FID at 38.2% lower cost. Larger output windows cover more pixels while the global sequence remains fixed;

the larger improvement over JiT-CM is consistent with this within-anchor allocation. On text-to-image generation, SpecSlot likewise surpasses JiT-B/8 on DPG-Bench by 1.06 at 40.6% lower forward cost.

Related Work

Diffusion and pixel-space generation. Diffusion and score-based models learn to reverse a gradual noising process and provide the iterative foundation of modern image synthesis [10, 11, 12]. Rectified flow turns the same idea into a straight noise-to-data path [13]. Latent diffusion moved this process into the latent space of a separately trained autoencoder, and improved autoencoders continue to strengthen this line [14, 15, 16]. DiT, U-ViT, and SiT showed that plain Transformers scale reliably in this setting [17, 18, 19]. Pixel-space generation removes the autoencoder, but then asks the model to process the full-resolution state. This line has followed several routes: Simple Diffusion and Hourglass DiT reduce the state internally, cascaded models and PixelFlow distribute generation across resolutions, and JiT shows that a direct large-patch Transformer can remain a strong pixel-space baseline [1, 20, 21, 22, 23]. Detail recovery is then assigned to extra components: PixelDiT adds a pixel-level DiT, DiP adds a detailer head, HyperDiT connects coarse and fine flows, and PixNerd decodes patch tokens with a neural field [2, 3, 4, 24]. These designs differ in where the compute goes, but share one premise: a token is a spatial patch, so finer detail and wider communication stay tied to the same spatial grid. SpecSlot complements this line with a single-backbone allocation: indexed slots carry fixed output and communication roles, so detail can be subdivided without making every output token global.

Frequency structure and transform-domain generation. Frequency offers a complementary view of this problem. It has entered generative modeling through the state or forward process, through generation order or conditioning, and through the training objective or analysis. In the first line, inverse heat dissipation, wavelet models, sparse transform codes, and DCTdiff change the representation or process used for generation [25, 26, 27, 28, 29]. In the second, VAR and NFIG organize generation by scale or frequency, while Spectral Forcing and Frequency-Forcing use frequency-related conditioning or guidance; Latent Forcing instead reorders the trajectory with a semantic auxiliary stream [9, 30, 31, 32, 33]. Focal frequency loss reweights spectral errors, and Fourier-space analyses characterize diffusion without changing the denoiser [34, 35]. Recent architectural methods make frequency roles part of the denoiser itself: DiMSUM alternates spatial and wavelet-subband processing, while DeCo assigns low and high frequencies to a DiT and a separate pixel decoder [36, 37]. Together these works show several ways for frequency to shape representation or generation, but the cited lines do not use fixed spectral groups as both per-token output ownership and communication eligibility within one pixel-space backbone. SpecSlot keeps the state, target, and loss in pixel space; fixed, disjoint DCT groups give slots spectral roles, and the role index also determines which slots communicate globally.

Visual tokenization and allocation. Vision Transformers tokenize an image into a spatial patch grid [38], and NaViT extends this idea to variable resolutions [39]. This line reduces sequence work by changing how many spatial tokens are retained or processed at each step. ToMe merges similar tokens, SparseDiT prunes them, DyDiT++ reschedules computation across denoising time and space, variable-length tokenization learns a global merge, and FlexiDiT spends fewer tokens at easier steps [40, 41, 42, 43, 44]. The spatial patch remains the allocation unit, so detail and communication stay coupled to the same grid. SpecSlot adds a second, spectral axis: each of $W$ anchors owns $K$ slots with disjoint coefficient groups, making the fixed budget $N = WK$ an explicit split between anchor density and spectral subdivision.

Global-local and sparse communication. The communication literature provides the other half of the problem by restricting how tokens exchange information. Swin uses local windows with shifted boundaries, Longformer combines local windows with a few global tokens, and Perceiver routes information through a small latent bottleneck [5, 6, 7]. In pixel-space generation, RINs exchange information through learned interface states, while trainable sparse attention and PixSGR learn or select interaction patterns from the data [8, 45, 46]. These approaches provide useful local or global topologies, but do not tie the communication eligibility to a fixed spectral output role. SpecSlot fixes the graph from the allocation: one lowest-frequency slot per anchor is the global carrier, and all slots interact locally, so cross-anchor communication is spent on the role that also carries the low-frequency scaffold.

Methodology

SpecSlot organizes pixel-space generation around one idea: tokenization as allocation—assigning each token a fixed output role and dividing a fixed token budget between anchor density and spectral subdivision. Global attention is then spent only on the role that needs it; the rectified-flow state, target, and loss remain in pixel space.

Shared Pixel-Space Rectified-Flow Objective

Following the JiT controls and objectives [1, 13], we mix an image $x \in[-1,1]^{3 \times H \times H}$ with noise $\epsilon\sim\mathcal{N}(0,I)$ on the linear path

where $t=\sigma(g)$ with $g \sim\mathcal{N}(-0.8,0.8^2)$ and $y$ is the class label. The network returns an image-shaped prediction $r_\theta(z_t,t,y)$, converted to a velocity by

where the clamp avoids the singularity at $t=1$, and training minimizes the velocity MSE

SpecSlot Architecture

Why is allocation needed? In a patch-grid model, adding finer detail means shrinking the patch, and the global sequence grows with it; the two cannot be adjusted separately. Prior designs keep the spatial patch as the only allocation unit. SpecSlot instead introduces the missing variables, forming $W$ spatial anchors and expanding each anchor into $K$ indexed slots (Figure 3):

The budget splits into $W$ anchors that read the input and $K$ slots per anchor that divide the output; each slot owns an equal share $E$ of the coefficients per channel, with $s$ the input cell side and $L$ the output window side. The two variables act separately: $W$ sets how densely the input is read, and $K$ how finely the output is divided. At fixed $N$, anchor density trades directly against spectral subdivision, and output detail can grow with $K while the global sequence stays at $W$ carriers. Equal $N$ does not imply equal cost, since communication and projection scale differently with the split.

Three operations implement this allocation, as shown in Figure 2: Read collects each anchor’s input evidence, Allocate assigns each slot a fixed output role, and Route lets the role decide communication.

Read: Spatial Anchors and Slot Expansion

The first question is where input evidence comes from; SpecSlot reads it on a grid of anchors, independent of how the output is divided. Let $\mathcal{P}_i$ extract the $i$-th non-overlapping $s \times s$ cell of $z_t$. A shared projection maps each cell to an anchor vector,

This projection is a patch embedding whose width $B$ is free, because the output count no longer depends on it. A zero-initialized residual depthwise $3 \times3$ convolution over the anchor grid—the input mixer—lets neighboring cells exchange evidence before it is split into roles. A shared $1 \times1$ projection expands each mixed anchor vector into $K$ hidden states,

where $P_2 \in\mathbb{R}^{Kd \times B}$; a two-dimensional anchor position plus a learned slot position are added. One anchor vector seeds all $K$ slots; the roles come from the slot positions, output heads, and routing, not from the input. All $WK$ slots are evaluated at every flow time.

Allocate: Radial Coefficients and Pixel Synthesis

Radial coefficient assignment. The second question is who produces what. SpecSlot assigns each slot a fixed share of the output in a DCT basis. Low radial frequencies carry broad structure and high frequencies fine detail, and natural-image spectra decay smoothly along this order [9]; a radial split therefore sorts the roles from global to local without any learned assignment. For an output window of side $L = 2s$, the DCT-II indices $(u,v)$ are ordered by $\sqrt{u^2 + v^2}$ and split into $K$ equal groups, requiring $K \mid L^2$; slot $k$ of each anchor owns group $k$, that is, $E = L^2/K$ coefficients per channel. The disjoint groups cover the grid: every coefficient is predicted exactly once. A shared affine head predicts the owned coefficients from the slot’s final hidden state $h_{i,k}^{\mathrm{out}}$:

Sharing the head is deliberate: the slot index, not per-band parameters, defines ownership.

Lapped synthesis. Decoding each window independently would leave neighbors free to disagree on their shared edge, and the mismatch shows as block seams, most visibly early in training. SpecSlot decodes by the synthesis half of a 50%-lapped DCT frame instead: inverse DCT, a square-root periodic Hann window, and normalized overlap-add. Neighboring windows share half their pixels, and normalized overlap-add mitigates cross-window discontinuities.

where each window is weighted before overlap-add. Let $g$ be the separable square-root periodic Hann window, $\mathcal{D}$ the orthonormal two-dimensional DCT-II, $R_i^T$ the placement of window $i$ at stride $s = L/2$, and $Q$ the crop to the image region after reflection padding by $s/2$. The normalized synthesis is

Here $\Lambda^{\dagger}$ is the reciprocal on the covered region. For the matched analysis frame, let $P$ reflect-pad the image by $s/2$ pixels and let $R_i$ extract the $i$-th window. Its coefficients are

Applying the synthesis above to these coefficients gives $S(\{c_i(x)\})=x$ up to floating-point error: the same window is used in analysis and synthesis, $\Lambda$ normalizes the overlap, and the crop satisfies $QP=I$.

Anchor-aligned spatial correction. The same slot states can also be read out directly in pixel space. A second head adds an anchor-aligned correction alongside the spectral base: slot $k$ writes tile $k$ of a deterministic row-major $K_h \times K_w$ tiling ($K_hK_w=K$) of its anchor cell (four $8 \times8$ quadrants for $K=4$, $s=16$). These projections are slot-specific and zero-initialized, so training starts from a pure spectral model. The correction adds output parameters but no token states or attention edges. Radial ownership applies only to the spectral base; the spatial term is a pixel readout with no frequency assignment. The two terms express the same prediction in different bases, and § 6.1 finds their content complementary. The prediction is

Route: Carrier and Local Attention

The third question is who needs to communicate globally. Sending every slot through global attention would make the global quadratic cost grow with $K$—reintroducing exactly the coupling between output detail and communication cost that the slot allocation set out to remove. The spectral roles suggest a cheaper split: the lowest radial group holds the broad structure that benefits from cross-anchor exchange, while finer groups are increasingly local; its signal also overtakes the noise earliest in the flow (§ 6.2). SpecSlot therefore routes by role: the owner of the lowest radial group, slot 0, is the carrier and the only slot entering the global stage; all $K$ slots then mix locally within their anchor. Since global and local blocks alternate, the carrier brings global information into its anchor and takes local detail back into the next global stage.

Let $Z \in\mathbb{R}^{W \times K \times d}$ be the slot tensor and $Y \in\mathbb{R}^{C \times d}$ the $C=32$ context states formed from the class embedding. The two routing sets are

Two consecutive blocks process the global set and two more process each local set; this global-global-local-local (GGLL) schedule repeats three times, giving six global and six local blocks of identical dimensions. Class context appears only in the global group. Non-carrier slots traverse the six local blocks, context states the six global ones, and carriers all twelve. On ImageNet-256 with $s=16$ and $K=4$, the global sequence holds $W+C=288$ tokens instead of $WK+C=1{,}056$.

Ignoring projections and elementwise operations, the quadratic terms are

where $d$ is the hidden width; the global quadratic term now grows with $W$, not $WK$. Measured GFLOPs include the full execution and decoder, not only these terms.

Because $W$ and $K$ can be varied independently, their effects on quality and compute can be measured separately.

Experiments

Experimental Setup

Models and training. We train class-conditional models on ImageNet-1K [47] and report results at ImageNet-256 and ImageNet-512. Unless stated otherwise, our models use the $W256$ anchor grid of § 3 with $B=1536$, where $B$ is the input-projection width; we write SpecSlot-K2 and SpecSlot-K4 for the two allocations. ImageNet-256 training lasts 120 epochs at effective batch 512, and the aligned models share the full training recipe.

Evaluation. We sample with 50 Heun intervals on the common legacy-uniform grid and use EMA¹ weights. Reported rows use the per-model best classifier-free guidance (CFG) [48] from a coarse-to-fine sweep (Appendix § 9); JiT rows use 2.9 instead of 3.0, also the swept optimum for JiT (Table 10), matching SF’s protocol [9]. The main ImageNet and text-to-image results use five independently trained seeds. Ablations use three seeds, except that we reuse the five-seed results for W256/K2 and W256/K4. For ImageNet, FID and Inception Score [49, 50] are computed at 50K images per seed, and compared checkpoints share class ordering and noise draws.

(a) CFG	FID-50K $\downarrow$	IS $\uparrow$
1.00	29.4893	53.4115
2.00	7.3136	154.0057
2.90	4.3579	225.1920
4.00	5.0462	281.4794
5.00	6.3617	312.9500
6.00	7.5583	334.1862
7.00	8.5149	349.8076
8.00	9.2679	359.5481
(b) CFG	FID-50K $\downarrow$	IS $\uparrow$
2.25	5.7930	176.5661
2.50	4.9153	197.2529
2.75	4.4765	215.3064
2.90	4.3579	225.1920
3.25	4.3611	246.4003
3.50	4.5283	259.3960
3.75	4.7617	270.9640

Compute accounting. We measure forward cost on one H800 in BF16 with whole-network TorchInductor, counting a multiply–add as two FLOPs (full profiler contract in Table 8). Throughput is the best Heun-50 value over a shared batch sweep.

Table 8 Implementation and training contract of the class-conditional W256/K4 models. Resource rows are measured on one H800 with BF16 whole-network TorchInductor at profiler batch 128; the ImageNet-512 GFLOPs and throughput are the main-table measurements, and its peak-memory row is the logged training peak per GPU at batch 128.

ImageNet-256
Quantity	Value
Image side $H$	256
Input cell / stride $s$	16 / 16
Spatial anchors $W$	256
Slots per anchor $K$	4
Image slots $N = WK$	1,024
Hidden / FFN width / heads / blocks	768 / 3,072 / 12 / 12
Block design	pre-RMSNorm ($\epsilon=10^{-6}$), RMS-normed q/k, SwiGLU FFN
Conditioning	sinusoidal-256 $t$ embedding; AdaLN shift/scale/gates from $t$ + 32 class-context slots
Evidence width $B$	1,536
Input mixer	zero-initialized residual depthwise 3 × 3
Spectral-base window / stride	32 / 16
Spatial correction tiles / anchor stride	zero-init, row-major 2 × 2 / 16 (K2: 1 × 2, K8: 2 × 4, K32: 4 × 8)
Routing	$(G,G,L,L) \times3$
Input DCT / frequency gate	none / none
Prediction / loss	image-shaped prediction / pixel velocity MSE
Training epochs / GPUs	120 / 8
Batch per GPU (effective)	64 (512)
Optimizer / schedule	AdamW / constant after warmup
Actual learning rate	$1 \times10^{-4}$
EMA decays	0.9999 / 0.9996
Time sampling / label dropout	logit-normal (−0.8, 0.8²) / 0.1
Reported training seeds	12345–12349
Trainable parameters	138.025728M
Profiler device / precision	H800 / BF16
Conditional forward GFLOPs	129.20
Conditional peak memory (batch 128)	5.56 GB
Heun-50 peak memory (batch 128)	5.84 GB
Heun-50 throughput (batch 128)	9.06 img/s

ImageNet-512
Quantity	Value
Image side $H$	512
Input cell / stride $s$	32 / 32
Spatial anchors $W$	256
Slots per anchor $K$	4
Image slots $N = WK$	1,024
Hidden / FFN width / heads / blocks	768 / 3,072 / 12 / 12
Block design	pre-RMSNorm ($\epsilon=10^{-6}$), RMS-normed q/k, SwiGLU FFN
Conditioning	sinusoidal-256 $t$ embedding; AdaLN shift/scale/gates from $t$ + 32 class-context slots
Evidence width $B$	4,096
Input mixer	zero-initialized residual depthwise 3 × 3
Spectral-base window / stride	64 / 32
Spatial correction tiles / anchor stride	zero-init, row-major 2 × 2 / 32
Routing	$(G,G,L,L) \times3$
Input DCT / frequency gate	none / none
Prediction / loss	image-shaped prediction / pixel velocity MSE
Training epochs / GPUs	120 / 8
Batch per GPU (effective)	128 (1,024)
Optimizer / schedule	AdamW / constant after warmup
Actual learning rate	$2 \times10^{-4}$
EMA decays	0.9999 / 0.9996
Time sampling / label dropout	logit-normal (−0.8, 0.8²) / 0.1
Reported training seeds	{0, 1, 2, 3, 4}
Trainable parameters	160.859904M
Profiler device / precision	H800 / BF16
Conditional forward GFLOPs	135.64
Training peak memory (batch 128 / GPU)	17.39 GB / GPU
Heun-50 throughput (batch 128)	12.69 img/s

Compute-matched JiT control. JiT-CM/124G keeps JiT-B/8’s raw 8 × 8 tokenizer and routes features through a 128-dimensional input bottleneck and 32 context tokens inserted at block 4 of the 12-block trunk; width 560 with 10 heads measures 12.24 GFLOPs under the profiler above. It is an implementation-defined control matched on forward GFLOPs rather than a published JiT size. The ImageNet-512 control JiT-CM/141G uses the corresponding 16 × 16 tokenizer at width 600 (141.21 GFLOPs); we write JiT-CM for the resolution-matched control in what follows.

ImageNet Results

On ImageNet-256, SpecSlot-K2 and SpecSlot-K4 are two measured, nondominated operating points between JiT-B/16 and JiT-B/8 (Table 1): K2 already improves over JiT-B/16, and K4 improves further while using 40.9% fewer FLOPs than JiT-B/8, which remains the quality leader. JiT-CM gives the closer compute comparison: SpecSlot-K4 matches its mean FID within the five-seed 95% interval at nearly identical FLOPs and throughput (Appendix § 11), and descends faster at early-to-mid checkpoints (Figure 4).

Model	Tokens	$B$	CFG	FID $\downarrow$	IS $\uparrow$	GFLOPs $\downarrow$	img/s $\uparrow$
ImageNet-256
JiT-B/16	256	–	2.90	7.51	174.81	50.45	33.08
JiT-B/8	1,024	–	2.90	4.34	225.72	218.50	8.15
SF-B/16	256	–	2.90	7.81	172.35	50.85	32.89
JiT-CM	1,024	–	2.90	6.68	182.56	124.24	8.94
SpecSlot (W256/K2)	512	512	5.00	7.27	219.49	82.63	21.32
SpecSlot (W256/K4)	1,024	1,536	4.00	6.59	215.20	129.20	9.06
ImageNet-512
JiT-B/32	256	–	2.90	15.88	162.12	51.50	30.61
JiT-B/16	1,024	–	2.90	8.45	196.61	219.56	7.93
SF-B/32	256	–	9.50	11.41	172.48	54.72	30.88
JiT-CM	1,024	–	2.90	11.98	178.72	141.21	8.51
SpecSlot (W256/K2)	512	3,072	10.50	10.94	183.69	94.52	18.32
SpecSlot (W256/K4)	1,024	4,096	8.25	9.14	208.24	135.64	12.69

At ImageNet-512, each anchor covers four times as many pixels as at ImageNet-256, while our global sequence stays at 288 tokens. Larger spectral windows and the spatial correction can accommodate the additional within-anchor detail without adding global communication, offering a possible explanation for SpecSlot’s larger margin at this resolution. SpecSlot-K4 improves FID by 2.84 over JiT-CM at slightly lower cost, and is only 0.69 FID behind JiT-B/16 at 38.2% lower cost and 1.6× higher throughput; it also attains the highest Inception Score in the panel, and both allocations outperform the input-side spectral baseline SF-B/32. The lower-cost SpecSlot-K2 stays within 1.04 FID of the control at 33.1% less compute.

Against the strongest JiT model in each panel, which uses about 1.7× our compute, our FID deficit narrows from 2.25 at ImageNet-256 to 0.69 at ImageNet-512, while our lead over JiT-CM grows from 0.09 to 2.84. These two operating points are consistent with the within-anchor allocation mechanism above, but do not by themselves isolate the cause of the larger margin.

Paired ImageNet-256 samples are shown in Figure 5; additional examples are provided in Appendix Figure 9.

Text-to-Image Generation

Protocol. We test whether the allocation and routing design transfers from class labels to text conditions. We train at $256 \times256$ on a mixture of the BLIP3o long-caption and short-caption shards [53] and JourneyDB shards [54]. A frozen OpenCLIP ViT-L/14 encoder [55] and a trainable 32-query adapter provide the denoiser context. SpecSlot reads this context through its carrier states during the global route, while local blocks continue to process image slots; JiT uses the same text encoder and adapter. We retain the ImageNet rectified-flow objective, optimizer, and EMA; the complete training contract is in Appendix Table 9. Evaluation uses the 50-step Heun sampler with CFG 4.0 for SpecSlot and 2.9 for JiT. Table 2 reports the results.

Model	CFG	GFLOPs ↓	DPG ↑	GenEval ↑
JiT-B/16	2.9	50.71	59.22	24.89
JiT-B/8	2.9	218.77	61.15	29.78
SpecSlot (W256/K4)	4.0	129.86	62.21	26.99

Results. Transferred to text conditioning, SpecSlot attains the highest DPG-Bench score, improving over both JiT baselines at 40.6% lower forward cost than the stronger JiT-B/8 (Table 2). On GenEval it improves over JiT-B/16, while JiT-B/8 remains the quality leader at 1.7× the forward cost—the same quality–compute trade-off as on ImageNet-256. This transfer requires no T2I-specific change to the SpecSlot architecture or training objective. Selected text-to-image samples are shown in Figure 1; additional examples are provided in Appendix Figure 10.

Ablation Study

Understanding the Design

Anchor Density and Coefficient Subdivision

Both allocation axes trade compute for quality, but they are not interchangeable: $W$ improves the read, $K$ refines the write (Table 4). Increasing $K$ at fixed $W$ lowers FID at higher cost with diminishing returns—the last doubling buys only 0.31 FID—while shifting the split toward more anchors at fixed $N$ lowers FID monotonically, with no clear saturation in this range. Within the tested $N=2{,}048$ configurations, these trends favor allocating more of the token budget to spatial anchors; we do not claim that this rule generalizes beyond the measured range.

(a)
Alloc.	FID ↓	IS ↑	GFLOPs ↓	img/s ↑
Fixed slot budget
W64/K32	7.28	218.52	195.43	9.00
W256/K8	6.28	219.52	220.02	6.55
W1024/K2	5.66	212.31	292.29	5.21
Fixed anchor grid
W256/K2	7.39	222.97	83.84	20.10
W256/K4	6.59	215.20	129.20	9.06
W256/K8	6.28	219.52	220.02	6.55
(b)
Input	Output	FID ↓	IS ↑	GFLOPs ↓ / img/s ↑
Single basis
Freq.	Freq.	7.34	222.46	129.30 / 9.35
Freq.	Spat.	7.23	215.73	129.20 / 9.63
Spat.	Freq.	7.10	221.37	129.20 / 9.06
Spat.	Spat.	6.99	215.56	129.10 / 9.34
Dual basis
Spat.	Freq.+ Spat.	6.59	215.20	129.20 / 9.06

At fixed $W$ and $B$, the entropy-effective slot fraction decreases monotonically from 0.803 at $K=2$ to 0.495 at $K=8 ({xref}Table 5`). The absolute effective rank still grows, but sublinearly, so token count alone does not measure effective capacity. This utilization pattern is consistent with the diminishing quality gains at larger $K$ and motivates our choice of K2 as the lower-cost point and K4 as the intermediate point.

Input and Output Representations

At the same spectral output, spatial input surpasses frequency input, and the dual-basis hybrid achieves the best FID: 6.59, a further 0.40 below the best single-basis variant at 6.99, without adding tokens or cross-anchor attention edges (Table 4(b)). This margin indicates that the two output bases are not redundant: the spatial correction supplies structure that the spectral base alone misses. § 6.1 localizes this complementarity over the flow and across frequency bands.

Carrier Routing

To test routing at a fixed forward budget, we train a full-global control (Dense-global-CM) on the same $W256/K4$ configuration, replacing the carrier/local schedule with 12 full-global blocks over all 1,024 image slots; its hidden width is 560, matching the measured GGLL forward cost. GGLL is both better and faster than this control at essentially identical FLOPs (Table 3). Full-global attention reaches a lower FID but at 74.8% higher cost and lower throughput. The carrier design therefore targets the intermediate, attention-constrained regime, saving 42.8% of the full-global cost. GLGL keeps the same carrier set and global budget but alternates the global/local order; its FID degrades at identical compute, showing that block order affects performance under this fixed carrier set. § 6.2 provides a possible motivation for early global processing: the lowest-radial role is recovered earliest in the flow.

Route	Cross-anchor image states	FID ↓	IS ↑	GFLOPs ↓	img/s ↑
GGLL	one carrier per anchor	6.59	215.20	129.20	9.06
GLGL	one carrier per anchor	6.95	218.95	129.20	9.00
Dense-global-CM	all image slots	6.81	210.20	128.96	5.80
Full-global	all image slots	5.84	237.54	225.82	5.11

Component Ablations

The following controls vary one implementation component at a time at the fixed anchor–slot contract. The best FID among the tested evidence widths is obtained at $B=1536$ (Table 6(a)). The residual mixer (Table 6(d)) and 50% overlap (Table 6(c)) improve FID at nearly unchanged forward cost, supporting their use in the final contract.

(a) Evidence width $B$	FID ↓	IS ↑	GFLOPs ↓	img/s ↑
$B=1024$	6.77	221.58	128.19	9.18
$B=1536$	6.59	215.20	129.20	9.06
$B=2048$	6.64	224.80	129.87	9.04
(b) Route	20	30	50
GGLL	21.02	13.68	8.20
Dense-global-CM	14.31	9.16	5.56
Full-global	12.35	8.01	4.79
(c) Output window	FID ↓	IS ↑	GFLOPs ↓	img/s ↑
Non-overlapping	6.78	211.23	129.19	9.13
50% overlapping	6.59	215.20	129.20	9.06
(d) Input mixer	FID ↓	IS ↑	GFLOPs ↓	img/s ↑
Identity	6.83	220.38	129.20	9.06
Residual depthwise mixer	6.59	215.20	129.20	9.06

Sampling Efficiency

The main tables report peak Heun-50 throughput. To check how route cost scales with the sampling budget, we also measured fixed-batch BF16 sampling at batch 512 at 20, 30, and 50 Heun intervals (Table 6(b)). GGLL is faster than the compute-matched dense-global control at every tested budget: its throughput is 1.47–1.49× higher.

Mechanistic Analysis

Spectral Base and Spatial Correction

Figure 7 shows the spectral term has the larger self-energy, while the spatial term’s share grows from 0.07 to 0.23 over the flow, and the normalized cross term stays positive—the two terms reinforce, not cancel. Projected through the matched lapped-DCT analysis frame (which does not confer radial coefficient ownership), the spatial term’s share is smallest in the lowest radial group and dominates the higher groups late in the flow. Removing either term with the trunk fixed raises velocity MSE: at $t=0.90$ by 32.1× (spatial) or 64.1× (spectral), and at $t=0.05$ by 1.1× and 2.1×. The spectral branch is the main scaffold throughout these probes; the spatial branch supplies an additional late, higher-frequency correction under this parameterization.

Carrier Identity and Routed Communication

We vary only which slot receives cross-anchor access, holding the $W256/K4$ allocation, GGLL graph, and compute fixed (Table 7). Replacing the lowest-radial carrier with slot 1, 2, or 3 raises mean FID in every case (Table 7): the assignment matters, and the lowest-radial role is also the one whose signal emerges earliest. In Figure 6, this band is the first to cross DNR $= 1$, motivating the assignment of slot 0 to the global path. Table 3 compares separately trained communication graphs, whereas the GGLL intervention curves in Figure 6 change only route edges inside the shared Transformer; Dense-global is a separately trained reference. On paired states, the lowest radial group is recovered earlier than the higher groups; removing cross-anchor carrier edges costs most early, while removing local cross-band edges costs most late.

Global carrier	FID ↓	IS ↑
Slot 0 (lowest radial)	6.59	215.20
Slot 1	6.91	220.38
Slot 2	7.21	203.90
Slot 3	6.97	214.87

Conclusion

SpecSlot recasts tokenization as allocation: a fixed token budget is split between anchor density and spectral subdivision, while global attention is assigned by role. On ImageNet-256, it reaches a comparable final FID to the FLOPs-matched JiT control and descends faster early in training; on ImageNet-512, it improves FID by 2.84 over that control and approaches JiT-B/16 at 38% lower cost. On text-to-image generation, the same design surpasses JiT-B/8 on DPG-Bench (62.21 vs. 61.15) at 40.6% lower forward cost. These results show that, in the evaluated pixel-space settings, output detail and global-communication cost can be decoupled through tokenization. Our diagnostics find the lowest-radial role is recovered earliest, supporting its use as the carrier. Allocation gives pixel-space models a new way to trade detail for communication.

Implementation Contract

Table 8 records the implementation and training contract of the final class-conditional models, and Table 9 that of the text-to-image model.

Table 9 Text-to-image implementation and training contract (§ 4.3).

Quantity	Value
Backbone configuration	W256/K4/B1536 at 256 × 256 (Table 8)
Text encoder	frozen OpenCLIP ViT-L/14, 77-token states
Text adapter	32-query single-layer cross-attention
Training data	BLIP3o long/short-caption + JourneyDB shards
Training epochs / GPUs	400 / 8 (64 images per GPU)
Optimizer / learning rate	AdamW / $2 \times10^{-4}$
Caption dropout / EMA	0.1 / 0.9999
Sampler / CFG	Heun-50 / 4.0 (SpecSlot), 2.9 (JiT)
Benchmarks	GenEval (553 prompts × 4), DPG-Bench (1,065 × 4)

Classifier-free Guidance Sweep

We use JiT-B/8 on ImageNet-256 as a concrete example of the guidance-scale search. Every point evaluates the same epoch-120 EMA1 checkpoint on 50K images with per-rank batch size 512, the legacy-uniform Heun sampler with 50 steps, generation seed 0, and guidance interval $[0.1, 1.0]$; only CFG changes. We first run a coarse grid at CFG $\in\{1.0, 2.0, 2.9, 4.0, 5.0, 6.0, 7.0, 8.0\}$ (see Table 10(a)). We then refine the low-FID region with a local $\pm1$ window and increments of 0.25. Table 10(b) lists the fine-grid evaluations retained in the archive; the coarse optimum 2.90 is also the fine optimum. We use CFG 2.9 for JiT in the main comparisons, rather than 3.0, to match the scale used by Spectral Forcing [9].

Sampling-step Latency Scaling

The main tables use 50 Heun intervals for quality evaluation. We additionally measured sampling latency at 20, 30, and 50 intervals for the trained $W256/K4$ model and two routing controls. The measurement uses one H800, BF16, a fixed sampling batch of 512, one warm-up repeat, and three timed repeats. The sampler is the same legacy-uniform Heun implementation used for the quality results. Under the profiler’s classifier-free-guidance accounting, the corresponding network-evaluation counts are 73, 113, and 187 per image. We report both quantities to avoid confusing an integration budget with a network-evaluation count. The route-specific CFG values are 4.0 for GGLL, 4.25 for Dense-global-CM, and 3.5 for Full-global.

Table 11 reports the median model-sampling latency. The compute-matched dense-global control has nearly the same per-evaluation FLOPs as GGLL, so it is the primary routing comparison; the full-global row uses a wider network and is included as a cost reference. At every tested budget, GGLL is faster than both controls. Relative to Dense-global-CM, its throughput is 1.47–1.49$\times$ higher across the three budgets; relative to full-global, it is 1.70–1.71$\times$ higher. Figure 8 shows the same scaling in latency.

Route	Heun	Evals/image	GFLOPs/eval	TFLOPs/image	ms/image	img/s
GGLL	20	73	128.59	9.39	47.56	21.02
Dense-global-CM	20	73	128.96	9.41	69.89	14.31
Full-global	20	73	225.22	16.44	80.99	12.35
GGLL	30	113	128.59	14.53	73.09	13.68
Dense-global-CM	30	113	128.96	14.57	109.20	9.16
Full-global	30	113	225.22	25.45	124.83	8.01
GGLL	50	187	128.59	24.05	121.88	8.20
Dense-global-CM	50	187	128.96	24.11	179.95	5.56
Full-global	50	187	225.22	42.12	208.60	4.79

These measurements isolate fixed-batch model-sampling latency under per-network compilation; the quality numbers in the main tables remain the 50-step evaluations defined in § 4.1.

Five-seed ImageNet-256 Results

The main tables report means over independently trained seeds. This section exposes the individual runs and the across-seed uncertainty for the two $W256$ allocations used in the ImageNet-256 comparison. Table 12 lists each training seed, while Table 13 reports the FID and Inception Score statistics. The training seed identifies the independently trained checkpoint; the generation seed is fixed to zero for every row. Evaluations use epoch-120 EMA1 weights, 50K generated images, the legacy-uniform 50-step Heun sampler, and the same class ordering and noise draws. The selected CFG is 5.0 for K2 and 4.0 for K4, as in Table 1; the K2 runs use evidence width $B=512$.

Allocation	Training seed	FID-50K ↓	IS ↑
$W256/K2$	12345	7.2191	214.2811
$W256/K2$	12346	7.0054	237.4801
$W256/K2$	12347	7.6025	206.7721
$W256/K2$	12348	7.3332	226.7448
$W256/K2$	12349	7.1983	212.1924
$W256/K4$	12345	6.4266	232.6772
$W256/K4$	12346	6.6197	212.0868
$W256/K4$	12347	6.4503	224.0468
$W256/K4$	12348	6.8389	201.4354
$W256/K4$	12349	6.6250	205.7543

Statistic	$W256/K2$	$W256/K4$
$n$	5	5
Mean FID	7.2717	6.5921
SD	0.2192	0.1660
SE	0.0981	0.0742
95% CI	[6.9995, 7.5439]	[6.3860, 6.7982]
Median	7.2191	6.6197
Min–Max	[7.0054, 7.6025]	[6.4266, 6.8389]
Statistic	$W256/K2$	$W256/K4$
$n$	5	5
Mean IS	219.4941	215.2001
SD	12.4339	12.9624
SE	5.5606	5.7970
95% CI	[204.0554, 234.9328]	[199.1051, 231.2951]
Median	214.2811	212.0868
Min–Max	[206.7721, 237.4801]	[201.4354, 232.6772]

The aligned five-seed mean trajectories in Figure 4, presented in the main comparison, complement the endpoint statistics above.

Additional Mechanistic Diagnostics

This appendix collects measurements that probe how SpecSlot uses its explicit roles. Unless stated otherwise, a probe loads a checkpoint and intervenes on its states without updating any parameter; these results characterize representation and communication rather than claim end-to-end quality gains. For the route probe, the intervention is made inside the shared Transformer after slot formation and before the output head, so its interpretation concerns the spectral-slot/GGLL submodule rather than a particular output basis.

Evidence timing and route interventions

We first measure an empirical data-to-noise ratio (DNR) for each radial coefficient group in the lapped frame. For group $k$, $\operatorname{DNR}_{k}(t)=t^{2}\mathbb{E}[\lVert A_{k}x\rVert^{2}]/[(1-t)^{2}\mathbb{E}[\lVert A_{k}\epsilon\rVert^{2}]]$, where $A_{k}$ denotes the matched lapped-frame analysis restricted to that group. The crossing time $t^{\star}$ is the first flow time at which the group reaches DNR $\ge1$; it is a descriptive measure of when the noisy input contains more signal than noise, not a learned gate. Table 14 reports the crossings for the paired W256/K8, W256/K4, and W1024/K2 checkpoints.

Allocation	$t^*$ for successive radial groups
$W256/K8$	$\{0.382, 0.836, 0.880, 0.904, 0.922, 0.933, 0.943, 0.959\}$
$W256/K4$	$\{0.468, 0.891, 0.928, 0.951\}$
$W1024/K2$	$\{0.549, 0.936\}$

The frontiers move when $W$ or $K$ changes: the low group becomes usable earlier, while progressively finer groups become usable later. This is consistent with assigning the early cross-anchor path to the lowest radial role. In the matched teacher-forced route intervention, removing the global carrier path primarily increases the low-group and early error; removing local mixing primarily increases detail-group error later in the trajectory. For example, the mid-time log-MSE penalties for the W256/K8 and W256/K4 probes are approximately 0.14/1.66 and 0.20/2.14 (global/local). The on-policy intervention has the same ordering: global removal changes the early coarse field (low-frequency change about 0.98, 0.99, and 1.05 for the three allocations), while local removal continues to change mid/high frequency content in the middle and late segments. These measurements are consistent with the role assignment used by GGLL.

Allocation and effective use of additional slots

To separate single-step fitting from the use of extra role slots, a paired fixed-$W = 256$, fixed-$B = 1536$ analysis compares $K \in\{2, 4, 8\}$. The models share the input, lapped output, and GGLL interfaces used by the registered $K$-rate diagnostic. For reproducibility, the effective-slot value is the entropy effective rank of each anchor’s slot Gram matrix, divided by $K$; the effective-local-key value is $\exp(H_q)/K$ averaged over local queries, with $H_q = -\sum_j p_{qj} \log p_{qj}$; and detail-to-carrier is the mean local attention probability from detail queries to the carrier key. Table 5 in § 5.1.1 shows that although the teacher-forced velocity error changes little, the fraction of effective slots and the fraction of local keys decline as the within-anchor sequence grows.

$K$	Velocity MSE	Effective slots/$K$	Effective local keys	Detail → carrier
2	.10670	.803	.729	.330
4	.10549	.620	.682	.312
8	.10497	.495	.621	.275

Thus adding slots can slightly improve teacher-forced fitting without providing the same number of independent, strongly coupled directions. This is consistent with an allocation sweet spot: $K$ controls how finely the coefficient ownership is divided, but it is not a promise that every added slot contributes an equal amount of information.

Paired Qualitative Comparisons

ImageNet examples

Figure 9 compares selected ImageNet-256 generations from JiT-CM and SpecSlot W256/K4 under the same class and noise draw. In these pairs, SpecSlot renders class-specific fine structure more coherently: feather boundaries are clearer for brambling, chickadee, and rooster; the cornet has more regular tubing and bell geometry; and the boathouse preserves sharper roofline and reflection structure. It also reduces visible local distortions in the anemone fish, banister, and barrel examples, where contours or repeated parts are less coherent in the JiT-CM samples. The hay example shows a more regular field-scale texture.

Text-to-image examples

Figure 10 shows additional SpecSlot samples generated from text prompts in DPG-Bench. The examples cover portraits, creatures, environments, architecture, and interiors; the prompt paired with each sample is included in the figure.

Prompt: Visually intriguing macro photography captures mystical waves, seemingly afloat and intertwined with delicate, hair-like particles. Shades from a psychedelic color palette create a stunning and surreal scene reminiscent of twilight. These vibrant waves offer a hyper-realistic glimpse into an otherworldly dimension.

Prompt: A grand, sprawling landscape is inspired by Hayao Miyazaki’s “Nausicaä of the Valley of the Wind” and “Breath of the Wild” from The Legend of Zelda series. Studio Ghibli’s post-apocalyptic setting blends with a vibrant open-world aesthetic. Towering ancient trees with twisted roots rise from the earth, while bioluminescent creatures add surreal luminance.

Prompt: An incredibly detailed digital artwork depicts an enormous skyscraper soaring into the sky above an industrial power station, as imagined by Peter Elson, Chris Moore, and Jim Burns. The skyscraper has intricate designs, reflective glass windows, and numerous protruding antennas. Pipes, wires, and glowing energy cores fill the power station at its base.

Prompt: A detailed portrait of a character concept titled “Under Dreaming Tree,” characterized by its symmetrical design and realistic texturing. The character is situated centrally before an ethereal backdrop with the namesake tree and expansive, leaf-laden branches. The visage displays a serene expression.

Prompt: A voluminous library, bathed in soft, muted blue hues, stretches across a spacious Victorian-style living room. Intricately carved wooden bookshelves line the walls, and a grand fireplace is the focal point. Plush patterned rugs, polished wooden tables, and antique armchairs complete the room.

Prompt: A rendered Celtic temple stands majestically in a digital landscape, exhibiting high detail and a photorealistic appearance. Intricate stone textures and realistic lighting define the temple, which is surrounded by a lush environment with dense foliage.