betaFeedback

WorksComputer Science

Asuka-FM: A System-Architectural Co-Design Framework for Efficient Text-to-Image Pretraining on Consumer Hardware

Abstract

Frontier text-to-image (T2I) foundation models predominantly rely on multi-billion parameter Diffusion Transformers (DiTs) pretrained across massive compute clusters, creating an accessibility barrier for academic laboratories. When training DiTs from scratch on single consumer GPUs (e.g., an NVIDIA RTX 4090 with 24,GB VRAM), practitioners encounter three primary bottlenecks: quadratic attention sequence costs, Just-In-Time (JIT) compilation graph breaks induced by variable aspect-ratio bucketing, and an autoencoder throughput wall where frozen Variational Autoencoder (VAE) operations dominate step latency. In this work, we introduce Asuka-FM, an end-to-end system and architectural framework demonstrating that competitive dual-stream DiTs (∼\sim206M parameters) can be pretrained from scratch on a single consumer GPU in ∼\sim160 GPU hours for under $55. Asuka-FM integrates four core designs: (i) an AdaLN-free dual-stream DiT backbone reducing parameter count; (ii) structured sequence sparsification via Sparse-Dense Residual Fusion (SPRINT) to cut attention FLOPs by over 50%; (iii) continuous aspect-normalized Rotary Position Embeddings (RoPE) coupled with shifted square crop training, guaranteeing static tensor shapes while unlocking zero-shot virtual camera control; and (iv) a pixel adaptation phase utilizing a lightweight (10M parameter) Patch Detailer Head operating under velocity regression (vv-prediction). Evaluated on open-domain benchmarks without benchmark-specific fine-tuning, Asuka-FM achieves GenEval 0.588 / DPG 78.96 scores at 256px and GenEval 0.609 / DPG 78.51 scores at 512px, setting a new Pareto-optimal efficiency standard for democratic foundation modeling.

Project Page · Code · Model · Data

ModelParamsHardware BudgetGenEval ↑DPG ↑
Industrial Foundation Models
SD3 Medium [16]2.0BMassive Cluster0.62084.08
Accessible Academic Baselines
MiniT2I-L/16 [43]0.91B8× H100 (27 Days)0.883†84.80
Asuka-FM (Consumer GPU, Open Data)
Asuka-512 (Ours)0.21B1× RTX 4090 (81h)0.60978.51

Table 1c. Benchmark Summary

Introduction

The emergence of Diffusion Transformers (DiTs) [34] trained via flow matching [27] has established a new state of the art in high-fidelity text-to-image (T2I) synthesis. However, state-of-the-art models, such as FLUX.1 (12B) [3], SD3 (8B) [16], and Qwen-Image (20B) [45], have scaled parameter counts and training datasets to regimes that require industrial-scale data centers. This trajectory has widened the accessibility gap, preventing independent researchers and academic groups with limited resources from training generative models from scratch.

When attempting to pretrain generative diffusion models under compute-constrained environments (e.g., a single workstation with an NVIDIA RTX 4090 with 24 GB GDDR6X), the standard pretraining paradigm encounters three fundamental systems bottlenecks:

  • (i) Quadratic Attention Sequence Redundancy: Standard homogeneous DiTs allocate identical full-token sequence lengths across every transformer block. However, recent representational analyses [33] reveal that early layers primarily capture local high-frequency textures, whereas deep middle blocks model global macro-semantics. Processing full-length sequences across deep layers wastes limited floating-point operations (FLOPs) on redundant spatial details, slowing convergence.

  • (ii) Just-In-Time (JIT) Compilation Graph Breaks vs. Aspect-Ratio Bucketing: Real-world images feature diverse aspect ratios. Conventional pipelines handle this via dynamic aspect-ratio bucketing, grouping images of varying shapes into specialized mini-batches. Under modern tracing and JIT compiler runtimes (e.g., torch.compile with TorchInductor), changing spatial dimensions dynamically triggers continuous kernel recompilations, invalidates static CUDA graphs, and severely degrades actual Model FLOPs Utilization (MFUact\mathrm{MFU}_{\mathrm{act}}).

  • (iii) The Autoencoder Throughput Wall: In conventional Latent Diffusion Models (LDMs) [36], an autoencoder (such as the FLUX.2 VAE [4]) compresses images into a regularized latent space. At high resolutions (1024×10241024 \times1024), storing pre-computed 32-channel float16 latents for hundreds of thousands of images requires multiple terabytes of high-speed storage, exceeding consumer NVMe capacities. Consequently, latents must be computed on-the-fly. On consumer silicon during the training stage, the frozen VAE forward pass consumes ≈60%\approx60\% of total wall-time, whereas a compiled 200M DiT forward-backward pass requires only ≈35%\approx35\% (Figure 7 in Appendix § 10). Furthermore, at inference, VAE decoding accounts for over 70% of wall-clock latency under few-step regimes, creating a computational bottleneck for downstream reinforcement learning (e.g., Flow-GRPO [28]) and distillation.

Figure 7. Figure 7: Per-step execution latency breakdown across resolutions on consumer hardware. While In-RAM latent caching eliminates online VAE overhead at 2562256^{2} (devoting > 80% of step latency to the DiT backbone), on-the-fly VAE encoding at 102421024^{2} (1MP) consumes 72.6% of wall-clock time (2,548.1 ms out of 3,507.6 ms). Decoupled latent-to-pixel adaptation permanently bypasses this throughput wall.

To address these challenges, we introduce Asuka-FM, an end-to-end system and architectural co-design framework proving that a competitive ∼\sim206M-parameter Dual-Stream DiT can be pretrained from scratch on a single desktop GPU in ∼\sim160 GPU hours for under $551. As showcased in Figure 2, our open-domain model achieves sharp textural coherence, rich lighting, and diverse multi-concept synthesis (additional domain-specialized anime samples are shown in Figure 19 in Appendix § 17). To achieve this efficiency, we propose a multi-stage training roadmap (Figure 3) coupling structured sequence sparsification, canvas scaling, and latent-to-pixel adaptation. Our primary contributions are:

High-fidelity zero-shot visual synthesis from scratch on a single consumer GPU

Figure 2. High-fidelity zero-shot visual synthesis from scratch on a single consumer GPU. Curated open-domain samples synthesized by Asuka-FM (206M backbone) following two-stage pretraining (2562→5122256^2 \to512^2) on a single NVIDIA RTX 4090 GPU in 149.0 hours ($49.17).

Comprehensive multi-stage training roadmap of Asuka-FM

Figure 3. Comprehensive multi-stage training roadmap of Asuka-FM. Pretraining advances through two latent resolution tiers (2562→5122256^2 \rightarrow512^2) coupling 75% SPRINT sequence sparsification with full-token dense refinement and Warmup-Stable-Decay (WSD). Stage 3 transitions the converged 512px foundation prior to pixel space using a 10M convolutional Patch Detailer Head (PDH) trained under native vv-prediction across synthetic and real datasets, permanently removing the VAE decode barrier during few-step inference.

Nine anime character portraits in a 3×3 grid

Figure 19. High-quality zero-shot samples from the anime-specialized Asuka-FM model. Our 206M dual-stream backbone demonstrates sharp lineart, intricate anatomical rendering, and character-concept binding pretrained from scratch on a single consumer RTX 4090 despite a compact open dataset.

  • System-Architectural Pretraining Recipe: We establish an engineering pipeline integrating in-RAM latent caching for low-resolution stages, network streaming for intermediate scales, and FlashAttention-2 kernel fusion within a fully static torch.compile execution graph. This setup sustains an average 75.6% physical MFUact\mathrm{MFU}_{\mathrm{act}} on a single consumer RTX 4090 GPU.

  • High-Efficiency Dual-Stream Architecture: We couple an AdaLN-free Dual-Stream DiT backbone with three synergistic mechanisms: (i) SPRINT sequence sparsification [33], routing 75%-sparse token sequences through 9 heavy middle blocks to slash attention FLOPs by over 50%;

(ii) Continuous Aspect-Normalized RoPE and shifted square cropping [44, 47], supported by a mathematical derivation proving that uncalibrated continuous RoPE undergoes spatial phase collapse, and a full-spectrum frequency calibration that ensures isotropic decay while conferring zero-shot virtual camera control; and (iii) Decoupled Latent-to-Pixel Adaptation, which attaches a lightweight (10M parameter) convolutional Patch Detailer Head (PDH) [10] under native velocity regression (vv-prediction) to eliminate the online VAE wall.

  • Empirical Ablations and Profiling with Systems Analysis: We discover an Amortization Asymmetry: while pixel adaptation eliminates the online VAE encoding wall during pretraining, its recurrent convolutional evaluation makes standard multi-step sampling (N≥13N \ge13) slower than latent ODE integration, delivering superior efficiency exclusively in few-step regimes (N≤8N \le8, yielding up to 3.34×3.34 \times speedups at N=1N = 1 step).

  • Open-Domain Validation: Trained on a balanced 625K multi-source open corpus without benchmark-specific fine-tuning (avoiding BLIP3o-60K overfitting), Asuka-FM achieves a GenEval score of 0.588 and DPG-Bench score of 78.96 at 256px (68h), converging to 0.609 GenEval and 78.51 DPG-Bench at 512px (81h).

Related Work

Diffusion Transformers and Conditioning Paradigms. The transition from U-Nets to Diffusion Transformers (DiTs) [34] established transformers as the standard generative backbone. Multi-Modal DiTs (MMDiT) in SD3 [16], FLUX.1 [3] utilize separate weights per modality with shared self-attention, while single-stream models (Lumina-T2X [18], Z-Image [40]) concatenate modalities directly. To eliminate parameter overhead from per-block Adaptive Layer Normalization (AdaLN-Zero), DiT-Air [6] shared AdaLN parameters globally. While Peebles & Xie [34] initially explored concatenating class tokens in single-stream models, i1i1 [49] showed that explicit modulation can be entirely omitted when stabilized by Sandwich RMSNorm [15] and SwiGLU activations [38]. Asuka-FM adopts this principle. by replacing AdaLN with in-context temporal conditioning, prepending a continuous sinusoidal timestep token directly to the multimodal sequence.

Sequence Sparsification and Token Dropping. Because self-attention complexity scales as O(N2)\mathcal{O}(N^2) with sequence length NN, token dropping provides significant acceleration. MaskDiT [50] and MDTv2 [19] introduced masked autoencoding objectives, while TREAD [25] bypassed subsets of tokens through inner layers. SPRINT [33] demonstrated that structural specialization—combining a dense shallow encoder (fθf_\theta), a 75%-sparse middle path (gθg_\theta), and a dense decoder (hθh_\theta) fused via linear skips—permits aggressive token dropping without fidelity loss. We extend SPRINT to open-domain multimodal conditioning and evaluate its scaling limits.

Looped Transformers: Parameter-Efficient Depth Scaling. Looped transformers [12, 14, 20] achieve parameter efficiency by repeatedly executing a shared group of middle transformer blocks. In Looped-DiT [12], cycling through shared blocks keeps weights resident in on-chip SRAM and L2 cache, mitigating memory bandwidth bottlenecks. While looped models compress parameter count across recurrent depth, Asuka-FM focuses on sequence-length compression via SPRINT, providing complementary architectural paths toward efficient pretraining.

Micro-Budget Foundation Modeling and Pixel Diffusion. Micro-budget training has gained significant attention. Stretching Each Dollar [37] demonstrated T2I pretraining from scratch for $1,890 on 8×8 \times H100s using mixture-of-experts and deferred masking. Concurrently, HDM [47] trained a 343M model for $535–620 on 4×4 \times RTX 5090 GPUs using shifted square cropping and aspect-normalized RoPE with learnable frequency parameters. In pixel-space diffusion, early works suffered from slow convergence [8, 22]. JiT [26] and Jiang et al. [24] advocated clean data prediction (x0x_0-prediction), but Iris-3B [5] showed that velocity regression (vv-prediction) provides equal stability. DiP [10] introduced the Patch Detailer Head (PDH) to bridge latent efficiency with pixel generation. Asuka-FM synthesizes these insights into an integrated low-cost training pipeline.

Preliminaries

Continuous Normalizing Flows and Flow Matching

We formulate image generation through Continuous Normalizing Flows (CNF) and Conditional Flow Matching (CFM) [27, 29]. Let x0∼p0(x)x_0 \sim p_0(x) denote an empirical sample from data distribution p0(x)p_0(x) on a low-dimensional manifold M⊂Rddata\mathcal{M} \subset\mathbb{R}^{d_{\mathrm{data}}} of ambient dimension ddatad_{\mathrm{data}}, and let x1∼p1(x)=N(0,I)x_1 \sim p_1(x) = \mathcal{N}(\mathbf{0}, \mathbf{I}) denote noise from a standard Gaussian prior with zero mean 0\mathbf{0} and identity covariance I\mathbf{I}. Under the optimal-transport linear probability path across continuous time t∈[0,1]t \in[0,1], intermediate noisy states xtx_t are constructed via:

xt=(1−t)x0+tx1,t∈[0,1].(1)x_t = (1-t)x_0 + tx_1,\quad t \in[0,1]. \tag*{(1)}

Differentiating Eq. (1) yields the conditional velocity field ut(xt∣x0,x1)=x1−x0u_t(x_t \mid x_0, x_1) = x_1 - x_0. A neural vector field regressor vθ(xt,t,c)v_\theta(x_t,t,c), conditioned on prompt context cc, is trained via the simulation-free objective:

LCFM(θ)=Et,x0,x1[∥vθ(xt,t,c)−(x1−x0)∥22].(2)\mathcal{L}_{\mathrm{CFM}}(\theta) = \mathbb{E}_{t,x_0,x_1}\left[\left\|v_\theta(x_t,t,c) - (x_1-x_0)\right\|_2^2\right]. \tag*{(2)}

At inference, novel samples are generated by drawing x1∼N(0,I)x_1 \sim\mathcal{N}(\mathbf{0},\mathbf{I}) and integrating the probability flow ODE: dxt=vθ(xt,t,c) dt\mathrm{d}x_t = v_\theta(x_t,t,c)\,\mathrm{d}t from t=1t=1 to t=0t=0.

Latent Diffusion and Dual-Stream Tokenization

Following the Latent Diffusion Model (LDM) framework [36], an autoencoder compresses raw pixel images of target training resolution H0×W0H_0 \times W_0 (e.g., H0=W0=256H_0 = W_0 = 256) into a latent space Z=RC×H×W\mathcal{Z} = \mathbb{R}^{C \times H \times W} with CC channels and spatial dimensions H×WH \times W via spatial downsampling factor f=H0/H=W0/Wf = H_0/H = W_0/W (e.g., f=8f = 8, C=32C = 32 for the FLUX.2 VAE). Following Diffusion Transformers (DiT) [34], noisy latent tensors zt∈RC×H×Wz_t \in\mathbb{R}^{C \times H \times W} at timestep tt are divided into non-overlapping spatial patches of size p×pp \times p (giving effective pixel patch size P=f⋅p=16P = f \cdot p = 16, producing a flattened sequence of image tokens zimg∈RN×Dz_{\mathrm{img}} \in\mathbb{R}^{N \times D}, where N=(H/p)(W/p)N = (H/p)(W/p) is the image token count and D=768D = 768 is the transformer hidden dimension.

In Dual-Stream DiTs [16], image tokens zimgz_{\mathrm{img}} and text tokens ctext∈RLtext×Dc_{\mathrm{text}} \in\mathbb{R}^{L_{\mathrm{text}} \times D} (of text sequence length LtextL_{\mathrm{text}}) are mapped through dedicated linear projections into image and context queries, keys, and values (QimgQ_{\mathrm{img}}, KimgK_{\mathrm{img}}, VimgV_{\mathrm{img}} and QctxQ_{\mathrm{ctx}}, KctxK_{\mathrm{ctx}}, VctxV_{\mathrm{ctx}}) while sharing joint attention blocks. These projections are concatenated along the token sequence dimension (∥\parallel) to form the joint matrices QQ, KK, VV:

Q=[Qimg∥Qctx],K=[Kimg∥Kctx],V=[Vimg∥Vctx].(3)Q = [Q_{\mathrm{img}} \parallel Q_{\mathrm{ctx}}], \quad K = [K_{\mathrm{img}} \parallel K_{\mathrm{ctx}}], \quad V = [V_{\mathrm{img}} \parallel V_{\mathrm{ctx}}]. \tag*{(3)}

Spatial layout is anchored via 2D axial Rotary Position Embeddings (RoPE) [39], which apply a block-diagonal rotation matrix Rp\mathbf{R}_{\mathbf{p}} parameterized by continuous coordinates p=(y,x)\mathbf{p} = (y, x) to yield rotated image queries and keys Q~img=RpQimg\widetilde{Q}_{\mathrm{img}} = \mathbf{R}_{\mathbf{p}}Q_{\mathrm{img}} and K~img=RpKimg\widetilde{K}_{\mathrm{img}} = \mathbf{R}_{\mathbf{p}}K_{\mathrm{img}}. Here, (y,x)(y, x) denotes the vertical and horizontal center coordinates of each patch in matrix row-major convention.

Timestep Sampling and Time-SNR Shifting

To counteract signal-to-noise ratio (SNR) compression when scaling image resolution, we introduce a resolution-dependent shift factor sshift∈{1.0,2.0}s_{\mathrm{shift}} \in\{1.0, 2.0\} and sample an unconstrained log-odds variable s∼N(μ,σs2)s \sim\mathcal{N}(\mu, \sigma_s^2) from a normal distribution with mean μ=−ln⁡(sshift)\mu= -\ln(s_{\mathrm{shift}}) and standard deviation σs\sigma_s, mapping it to t=sigmoid⁡(s)t = \operatorname{sigmoid}(s) [16]. We then compute the shifted timestep tshifted=(t⋅sshift)/[1+(sshift−1)t]t_{\mathrm{shifted}} = (t \cdot s_{\mathrm{shift}})/[1 + (s_{\mathrm{shift}} - 1)t] for Eq. (1), biasing sampling toward the noise-dominant regime (t→1t \to1).

Asuka-FM Framework

Asuka-FM establishes an end-to-end system and architectural framework designed to train high-capacity Diffusion Transformers (DiTs) from scratch within limited consumer-compute budgets. Our work extends the Micro-Budget Foundation Modeling research to an even more accessible boundary: pretraining a 206M Dual-Stream DiT on a single consumer workstation equipped with an RTX 4090 GPU in 161.2 hours. Furthermore, the random square crops training [44] and the multi-captioning strategy of i1 [49] demonstrate that a capable foundation model can be trained from scratch on fewer than one million images.

Rather than relying on massive parameter scaling or distributed multi-node infrastructure, the framework co-designs the model architecture, memory hierarchy, and training curriculum around four complementary pillars:

  1. Minimalist Dual-Stream Backbone (§ 4.1): Completely eliminates per-block modulation overhead reducing compute and parameters.

  1. Structured Sequence Sparsification via SPRINT (§ 4.2): Reduces attention FLOPs by over 50% while providing strong regularization.

  1. Continuous Aspect-Normalized RoPE & Shifted Cropping (§ 4.3): Prevents rotary phase collapse, enforces static tensor shapes to eliminate JIT compilation breaks, and provides zero-shot virtual camera controls (pan and zoom).

  1. Decoupled Latent-to-Pixel Adaptation (§ 4.5): Transfers converged latent priors directly into raw pixel space, permanently bypassing the autoencoder decoding latency wall during downstream inference and reinforcement learning rollouts.

Minimalist Conditioning and Dual-Stream Backbone

Asuka-FM employs a 206M-parameter Dual-Stream DiT backbone optimized for single-GPU execution (Table 1). Standard AdaLN-Zero requires projecting conditioning embeddings into six independent

Architectural HyperparameterSpecification / Value
Latent Channel Dimension (CC)32 (FLUX.2-dev VAE, f=8f = 8)
Latent Patch Dimension (pp)2×22 \times2 (Effective P=16P = 16)
Hidden Dimension (DD)768
Number of Attention Heads (HH)12
Attention Head Dimension (dhd_h)64
Feedforward Network (FFN)SwiGLU (Dffn=2048D_{\mathrm{ffn}} = 2048)
Total Transformer Blocks (LL)13 Blocks (2 Dense + 9 Sparse + 2 Dense)
Positional EmbeddingCalibrated Aspect RoPE (ω∈[π/4,2π]\omega\in[\pi/4, 2\pi])
Text EncoderLiquidAI/LFM2.5-230M (Frozen, Dctx=640D_{\mathrm{ctx}} = 640)
Text Adapter2-Layer Transformer (D=768D = 768, 6.09M params)
Patch Detailer Head (DiP)4-Stage Conv U-Net (+10.09M params)
Total Pretraining Parameters206.12M (200.03M Backbone + 6.09M Adapter)

caption: Asuka-FM architecture specifications. 206M-parameter Dual-Stream SPRINT DiT backbone.

:::

affine modulation parameters per residual stream across every layer, introducing non-compute parameter overhead and intermediate memory traffic. Following i1 [49], we eliminate per-block AdaLN modulation layers. To retain temporal guidance, the continuous scalar timestep t∈[0,1]t \in[0, 1] is mapped via a sinusoidal MLP into a single temporal token ztime∈R1×Dz_{\mathrm{time}} \in\mathbb{R}^{1 \times D} and directly prepended to the text sequence ctext∈RLtext×Dc_{\mathrm{text}} \in\mathbb{R}^{L_{\mathrm{text}} \times D}, forming the combined context sequence c=[ztime∥ctext]∈R(1+Ltext)×Dc = [z_{\mathrm{time}} \parallel c_{\mathrm{text}}] \in\mathbb{R}^{(1+L_{\mathrm{text}}) \times D}. Time conditioning is broadcast through shared attention at near-zero parameter cost.

Text representations are extracted from a frozen language model (LiquidAI/LFM2.5-230M [1], output dimension Dctx=640D_{\mathrm{ctx}} = 640) and mapped into the DiT hidden dimension D=768D = 768 via a 2-layer bidirectional Transformer Text Adapter (6.09M params). Following attention mixing, image and context representations are updated via independent SwiGLU feedforward networks [38] with intermediate dimension Dffn=2048D_{\mathrm{ffn}} = 2048. Sandwich RMSNorm [15] stabilizes bfloat16 mixed precision. In the final decoder block, projections for the text stream are frozen and pruned from the backward pass, avoiding dead gradient calculations.

Sparse-Dense Residual Fusion (SPRINT)

To eliminate quadratic attention overhead during pretraining, we integrate SPRINT [33], partitioning our L=13L = 13-layer transformer into three functional stages: a dense shallow encoder fθf_\theta (Lenc=2L_{\mathrm{enc}} = 2), a sparse deep middle path gθg_\theta (Lmid=9L_{\mathrm{mid}} = 9), and a dense decoder hθh_\theta (Ldec=2L_{\mathrm{dec}} = 2).

Structured Group-Wise Subsampling. Following SPRINT [33], our 13-layer transformer is partitioned into three functional stages (Figure 4):

Asuka-FM system-architectural co-design pipeline

Figure 4. Asuka-FM system-architectural co-design pipeline. Top Row (Stage 1): Raw images IorigI_{\mathrm{orig}} undergo shifted square cropping to enforce static tensor dimensions (X×XX \times X) and eliminate JIT recompilations, while continuous aspect-normalized RoPE prevents phase collapse. Precomputed FLUX.2 VAE latents ztz_t receive prepended in-context temporal conditioning (ztimez_{\mathrm{time}}) alongside text representations from a 2-layer adapter without per-block AdaLN parameters. Bottom Row (Stage 2): The 13-layer dual-stream DiT executes SPRINT sequence sparsification: shallow blocks (fθf_\theta) supply high-frequency dense skips ftf_t, deep middle blocks (gθg_\theta) process macro-semantics over a 75%-sparse grid, and dense decoder blocks (hθh_\theta) fuse representations to predict flow trajectories. Finally, the lightweight 10.09M Patch Detailer Head (PDH) transfers representations directly to RGB pixels under native vv-prediction.

  1. (i) a dense shallow encoder fθf_\theta (Lenc=2L_{\mathrm{enc}} = 2) processing the batched full token sequence xt∈RB×N×Dx_t \in\mathbb{R}^{B \times N \times D} (for batch size BB and NN image tokens) to preserve local edge representations ft∈RB×N×Df_t \in\mathbb{R}^{B \times N \times D}.

  1. (ii) a sparse deep middle path gθg_\theta (Lmid=9L_{\mathrm{mid}} = 9, 78.4% of backbone weights) processing only the active subsampled tokens gtdrop∈RB×Nsp×Dg_t^{\mathrm{drop}} \in\mathbb{R}^{B \times N_{\mathrm{sp}} \times D} by deterministically retaining k=1k = 1 token per n×n=2×2n \times n = 2 \times2 spatial group (75% drop ratio r=0.75r = 0.75, leaving Nsp=(1−r)N=0.25NN_{\mathrm{sp}} = (1-r)N = 0.25N active tokens) to route macro-semantics at low FLOP cost.

  1. (iii) a dense decoder hθh_\theta (Ldec=2L_{\mathrm{dec}} = 2) that restores sequence length by inserting a learnable mask vector M∈RD\mathbf{M} \in\mathbb{R}^{D} at dropped positions to yield gtpad∈RB×N×Dg_t^{\mathrm{pad}} \in\mathbb{R}^{B \times N \times D}, fuses shallow and deep paths via a linear projection with weight Wfuse∈RD×2DW_{\mathrm{fuse}} \in\mathbb{R}^{D \times2D} and bias bfuse∈RDb_{\mathrm{fuse}} \in\mathbb{R}^{D} into ht=Wfuse[ft∥gtpad]+bfuseh_t = W_{\mathrm{fuse}}[f_t \parallel g_t^{\mathrm{pad}}] + b_{\mathrm{fuse}}, and predicts the velocity field v^t\hat{v}_t.

Analytical Speedup Model. At 512×512512 \times512 (N=1024N = 1024, Nsp=0.25N=256N_{\mathrm{sp}} = 0.25N = 256, D=768D = 768, Dffn=2048D_{\mathrm{ffn}} = 2048), neglecting the short text sequence, the theoretical FLOP complexity CSPRINTC_{\mathrm{SPRINT}} under SPRINT is:

CSPRINT=(Lenc+Ldec)(2ND(2D+N+4Dffn))+Lmid(2NspD(2D+Nsp+4Dffn))+2ND2.(4)\begin{aligned} C_{\mathrm{SPRINT}} ={}& (L_{\mathrm{enc}} + L_{\mathrm{dec}})\left(2ND(2D + N + 4D_{\mathrm{ffn}})\right) \\ &+ L_{\mathrm{mid}}\left(2N_{\mathrm{sp}}D(2D + N_{\mathrm{sp}} + 4D_{\mathrm{ffn}})\right) \\ &+ 2ND^2. \tag*{(4)} \end{aligned}

Compared to a homogeneous dense DiT of complexity CdenseC_{\mathrm{dense}} (evaluating all L=13L = 13 blocks with NN tokens and no fusion overhead), Eq. (4) yields a theoretical FLOP ratio of αFLOP=CSPRINT/Cdense≈0.474\alpha_{\mathrm{FLOP}} = C_{\mathrm{SPRINT}}/C_{\mathrm{dense}} \approx0.474 (2.11×\times arithmetic acceleration, with self-attention FLOPs reduced by 64.9%). Accounting for memory bus traffic, text-stream compute, and kernel launch overheads on physical silicon (RTX 4090), the model predicts an achievable speedup of 1.65×\times–1.95×\times, closely matching our empirical measurement of 1.67×\times (108.5 img/s sparse vs. 64.8 img/s dense at 5122512^2, Table 2).

Stage / PhaseRes. (X2X^2)Drop (rr)SamplesThroughput (img/s)MFU (%) Theo / Act†Compute (EFLOPs)*Wall TimeCost ($)
Stage 1: Base Semantic Pretraining (256 × 256, Latent Patch Grid 16 × 16 = 256 Image Tokens)
Offline VAE Encoding‡2562256^2–0.625M45.2––3.8 h$1.27
1A: SPRINT Sparse2562256^20.7546.88M282.674.4% / 86.8%23.7447.0 h$15.51
1B: Dense Refine2562256^20.0012.50M172.562.9% / 73.4%8.7321.0 h$6.93
Stage 2: High-Resolution Adaptation (512 × 512, Latent Patch Grid 32 × 32 = 1,024 Image Tokens)
Offline VAE Encoding‡5122512^2–0.625M20.6––8.4 h$2.78
2A: High-Res Sparse5122512^20.7515.63M108.558.5% / 68.3%16.2440.0 h$13.20
2B: Dense Refine5122512^20.009.38M64.861.0% / 71.2%16.9441.0 h$13.53
Cumulative DiT Training Total (Stages 1 + 2)84.38M–64.8% / 75.6%65.65149.0 h$49.17
Cumulative Grand Total (Including Offline VAE Pre-Encoding)–––65.65161.2 h$53.20

Table 2. Pretraining progression and compute metrics of Asuka-FM on a single consumer NVIDIA RTX 4090 GPU (24GB). All stages use continuous aspect-normalized RoPE, bfloat16, and torch.compile. Pretraining uses an effective batch size of 256. Unabridged configurations and token processing rates are detailed in Table 8 (Appendix § 10). Evaluated at $0.33/GPU-hour.

Calibrated Continuous Aspect-Normalized RoPE

Standard discrete integer positional encodings fail when generalizing across variable aspect ratios, inducing artifacts. To resolve this, previous work [47] treats vertical and horizontal spatial frequencies as learnable parameters (ωh,ωw∈RNH×(dh/4)\omega_h, \omega_w \in\mathbb{R}^{N_H \times(d_h/4)} for patch-grid height NHN_H and head dimension dhd_h) initialized via log-spaced heuristics. In contrast, Asuka-FM establishes a parameter-free continuous coordinate formulation. We mathematically diagnose why naive continuous RoPE undergoes spatial phase collapse and derive a closed-form, calibrated frequency spectrum that achieves isotropic decay without optimizer overhead.

Continuous Coordinate Mapping. For a latent patch grid of height NH=H/pN_H = H/p and width NW=W/pN_W = W/p, aspect-ratio scale factors rHr_H and rWr_W preserving unit canvas area satisfy rH⋅rW=1.0r_H \cdot r_W = 1.0 and rH/rW=NH/NWr_H/r_W = N_H/N_W, yielding rH=NH/NWr_H = \sqrt{N_H/N_W} and rW=NW/NHr_W = \sqrt{N_W/N_H}. Continuous normalized coordinates (ypos,xpos)(y_{\mathrm{pos}}, x_{\mathrm{pos}}) for a patch centered at (ycenter,xcenter)(y_{\mathrm{center}}, x_{\mathrm{center}}) are bounded within [−rH,rH]×[−rW,rW][-r_H, r_H] \times[-r_W, r_W] via:

ypos=(ycenterH)2rH−rH,xpos=(xcenterW)2rW−rW.(5)y_{\mathrm{pos}} = \left(\frac{y_{\mathrm{center}}}{H}\right)2r_H - r_H,\qquad x_{\mathrm{pos}} = \left(\frac{x_{\mathrm{center}}}{W}\right)2r_W - r_W. \tag*{(5)}

Continuous RoPE Phase Collapse. In 2D axial RoPE, the head dimension dhd_h is partitioned equally across vertical and horizontal axes, yielding ds=dh/4d_s = d_h/4 two-dimensional rotation channels per axis. Let ωk\omega_k denote the spatial frequency for channel k∈{0,…,ds−1}k \in\{0, \ldots, d_s - 1\}. The normalized spatial attention kernel K(pi,pj)K(\mathbf{p}_i, \mathbf{p}_j) between patches at coordinates pi=(yi,xi)\mathbf{p}_i = (y_i, x_i) and pj=(yj,xj)\mathbf{p}_j = (y_j, x_j) evaluates to:

K(pi,pj)=∑k=0ds−1[cos⁡((yi−yj)ωk)+cos⁡((xi−xj)ωk)]2ds.(6)K(\mathbf{p}_i, \mathbf{p}_j) = \frac{\sum_{k=0}^{d_s-1} \left[\cos\left((y_i-y_j)\omega_k\right) + \cos\left((x_i-x_j)\omega_k\right)\right]}{2d_s}. \tag*{(6)}

In standard discrete RoPE [39], adjacent vertical patch separation is fixed at Δy=1.0\Delta y = 1.0. However, when mapped to continuous normalized coordinates y∈[−rH,rH]y \in[-r_H, r_H] on an NH×NWN_H \times N_W patch canvas, the normalized inter-patch separation contracts to Δynorm=2rH/NH≪1.0\Delta y_{\mathrm{norm}} = 2r_H/N_H \ll1.0. Under standard geometric base frequencies (ωk=10000−k/ds≤1.0\omega_k = 10000^{-k/d_s} \le1.0 across 2ds=dh/22d_s = d_h/2 dimensions per axis), the adjacent phase shift Δθk=Δynorm⋅ωk\Delta\theta_k = \Delta y_{\mathrm{norm}} \cdot\omega_k contracts to Δθk≤2rH/NH\Delta\theta_k \le2r_H/N_H. Evaluating the second-order Taylor expansion of the cosine kernel around Δθk≈0\Delta\theta_k \approx0 between adjacent patches pi\mathbf{p}_i and pi+1\mathbf{p}_{i+1} (shown for a diagonal step Δxnorm=Δynorm\Delta x_{\mathrm{norm}} = \Delta y_{\mathrm{norm}}; an axial step yields prefactor 14ds\frac{1}{4d_s}) gives:

K(pi,pi+1)=1−12ds∑k=0ds−1(Δθk)2+O(NH−4)=1.0−O(NH−2).(7)\begin{aligned} K(\mathbf{p}_i, \mathbf{p}_{i+1}) &= 1 - \frac{1}{2d_s}\sum_{k=0}^{d_s-1}(\Delta\theta_k)^2 + O(N_H^{-4}) \\ &= 1.0 - O(N_H^{-2}). \tag*{(7)} \end{aligned}

Here, O(⋅)O(\cdot) denotes Landau’s big-OO notation, characterizing the asymptotic upper bound on the Taylor expansion truncation error as grid resolution NH→∞N_H \to\infty. Specifically, since cos⁡(u)=1−u2/2+O(u4)\cos(u) = 1 - u^2/2 + O(u^4) and adjacent phase shifts scale inversely with resolution (Δθk∝NH−4\Delta\theta_k \propto N_H^{-4}), the remainder terms vanish at least as fast as NH−4N_H^{-4}. Summing over all dsd_s frequency channels proves that the kernel’s deviation from identity decays asymptotically as O(NH−2)O(N_H^{-2}).

Consequently, as canvas resolution NHN_H scales, K(pi,pi+1)→1.0K(\mathbf{p}_i, \mathbf{p}_{i+1}) \to1.0 uniformly across the grid (Figure 5D). Adjacent tokens lose spatial discriminability (Δθk→0\Delta\theta_k \to0), causing phase collapse and anatomical tiling artifacts.

Figure 5. Continuous RoPE coordinate geometry and frequency calibration analysis. Top Row: (A) Adjacent patch phase step Δθk\Delta\theta_k across channels k∈[0,7]k \in[0, 7]: uncalibrated normalized coordinates collapse phase shifts to <7.16∘< 7.16^\circ (K→1.0K \to1.0), while calibrated frequencies (ω∈[π/4,2π]\omega\in[\pi/4, 2\pi]) restore angular separation (22.5∘22.5^\circ–45∘45^\circ). (B) Center-row coordinate trajectories comparing discrete integer indices [0,15][0, 15] against continuous coordinates [−rW,rW][-r_W, r_W]. Bottom Row: Spatial attention kernel heatmaps K(pi,pj)K(\mathbf{p}_i, \mathbf{p}_j) centered at probe patch (8,8)(8, 8) (cyan star). (C) Discrete integer baseline exhibits natural radial decay. (D) Uncalibrated continuous RoPE undergoes phase collapse (K≈1.0K \approx1.0). (E) Calibrated continuous RoPE restores sharp isotropic radial decay across arbitrary canvas dimensions without learned parameters.

Full-Spectrum Frequency Calibration. To guarantee isotropic radial decay without harmonic aliasing (Δx⋅ωmax⁡=2π\Delta x \cdot\omega_{\max} = 2\pi for horizontal coordinate separation Δx\Delta x and peak frequency ωmax⁡=fmax⁡\omega_{\max} = f_{\max}), spatial frequencies must satisfy two boundary constraints between upper bound fmax⁡f_{\max} and lower bound fmin⁡f_{\min}:

  • Local Dispersion Bound (fmax⁡f_{\max}): On the canonical 2562256^2 square patch grid (NH=16N_H = 16, rH=1.0r_H = 1.0), adjacent patches separated by Δpmin⁡=2rH/NH=2.0/16\Delta p_{\min} = 2r_H/N_H = 2.0/16 must achieve angular dispersion Δθ∈[π/4,π/2]\Delta\theta\in[\pi/4, \pi/2] to maintain local boundary discriminability without exceeding the Nyquist limit π\pi, yielding fmax⁡=π/42.0/16=2π≈6.283f_{\max} = \frac{\pi/4}{2.0/16} = 2\pi\approx6.283 (Figure 5A).

  • Global Monotonicity Bound (fmin⁡f_{\min}): Across the maximum canvas separation Δpmax⁡=2rH≈2.0\Delta p_{\max} = 2r_H \approx2.0, the slowest channel must not complete a half-cycle (<π< \pi), preventing periodic harmonic resonance across the canvas and setting fmin⁡=π/22.0=π4≈0.785f_{\min} = \frac{\pi/2}{2.0} = \frac{\pi}{4} \approx0.785.

Frequencies ωk\omega_k are interpolated log-linearly from fmax⁡f_{\max} to fmin⁡f_{\min} across channels k∈{0,…,ds−1}k \in\{0, \ldots, d_s - 1\}:

ωk=exp⁡(ln⁡(fmax⁡)−kds−1ln⁡(fmax⁡fmin⁡)).(8)\omega_k = \exp\left(\ln(f_{\max}) - \frac{k}{d_s - 1}\ln\left(\frac{f_{\max}}{f_{\min}}\right)\right). \tag*{(8)}

This closed-form formulation enforces sharp radial decay across arbitrary aspect ratios without requiring learned parameters or optimizer tracking (Figure 5E vs. discrete integer baseline in Figure 5C). These frequencies are registered as static buffers (register_buffer), avoiding optimizer tracking entirely.

Shifted Square Crop and Camera Dynamics

Variable aspect-ratio training typically requires padding or dynamic batching, triggering continuous JIT graph recompilations under torch.compile. Following Patch Diffusion [44] and HDM [47], we train strictly on fixed square crops while conditioning on continuous coordinates:

  1. An input image IorigI_{\mathrm{orig}} with native resolution Hres×WresH_{\mathrm{res}} \times W_{\mathrm{res}} is isotropically scaled to Hscaled×WscaledH_{\mathrm{scaled}} \times W_{\mathrm{scaled}} such that its shorter side matches training dimension XX (e.g., X=256X = 256 or 512512), i.e., min⁡(Hscaled,Wscaled)=X\min(H_{\mathrm{scaled}}, W_{\mathrm{scaled}}) = X.

  1. Continuous 2D coordinates Pcoord∈RNH,scaled×NW,scaled×2\mathbf{P}_{\mathrm{coord}} \in\mathbb{R}^{N_{H,\mathrm{scaled}} \times N_{W,\mathrm{scaled}} \times2} are calculated across the scaled patch grid (NH,scaled=Hscaled/PN_{H,\mathrm{scaled}} = H_{\mathrm{scaled}}/P, NW,scaled=Wscaled/PN_{W,\mathrm{scaled}} = W_{\mathrm{scaled}}/P, with effective patch stride P=16P = 16) using Eq. (5).

  1. A random X×XX \times X window is extracted, yielding the aligned pair of cropped image and cropped position map (Icrop,Pcrop)(I_{\mathrm{crop}}, \mathbf{P}_{\mathrm{crop}}). Because tensor shapes remain strictly static (X×XX \times X), GPU execution graphs execute without recompilations or CUDA graph invalidations.

Zero-Shot Virtual Camera Inversion. Because the model is conditioned on continuous coordinates rather than discrete token indices, affine transformations parameterized by an optical magnification factor zoom∈R+\mathrm{zoom} \in\mathbb{R}^{+} and normalized pan offsets (xshift,yshift)(x_{\mathrm{shift}}, y_{\mathrm{shift}}) can be applied to the evaluation coordinate mesh (xpos,ypos)(x_{\mathrm{pos}}, y_{\mathrm{pos}}) at inference time to steer the virtual camera viewport (xpos′,ypos′)(x'_{\mathrm{pos}}, y'_{\mathrm{pos}}) without parameter fine-tuning:

ypos′=yposzoom+(yshift⋅2rH),xpos′=xposzoom+(xshift⋅2rW).(9)y'_{\mathrm{pos}} = \frac{y_{\mathrm{pos}}}{\mathrm{zoom}} + (y_{\mathrm{shift}} \cdot2r_H), \qquad x'_{\mathrm{pos}} = \frac{x_{\mathrm{pos}}}{\mathrm{zoom}} + (x_{\mathrm{shift}} \cdot2r_W). \tag*{(9)}

Mathematically, zoom⁡>1.0\operatorname{zoom} > 1.0 contracts the continuous coordinate query bounds from [−rW,rW][-r_W, r_W] to [−rWzoom⁡,rWzoom⁡]\left[-\frac{r_W}{\operatorname{zoom}}, \frac{r_W}{\operatorname{zoom}}\right]. Because the transformer outputs tokens over a fixed NN-patch grid, querying this contracted coordinate neighborhood forces the model to synthesize a smaller spatial sub-region across the full canvas, creating continuous optical focal magnification (zoom in). Conversely, zoom⁡<1.0\operatorname{zoom} < 1.0 expands the coordinate field to incorporate wide-angle scene context. Concurrently, (xshift,yshift)(x_{\mathrm{shift}}, y_{\mathrm{shift}}) applies a linear translation vector (Δx,Δy)=(xshift⋅2rW,yshift⋅2rH)(\Delta x, \Delta y) = (x_{\mathrm{shift}} \cdot2r_W, y_{\mathrm{shift}} \cdot2r_H) to the coordinate center, shifting the camera frustum horizontally and vertically across identical noise seeds (Appendix § 13).

Latent-to-Pixel Decoupling via DiP Head

While latent diffusion provides rapid semantic pretraining, the frozen VAE decoder imposes latency bottlenecks during few-step inference and online reinforcement learning rollouts (e.g., Flow-GRPO [28]). We bridge this trade-off by pretraining in latent space and transferring to raw pixel space via a lightweight convolutional Patch Detailer Head (PDH) based on DiP [10].

PDH Architecture. Patch-level visual tokens si∈RDs_i \in\mathbb{R}^{D} (i∈{1,…,N}i \in\{1, \ldots, N\}) emitted by the DiT decoder hθh_\theta are mapped into raw RGB patches via a 4-stage convolutional U-Net adding only 10.09M parameters (≈5×\approx5\times smaller than standard autoencoder decoders). To prevent high-frequency aliasing from transposed convolutions, Asuka-FM employs nearest-neighbor upsampling coupled with 3×33 \times3 convolutions (Upsample + Conv2d). Reconstructed patches are folded back into the full RGB pixel canvas of shape [B,3,H0,W0][B, 3, H_0, W_0] (for batch size BB and 3 RGB channels).

Direct Velocity Regression vs. Clean Data Prediction. Previous pixel diffusion studies [24, 26] advocated clean data prediction (x0x_0-prediction). However, converting predicted clean states x^0\hat{x}_0 into velocity targets vtv_t from noisy pixel states xtx_t at timestep tt via Eq. (1), vt=(xt−x^0)/tv_t = (x_t - \hat{x}_0)/t, introduces mathematical instability. As t→0t \to0, this conversion suffers from division-by-zero singularities under bfloat16, requiring artificial clamping and dynamic percentile clipping that flattens dynamic range and induces chromatic blowout. In contrast, our ablations (corroborated by Iris-3B [5]) demonstrate that direct velocity regression (vv-prediction) is superior. Retaining native vv-prediction preserves continuous probability paths, stabilizes optimization, and eliminates the need for dynamic activation clipping.

Progressive Training and Systems Engineering

Curated Multi-Source Dataset and Ingestion Pipeline

Open-Domain General Corpus (625K). Following empirical findings from the i1i1 [49], training on balanced mixtures with 100k–400k high-quality samples per source yields competitive generative performance without massive web scraping. We curate an open-domain dataset of 625,000 samples by extracting exactly 125,000 samples across five sources including synthetic and real samples: (i) FLUX-Reason-6M [17] (complex composition); (ii) GPT-Edit-Simpler (paired instructional edits); (iii) Midjourney-v6-Recap (artistic lighting and textures); (iv) Pexels (natural photography and anatomy); and (v) TextAtlas5M [41] (text-rich scenes). All images satisfy min⁡(H,W)≥512\min(H, W) \ge512 px.

VLM Multi-Captioning. We maintain five synthetic captions per image generated via Qwen3-VL-30B-A3B from i1i1 [49] dataset. At each epoch, a caption is sampled dynamically, providing continuous semantic regularization that prevents text memorization. Training context is partitioned into three discrete tiers ([77, 152, 227] tokens), reducing padding overhead by up to 66% and triggering TorchInductor recompilation exactly twice during warmup.

Curated Danbooru Corpus (337K) & Tag-Rarity Weighting. To investigate domain specialization and high-density concept binding, we curate 337,000 illustrations5 from Danbooru2023 [2]. Because community tags often lack structural details, each image is processed through a SwinV2 tagger [30] to upsample descriptive metadata. Prompts are formatted into a canonical prefix structure with dynamically shuffled general tags:

[character count], [character name], [series], [artist], [rating], [aesthetic tags], [year], [shuffled general tags]

To prevent frequent tags (e.g., 1girl, solo) from dominating gradient updates while rare artist styles remain undertrained, we scale the flow matching loss LCFM\mathcal{L}_{\mathrm{CFM}} by an inverse tag-frequency prompt weight wpromptw_{\mathrm{prompt}} to obtain the weighted loss Lweighted\mathcal{L}_{\mathrm{weighted}}. Given the active tag set TT of cardinality ∣T∣|T|, dataset size NtotalN_{\mathrm{total}}, and corpus frequency NkN_k of tag k∈Tk \in T, we define:

wprompt=1.0+1∣T∣∑k∈Tln⁡(NtotalNk+1),Lweighted=wprompt⋅LCFM.(10)w_{\mathrm{prompt}} = 1.0 + \frac{1}{|T|}\sum_{k\in T}\ln\left(\frac{N_{\mathrm{total}}}{N_k + 1}\right), \qquad\mathcal{L}_{\mathrm{weighted}} = w_{\mathrm{prompt}}\cdot\mathcal{L}_{\mathrm{CFM}}. \tag*{(10)}

where TT is the active tag set, NtotalN_{\mathrm{total}} is the dataset size, and NkN_k is the corpus frequency of tag kk.

We use this dataset to train an anime-specialized Asuka-FM model6 in order to validate our Open-Domain dataset results.

Progressive Pretraining Recipe

Pretraining advances through two core resolution tiers (Table 2; see Table 8 in Appendix § 10 for the complete progression matrix including batch sizes, epoch schedules, and average token rates): base semantic anchoring at 256 × 256 and texture synthesis at 512 × 512. All stages employ 8-bit AdamW (β1=0.9\beta_1 = 0.9, β2=0.95\beta_2 = 0.95, weight decay λ=0.01\lambda= 0.01) via bitsandbytes, mixed-precision bfloat16 with TF32 tensor cores, and FlashAttention-2 inside torch.compile. Classifier-Free Guidance dropout is set to puncond=0.10p_{\mathrm{uncond}} = 0.10.

Table 8: Comprehensive pretraining progression and compute metrics of Asuka-FM on a single consumer NVIDIA RTX 4090 GPU (24GB). Unabridged reference for Table 2. Sequence lengths account for visual image tokens plus context (227 text + 1 time = 228 tokens). MFUact_{\mathrm{act}} evaluates actual tensor operations on silicon under selective SwiGLU activation checkpointing (7PD7PD work factor, where PP denotes active model parameters and DD denotes tokens per sample); MFUtheo_{\mathrm{theo}} denotes standard theoretical algorithmic utilization (6PD6PD factor). Hardware costs evaluated at $0.33/GPU-hour.

Stage / PhaseRes. (X2X^2)Batch SizeDrop (rr)EpochsSamplesAvg TokensThroughput (img/s)Token Rate (tok/s)MFU (%) Theo / Act†Compute (EFLOPs)∗Wall TimeCost ($)
Stage 1: Base Semantic Pretraining (256 × 256, Latent Patch Grid 16 × 16 = 256 Image Tokens)
Offline VAE Encoding‡2562256^264––0.625M–45.2–––3.8 h$1.27
1A: SPRINT Sparse2562256^22560.757546.88M351.1 tok282.699.2k74.4% / 86.8%23.7447.0 h$15.51
1B: Dense Refine2562256^22560.002512.50M484.0 tok172.583.5k62.9% / 73.4%8.7321.0 h$6.93
Stage 2: High-Resolution Adaptation (512 × 512, Latent Patch Grid 32 × 32 = 1,024 Image Tokens)
Offline VAE Encoding‡5122512^232––0.625M–20.6–––8.4 h$2.78
2A: High-Res Sparse5122512^22560.752515.63M720.3 tok108.578.1k58.5% / 68.3%16.2440.0 h$13.20
2B: Dense Refine5122512^22560.00159.38M1,252.0 tok64.881.1k61.0% / 71.2%16.9441.0 h$13.53
Cumulative DiT Training Total (Stages 1 + 2)14084.38M–––64.8% / 75.6%65.65149.0 h$49.17
Cumulative Grand Total (Including Offline VAE Pre-Encoding)––––––––65.65161.2 h$53.20

Table 8.

Execution Breakdown. Cumulatively, 149.0 hours of DiT latent pretraining ($49.17 cost) executed 65.65 ExaFLOPs (0.0657 ZFLOPs) at an average sustained utilization of MFUact=75.6%\mathrm{MFU}_{\mathrm{act}} = 75.6\%. Including one-time offline VAE pre-encoding (12.2h, $4.03), the entire latent pretraining pipeline converges in 161.2 total GPU hours ($53.20).

Memory Hierarchy Optimizations

Pretraining experiments were executed on a single workstation: NVIDIA RTX 4090 GPU (24GB GDDR6X, 165.2 TFLOPS peak), AMD Ryzen 9 7950X CPU, and 128GB DDR5 RAM under Ubuntu 24.04 and PyTorch 2.8.

(i) In-RAM Latent Caching (2562256^{2}): Pre-encoded latents for all 625k samples at 2562256^{2} require ≈52\approx52 GB. Caching latents directly in system RAM eliminated disk I/O and autoencoder latency, sustaining 282.6 img/s (86.8% MFUact_{\mathrm{act}}). (ii) Network Latent Streaming (5122512^{2}): At 5122512^{2}, latents expand to ≈203\approx203 GB, exceeding system RAM. Latents were pre-encoded offline (8.4h) and streamed asynchronously via dynamic FIFO queues, sustaining 108.5 img/s (68.3% MFUact_{\mathrm{act}}). (iii) Selective Activation Checkpointing: Checkpointing only the SwiGLU blocks while retaining joint self-attention tensors reduced peak activation memory by ≈30%\approx30\%, enabling micro-batch size 64 at 5122512^{2} within 24 GB VRAM. Recomputing MLP projections adds +1PD+1PD work factor (7PD7PD on silicon vs. 6PD6PD theoretical, where PP and DD here denote active model parameters and tokens per sample).

Experiments and Empirical Analysis

We ablated our architectural components and the SPRINT algorithm on a controlled toy T2I benchmark (details in Appendix § 11); here, we focus on open-domain foundation benchmarks and micro-architectural systems profiling.

Open-Domain Benchmark Evaluation

We evaluate Asuka-FM across two benchmarks: GenEval [21] and DPG-Bench [23] (Table 3; the complete GenEval breakdown is provided in Table 9 of Appendix § 10). At 256×256256 \times256, Asuka-FM achieves GenEval of 0.588 and DPG-Bench of 78.96. When scaling to 512×512512 \times512 under 75% SPRINT token dropping (Phase 2A), performance temporarily drops to 0.480 on GenEval and 68.40 on DPG-Bench due to fine spatial binding degradation (color attribution drops from 41.75% to 16.25%). However, full-sequence dense refinement (Phase 2B) rapidly restores high-frequency fidelity: within 20,000 steps, DPG-Bench surges to 79.49, converging at Epoch 139 to 0.609 GenEval and 78.51 DPG-Bench.

ModelParamsHardware BudgetTraining DataGenEval Total ↑DPG-Bench Total ↑
Industrial Foundation Models (Multi-Billion Parameters, Datacenter Compute)
SDXL Base [35]2.6BHundreds of A100sMillions (Web)0.406*72.40*
SD3 Medium [16]2.0BMassive ClusterMillions (Web)0.620*84.08*
FLUX.1 [Dev] [3]12.0BMassive ClusterWeb-Scale0.660*83.84*
Janus-Pro-7B [9]7.0BLarge ClusterMultimodal Web0.800*84.19*
i1 (Open Foundation) [49]3.0BTPU v5p-128 PodCurated Open0.84086.73
Accessible Academic Baselines (Sub-2B Parameters, Finetuned on BLIP3o-60K†)
PixNerd-XXL [42]1.2BMulti-GPU ClusterPretrain + BLIP3o†0.730*80.90*
PixelGen-XXL [32]1.1B8 × H800 (6 Days)36M + BLIP3o†0.790*79.36
DeCo-XXL [31]1.1BMulti-GPU ClusterPretrain + BLIP3o†0.860*81.40*
Looped-DiT B/16 [12]0.26B8 × H100 (3 Days)CC12M + BLIP3o†0.874*87.00*
MiniT2I-L/16 [43]0.91B8 × H100 (27 Days)CC12M + BLIP3o†0.883*84.80*
Asuka-FM (Ours: Single Consumer RTX 4090, 625K Balanced Dataset, No BLIP3o Finetuning)
Asuka-256 (Dense Final)0.21B1 × RTX 4090 (68h)625K Open Curated0.58878.96
Asuka-512 (75% Sparse)0.21B1 × RTX 4090 (40h)625K Open Curated0.48068.40
Asuka-512 (Dense FT 20k Steps)0.21B1 × RTX 4090 (60h)625K Open Curated0.57579.49
Asuka-512 (Dense Final, Ep 139)0.21B1 × RTX 4090 (81h)625K Open Curated0.60978.51

Table 3. Quantitative comparison on representative text-to-image benchmarks (GenEval and DPG-Bench). Results marked with (* ) denote numbers reported in prior literature [12, 40, 43] [49]. Models marked with † indicate fine-tuning on the BLIP3o-60K dataset, which specifically targets GenEval metrics. Asuka-FM is trained from scratch on open data without benchmark-specific fine-tuning. The full GenEval breakdown is provided in Table 9 in Appendix § 10.

ModelParamsHardware BudgetTraining DataSingleTwo-ObjCountColorsPositionColor-AttrGenEval Overall ↑DPG-Bench Total Score ↑
Industrial Foundation Models (Multi-Billion Parameters, Datacenter Compute)
SDXL Base [35]2.6BHundreds of A100sMillions (Web)––––––0.406*72.40*
SD3 Medium [16]2.0BMassive ClusterMillions (Web)0.980.840.660.740.400.430.620*84.08*
FLUX.1 [Dev] [3]12.0BMassive ClusterWeb-Scale0.990.810.790.740.200.470.660*83.84*
Janus-Pro-7B [9]7.0BLarge ClusterMultimodal Web––––––0.800*84.19*
i1 (Open Foundation) [49]3.0BTPU v5p-128 PodCurated Open––––––0.84086.73
Accessible Academic Baselines (Sub-2B Parameters, Finetuned on BLIP3o-60K†)
PixNerd-XXL [42]1.2BMulti-GPU ClusterPretrain + BLIP3o†0.970.860.440.830.710.530.730*80.90*
PixelGen-XXL [32]1.1B8× H800 (6 Days)36M + BLIP3o†0.990.880.590.900.700.700.790*79.36
DeCo-XXL [31]1.1BMulti-GPU ClusterPretrain + BLIP3o†––––––0.860*81.40*
Looped-DiT B/16 [12]0.26B8× H100 (3 Days)CC12M + BLIP3o†––––––0.874*87.00*
MiniT2I-L/16 [43]0.91B8× H100 (27 Days)CC12M + BLIP3o†––––––0.883*84.80*
Asuka-FM (Ours: Single Consumer RTX 4090, 625K Balanced Dataset, No BLIP3o Finetuning)
Asuka-256 (Dense Final)0.21B1× RTX 4090 (68h)625K Open Curated0.9500.7500.3190.7820.3080.4180.58878.96
Asuka-512 (75% Sparse)0.21B1× RTX 4090 (40h)625K Open Curated0.9630.6390.2530.6730.1900.1630.48068.40
Asuka-512 (Dense FT 20k Steps)0.21B1× RTX 4090 (60h)625K Open Curated0.9530.7880.3310.7150.3050.3580.57579.49
Asuka-512 (Dense Final, Ep 139)0.21B1× RTX 4090 (81h)625K Open Curated0.9500.8110.3470.8170.2780.4500.60978.51

Table 9. Table 9: Comprehensive quantitative comparison on representative text-to-image benchmarks (GenEval and DPG-Bench) with full task breakdown. Unabridged reference for Table 3. Results marked with (*) denote numbers reported in prior literature. Models marked with † indicate fine-tuning on the BLIP3o-60K instruction dataset. Asuka-FM is trained from scratch on open-source data without benchmark-specific fine-tuning.

Evaluation Protocol and Checkpoint selection. To ensure strict reproducibility, benchmarks are evaluated using a 100-step Euler ODE solver. Classifier-Free Guidance (CFG) scale is set to wcfg=8.0w_{\mathrm{cfg}} = 8.0 at 2562256^2 and reduced to wcfg=6.0w_{\mathrm{cfg}} = 6.0 at 5122512^2 to prevent contrast blowout. We explicitly select Epoch 139 over the 20k-step checkpoint despite the slight 0.98-point variance in DPG: extended training improves fine-grained multi-object binding, with color attribution rising from 0.358 to 0.450 and two-object layout rising from 0.788 to 0.811 (Table 9).

Contextualizing Benchmark Fine-Tuning. As documented in prior evaluations [46] [49], several academic baselines (e.g., Looped-DiT, MiniT2I, DeCo) report scores following fine-tuning on the BLIP3o-60K prompt dataset [7], which directly aligns models to GenEval syntactic patterns. In contrast, Asuka-FM is evaluated strictly as an open-domain foundation model trained on general multi-source data without benchmark-specific fine-tuning. Even without specialized prompt tuning, Asuka-FM’s DPG-Bench score (78.51–79.49) is highly competitive with models trained on massive clusters (DeCo at 81.40, SDXL at 72.40), while outperforming SDXL on GenEval (0.609 vs. 0.406).

Compute Budget Comparison

Table 4 compares Asuka-FM against modern foundation models. While sub-2B models require multi-GPU nodes costing $2,000–$5,000, Asuka-FM completes its latent two-stage curriculum in 149.0 hours on a single RTX 4090 GPU ($49.17), establishing an order-of-magnitude cost reduction. On the Pareto frontier (Figure 1, Panel A), Asuka-FM executes only 0.0657 ZFLOPs (56.27 EFLOPs theoretical), sitting at the extreme low-compute anchor while maintaining high fidelity.

Asuka-FM framework overview showing the compute Pareto frontier and Asuka-FM samples

Figure 1. Asuka-FM framework overview. (a) Denoiser training compute vs. compositional alignment (DPG-Bench). (b) High-fidelity visual synthesis from our 206M dual-stream backbone. (c) Representative performance summary across industrial models, academic baselines, and Asuka-FM (†denotes finetuning on BLIP3o-60K).

ModelParamsHardwareWall TimeGPU-hCost ($)
FLUX.1 [Dev]12.0BH100 SXM5 PodWeeks> 100k> $175k
SD3 Medium2.0BH100 SXM5 PodWeeks> 50k> $70k
i1.3.0BTPU v5p-12829.5 Days90.6k≈ $380k
PixelGen-XXL1.1B8 × H8006.0 Days1,152≈ $2,880
MiniT2I-B/320.26B8 × H1003.0 Days576≈ $2,016
Looped-DiT0.26B8 × H1003.0 Days576≈ $2,016
Asuka-2560.21B1 × RTX 409068.0 h68.0$22.44
Asuka-5120.21B1 × RTX 409081.0 h81.0$26.73
Asuka-FM DiT Total0.21B1 × RTX 4090149.0 h149.0$49.17
Asuka-FM Full (Inc. VAE)0.21B1 × RTX 4090161.2 h161.2$53.20

Table 4. Pretraining hardware and financial cost comparison across foundation T2I models. Cloud rates: $0.33/hr (RTX 4090), $1.50/hr (H800), $2.89/hr (H100), $537.60/hr (TPU v5p-128 pod).

Hardware Profiling: RTX 4090 vs. NVIDIA A40

We profile kernel execution across consumer and datacenter silicon: RTX 4090 (24GB GDDR6X, Ada Lovelace) and NVIDIA A40 (48GB GDDR6, Ampere) under identical software stacks (Table 5; see Table 10 in Appendix § 10 for complete resolution and token sequence specifications).

Hardware DevicePeak VRAM AllocatedData / IOOnline VAEText Enc.Backbone (F+B)Opt. & ClipTotal Step LatencyBackbone ThroughputMFUact_{\mathrm{act}} (%) (Isolated / Sys)
Production Latent Pretraining: Kernel-Fused Tensor Saturation (256×256256 \times256, In-RAM Latents)
NVIDIA A4013,254.5 MB1.44 ms–99.98 ms469.88 ms36.33 ms607.65 ms44.43 TFLOPS26.90% / 31.71%
RTX 409015,249.5 MB0.27 ms0.28 ms38.94 ms247.91 ms20.87 ms308.27 ms84.22 TFLOPS50.98% / 63.25%
Online Autoencoder Wall (512×512512 \times512, FLUX.2 VAE On-the-Fly Encoding, Dense Tokens)
NVIDIA A4035,201.0 MB0.33 ms2,263.52 ms111.34 ms1,396.95 ms36.28 ms3,808.42 ms51.39 TFLOPS31.11% / 15.27%

Table 5. Micro-architectural profiling across hardware architectures at 2562256^2 and 5122512^2. Profiled across 30 active iterations via PyTorch FlopCounterMode (165.2 TFLOPS peak reference, micro-batch 64 with 16,384 tokens/step at 2562256^2 and 65,536 tokens/step at 5122512^2). Canvas dimensions and token lengths are reported in Table 10 (Appendix A).

Hardware DeviceRes. CanvasBatch Size (Tokens/Step)Peak VRAM AllocatedData/IOOnline VAEText Enc.Backbone (F+B)Opt. & ClipTotal Step LatencyBackbone ThroughputMFUact_{\mathrm{act}} (%) (Iso. / Sys)
Production Latent Pretraining: Kernel-Fused Tensor Saturation (256×256256 \times256, In-RAM Latents)
NVIDIA A402562256^264 (16,384)13,254.5 MB1.44 ms–99.98 ms469.88 ms36.33 ms607.65 ms44.43 TFLOPS26.90% / 31.71%
RTX 40902562256^264 (16,384)15,249.5 MB0.27 ms0.28 ms38.94 ms247.91 ms20.87 ms308.27 ms84.22 TFLOPS50.98% / 63.25%
Online Autoencoder Wall (512×512512 \times512, FLUX.2 VAE On-the-Fly Encoding, Dense Tokens)
NVIDIA A405122512^264 (65,536)35,201.0 MB0.33 ms2,263.52 ms111.34 ms1,396.95 ms36.28 ms3,808.42 ms51.39 TFLOPS31.11% / 15.27%

Table 10. Comprehensive micro-architectural profiling across hardware architectures at 2562256^2 and 5122512^2. Unabridged reference for Table 5. Profiled across 30 active iterations via PyTorch FlopCounterMode (165.2 TFLOPS peak reference). Reports explicit spatial dimensions, per-step token volumes, and isolated backbone (‘Iso.’) vs. end-to-end system (‘Sys’) MFUact_{\mathrm{act}}.

The VRAM and Peak TFLOPS Analysis. The A40 provides double the physical VRAM (48GB vs. 24GB) and comparable rated compute (~150 TFLOPS). However, at batch size 64, the RTX 4090 completes each compiled step in 308.27 ms (84.22 TFLOPS, MFUact=50.98%\mathrm{MFU}_{\mathrm{act}} = 50.98\%), whereas the A40 requires 607.65 ms (44.43 TFLOPS, MFUact=26.90%\mathrm{MFU}_{\mathrm{act}} = 26.90\%). The consumer GPU executes identical workloads 1.97× faster. Peak memory allocation reaches only 15.25 GB on the RTX 4090; the extra 34.75 GB on the A40 remains completely idle.

Micro-Architectural Drivers. This divergence is driven by two factors: (i) 12× L2 Cache Differential (72 MB vs. 6 MB): Under torch.compile, TorchInductor fuses projections, norms, and SwiGLU gating into compound Triton kernels. On Ada Lovelace, the 72 MB L2 cache keeps intermediate activations resident in on-chip SRAM across kernel invocations. On Ampere (6 MB L2), batched sequence activations (Nbatch=B⋅N=16,384N_{\mathrm{batch}} = B \cdot N = 16{,}384 tokens/step) overflow, thrashing memory buses. (ii) DRAM Bandwidth Starvation (1,008 GB/s vs. 696 GB/s): When cache overflows, the A40 falls back to GDDR6 memory (31% slower). Memory-bound phases suffer heavy penalties: text encoder forward is 2.57× slower on A40 (99.98 ms vs. 38.94 ms), and AdamW steps are 1.74× slower (36.33 ms vs. 20.87 ms).

The Online VAE Wall at 512px and 1MP. Profiling at 512×512512 \times512 on the A40 under online VAE encoding (Table 5) shows that in-place VAE encoding consumes 2,263.52 ms out of 3,808.42 ms per step (59.4% of wall-clock time; see Figure 7 in Appendix § 10 for resolution breakdown), dragging end-to-end system MFU down to 15.27%. This bottleneck intensifies at native 1-megapixel (1024×10241024 \times1024, N=4096N = 4096 tokens): caching uncompressed 32-channel float16 latents for 337k samples requires over 1.2 TB of storage, exceeding consumer NVMe capacities and forcing on-the-fly encoding. Profiling at micro-batch size B=8B = 8 on a single RTX 4090 reveals that the frozen VAE encoder consumes 164.92 ms per step (62.0% of total step latency, 265.44 ms compiled), whereas the DiT forward and backward passes execute in 23.70 ms and 44.50 ms respectively (<70< 70 ms combined). Because the frozen autoencoder runs 2.42× slower than the generative backbone, physical hardware MFU is capped at 14.5%–26.2%—establishing the primary systems motivation for our autoencoder-free latent-to-pixel adaptation strategy (preliminary 1MP SPRINT scaling diagnostics on the Danbooru corpus are deferred to Appendix § 16).

Latent-to-Pixel Adaptation at 512px Resolution

Following our pretraining recipe, we adapt the converged 512px latent model directly to raw RGB pixel space using the 10.09M convolutional Patch Detailer Head (PDH) across three progressive stages (Table 6):

Stage / PhaseRes. (X2X^2)Drop (rr)SamplesThroughput (img/s)MFU (%) Theo / Act†Compute (EFLOPs)*Wall TimeCost
Phase 0: Offline Synth Generation (A40)‡5122512^2–0.10M1.85 img/s––15.0 h$6.00
Stage 1: Decoupled PDH Warmup (Synth)5122512^20.750.50M61.8 img/s62.4% / 72.8%0.522.25 h$0.74
Stage 2A: Joint SPRINT Sparse (Synth)5122512^20.752.50M57.7 img/s58.1% / 67.8%2.6012.0 h$3.98
Stage 2B: Full-Token Dense Refine (Synth)5122512^20.001.50M44.4 img/s59.8% / 69.8%2.719.4 h$3.10
Stage 3: Real-Data SFT Refinement (Real)5122512^20.005.16M42.1 img/s56.7% / 66.2%9.3234.0 h$11.22
Cumulative Pixel DiT Adaptation Total (Stages 1–3)9.66M46.5 img/s57.8% / 67.5%15.1557.65 h$19.04
Cumulative Grand Total (Including Offline Synth Dataset Generation)–––15.1572.65 h$25.04
Progressive Token Compression via Large Patch Adaptation (P=16→32P = 16 \to32)
ps32-adapt16 (Large Patch Engine)5122512^20.009.92M38.3 img/s54.2% / 63.3%17.9272.0 h23.7623.76

Table 6. Latent-to-Pixel adaptation progression and compute metrics of Asuka-FM on a single consumer NVIDIA RTX 4090 GPU (24GB). Adapts the converged 512px latent foundation model to direct RGB pixel space using a 10.09M convolutional Patch Detailer Head (PDH) under native vv-prediction at effective batch size 128. Hardware costs evaluated at 0.330.33/GPU-hour for RTX 4090 and 0.400.40/GPU-hour for A40 (Phase 0). Unabridged telemetry and hyperparameter matrices are detailed in Table 13 in Appendix § 12.

  • Phase 0 (Offline Synthetic Dataset Generation): Using our converged Asuka-512 latent foundation checkpoint, we generate a support corpus of 100,000 synthetic images across diverse prompts. On an NVIDIA A40 GPU, this offline generation phase completed in 15.0 hours at a sustained rate of 1.85 img/s ($6.00 compute cost).

  • Stage 1 (Decoupled PDH Warmup): To suppress initial gradient shocks from randomly initialized convolutional filters, the 206M DiT backbone is frozen. Only the input patch embedder (x_embedder) and the PDH convolutional U-Net are trained on 100k synthetic images for 5 epochs (0.50M samples total) under 75% SPRINT token dropping (r=0.75r = 0.75, peak lr 10−410^{-4} with 25% linear warmup). This phase converged in 2.25 hours (2h 14m 45s) on an RTX 4090, sustaining 61.8 img/s (MFUact=72.8%_{\mathrm{act}} = 72.8\%, $0.74 cost).

  • Stage 2A (Joint Synthetic SPRINT Adaptation): The full network is unfrozen. Training proceeds for 25 epochs (2.50M samples) on synthetic data under r=0.75r = 0.75 token dropping (DiT backbone learning-rate multiplier 1.25×, base lr 6×10−56 \times10^{-5} with 10% warmup), completing in 12.0 hours (12h 02m 42s) at 57.7 img/s ($3.98 cost).

  • Stage 2B (Full-Token Dense Refinement): The network switches to dense tokens (r=0.00r = 0.00, micro-batch size 8, effective batch size 128) for 15 epochs (1.50M samples) with a 10% WSD linear decay schedule. This phase converged in 9.4 hours (9h 23m 31s) at 44.4 img/s ($3.10 cost).

  • Stage 3 (Real-Data SFT Refinement): Synthetic data is discarded. The model is fine-tuned strictly on full tokens (r=0.00r = 0.00) over a curated high-aesthetic real corpus of 396,635 samples (min⁡(H,W)≥1024\min(H,W) \ge1024px) extracted from the i1 dataset. Training for 13 epochs (5.16M samples) completed in 34.0 hours on a single RTX 4090 at 42.1 img/s ($11.22 cost).

Cumulatively, the three DiT pixel adaptation stages executed 15.15 EFLOPs over 57.65 wall-clock hours ($19.04 compute cost) at an average sustained utilization of MFUact=67.5%\mathrm{MFU}_{\mathrm{act}} = 67.5\%. Including offline synthetic dataset generation, the entire latent-to-pixel adaptation pipeline converges in 72.65 GPU hours ($25.04).

Empirical Analysis of Latent-to-Pixel Mechanics

Quantitative Convergence and Benchmark Recovery. As documented in Table 7, the final converged Pixel DiT (P=16P = 16, Stage 3 SFT) achieves 75.31 DPG-Bench and 0.568 GenEval, recovering over 95% of the generative compositional fidelity of the source latent foundation model (78.51 DPG / 0.609 GenEval) while permanently bypassing the autoencoder decoding wall.

Configuration / StagePatch Size (PP)Target ObjectiveTraining ScheduleGenEval Total ↑\uparrowDPG-Bench Total ↑\uparrowSampling Latency / Img (20-step / 30-step)
Baseline Latent Foundation Model (Pretraining Anchor)
Asuka-512 (Base Latent)P=16P = 16vv-pred139 Epochs (81.0h)0.60978.510.250 s / 0.374 s
Latent-to-Pixel Adaptation Progression (DiP Architecture, Zero VAE Decode)
Stage 2A: Joint SPRINT Sparse (r=0.75r = 0.75)P=16P = 16vv-pred25 Epochs (12.0h)0.46569.630.329 s / 0.493 s
Stage 2B: Full-Token Refine (Synth, vv-pred)P=16P = 16vv-pred15 Epochs (9.4h)0.50572.350.329 s / 0.493 s
Stage 2B: Full-Token Refine (Synth, x0x_0-pred)P=16P = 16x0x_0-pred15 Epochs (9.4h)0.47771.490.329 s / 0.493 s
Stage 3 SFT (x0x_0-pred Ablation)P=16P = 16x0x_0-pred13 Epochs (34.0h)0.54576.320.329 s / 0.493 s
Stage 3 SFT (vv-pred, Final Pixel DiT)P=16P = 16vv-pred13 Epochs (34.0h)0.56875.310.329 s / 0.493 s
Progressive Token Compression via Large Patch Adaptation (P=16→32P = 16 \to32)
ps32-adapt16 (Large Patch Pixel Engine)P=32P = 32vv-pred+25 Epochs (72.0h)0.50469.870.187 s / 0.277 s

Table 7. Quantitative benchmark summary of Latent-to-Pixel adaptation on GenEval and DPG-Bench. Evaluated at 5122512^2 canvas resolution using a 100-step Euler ODE solver with CFG scale wcfg=6.0w_{\mathrm{cfg}} = 6.0. Sampling throughput and per-image latency profiled on a single NVIDIA A40 GPU at batch size 32. Full task-by-task GenEval breakdowns are provided in Table 14 in Appendix § 12.

vv-Prediction vs. x0x_0-Prediction. To rigorously evaluate prediction objectives under identical training compute, we benchmarked x0x_0-prediction against vv-prediction across Stages 2B and 3. As detailed in Table 7, while x0x_0-prediction reached a marginally higher overall DPG score (76.32 vs. 75.31), vv-prediction delivered strictly superior performance on GenEval (0.568 vs. 0.545) and fine-grained color attribution (0.420 vs. 0.348; Table 14).

Table 14: Comprehensive benchmark evaluation of latent-to-pixel adaptation configurations on GenEval and DPG-Bench across training stages. All models evaluate at 5122512^2 canvas resolution using a 100-step Euler ODE solver with CFG scale wcfg=6.0w_{\mathrm{cfg}} = 6.0. Sampling throughput and per-image latency profiled on a single NVIDIA A40 GPU at batch size 32. Intermediate stages demonstrate that vv-prediction achieves faster initial convergence and prevents color desaturation.

Configuration / StagePatch Size (PP)Target Obj.Steps / EpochsSingleTwo-ObjCountColorsPositionColor-AttrGenEval Overall ↑\uparrowDPG-Bench Total ↑\uparrowLatency / Img (20-step / 30-step)
Baseline Latent Foundation Model (Pretraining Anchor)
Asuka-512 (Base Latent)P=16P = 16v-pred139 Ep0.9500.8110.3470.8170.2780.4500.60978.510.250 s / 0.374 s
Stage 2: Synthetic Data SPRINT Adaptation Progression (100k Synthetic Images)
Stage 2A: SPRINT Sparse (r=0.75r = 0.75)P=16P = 16v-pred25 Ep (12.0h)0.9280.5430.2590.6620.1830.2150.46569.630.329 s / 0.493 s
Stage 2B: Dense Refine (r=0.00r = 0.00)P=16P = 16x0x_0-pred15 Ep (9.4h)0.9280.5860.2910.6280.1950.2330.47771.490.329 s / 0.493 s
Stage 2B: Dense Refine (r=0.00r = 0.00)P=16P = 16x0x_0-pred15 Ep (9.4h)0.9410.6140.2840.7260.1930.2730.50572.350.329 s / 0.493 s
Stage 3: Real-Data SFT Convergence (396k Curated Images, min⁡(H,W)≥1024px\min(H,W) \ge1024\mathrm{px})
Stage 3 SFT (x0x_0-pred Ablation)P=16P = 16x0x_0-pred70k (13 Ep)0.9500.6990.2970.7070.2700.3480.54576.320.329 s / 0.493 s
Stage 3 SFT (v-pred, Final Ours)P=16P = 16v-pred70k (13 Ep)0.9500.7800.3350.7500.2850.4200.56875.310.329 s / 0.493 s
Progressive Token Compression via Large Patch Adaptation (P=16→32P = 16 \to32)
ps32-adapt16 (Large Patch)P=32P = 32v-pred+75k (25 Ep)0.9380.5760.3250.7390.1950.2500.50469.870.187 s / 0.277 s

Table 14.

Critically, converting predicted clean states x^0\hat{x}_0 into velocity targets vt=(xt−x^0)/tv_t = (x_t - \hat{x}_0)/t introduces a fatal singularity as t→0t \to0 in bfloat16, requiring dynamic percentile clipping that flattens dynamic range and induces chromatic blowout. In contrast, native vv-prediction maintained smooth optimization trajectories with zero activation clipping, completely avoiding color desaturation.

Progressive Token Compression (P=16→32P = 16 \to32). To accelerate multi-step pixel inference, we evaluated progressive patch widening to P=32P = 32 (4×\times token compression, sequence length N=256N = 256 at 5122512^2; 25 epochs, 72.0h). While compositional binding declines under the larger patch stride (DPG-Bench drops to 69.87, GenEval to 0.504), sampling latency at 5122512^2 on an NVIDIA A40 drops to 0.187 s (20 steps) and 0.277 s (30 steps, a 1.78×\times speedup over P=16P = 16), establishing an ultra-fast sampling engine for compute-critical applications.

Inference Profiling: The Amortization Asymmetry

A common assumption in pixel diffusion literature [10] is that eliminating the VAE decoder accelerates inference. Profiling across N∈{20,30}N \in\{20, 30\} sampling steps on an NVIDIA A40 (Table 11) reveals an Amortization Asymmetry: for standard multi-step sampling (N∈{20,30}N \in\{20, 30\}), pixel diffusion is actually 9%–31% slower than latent diffusion. At 102421024^2 (batch size B=16B = 16), generating 30-step trajectories requires 46.43 s in latent mode versus 53.28 s in pixel mode (14.8% slower).

ResolutionModeStepsTotal TimeLatency / ImageStep Cost (tstept_{\mathrm{step}})Fixed Overhead (tfixedt_{\mathrm{fixed}})Peak VRAMRate
1024×10241024 \times1024Latent + VAE3046.43 s2.902 s / img1.387 s / step4.82 s (VAE Decode)6.75 GB0.34 img/s
1024×10241024 \times1024Latent + VAE2032.56 s2.035 s / img1.387 s / step4.82 s (VAE Decode)6.75 GB0.49 img/s
1024×10241024 \times1024Pixel (DiP)3053.28 s3.330 s / img1.773 s / step0.09 s (Zero VAE)13.22 GB0.30 img/s
1024×10241024 \times1024Pixel (DiP)2035.55 s2.222 s / img1.773 s / step0.09 s (Zero VAE)13.22 GB0.45 img/s
512×512512 \times512Latent + VAE305.99 s0.374 s / img0.181 s / step0.56 s (VAE Decode)2.78 GB2.67 img/s
512×512512 \times512Pixel (DiP)307.89 s0.493 s / img0.260 s / step0.09 s (Zero VAE)4.40 GB2.03 img/s

Table 11. Sampling execution profile across latent and pixel-space adaptation on NVIDIA A40 (B=16B = 16 samples, TorchInductor compilation, FlashAttention-2, bfloat16). Latent mode evaluates 206M DiT followed by a terminal VAE decode. Pixel mode evaluates 216M DiT + PDH without an autoencoder.

Deconstructing the Asymmetry. Total sampling latency over NN ODE steps decomposes into per-step evaluation latency tstept_{\mathrm{step}} and fixed terminal decoding overhead tfixedt_{\mathrm{fixed}}: Total Time⁡(N)=N⋅tstep+tfixed\operatorname{Total\ Time}(N) = N \cdot t_{\mathrm{step}} + t_{\mathrm{fixed}}. In latent sampling, terminal VAE decoding incurs a one-time fixed penalty (tfixed,latent=4.82t_{\mathrm{fixed, latent}} = 4.82 s at 102421024^2), while per-step latency remains low (tstep,latent=1.387t_{\mathrm{step, latent}} = 1.387 s per batch step). In pixel sampling, while the VAE overhead is eliminated (tfixed,pixel=0.09t_{\mathrm{fixed, pixel}} = 0.09 s ≈0\approx0), the convolutional PDH head must evaluate recurrently at every step (and twice per step under CFG), increasing per-step latency to tstep,pixel=1.773t_{\mathrm{step, pixel}} = 1.773 s (+27.8%). Across 30 steps, this recurrent overhead accumulates to 11.58 s, completely eclipsing the 4.82 s VAE decode savings.

Crossover Dynamics and Operational Regimes. Equating both linear trajectories yields an empirical break-even crossover threshold of N∗≈12.25N^* \approx12.25 sampling steps (formal derivation in Appendix § 10.§ 4), cleanly partitioning operational utility (Figure 6). For multi-step regimes (N≥13N \ge13 steps), latent sampling is strictly faster because the O(1)O(1) VAE decode is amortized across many steps. Conversely, in few-step regimes (N≤8N \le8 steps), pixel diffusion is decisively superior: under 4-step distilled sampling (DMD2 [48]), pixel latency is 7.18 s (0.449 s/img) versus 10.37 s (0.648 s/img) for latent mode (a 1.44×\times speedup). At N=1N = 1 step, pixel mode achieves a 3.34×\times acceleration, establishing pixel models as natural engines for distilled few-step rollout.

Figure 6. Sampling latency trade-off at 1MP: Latent DiT vs. DiP Pixel Engine on NVIDIA A40. Pixel sampling is faster in few-step regimes (N≤8N \le8, 3.34× speedup at 1 step), while latent sampling is faster for multi-step ancestral sampling (N≥13N \ge13).

Manifold Support: Square Synthetics vs. Multi-Aspect Corpora

An empirical discovery is the divergence in camera controllability across training distributions. Models trained on synthetic open-domain datasets (e.g., FLUX-Reason-6M, Midjourney-v6) exhibit geometric warping and background collapse on some prompts during coordinate manipulation. Synthetic datasets are mainly generated as centered 1024×10241024 \times1024 squares. Extracting shifted square crops from square-native images forcibly clips through canonical focal points, teaching the model that off-center coordinate queries correlate with truncated subjects.

In contrast, our curated Danbooru corpus naturally spans non-square aspect ratios (1 : 2 portrait pin-ups to 21 : 9 panoramic spreads). Performing shifted cropping on raw rectangular illustrations forces the continuous RoPE manifold to encounter the entire coordinate spectrum [−rH,rH]×[−rW,rW][-r_H, r_H] \times[-r_W, r_W] over diverse, naturally framed focal points and perspective depths. Consequently, the anime-specialized model establishes genuine spatial invariance, unlocking robust zero-shot virtual camera manipulation without auxiliary conditioning tokens.

Discussion, Limitations, and Future Work

Limitations. While Asuka-FM achieves competitive synthesis on a single consumer GPU, several limitations remain: (i) Pretraining was conducted on 625,000 samples; multi-billion parameter models trained on hundreds of millions of images remain superior on complex crowd scenes and long-tail concepts. (ii) Although our corpus includes TextAtlas5M [41], the 206M backbone does not consistently render multi-word text strings. (iii) In pixel space, multi-step sampling (N≥13N \ge13 steps) without patch enlargement (P=16P = 16) exhibits higher latency than latent diffusion due to recurrent PDH evaluation.

Future Work. Immediate research will expand along two directions: (i) Native 1MP Scaling Curriculum: Building on our preliminary 1MP profiling (Appendix § 16). (ii) Zero-Decode Downstream Post-Training (Flow-GRPO): In Flow-GRPO reinforcement learning [28], generating G=12G = 12 rollouts per prompt requires 12 independent VAE decodes in latent mode (>60%> 60\% of GPU memory). By emitting direct RGB tensors, our pixel model eliminates this decoding bottleneck, enabling scalable online RL on consumer workstations.

Conclusion

Asuka-FM demonstrates that high-resolution text-to-image foundation models do not fundamentally require datacenter-scale compute clusters. By synthesizing an AdaLN-free dual-stream backbone with prepended temporal conditioning, structured sequence sparsification via SPRINT, continuous aspect-normalized RoPE with shifted square crop training, and decoupled latent-to-pixel adaptation via a lightweight Patch Detailer Head, Asuka-FM successfully trains a 206M latent generative foundation model from scratch on a single consumer NVIDIA RTX 4090 GPU in 161.2 wall-clock hours, executing 65.65 ExaFLOPs for only $53.2 in compute.

Operating on the Pareto frontier of denoiser compute (0.0657 ZFLOPs), Asuka-FM achieves a GenEval score of 0.609 and DPG-Bench score of 78.51 at 512px without benchmark-specific fine-tuning. Micro-architectural profiling reveals an Amortization Asymmetry: while pixel adaptation eliminates the online VAE wall during pretraining and delivers up to 3.34×3.34 \times speedups in few-step regimes (N≤4N \le4 sampling steps), latent sampling remains faster for multi-step trajectories (N≥13N \ge13 steps). Asuka-FM provides an empirical and systems foundation for accessible, low-budget generative foundation modeling across academic and independent research laboratories.

Acknowledgments

We gratefully acknowledge the Department of Signal Theory and Communications at Universidad Carlos III de Madrid (UC3M) for providing the dedicated GPU workstations and computing infrastructure that made these extensive pretraining experiments possible.

Víctor Rodrigo González and Alejandro Lancho have received funding from the Ministerio de Ciencia, Innovación y Universidades, Spain, under Grant Agreement No. PID2023-148856OA-I00 (funded by MICIU/AEI/10.13039/501100011033 and by ERDF/UE). Alejandro Lancho and Pablo M. Olmos have received funding from the Comunidad de Madrid through the IDEA-CM project (Grant Agreement No. TEC-2024/COM-89).

Additionally, Alejandro Lancho has received funding from the Comunidad de Madrid through the CARE-CM project (Grant Agreement No. SYG-2024/COM-870). Ignacio Peis acknowledges support from the Danish Data Science Academy, which is funded by the Novo Nordisk Foundation (NNF21SA0069429), and from the “la Caixa” Foundation (ID 100010434) through the Junior Leader Incoming fellowship LCF/BQ/PI26/12120010. Pablo M. Olmos has additionally been supported by the Ministerio de Ciencia, Innovación y Universidades, Spain, under Grant PID2024-157856NB-I00 (CARTESIAN), funded by MCIN/AEI/10.13039/501100011033 and, as appropriate, by “ERDF A way of making Europe”; by the ELLIS Unit Madrid (European Laboratory for Learning and Intelligent Systems); and by the Comunidad de Madrid under grant IND2024/TIC-34728.

Our work stands on the shoulders of the open machine learning research community. In particular, we express our sincere appreciation to the authors of:

  • HDM ([47]), for pioneering consumer-scale anime foundation pretraining and demonstrating the viability of shifted square cropping and continuous RoPE.

  • SPRINT ([33]), for the sparse-dense residual fusion paradigm that fundamentally slashed our attention sequence complexity.

  • i1 ([49]), for establishing an exemplary open recipe for multi-modal diffusion transformers and demonstrating that explicit AdaLN modulation can be omitted, and for publicly releasing the synthetic captions used for training.

  • DiP ([10]) and the empirical pixel diffusion studies by [11, 24], for illuminating the dynamics of the Patch Detailer Head and pixel-space velocity trajectories.

We note that while we did not have direct communication or collaboration with the authors of these prior works, our system-architectural framework was made possible through their transparent public papers, released checkpoints, and open codebases.

We also extend our gratitude to the broader anime and open-source community, particularly the crowdsourced annotators and maintainers of the Danbooru platform [2] whose decades of detailed image tagging provided the rich semantic metadata necessary for concept learning, as well as the maintainers of bitsandbytes, FlashAttention [13], and PyTorch.

Finally, the project Asuka-FM is dedicated to Asuka Soryuu Langley from Neon Genesis Evangelion, whose unyielding determination inspired our resolve through countless errors and pretraining milestones.

†MFUact_{\mathrm{act}} evaluates actual tensor ops (7PD7PD per sample, for active parameter count PP and tokens per sample DD) under SwiGLU recomputation; MFUtheo_{\mathrm{theo}} denotes standard theoretical utilization (6PD6PD).

∗1 EFLOP = 101810^{18} FLOPs = 10310^3 PFLOPs = 10−310^{-3} ZFLOPs. Cumulative theoretical algorithmic compute is 56.27 EFLOPs.

‡One-time offline encoding using FLUX.2 VAE (C=32C = 32, f=8f = 8) permanently removing autoencoder overhead from generative pretraining.

Extended Benchmark Breakdowns and Systems Telemetry

This section provides the complete, unabridged numerical tables corresponding to the condensed performance summaries in the main text.

Full Pretraining Lifecycle and Token Telemetry

Table 8 provides the unabridged pretraining lifecycle metrics of Asuka-FM across both square training resolutions X×XX \times X (2562256^2 and 5122512^2) and SPRINT token drop ratios rr, detailing per-stage batch sizes, epoch allocations, cumulative token sequences, and physical hardware throughputs. As discussed in § 5, the token processing rate demonstrates the arithmetic efficiency of SPRINT: by processing only 25% of visual tokens in the middle blocks (r=0.75r = 0.75), the system achieves 99.2k tok/s in Phase 1A and 78.1k tok/s in Phase 2A, substantially mitigating memory bandwidth saturation on consumer-grade GDDR6X buses.

Fine-Grained GenEval Task Breakdown

Table 9 presents the complete compositional task breakdown on GenEval [21] alongside DPG-Bench [23]. The individual columns evaluate basic object synthesis (Single Object), relational spatial semantics (Two Objects, Position), numerical consistency (Counting), and chromatic attribute binding (Colors, Color Attribution).

Crucially, the breakdown contextualizes the dynamics of SPRINT during pretraining. In Phase 2A (75% token drop), the model maintains strong single-object recognition (0.963) and two-object layout (0.639), but suffers a temporary drop in color-attribute binding (0.163). Once terminal dense refinement is initiated in Stage 2B, full spatial cross-attention across all 1,024 tokens recovers color attribution to 0.450, directly proving that macro-semantics can be learned sparsely while fine attribute binding requires dense token interaction.

Full Hardware Profiling and Micro-Architectural Telemetry

Table 10 details the step latency, VRAM allocation, and tensor core utilization—reported both for the isolated DiT backbone (‘Iso.’) and end-to-end training step (‘Sys’)—across the RTX 4090 and NVIDIA A40 GPUs, retaining the explicit token sequence dimensions per step.

This full view illustrates the hardware architectural difference: at 2562256^2 (16,384 tokens/step), the RTX 4090’s large 72MB L2 cache keeps the entire fused Inductor kernel resident on-chip, sustaining

84.22 TFLOPS and cutting total step latency to 308.27 ms. In contrast, the A40 overflows its 6MB L2 cache, incurring frequent spills to GDDR6 DRAM (696 GB/s) and doubling total step time to 607.65 ms. Figure 7 further contrasts In-RAM latent caching at 2562256^{2} against online VAE encoding at 5122512^{2} and 102421024^{2}, highlighting why high-resolution online autoencoding stalls the generative backbone.

Analytical Derivation of the Amortization Asymmetry

To formally model the sampling latency trade-off between latent diffusion (with a terminal autoencoder) and pixel diffusion (with a per-step convolutional detailer head), let NN denote the number of ODE sampling steps, tstep,latentt_{\mathrm{step, latent}} the per-step latent DiT evaluation latency, tfixed,VAEt_{\mathrm{fixed, VAE}} the one-time terminal VAE decoding latency, Δthead\Delta t_{\mathrm{head}} the marginal per-step latency added by evaluating the Patch Detailer Head (PDH), and tfixed,pixel≈0t_{\mathrm{fixed,pixel}} \approx0 the negligible fixed pixel tensor formatting overhead. Total generation latencies Tlatent(N)T_{\mathrm{latent}}(N) and Tpixel(N)T_{\mathrm{pixel}}(N) for a batch of BB images over NN sampling steps are then expressed as:

Tlatent(N)=N⋅tstep,latent+tfixed,VAE,(11)T_{\mathrm{latent}}(N) = N \cdot t_{\mathrm{step,latent}} + t_{\mathrm{fixed,VAE}}, \tag*{(11)}
Tpixel(N)=N⋅(tstep,latent+Δthead)+tfixed,pixel.(12)T_{\mathrm{pixel}}(N) = N \cdot(t_{\mathrm{step,latent}} + \Delta t_{\mathrm{head}}) + t_{\mathrm{fixed,pixel}}. \tag*{(12)}

Defining the per-step pixel evaluation latency as tstep,pixel=tstep,latent+Δtheadt_{\mathrm{step,pixel}} = t_{\mathrm{step,latent}} + \Delta t_{\mathrm{head}}, setting Tlatent(N∗)=Tpixel(N∗)T_{\mathrm{latent}}(N^*) = T_{\mathrm{pixel}}(N^*) yields the analytical break-even step threshold N∗N^*:

N∗=tfixed,VAE−tfixed,pixelΔthead=tfixed,VAE−tfixed,pixeltstep,pixel−tstep,latent.(13)N^* = \frac{t_{\mathrm{fixed,VAE}} - t_{\mathrm{fixed,pixel}}}{\Delta t_{\mathrm{head}}} = \frac{t_{\mathrm{fixed,VAE}} - t_{\mathrm{fixed,pixel}}}{t_{\mathrm{step,pixel}} - t_{\mathrm{step,latent}}}. \tag*{(13)}

Substituting the empirical latency parameters benchmarked on an NVIDIA A40 at 1024×10241024 \times1024 resolution with batch size B=16B = 16 (Table 11):

tfixed,VAE=4.82 s,tfixed,pixel=0.09 s,tstep,latent=1.387 s,tstep,pixel=1.773 s⟹Δthead=0.386 s.\begin{aligned} t_{\mathrm{fixed,VAE}} &= 4.82\ \mathrm{s}, \qquad t_{\mathrm{fixed,pixel}} = 0.09\ \mathrm{s},\\ t_{\mathrm{step,latent}} &= 1.387\ \mathrm{s}, \qquad t_{\mathrm{step,pixel}} = 1.773\ \mathrm{s} \quad\Longrightarrow\quad\Delta t_{\mathrm{head}} = 0.386\ \mathrm{s}. \end{aligned}

Evaluating Eq. (13) yields:

N∗=4.82−0.091.773−1.387=4.730.386≈12.25 steps.(14)N^* = \frac{4.82 - 0.09}{1.773 - 1.387} = \frac{4.73}{0.386} \approx12.25\ \text{steps}. \tag*{(14)}

When N>N∗N > N^*, the accumulated per-step penalty N⋅ΔtheadN \cdot\Delta t_{\mathrm{head}} exceeds the one-time VAE decoding cost tfixed,VAEt_{\mathrm{fixed,VAE}}, proving that pixel-space diffusion with recurrent convolutional heads is mathematically sub-optimal for multi-step trajectories without architectural modifications such as patch enlargement (effective pixel patch size P≥32P \ge32).

Toy-Diffusion Architectural Ablation Study

Prior to full-scale foundation pretraining on the 625K dataset, we established the controlled toy-diffusion benchmark on 87,000 image crops (256 ×\times 256) from the Anime Faces dataset7 to isolate structural components (Table 12).8

Backbone / Architecture VariantParamsBSEpochsFID ↓\downarrow
Part I: Latent Architecture & Conditioning Ablations
Single-Stream DiT (D=768D = 768, L=14L = 14)218M2566003.89
Efficient U-Net Baseline185M2564003.45
Dual-Stream DiT + MLP Text Adapter200M2566003.42
Dual-Stream DiT + LLM (Frozen LFM2.5)206M51210003.12
Dual-Stream Baseline (Constant LR)206M5126003.06
Dual-Stream Baseline (Large Batch)206M102414002.42
Dual-Stream Wide (D=1024D = 1024, L=10L = 10)352M5127502.85
SPRINT Dual-Stream (r=0.75r = 0.75, Sparse Only)206M2567005.56
SPRINT Dual-Stream (r=0.75→r = 0.75 \to Dense FT)206M25610003.96
Part II: Latent-to-Pixel DiP Adaptation (Toy Scale)
Stage 1: PDH Only (Frozen DiT Backbone)+10M2561528.40
Stage 2: Joint FT (P=16P = 16, Hard Clamp)216M2565019.85
Stage 2: Joint FT (P=16P = 16, Dynamic Clip)216M2562018.20
Stage 2: Joint FT (P=16P = 16, Dynamic Clip)216M2565016.00
Stage 2: Joint FT (P=16P = 16, Dynamic Clip)216M2568514.39
Stage 2: Staged Patch Widening (P=16→32P = 16 \to32)216M2565033.80

Table 12. Architectural and conditioning ablations on the toy-diffusion framework (evaluated on 87,000 facial portraits at 256×256256 \times256, patch size P=16P = 16).

Dual-Stream vs. Single-Stream Efficiency. Dual-stream cross-attention decisively outperformed single-stream concatenation. Concatenating disparate text and visual embeddings into shared attention matrices induced modality interference, capping convergence at FID = 3.89. Decoupling modalities into dedicated projections achieved higher sustained hardware MFU (70.4% vs. 49.8%) and improved distributional fidelity to FID = 3.06. Furthermore, eliminating per-block AdaLN modulation in favor of prepending continuous sinusoidal time tokens directly to the context stream reduced backbone parameters by 28% without degrading generative quality.

Bidirectional Text Adapters. Replacing a static linear/MLP projection with a 2-layer bidirectional Transformer Text Adapter (6.09M parameters) reduced FID from 3.42 to 3.12. The adapter aligns unidirectional causal representations emitted by the frozen LFM2.5-230M [1] LLM into an expressive spatial conditioning subspace.

SPRINT Token-Dropping Dynamics. Under structured 2×22 \times2 patch dropping (r=0.75r = 0.75), SPRINT reduced training iteration latency by 52%. While sparse pretraining alone plateaued at FID = 5.56, executing a terminal dense refinement phase (r=0.00r = 0.00) dropped the metric to 3.96, matching dense baselines within 0.5 FID while saving over half the cumulative pretraining FLOPs (Figure 8).

FID versus Epoch line plot showing dual-stream and SPRINT convergence

Figure 8. Quantitative convergence on the toy-diffusion benchmark. Evaluation of SPRINT token drop schedules (r=0.75→0.00r = 0.75 \to0.00) on FID.

Parameter Cooldown via WSD Scheduling. Diffusion transformers are predominantly trained with static learning rates. We evaluated Warmup-Stable-Decay (WSD), transitioning from a flat learning rate to linear decay in the terminal 20% of epochs. For the D=1024D = 1024 backbone, parameter cooldown dropped FID from 3.27 to 2.85; on the canonical D=768D = 768 model, FID dropped from 3.27 to 3.06, proving that scheduled learning rate decay is essential for optimal flow trajectory convergence.

Comprehensive Latent-to-Pixel Adaptation Telemetry and Breakdowns

This section provides the complete, unabridged numerical tables corresponding to the condensed Latent-to-Pixel adaptation summaries in § 6.4.

Complete Pixel Adaptation Lifecycle and Hyperparameter Matrix

Table 13 details the complete optimization progression across all pixel adaptation stages, specifying the trained modules, effective pixel patch size PP, SPRINT token drop ratio rr, per-GPU micro-batch size (‘Micro-BS’), peak learning rate η\eta, warmup/decay schedule, epoch and sample counts, and physical silicon throughputs.

Table 13: Complete pretraining and adaptation progression for Latent-to-Pixel transfer at 512×512512 \times512 on NVIDIA RTX 4090 (24GB). Hardware evaluated at $0.33/GPU-hour ($0.40/h for A40 in Phase 0). Effective batch size is maintained at 128 across all stages using gradient accumulation.

Stage / PhaseModules TrainedPatch Size (PP)Drop (rr)Micro-BS (Per GPU)Peak LR (η\eta)Warmup / ScheduleEpochsSamplesThroughput (img/s)MFU (%) Theo / ActWall TimeCost
Phase 0: Synth Gen (A40)– (Sampling)––32–––0.10M1.85–15.0 h$6.00
Stage 1: PDH WarmupEmbed + PDH160.75161.0×10−41.0 \times10^{-4}25% Const50.50M61.862.4% / 72.8%2.25 h$0.74
Stage 2A: Joint SPRINTFull Model160.75166.0×10−56.0 \times10^{-5}10% Const252.50M57.758.1% / 67.8%12.0 h$3.98
Stage 2B: Dense RefineFull Model160.0086.0×10−56.0 \times10^{-5}10% WSD151.50M44.459.8% / 69.8%9.4 h$3.10
Stage 3: Real Data SFTFull Model160.0086.0×10−56.0 \times10^{-5}10% WSD135.16M42.156.7% / 66.2%34.0 h$11.22
Cumulative Pixel DiT Adaptation Total (Stages 1–3)589.66M46.557.8% / 67.5%57.65 h$19.04
Cumulative Grand Total (Including Offline Synth Generation)––––72.65 h$25.04
Progressive Token Compression via Large Patch Adaptation (P=16→32P = 16 \to32)
ps32-adapt16 (Large Patch)Full Model320.00164.0×10−54.0 \times10^{-5}10% WSD259.92M38.354.2% / 63.3%72.0 h$23.76

Table 13.

Fine-Grained GenEval and DPG-Bench Task Breakdowns

Table 14 isolates the specific compositional sub-metrics across training checkpoints, comparing native velocity regression (vv-prediction) against clean-data prediction (x0x_0-prediction) under identical compute durations across Stage 2 (synthetic anchor adaptation) and Stage 3 (real-data SFT refinement).

Across Stage 2B, switching from sparse (r=0.75r = 0.75) to dense (r=0.00r = 0.00) tokens under vv-prediction improves two-object layout (0.543 →\rightarrow 0.614), color accuracy (0.662 →\rightarrow 0.726), and color attribution (0.215 →\to 0.273), consistently outperforming x0x_0-prediction (0.505 vs. 0.477 overall GenEval). As illustrated qualitatively in Figure 9, x0x_0-prediction requires dynamic percentile clipping to suppress t→0t \to0 singularities in bfloat16 when recovering velocities vt=(xt−x^0)/tv_t = (x_t - \hat{x}_0)/t, which flattens dynamic range and desaturates colors, whereas native vv-prediction preserves vibrant chromatic fidelity with zero activation clipping.

Giraffe beside a stop sign

(a) Stage 3 SFT (x0x_0-Prediction Ablation, 5122512^2): Converting x^0\hat{x}_0 into velocity targets vt=(xt−x^0)/tv_t = (x_t-\hat{x}_0)/t triggers t→0t \to0 singularities in bfloat16, forcing dynamic percentile clipping that flattens dynamic range and desaturates colors.

Giraffe beside a stop sign

(b) Stage 3 SFT (Native vv-Prediction, Final Pixel DiT): Direct velocity regression maintains smooth probability paths with zero activation clipping, recovering >95%> 95\% of the latent anchor while boosting Two-Object (0.780 vs. 0.699) and Color Attribution.

Figure 9. Qualitative and quantitative comparison of x0x_0-prediction vs. native vv-prediction in 5122512^2 Latent-to-Pixel adaptation under identical random seeds. Native vv-prediction eliminates dynamic clipping artifacts and color blowout.

Camera Dynamics, Spatial Disentanglement, and Failure Modes

Our continuous aspect-normalized RoPE coordinate manifold (Eq. (5)) and shifted square crop pretraining confer zero-shot virtual camera controllability at inference time without requiring auxiliary camera tokens, fine-tuning, or structural modifications.

Zero-Shot Virtual Viewport Manipulation

Applying affine transformations (Eq. (9)) directly to the continuous evaluation coordinate mesh (xpos,yposx_{\mathrm{pos}}, y_{\mathrm{pos}}) parameterized by normalized horizontal and vertical pan offsets (xshift,yshiftx_{\mathrm{shift}}, y_{\mathrm{shift}}) and an optical magnification factor z≡zoom∈R+z \equiv\mathrm{zoom} \in\mathbb{R}^{+} enables three independent, continuous spatial degrees of freedom under identical random seeds:

  • Horizontal Panning (XX-Shift): Figure 10 and Figure 13 (Row 1) illustrate horizontal camera panning. Applying non-zero offsets xshift∈{−0.25,+0.25}x_{\mathrm{shift}} \in\{-0.25, +0.25\} translates the virtual viewport horizontally across the scene manifold relative to the centered baseline (xshift=0.00x_{\mathrm{shift}} = 0.00), shifting subject framing and revealing peripheral background context while preserving fine anatomical structures, eye catchlights, and woven fabric textures without artifacting.

Dachshund in snow with a knitted cap, shifted left

(a) (a) xshift=−0.25x_{\mathrm{shift}} = -0.25

Dachshund in snow with a knitted cap, centered

(b) (b) xshift=0.00x_{\mathrm{shift}} = 0.00

Dachshund in snow with a knitted cap, shifted right

(c) (c) xshift=+0.25x_{\mathrm{shift}} = +0.25

Figure 10. Horizontal (XX-axis) viewport panning under identical noise seeds. Modulating the horizontal offset xshiftx_{\mathrm{shift}} of the continuous coordinate mesh translates the virtual viewport across the manifold, shifting the dachshund horizontally relative to the centered baseline (xshift=0.00x_{\mathrm{shift}} = 0.00) while preserving knit cap details and snow texture geometry.

Zero-shot continuous virtual camera manipulation on anime illustrations under identical random seeds

Figure 13. Zero-shot continuous virtual camera manipulation on anime illustrations under identical random seeds. Manipulating the continuous coordinate mesh allows continuous horizontal panning (Row 1), vertical elevation adjustment (Row 2), and optical zoom (Row 3) purely via position map manipulation, without retraining, camera adapters, or prompt modifications.

  • Vertical Panning (YY-Shift): Figure 11 and Figure 13 (Row 2) demonstrate vertical framing control. Injecting a positive offset (yshift=+0.25y_{\mathrm{shift}} = +0.25) lowers the virtual camera viewpoint, grounding the

Three views of a decorated wedding cake with different vertical viewport offsets

Figure 11. Vertical (YY-axis) viewport framing under identical noise seeds. Adjusting the vertical offset yshifty_{\mathrm{shift}} alters the virtual camera elevation: yshift=−0.25y_{\mathrm{shift}}=-0.25 elevates the framing to resolve upper floral tiers and the ballroom chandelier, whereas yshift=+0.25y_{\mathrm{shift}}=+0.25 lowers the viewpoint to ground the ornate metallic pedestal stand.

composition by resolving the ornate metallic pedestal stand, whereas a negative offset (yshift=−0.25y_{\mathrm{shift}} = -0.25) elevates the viewpoint to capture the upper floral tiers and the ballroom chandelier.

  • Continuous Optical Zoom: Figure 12 and Figure 13 (Row 3) demonstrate continuous focal magnification. Compressing the evaluation grid via z=1.25×z = 1.25\times concentrates the DiT’s spatial attention on the central wreath cavity, magnifying individual porcelain teacup rims and petal folds without geometric distortion, while z=0.75×z = 0.75\times expands the field of view for wide-angle framing.

Three views of a rose heart with different zoom levels

Figure 12. Continuous focal zoom scaling via RoPE coordinate grid compression. Rescaling the continuous evaluation mesh by optical magnification factor z∈{0.75×,1.00×,1.25×}z \in\{0.75\times, 1.00\times, 1.25\times\} simulates continuous wide-angle expansion and focal zoom under identical latent noise initializations. At z=1.25×z=1.25\times, transformer attention capacity concentrates on local topological features, resolving fine rose petal contours and porcelain teacup glazes while strictly maintaining global structural alignment.

Failure Modes and Generation Artifacts

While Asuka-FM achieves competitive open-domain synthesis (0.609 GenEval, 78.51 DPG-Bench) within a limited compute envelope (∼160\sim160 GPU hours), the model exhibits localized failure modes when subjected to out-of-distribution prompts. Figure 14 provides an explicit comparative ablation.

Comparison of unzoomed and RoPE zoomed organic compositions

Figure 14. Failure mode comparison under continuous RoPE zoom on a complex organic composition. (a) At canonical scale (z=1.0×z = 1.0\times), the 206M dual-stream backbone synthesizes a coherent scene with sharp lighting, structured perspective, and clean floral silhouettes. (b) Applying continuous RoPE focal zoom (z>1.0×z > 1.0\times) compresses the evaluation grid, inducing: (i) painterly high-frequency texture smearing on the stone pathway, (ii) semantic attribute bleeding (pink pigment leaking into ground cobblestones), and (iii) spatial non-rigidity and warping along linear wooden bench slats.

A systematic analysis reveals that these artifacts are not structural flaws of the base generator, but rather emerge from the physical limits of continuous coordinate scaling:

  1. High-Frequency Texture Smearing via Semantic Dilation. As demonstrated in Figure 14(a), the base model readily generates coherent cobblestones, realistic foliage distributions, and consistent depth-of-field. However, when focal zoom (z>1.0×z>1.0\times) is applied (Figure 14(b)), the coordinate grid contracts, forcing each patch of effective pixel stride P=16P=16 to cover a smaller physical subregion of the scene. Because the compact 206M backbone lacks multi-scale texture detailers or sub-patch supervision, it cannot hallucinate micro-surface details (e.g., individual stone crevices or fine wood grain). The model consequently defaults to smooth velocity trajectories, producing the waxy, painterly swirls visible in the

foreground path.

2. Spatial Non-Rigidity via Rotary Phase Dampening. Recall that rWr_W denotes the aspect-ratio width scale factor, NWN_W the horizontal patch-grid count, and ωk∈[π/4,2π]\omega_k \in[\pi/4, 2\pi] the spatial frequency for rotation channel kk. While the calibrated spectrum guarantees isotropic decay at unit scale (z=1.0×z = 1.0\times), compressing the coordinate mesh by zoom factor zz reduces the effective inter-patch phase difference to Δθk=(2rW/(NW⋅z))ωk\Delta\theta_k = (2r_W/(N_W \cdot z))\omega_k. This phase dampening slightly weakens relative positional discriminability between adjacent visual tokens in the spatial attention kernel (K(pi,pi+1)→1.0K(p_i, p_{i+1}) \to1.0, Eq. (6)), resulting in geometric relaxation. While organic structures (foliage, clouds) tolerate this relaxation, man-made rigid objects (such as the linear wooden slats of the bench) exhibit noticeable curvature and structural warping.

3. Semantic Attribute Bleeding under Compressed Receptive Fields. In dense compositional scenes containing strong complementary colors (vibrant pink petals against neutral grey stones and brown wood), joint self-attention in the shallow encoder fθf_\theta tends to disperse dominant color tokens. Under focal zoom, where coordinate density is artificially compressed, the effective receptive field of self-attention spans a broader semantic neighborhood in normalized space. This causes the pink floral pigment to bleed into adjacent latents, tinting the cobblestone path and staining the bench wood.

Positional Representation Robustness

During early exploratory pretraining on the 337K Danbooru corpus, Asuka-FM was initially optimized with discrete integer position IDs (pos⁡∈{0,…,N−1}\operatorname{pos} \in\{0, \ldots, N - 1\}, where NN is the image token count). Deviating from the square aspect ratio during inference exposed an architectural failure: because inter-patch indices outside the training grid are unseen, the model interpreted the expanded canvas as additional spatial instances, replicating heads and torsos (Figure 15a).

Rather than restarting pretraining from scratch, we investigated positional representation plasticity: whether a partially trained diffusion transformer could adapt mid-flight to continuous aspect-normalized RoPE without catastrophic forgetting of visual and character priors.

Discrete RoPE Collapse

(a) Discrete RoPE Collapse: Unseen integer positions outside the square grid induce harmonic repetition, causing duplicated heads and fragmented torsos along the vertical canvas.

Continuous Calibrated RoPE

(b) Continuous Calibrated RoPE: Aspect-normalized continuous coordinates bound the manifold within [−rH,rH]×[−rW,rW][-r_H,r_H] \times[-r_W,r_W], yielding single-subject anatomical consistency.

Figure 15. Ablation of RoPE positional formulations across non-square aspect ratios under identical random seeds. Switching to continuous aspect-normalized coordinates with frequency spectrum ω∈[π/4,2π]\omega\in[\pi/4,2\pi] completely resolves anatomical tiling without requiring pretraining restarts.

As detailed in Table 15, the RoPE adaptation phase (Phase 1B of the swapped run) realigned the model to continuous coordinates within only 20 epochs (5.6 GPU hours). The attention query-key projection matrices smoothly adapted to continuous phase rotations without loss of text-image alignment or character identities, and subsequent dense fine-tuning (Phases 1C and 1D) completely eradicated replication artifacts across arbitrary aspect ratios (Figure 15b).

PhaseEpochsDropRoPE ModeSamplesPatchesPFLOPsexec_{\mathrm{exec}}Time
1A: Base Sparse0–1000.75Discrete33.70M8.63B7.228.0 h
1B: RoPE Adaptation100–1200.75Continuous6.74M1.73B1.45.6 h
1C: Dense FT120–1450.00Continuous8.43M2.16B4.210.0 h
1D: Cosine Decay145–1700.00Continuous8.43M2.16B4.210.0 h
Total Stage 1 (Swapped Run)57.30M14.68B17.053.6 h

Table 15. Stage 1 RoPE swap progression on the 337K Danbooru corpus: Successful coordinate adaptation from discrete integer grid to continuous aspect-normalized RoPE within 20 epochs (≈6.74M samples) on a single RTX 4090.

Semantic Attribute Disentanglement

A hallmark of a well-conditioned multi-modal transformer is the mathematical decoupling of semantic attributes (e.g., color, apparel) from spatial and geometric priors (e.g., body pose, lighting, perspective). To evaluate whether Asuka-FM achieves clean semantic disentanglement, we sample images under identical standard Gaussian prior noise initializations x1∼N(0,I)x_1 \sim\mathcal{N}(0, I) while altering a single descriptive attribute in the prompt context. As demonstrated in Figure 16, toggling the tag from pink_hair to blue_hair selectively alters only the chromatic values of the hair strands. The complex finger pose, ribbon folds, facial morphology, and specular lighting remain invariant down to the pixel level. This confirms that the Dual-Stream DiT architecture effectively prevents cross-attention attribute leakage across distinct token subspaces.

Four anime character images comparing pink hair and blue hair prompt conditioning

Figure 16. (a) Prompt conditioning: pink_hair, cat_ears, maid_outfit (b) Prompt conditioning: blue_hair, cat_ears, maid_outfit. Strict semantic color disentanglement under identical latent initializations. Modifying exclusively the hair color prompt token updates the pigment channels without perturbing facial geometry, eye catchlights, clothing ruffles, hand gestures, or background sparkle lighting.

Visual Dynamics of SPRINT: Sparse Attenuation and Dense Recovery

To understand how sequence sparsification affects visual representation learning, we examine the qualitative representations produced during the 75% sparse phase (token drop ratio r=0.75r = 0.75) versus the final dense refinement phase (r=0.00r = 0.00).

Structured Group-Wise vs. Uniform Random Token Dropping. Uniform random token dropping produces irregular spatial holes that destabilize continuous RoPE phase shifts, as adjacent patch distances vary stochastically across the sequence. For an input canvas of resolution H0×W0H_0 \times W_0 and effective patch size P=16P = 16, SPRINT avoids irregular holes by partitioning the (H0/P)×(W0/P)(H_0/P) \times(W_0/P) patch grid into non-overlapping n×n=2×2n \times n = 2 \times2 windows (n=2n = 2), deterministically sampling k=1k = 1 active token per group. This preserves a regular, stride-2 downsampled grid that acts as a coherent, coarser canvas for the sparse deep middle path gθg_\theta.

SPRINT Sparse Phase

(a) SPRINT Sparse Phase (r=0.75r = 0.75): Macro-composition, character silhouette, and color blocking converge rapidly, but high-frequency lineart, eye highlights, and fringe hair details exhibit transient attenuation.

Dense Refined Phase

(b) Dense Refined Phase (r=0.00r = 0.00): Full token interaction restores high-frequency lineart, resolves complex eye iris stars, sharpens finger contours, and stabilizes intricate stage lighting.

Figure 17. Visual impact of SPRINT sequence sparsification during training under identical noise seeds. Sparse pretraining establishes robust macro-semantic layout at >50%> 50\% compute savings, after which dense refinement recovers razor-sharp textural boundaries.

High-Frequency Attenuation in Sparse Regimes. Because 75% of visual tokens are replaced with the learnable mask vector M∈RDM \in\mathbb{R}^{D} inside the middle blocks gθg_{\theta}, the model relies on the shallow dense encoder features ftf_t (produced by fθf_{\theta}) and the linear skip fusion ht=Wfuse[ft∥gpad]+bfuseh_t = W_{\mathrm{fuse}}[f_t \parallel g^{\mathrm{pad}}] + b_{\mathrm{fuse}} (where gpadg^{\mathrm{pad}} denotes the mask-padded deep representation, and WfuseW_{\mathrm{fuse}}, bfuseb_{\mathrm{fuse}} are the fusion projection parameters) to reconstruct fine edges. As seen in Figure 17a, sparse checkpoints capture overall pose and vibrant coloring but lack micro-structural crispness. Transitioning to dense fine-tuning (Figure 17b) allows attention weights across all 1,024 patches (at 5122512^2) to re-engage, completely recovering intricate lineart.

Multi-Character Scaling at 1MP Resolution. At native 1-megapixel resolution (1024 ×\times 1024, total image token count N=4096N = 4096), 75% token dropping discards 3,072 patches per step. In complex multi-character compositions, this creates spatial blind spots wider than individual faces or hands. As illustrated in Figure 18(a), early sparse epochs on multi-character compositions induced anatomical fractures and color leakage across character boundaries. Executing subsequent dense fine-tuning (Phase 3B in Table 16) re-established global semantic coordination (Figure 18(b)), proving that dense refinement is mandatory when scaling sequence sparsification to megapixel regimes.

Resolution scaling dynamics at native 1MP across sparse and dense pretraining phases

Figure 18. (a) 1MP SPRINT Sparse (r=0.75r = 0.75, 3 Epochs): Spatial stride of 32px induces multi-character anatomical fracturing and blurred limb interactions. (b) 1MP Dense Finetuned (r=0.00r = 0.00, 4 Epochs): Restores complete multi-subject anatomical separation, fine ribbon details, and environmental clarity.

Stage / PhaseRes. (X2X^2)Drop (rr)EpochsSamplesThroughput (img/s)MFU (%) Theo / Act†Compute (EFLOPs)*Wall TimeCost ($)
Stage 1: Base Semantic Pretraining (256 × 256, Latent Patch Grid 16 × 16 = 256 Image Tokens)
1A: SPRINT Sparse2562256^20.7510033.70M334.369.1% / 80.6%13.4228.0 h$9.24
1B: Dense Refine2562256^20.005016.85M234.071.7% / 83.6%9.9420.0 h$6.60
1C: WSD Cooldown2562256^20.00258.43M234.071.7% / 83.6%4.9710.0 h$3.30
Stage 2: High-Resolution Adaptation (512 × 512, Latent Patch Grid 32 × 32 = 1,024 Image Tokens)
2A: High-Res Sparse5122512^20.75258.43M70.934.3% / 40.0%7.8433.0 h$10.89
2B: Dense Finetune5122512^20.00103.37M72.063.4% / 74.0%5.7213.0 h$4.29
2C: WSD Cooldown5122512^20.0051.69M78.068.7% / 80.2%2.866.0 h$1.98
Stage 3: Native Megapixel Scaling (1024 × 1024, Latent Patch Grid 64 × 64 = 4,096 Image Tokens)
3A: SPRINT Sparse102421024^20.7531.01M7.8012.4% / 14.5%3.1036.0 h$11.88
3B: Dense Finetune102421024^20.0041.35M7.0722.5% / 26.2%8.2653.0 h$17.49
Cumulative 5122512^2 Subtotal (Stages 1 + 2)21572.46M–58.7% / 68.4%44.77110.0 h$36.30
Cumulative Grand Total (Stages 1 + 2 + 3)22274.81M–40.6% / 47.4%56.13199.0 h$65.67

Table 16. Pretraining progression and compute metrics of Asuka-FM on the curated anime dataset (Ntotal=337,000N_{\mathrm{total}} = 337{,}000) on a single consumer NVIDIA RTX 4090 GPU (24GB). Formatted symmetrically with Table 2. Pretraining executed using an effective batch size of 256 for Stages 1 and 2 (micro-batch 64 with 4 gradient accumulation steps), and effective batch size 128 for Stage 3 (micro-batch 16 with 8 accumulation steps). Throughput, compute, and MFU are derived directly from per-stage epochs and wall-clock hours using the medium context tier (152 text + 1 time = 153 tokens). Hardware costs evaluated at $0.33/GPU-hour. †MFUact_{\mathrm{act}} evaluates actual tensor ops (7PD7PD per sample, for active parameter count P=206.12P = 206.12M and tokens per sample DD) under SwiGLU recomputation; MFUtheo_{\mathrm{theo}} denotes standard algorithmic utilization (6PD6PD). *1 EFLOP = 101810^{18} FLOPs = 10310^3 PFLOPs = 10−310^{-3} ZFLOPs. Cumulative theoretical algorithmic compute (6PD6PD) is 38.37 EFLOPs (Stages 1–2) and 48.11 EFLOPs (Stages 1–3).

Curated Anime Dataset and Multi-Stage Pretraining Progression

To evaluate domain specialization, extreme aspect-ratio scaling, and continuous coordinate geometry, we established the curated-danbooru-20269 testbed. This dataset comprises Ntotal=337,000N_{\mathrm{total}} = 337{,}000 high-density anime-style illustrations sampled from Danbooru2023 [2].

Table 16 documents the complete computational expenditure, physical throughput, and financial accounting across the three resolution tiers (2562→5122→10242256^2 \rightarrow512^2 \rightarrow1024^2) under the same 7PD/6PD7PD / 6PD accounting as Table 2 (for P=206.12MP = 206.12\mathrm{M} active parameters and DD tokens per sample). Reflecting the bimodal prompt-length distribution of Danbooru tags, we evaluate sequence compute at the representative medium context tier (152 text + 1 time = 153 context tokens, vs. 228 tokens in the 625K corpus). Consequently, Stages 1 and 2B/2C achieve higher image throughput (234.0–334.3 img/s at 2562256^2 and 72.0–78.0 img/s at 5122512^2 dense) while sustaining 68.4% cumulative MFUact\mathrm{MFU}_{\mathrm{act}} across Stages 1–2. Note that Stage 2A (70.9 img/s, MFUact=40.0%\mathrm{MFU}_{\mathrm{act}} = 40.0\%) did not realize the full SPRINT speedup achieved in the 625K pipeline (108.5 img/s) due to network speed during latent streaming from HuggingFace, while Stage 3 (102421024^2) drops to 14.5%–26.2% MFUact\mathrm{MFU}_{\mathrm{act}} due to the online VAE bottleneck (Section G).

SwinV2 Prompt Upsampling and Dynamic Prefix Formatting. Because crowdsourced tags frequently suffer from omissions, every training image is tagged via a SwinV2-Transformer tagger [30], extracting fine-grained descriptors across apparel, hair styling, accessories, and atmospheric lighting. As introduced in § 5, tags are assembled into the canonical prefix format:

[character count], [character name], [series], [artist], [rating], [aesthetic tags], [year], [shuffled general tags]

General tags are shuffled per epoch to eliminate sequence order biases.

Tag-Rarity Loss Weighting. To prevent high-frequency tags (1girl, solo) from drowning out gradient signals for rare characters and artist signatures, we apply the inverse tag-frequency weighting of Eq. (10). Given the active prompt tag set T\mathcal{T} of cardinality ∣T∣|\mathcal{T}|, dataset size Ntotal=337,000N_{\mathrm{total}} = 337{,}000, and corpus frequency NkN_k of tag k∈Tk \in\mathcal{T}, we compute the per-sample prompt weight wpromptw_{\mathrm{prompt}} and scale the base flow-matching objective LCFM\mathcal{L}_{\mathrm{CFM}} to obtain the weighted loss Lweighted\mathcal{L}_{\mathrm{weighted}}:

wprompt=1.0+1∣T∣∑k∈Tln⁡(NtotalNk+1),Lweighted=wprompt⋅LCFM,(15)w_{\mathrm{prompt}} = 1.0 + \frac{1}{|\mathcal{T}|}\sum_{k \in\mathcal{T}} \ln\left(\frac{N_{\mathrm{total}}}{N_k + 1}\right), \qquad\mathcal{L}_{\mathrm{weighted}} = w_{\mathrm{prompt}} \cdot\mathcal{L}_{\mathrm{CFM}}, \tag*{(15)}

where T\mathcal{T} represents the prompt tag set, Ntotal=337,000N_{\mathrm{total}} = 337{,}000, and NkN_k is the corpus tag frequency.

Hierarchical Aesthetic Tiering. Samples are classified into four quality tiers based on community scores and favorite counts: masterpiece, good score, bad score, and worse score. To preserve distinctive low-visibility styles, curated artist works are capped against falling into lower aesthetic classes.

Native 1MP Scaling and Hardware Bottleneck Profiling

While prior evaluations of SPRINT were restricted to ImageNet-1K at 512×512512 \times512, we investigated extending Asuka-FM directly to native 1-megapixel (1024×10241024 \times1024, total image token count N=4096N = 4096) generation on single GPUs. Pretraining under 75% SPRINT token dropping (drop ratio r=0.75r = 0.75, Phase 3A) followed by dense fine-tuning (r=0.00r = 0.00, Phase 3B) revealed severe systems barriers:

The Online VAE Wall at Scale. At 1024×10241024 \times1024, storing uncompressed 32-channel float16 latents for hundreds of thousands of samples requires >1.2> 1.2 TB of high-speed NVMe storage, exceeding consumer workstation capacity. Consequently, latents must be encoded from raw pixels on-the-fly.

As detailed in Table 17, profiling on the NVIDIA A40 demonstrates that across both micro-batch regimes, the frozen in-place VAE forward pass consumes an overwhelming 72.5%–72.6% of total step latency (2,548.10 ms at micro-batch 16, 4,906.91 ms at micro-batch 32). In contrast, the entire 206M generative backbone forward-backward pass executes in only 888.96 ms at micro-batch 16 and 1,756.68 ms at micro-batch 32. Consistent with our RTX 4090 profiling at micro-batch 8 (§ 6.3, where the frozen VAE accounts for 62.0% of step latency and runs 2.42×\times slower than the DiT backbone), the frozen autoencoder on the A40 runs nearly 3×\times slower than the trainable generative backbone, causing tensor cores to idle for almost three-quarters of execution time and collapsing end-to-end system MFU to 3.66%–3.79%. This finding reinforces that consumer-scale generative modeling cannot rely on online autoencoding at high resolutions, providing decisive systems justification for our decoupled latent-to-pixel adaptation strategy.

Pipeline ComponentMicro-BS 16 (Accum 8)Micro-BS 32 (Accum 4)
Latency (ms)Share (%)Latency (ms)Share (%)
Data Loading / IO (CPU)0.16 ms0.0%0.18 ms0.0%
Online Frozen VAE Encoding2,548.10 ms72.6%4,906.91 ms72.5%
Text Encoder Forward Pass31.84 ms0.9%63.23 ms0.9%
Backbone Forward Pass (Sparse)229.31 ms6.5%457.20 ms6.8%
Backbone Backward Pass (Sparse)659.65 ms18.8%1,299.48 ms19.2%
Optimizer Step & Norm Clip38.49 ms1.1%37.83 ms0.6%
Total Iteration Latency3,507.55 ms100.0%6,764.83 ms100.0%
Backbone Isolated Throughput46.42 TFLOPS (MFUact_{\mathrm{act}} = 28.10%)46.98 TFLOPS (MFUact_{\mathrm{act}} = 28.44%)
End-to-End System MFU3.66%3.79%

Table 17. Execution profiling breakdown of Asuka-FM at native 1024×10241024 \times1024 resolution on a single NVIDIA A40 GPU across variable micro-batch sizes (FLUX.2 VAE, N=4096N = 4096 tokens, TorchInductor compilation, bfloat16 precision).

The frozen autoencoder runs nearly 3×\times slower than the trainable generative backbone, causing tensor cores to idle for almost three-quarters of execution time and collapsing end-to-end system MFU to an unacceptable 3.66%–3.79%. This finding reinforces that consumer-scale generative modeling cannot rely on online autoencoding at high resolutions, providing decisive systems justification for our decoupled latent-to-pixel adaptation strategy.

Shallow Receptive-Field Starvation in 1MP SPRINT. SPRINT extracts dense spatial representations in the initial shallow encoder (Lenc=2L_{\mathrm{enc}} = 2 layers) before subsampling. At 256×256256 \times256, two self-attention layers with effective patch size P=16P = 16 span a local receptive field covering 12.5% of the linear canvas span (capturing full facial landmarks). At 1024×10241024 \times1024, however, that same two-layer receptive field covers only 3.1% of the linear canvas span.

Subsampling 75% of tokens (r=0.75r = 0.75) immediately after Layer 2 forces the sparse middle blocks gθg_{\theta} to route global semantic composition from ungrounded micro-edges with an effective spatial stride of 32 px. During Phase 3A, this receptive-field starvation induced anatomical fracturing in complex multi-character scenes (Figure 18(a)). While subsequent dense fine-tuning (Phase 3B, Figure 18(b)) restored anatomical coherence, this degradation demonstrates that sequence sparsification at 1MP could require a density curriculum (starting at r=0.50r = 0.50) or a deeper shallow encoder (Lenc≥4L_{\mathrm{enc}} \ge4). These empirical findings validate our design decision to focus consumer-scale latent-to-pixel adaptation within the 512×512512 \times512 regime, where the autoencoder throughput wall is permanently bypassed.

Curated Anime Qualitative Generation Gallery

In Figure 19, we provide high-resolution zero-shot samples synthesized by our domain-specialized 206M Asuka-FM model pretrained from scratch on the curated 337K Danbooru corpus on a single consumer RTX 4090 GPU, demonstrating sharp lineart, anatomical rendering, and character-concept binding.

References

  1. [1]Liquid AI. “LFM2.5-230M: Built to Run Anywhere”. In: Liquid AI Blog (2026). www.liquid.ai/blog/lfm2-5-230m (cit. on pp. 7, 27).
  2. [2]Anonymous and Danbooru Community. Danbooru2023: A large-scale crowdsourced & tagged anime illustration dataset. 2023. URL: https://huggingface.co/datasets/nyanko7/danbooru2023 (cit. on pp. 12, 23, 37).
  3. [3]Black Forest Labs. FLUX.1: Text-to-Image Generation Models. https://github.com/black-forest-labs/flux. 2024 (cit. on pp. 3, 4, 13, 25).
  4. [4]Black Forest Labs. FLUX.2: Analyzing and Enhancing the Latent Space of FLUX – Representation Comparison. 2025. URL: https://bfl.ai/research/representation-comparison (cit. on p. 3).
  5. [5]Hanqiu Li Cai and Chema Garabito. Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning. 2026. arXiv: 2610.09450 [cs.CV]. URL: https://arxiv.org/abs/2610.09450 (cit. on pp. 5, 11).
  6. [6]Chen Chen et al. DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture Design in Text to Image Generation. 2025. arXiv: 2503.10618 [cs.CV]. URL: https://arxiv.org/abs/2503.10618 (cit. on p. 4).
  7. [7]Jiuhai Chen et al. BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset. 2025. arXiv: 2505.09568 [cs.CV]. URL: https://arxiv.org/abs/2505.09568 (cit. on p. 14).
  8. [8]Shoufa Chen, Chongjian Ge, Shilong Zhang, Peize Sun, and Ping Luo. PixelFlow: Pixel-Space Generative Models with Flow. 2025. arXiv: 2504.07963 [cs.CV]. URL: https://arxiv.org/abs/2504.07963 (cit. on p. 5).
  9. [9]Xiaokang Chen et al. Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling. 2025. arXiv: 2501.17811 [cs.AI]. URL: https://arxiv.org/abs/2501.17811 (cit. on pp. 13, 25).
  10. [10]Zhennan Chen et al. DiP: Taming Diffusion Models in Pixel Space. 2026. arXiv: 2511.18822 [cs.CV]. URL: https://arxiv.org/abs/2511.18822 (cit. on pp. 4, 5, 11, 17, 23).
  11. [11]Zhennan Chen et al. L2P: Unlocking Latent Potential for Pixel Generation. 2026. arXiv: 2605.12013 [cs.CV]. URL: https://arxiv.org/abs/2605.12013 (cit. on p. 23).
  12. [12]Yong Xieng Cheng et al. Looped Diffusion Transformer. 2026. arXiv: 2609.40305 [cs.CV]. URL: https://arxiv.org/abs/2609.40305 (cit. on pp. 5, 13, 25).
  13. [13]Tri Dao. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. 2023. arXiv: 2307.08691 [cs.LG]. URL: https://arxiv.org/abs/2307.08691 (cit. on p. 23).
  14. [14]Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal Transformers. 2019. arXiv: 1807.03819 [cs.CL]. URL: https://arxiv.org/abs/1807.03819 (cit. on p. 5).
  15. [15]Ming Ding et al. CogView: Mastering Text-to-Image Generation via Transformers. 2021. arXiv: 2105.13290 [cs.CV]. URL: https://arxiv.org/abs/2105.13290 (cit. on pp. 4, 7).
  16. [16]Patrick Esser et al. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. 2024. arXiv: 2403.03206 [cs.CV]. URL: https://arxiv.org/abs/2403.03206 (cit. on pp. 1, 3, 4, 6, 13, 25).
  17. [17]Rongyao Fang et al. FLUX-Reason-6M and PRISM-Bench: A Million-Scale Text-to-Image Reasoning Dataset and Comprehensive Benchmark. 2025. arXiv: 2509.09680 [cs.CV]. URL: https://arxiv.org/abs/2509.09680 (cit. on p. 12).
  18. [18]Peng Gao et al. Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers. 2024. arXiv: 2405.05945 [cs.CV]. URL: https://arxiv.org/abs/2405.05945 (cit. on p. 4).
  19. [19]Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer. 2024. arXiv: 2303.14389 [cs.CV]. URL: https://arxiv.org/abs/2303.14389 (cit. on p. 5).
  20. [20]Jonas Geiping et al. Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. 2025. arXiv: 2502.05171 [cs.LG]. URL: https://arxiv.org/abs/2502.05171 (cit. on p. 5).
  21. [21]Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. “GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment”. In: Advances in Neural Information Processing Systems (NeurIPS). Vol. 36. 2023 (cit. on pp. 13, 24).arxiv.org/abs/2310.11513
  22. [22]Jonathan Ho, Ajay Jain, and Pieter Abbeel. “Denoising Diffusion Probabilistic Models”. In: Advances in Neural Information Processing Systems. Vol. 33. 2020, pp. 6840–6851. URL: https://proceedings.neurips.cc/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf (cit. on p. 5).DOI
  23. [23]Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment. 2024. arXiv: 2403.05135 [cs.CV]. URL: https://arxiv.org/abs/2403.05135 (cit. on pp. 13, 24).
  24. [24]Dengyang Jiang et al. An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models. 2026. arXiv: 2608.16887 [cs.CV]. URL: https://arxiv.org/abs/2608.16887 (cit. on pp. 5, 11, 23).
  25. [25]Felix Krause, Timy Phan, Ming Gui, Stefan Andreas Baumann, Vincent Tao Hu, and Björn Ommer. TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Training. 2025. arXiv: 2501.04765 [cs.CV]. URL: https://arxiv.org/abs/2501.04765 (cit. on p. 5).
  26. [26]Tianhong Li and Kaiming He. Back to Basics: Let Denoising Generative Models Denoise. 2026. arXiv: 2511.13720 [cs.CV]. URL: https://arxiv.org/abs/2511.13720 (cit. on pp. 5, 11).
  27. [27]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. “Flow Matching for Generative Modeling”. In: The Eleventh International Conference on Learning Representations. 2023. URL: https://openreview.net/forum?id=PqvMRDCJT9t (cit. on pp. 3, 5).DOI
  28. [28]Jie Liu et al. Flow-GRPO: Training Flow Matching Models via Online RL. 2025. arXiv: 2505.05470 [cs.CV]. URL: https://arxiv.org/abs/2505.05470 (cit. on pp. 3, 11, 18).
  29. [29]Xingchao Liu, Chengyue Gong, and Qiang Liu. “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow”. In: The Eleventh International Conference on Learning Representations. 2023. URL: https://openreview.net/forum?id=XVjTT1nw5z (cit. on p. 5).arxiv.org/abs/2209.03003
  30. [30]Ze Liu et al. “Swin Transformer V2: Scaling Up Capacity and Resolution”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022, pp. 12009–12019 (cit. on pp. 12, 37).arxiv.org/abs/2111.09883
  31. [31]Zehong Ma, Longhui Wei, Shuai Wang, Shiliang Zhang, and Qi Tian. “DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2026 (cit. on pp. 13, 25).arxiv.org/abs/2511.19365
  32. [32]Zehong Ma, Ruihan Xu, and Shiliang Zhang. PixelGen: Improving Pixel Diffusion with Perceptual Supervision. 2026. arXiv: 2602.02493 [cs.CV]. URL: https://arxiv.org/abs/2602.02493 (cit. on pp. 13, 25).
  33. [33]Dogyun Park et al. Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers. 2026. arXiv: 2510.21986 [cs.CV]. URL: https://arxiv.org/abs/2510.21986 (cit. on pp. 3, 5, 7, 23).
  34. [34]William Peebles and Saining Xie. Scalable Diffusion Models with Transformers. 2023. arXiv: 2212.09748 [cs.CV]. URL: https://arxiv.org/abs/2212.09748 (cit. on pp. 3–5).
  35. [35]Dustin Podell et al. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. 2023. arXiv: 2307.01952 [cs.CV]. URL: https://arxiv.org/abs/2307.01952 (cit. on pp. 13, 25).
  36. [36]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. “High-Resolution Image Synthesis with Latent Diffusion Models”. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2022, pp. 10674–10685 (cit. on pp. 3, 5).arxiv.org/abs/2112.10752
  37. [37]Vikash Sehweg, Xianghao Kong, Jingtao Li, and Michael Spranger. “Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2025 (cit. on p. 5).
  38. [38]Noam Shazeer. GLU Variants Improve Transformer. 2020. arXiv: 2002.05202 [cs.LG]. URL: https://arxiv.org/abs/2002.05202 (cit. on pp. 4, 7).
  39. [39]Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. RoFormer: Enhanced Transformer with Rotary Position Embedding. 2023. arXiv: 2104.09864 [cs.CL]. URL: https://arxiv.org/abs/2104.09864 (cit. on pp. 6, 9).
  40. [40]Image Team et al. Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer. 2026. arXiv: 2511.22699 [cs.CV]. URL: https://arxiv.org/abs/2511.22699 (cit. on pp. 4, 13).
  41. [41]Alex Jinpeng Wang et al. TextAtlas5M: A Large-scale Dataset for Dense Text Image Generation. 2025. arXiv: 2502.07870 [cs.CV]. URL: https://arxiv.org/abs/2502.07870 (cit. on pp. 12, 18).
  42. [42]Shuai Wang, Ziteng Gao, Chenhui Zhu, Weilin Huang, and Limin Wang. “PixNerd: Pixel Neural Field Diffusion”. In: International Conference on Learning Representations (ICLR). 2026 (cit. on pp. 13, 25).arxiv.org/abs/2507.23268
  43. [43]Xianbang Wang, Hanhong Zhao, Yiyang Lu, Kangyang Zhou, Linrui Ma, and Kaiming He. MiniT2I: A Minimalist Baseline for Text-to-Image Generation. 2026. URL: https://peppaking8.github.io/minit2i/ (cit. on pp. 1, 13, 25).
  44. [44]Zhendong Wang et al. Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models. 2023. arXiv: 2304.12526 [cs.CV]. URL: https://arxiv.org/abs/2304.12526 (cit. on pp. 4, 6, 10).
  45. [45]Chenfei Wu et al. Qwen-Image Technical Report. 2025. arXiv: 2508.02324 [cs.CV]. URL: https://arxiv.org/abs/2508.02324 (cit. on p. 3).
  46. [46]Size Wu et al. OpenUni: A Simple Baseline for Unified Multimodal Understanding and Generation. 2025. arXiv: 2505.23661 [cs.CV]. URL: https://arxiv.org/abs/2505.23661 (cit. on p. 14).
  47. [47]Shih-Ying Yeh. Home-made Diffusion Model from Scratch to Hatch. 2025. arXiv: 2509.06068 [cs.CV]. URL: https://arxiv.org/abs/2509.06068 (cit. on pp. 4, 5, 8, 10, 23).

Paper details

vlnc:W_hjr639c9
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 11

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents