WorksComputer Science
Asuka-FM: A System-Architectural Co-Design Framework for Efficient Text-to-Image Pretraining on Consumer Hardware
Abstract
Frontier text-to-image (T2I) foundation models predominantly rely on multi-billion parameter Diffusion Transformers (DiTs) pretrained across massive compute clusters, creating an accessibility barrier for academic laboratories. When training DiTs from scratch on single consumer GPUs (e.g., an NVIDIA RTX 4090 with 24,GB VRAM), practitioners encounter three primary bottlenecks: quadratic attention sequence costs, Just-In-Time (JIT) compilation graph breaks induced by variable aspect-ratio bucketing, and an autoencoder throughput wall where frozen Variational Autoencoder (VAE) operations dominate step latency. In this work, we introduce Asuka-FM, an end-to-end system and architectural framework demonstrating that competitive dual-stream DiTs (206M parameters) can be pretrained from scratch on a single consumer GPU in 160 GPU hours for under $55. Asuka-FM integrates four core designs: (i) an AdaLN-free dual-stream DiT backbone reducing parameter count; (ii) structured sequence sparsification via Sparse-Dense Residual Fusion (SPRINT) to cut attention FLOPs by over 50%; (iii) continuous aspect-normalized Rotary Position Embeddings (RoPE) coupled with shifted square crop training, guaranteeing static tensor shapes while unlocking zero-shot virtual camera control; and (iv) a pixel adaptation phase utilizing a lightweight (10M parameter) Patch Detailer Head operating under velocity regression (-prediction). Evaluated on open-domain benchmarks without benchmark-specific fine-tuning, Asuka-FM achieves GenEval 0.588 / DPG 78.96 scores at 256px and GenEval 0.609 / DPG 78.51 scores at 512px, setting a new Pareto-optimal efficiency standard for democratic foundation modeling.
Project Page · Code · Model · Data
Table 1c. Benchmark Summary
Introduction
The emergence of Diffusion Transformers (DiTs) [34] trained via flow matching [27] has established a new state of the art in high-fidelity text-to-image (T2I) synthesis. However, state-of-the-art models, such as FLUX.1 (12B) [3], SD3 (8B) [16], and Qwen-Image (20B) [45], have scaled parameter counts and training datasets to regimes that require industrial-scale data centers. This trajectory has widened the accessibility gap, preventing independent researchers and academic groups with limited resources from training generative models from scratch.
When attempting to pretrain generative diffusion models under compute-constrained environments (e.g., a single workstation with an NVIDIA RTX 4090 with 24 GB GDDR6X), the standard pretraining paradigm encounters three fundamental systems bottlenecks:
(i) Quadratic Attention Sequence Redundancy: Standard homogeneous DiTs allocate identical full-token sequence lengths across every transformer block. However, recent representational analyses [33] reveal that early layers primarily capture local high-frequency textures, whereas deep middle blocks model global macro-semantics. Processing full-length sequences across deep layers wastes limited floating-point operations (FLOPs) on redundant spatial details, slowing convergence.
(ii) Just-In-Time (JIT) Compilation Graph Breaks vs. Aspect-Ratio Bucketing: Real-world images feature diverse aspect ratios. Conventional pipelines handle this via dynamic aspect-ratio bucketing, grouping images of varying shapes into specialized mini-batches. Under modern tracing and JIT compiler runtimes (e.g.,
torch.compilewith TorchInductor), changing spatial dimensions dynamically triggers continuous kernel recompilations, invalidates static CUDA graphs, and severely degrades actual Model FLOPs Utilization ().(iii) The Autoencoder Throughput Wall: In conventional Latent Diffusion Models (LDMs) [36], an autoencoder (such as the FLUX.2 VAE [4]) compresses images into a regularized latent space. At high resolutions (), storing pre-computed 32-channel float16 latents for hundreds of thousands of images requires multiple terabytes of high-speed storage, exceeding consumer NVMe capacities. Consequently, latents must be computed on-the-fly. On consumer silicon during the training stage, the frozen VAE forward pass consumes of total wall-time, whereas a compiled 200M DiT forward-backward pass requires only (Figure 7 in Appendix § 10). Furthermore, at inference, VAE decoding accounts for over 70% of wall-clock latency under few-step regimes, creating a computational bottleneck for downstream reinforcement learning (e.g., Flow-GRPO [28]) and distillation.
Figure 7. Figure 7: Per-step execution latency breakdown across resolutions on consumer hardware. While In-RAM latent caching eliminates online VAE overhead at (devoting > 80% of step latency to the DiT backbone), on-the-fly VAE encoding at (1MP) consumes 72.6% of wall-clock time (2,548.1 ms out of 3,507.6 ms). Decoupled latent-to-pixel adaptation permanently bypasses this throughput wall.
To address these challenges, we introduce Asuka-FM, an end-to-end system and architectural co-design framework proving that a competitive 206M-parameter Dual-Stream DiT can be pretrained from scratch on a single desktop GPU in 160 GPU hours for under $551. As showcased in Figure 2, our open-domain model achieves sharp textural coherence, rich lighting, and diverse multi-concept synthesis (additional domain-specialized anime samples are shown in Figure 19 in Appendix § 17). To achieve this efficiency, we propose a multi-stage training roadmap (Figure 3) coupling structured sequence sparsification, canvas scaling, and latent-to-pixel adaptation. Our primary contributions are:

Figure 2. High-fidelity zero-shot visual synthesis from scratch on a single consumer GPU. Curated open-domain samples synthesized by Asuka-FM (206M backbone) following two-stage pretraining () on a single NVIDIA RTX 4090 GPU in 149.0 hours ($49.17).

Figure 3. Comprehensive multi-stage training roadmap of Asuka-FM. Pretraining advances through two latent resolution tiers () coupling 75% SPRINT sequence sparsification with full-token dense refinement and Warmup-Stable-Decay (WSD). Stage 3 transitions the converged 512px foundation prior to pixel space using a 10M convolutional Patch Detailer Head (PDH) trained under native -prediction across synthetic and real datasets, permanently removing the VAE decode barrier during few-step inference.

Figure 19. High-quality zero-shot samples from the anime-specialized Asuka-FM model. Our 206M dual-stream backbone demonstrates sharp lineart, intricate anatomical rendering, and character-concept binding pretrained from scratch on a single consumer RTX 4090 despite a compact open dataset.
System-Architectural Pretraining Recipe: We establish an engineering pipeline integrating in-RAM latent caching for low-resolution stages, network streaming for intermediate scales, and FlashAttention-2 kernel fusion within a fully static
torch.compileexecution graph. This setup sustains an average 75.6% physical on a single consumer RTX 4090 GPU.High-Efficiency Dual-Stream Architecture: We couple an AdaLN-free Dual-Stream DiT backbone with three synergistic mechanisms: (i) SPRINT sequence sparsification [33], routing 75%-sparse token sequences through 9 heavy middle blocks to slash attention FLOPs by over 50%;
(ii) Continuous Aspect-Normalized RoPE and shifted square cropping [44, 47], supported by a mathematical derivation proving that uncalibrated continuous RoPE undergoes spatial phase collapse, and a full-spectrum frequency calibration that ensures isotropic decay while conferring zero-shot virtual camera control; and (iii) Decoupled Latent-to-Pixel Adaptation, which attaches a lightweight (10M parameter) convolutional Patch Detailer Head (PDH) [10] under native velocity regression (-prediction) to eliminate the online VAE wall.
Empirical Ablations and Profiling with Systems Analysis: We discover an Amortization Asymmetry: while pixel adaptation eliminates the online VAE encoding wall during pretraining, its recurrent convolutional evaluation makes standard multi-step sampling () slower than latent ODE integration, delivering superior efficiency exclusively in few-step regimes (, yielding up to speedups at step).
Open-Domain Validation: Trained on a balanced 625K multi-source open corpus without benchmark-specific fine-tuning (avoiding BLIP3o-60K overfitting), Asuka-FM achieves a GenEval score of 0.588 and DPG-Bench score of 78.96 at 256px (68h), converging to 0.609 GenEval and 78.51 DPG-Bench at 512px (81h).
Related Work
Diffusion Transformers and Conditioning Paradigms. The transition from U-Nets to Diffusion Transformers (DiTs) [34] established transformers as the standard generative backbone. Multi-Modal DiTs (MMDiT) in SD3 [16], FLUX.1 [3] utilize separate weights per modality with shared self-attention, while single-stream models (Lumina-T2X [18], Z-Image [40]) concatenate modalities directly. To eliminate parameter overhead from per-block Adaptive Layer Normalization (AdaLN-Zero), DiT-Air [6] shared AdaLN parameters globally. While Peebles & Xie [34] initially explored concatenating class tokens in single-stream models, [49] showed that explicit modulation can be entirely omitted when stabilized by Sandwich RMSNorm [15] and SwiGLU activations [38]. Asuka-FM adopts this principle. by replacing AdaLN with in-context temporal conditioning, prepending a continuous sinusoidal timestep token directly to the multimodal sequence.
Sequence Sparsification and Token Dropping. Because self-attention complexity scales as with sequence length , token dropping provides significant acceleration. MaskDiT [50] and MDTv2 [19] introduced masked autoencoding objectives, while TREAD [25] bypassed subsets of tokens through inner layers. SPRINT [33] demonstrated that structural specialization—combining a dense shallow encoder (), a 75%-sparse middle path (), and a dense decoder () fused via linear skips—permits aggressive token dropping without fidelity loss. We extend SPRINT to open-domain multimodal conditioning and evaluate its scaling limits.
Looped Transformers: Parameter-Efficient Depth Scaling. Looped transformers [12, 14, 20] achieve parameter efficiency by repeatedly executing a shared group of middle transformer blocks. In Looped-DiT [12], cycling through shared blocks keeps weights resident in on-chip SRAM and L2 cache, mitigating memory bandwidth bottlenecks. While looped models compress parameter count across recurrent depth, Asuka-FM focuses on sequence-length compression via SPRINT, providing complementary architectural paths toward efficient pretraining.
Micro-Budget Foundation Modeling and Pixel Diffusion. Micro-budget training has gained significant attention. Stretching Each Dollar [37] demonstrated T2I pretraining from scratch for $1,890 on H100s using mixture-of-experts and deferred masking. Concurrently, HDM [47] trained a 343M model for $535–620 on RTX 5090 GPUs using shifted square cropping and aspect-normalized RoPE with learnable frequency parameters. In pixel-space diffusion, early works suffered from slow convergence [8, 22]. JiT [26] and Jiang et al. [24] advocated clean data prediction (-prediction), but Iris-3B [5] showed that velocity regression (-prediction) provides equal stability. DiP [10] introduced the Patch Detailer Head (PDH) to bridge latent efficiency with pixel generation. Asuka-FM synthesizes these insights into an integrated low-cost training pipeline.
Preliminaries
Continuous Normalizing Flows and Flow Matching
We formulate image generation through Continuous Normalizing Flows (CNF) and Conditional Flow Matching (CFM) [27, 29]. Let denote an empirical sample from data distribution on a low-dimensional manifold of ambient dimension , and let denote noise from a standard Gaussian prior with zero mean and identity covariance . Under the optimal-transport linear probability path across continuous time , intermediate noisy states are constructed via:
Differentiating Eq. (1) yields the conditional velocity field . A neural vector field regressor , conditioned on prompt context , is trained via the simulation-free objective:
At inference, novel samples are generated by drawing and integrating the probability flow ODE: from to .
Latent Diffusion and Dual-Stream Tokenization
Following the Latent Diffusion Model (LDM) framework [36], an autoencoder compresses raw pixel images of target training resolution (e.g., ) into a latent space with channels and spatial dimensions via spatial downsampling factor (e.g., , for the FLUX.2 VAE). Following Diffusion Transformers (DiT) [34], noisy latent tensors at timestep are divided into non-overlapping spatial patches of size (giving effective pixel patch size , producing a flattened sequence of image tokens , where is the image token count and is the transformer hidden dimension.
In Dual-Stream DiTs [16], image tokens and text tokens (of text sequence length ) are mapped through dedicated linear projections into image and context queries, keys, and values (, , and , , ) while sharing joint attention blocks. These projections are concatenated along the token sequence dimension () to form the joint matrices , , :
Spatial layout is anchored via 2D axial Rotary Position Embeddings (RoPE) [39], which apply a block-diagonal rotation matrix parameterized by continuous coordinates to yield rotated image queries and keys and . Here, denotes the vertical and horizontal center coordinates of each patch in matrix row-major convention.
Timestep Sampling and Time-SNR Shifting
To counteract signal-to-noise ratio (SNR) compression when scaling image resolution, we introduce a resolution-dependent shift factor and sample an unconstrained log-odds variable from a normal distribution with mean and standard deviation , mapping it to [16]. We then compute the shifted timestep for Eq. (1), biasing sampling toward the noise-dominant regime ().
Asuka-FM Framework
Asuka-FM establishes an end-to-end system and architectural framework designed to train high-capacity Diffusion Transformers (DiTs) from scratch within limited consumer-compute budgets. Our work extends the Micro-Budget Foundation Modeling research to an even more accessible boundary: pretraining a 206M Dual-Stream DiT on a single consumer workstation equipped with an RTX 4090 GPU in 161.2 hours. Furthermore, the random square crops training [44] and the multi-captioning strategy of i1 [49] demonstrate that a capable foundation model can be trained from scratch on fewer than one million images.
Rather than relying on massive parameter scaling or distributed multi-node infrastructure, the framework co-designs the model architecture, memory hierarchy, and training curriculum around four complementary pillars:
Minimalist Dual-Stream Backbone (§ 4.1): Completely eliminates per-block modulation overhead reducing compute and parameters.
Structured Sequence Sparsification via SPRINT (§ 4.2): Reduces attention FLOPs by over 50% while providing strong regularization.
Continuous Aspect-Normalized RoPE & Shifted Cropping (§ 4.3): Prevents rotary phase collapse, enforces static tensor shapes to eliminate JIT compilation breaks, and provides zero-shot virtual camera controls (pan and zoom).
Decoupled Latent-to-Pixel Adaptation (§ 4.5): Transfers converged latent priors directly into raw pixel space, permanently bypassing the autoencoder decoding latency wall during downstream inference and reinforcement learning rollouts.
Minimalist Conditioning and Dual-Stream Backbone
Asuka-FM employs a 206M-parameter Dual-Stream DiT backbone optimized for single-GPU execution (Table 1). Standard AdaLN-Zero requires projecting conditioning embeddings into six independent
| Architectural Hyperparameter | Specification / Value |
| Latent Channel Dimension () | 32 (FLUX.2-dev VAE, ) |
| Latent Patch Dimension () | (Effective ) |
| Hidden Dimension () | 768 |
| Number of Attention Heads () | 12 |
| Attention Head Dimension () | 64 |
| Feedforward Network (FFN) | SwiGLU () |
| Total Transformer Blocks () | 13 Blocks (2 Dense + 9 Sparse + 2 Dense) |
| Positional Embedding | Calibrated Aspect RoPE () |
| Text Encoder | LiquidAI/LFM2.5-230M (Frozen, ) |
| Text Adapter | 2-Layer Transformer (, 6.09M params) |
| Patch Detailer Head (DiP) | 4-Stage Conv U-Net (+10.09M params) |
| Total Pretraining Parameters | 206.12M (200.03M Backbone + 6.09M Adapter) |
caption: Asuka-FM architecture specifications. 206M-parameter Dual-Stream SPRINT DiT backbone.
:::
affine modulation parameters per residual stream across every layer, introducing non-compute parameter overhead and intermediate memory traffic. Following i1 [49], we eliminate per-block AdaLN modulation layers. To retain temporal guidance, the continuous scalar timestep is mapped via a sinusoidal MLP into a single temporal token and directly prepended to the text sequence , forming the combined context sequence . Time conditioning is broadcast through shared attention at near-zero parameter cost.
Text representations are extracted from a frozen language model (LiquidAI/LFM2.5-230M [1], output dimension ) and mapped into the DiT hidden dimension via a 2-layer bidirectional Transformer Text Adapter (6.09M params). Following attention mixing, image and context representations are updated via independent SwiGLU feedforward networks [38] with intermediate dimension . Sandwich RMSNorm [15] stabilizes bfloat16 mixed precision. In the final decoder block, projections for the text stream are frozen and pruned from the backward pass, avoiding dead gradient calculations.
Sparse-Dense Residual Fusion (SPRINT)
To eliminate quadratic attention overhead during pretraining, we integrate SPRINT [33], partitioning our -layer transformer into three functional stages: a dense shallow encoder (), a sparse deep middle path (), and a dense decoder ().
Structured Group-Wise Subsampling. Following SPRINT [33], our 13-layer transformer is partitioned into three functional stages (Figure 4):

Figure 4. Asuka-FM system-architectural co-design pipeline. Top Row (Stage 1): Raw images undergo shifted square cropping to enforce static tensor dimensions () and eliminate JIT recompilations, while continuous aspect-normalized RoPE prevents phase collapse. Precomputed FLUX.2 VAE latents receive prepended in-context temporal conditioning () alongside text representations from a 2-layer adapter without per-block AdaLN parameters. Bottom Row (Stage 2): The 13-layer dual-stream DiT executes SPRINT sequence sparsification: shallow blocks () supply high-frequency dense skips , deep middle blocks () process macro-semantics over a 75%-sparse grid, and dense decoder blocks () fuse representations to predict flow trajectories. Finally, the lightweight 10.09M Patch Detailer Head (PDH) transfers representations directly to RGB pixels under native -prediction.
(i) a dense shallow encoder () processing the batched full token sequence (for batch size and image tokens) to preserve local edge representations .
(ii) a sparse deep middle path (, 78.4% of backbone weights) processing only the active subsampled tokens by deterministically retaining token per spatial group (75% drop ratio , leaving active tokens) to route macro-semantics at low FLOP cost.
(iii) a dense decoder () that restores sequence length by inserting a learnable mask vector at dropped positions to yield , fuses shallow and deep paths via a linear projection with weight and bias into , and predicts the velocity field .
Analytical Speedup Model. At (, , , ), neglecting the short text sequence, the theoretical FLOP complexity under SPRINT is:
Compared to a homogeneous dense DiT of complexity (evaluating all blocks with tokens and no fusion overhead), Eq. (4) yields a theoretical FLOP ratio of (2.11 arithmetic acceleration, with self-attention FLOPs reduced by 64.9%). Accounting for memory bus traffic, text-stream compute, and kernel launch overheads on physical silicon (RTX 4090), the model predicts an achievable speedup of 1.65–1.95, closely matching our empirical measurement of 1.67 (108.5 img/s sparse vs. 64.8 img/s dense at , Table 2).
| Stage / Phase | Res. () | Drop () | Samples | Throughput (img/s) | MFU (%) Theo / Act† | Compute (EFLOPs)* | Wall Time | Cost ($) |
| Stage 1: Base Semantic Pretraining (256 × 256, Latent Patch Grid 16 × 16 = 256 Image Tokens) | ||||||||
| Offline VAE Encoding‡ | – | 0.625M | 45.2 | – | – | 3.8 h | $1.27 | |
| 1A: SPRINT Sparse | 0.75 | 46.88M | 282.6 | 74.4% / 86.8% | 23.74 | 47.0 h | $15.51 | |
| 1B: Dense Refine | 0.00 | 12.50M | 172.5 | 62.9% / 73.4% | 8.73 | 21.0 h | $6.93 | |
| Stage 2: High-Resolution Adaptation (512 × 512, Latent Patch Grid 32 × 32 = 1,024 Image Tokens) | ||||||||
| Offline VAE Encoding‡ | – | 0.625M | 20.6 | – | – | 8.4 h | $2.78 | |
| 2A: High-Res Sparse | 0.75 | 15.63M | 108.5 | 58.5% / 68.3% | 16.24 | 40.0 h | $13.20 | |
| 2B: Dense Refine | 0.00 | 9.38M | 64.8 | 61.0% / 71.2% | 16.94 | 41.0 h | $13.53 | |
| Cumulative DiT Training Total (Stages 1 + 2) | 84.38M | – | 64.8% / 75.6% | 65.65 | 149.0 h | $49.17 | ||
| Cumulative Grand Total (Including Offline VAE Pre-Encoding) | – | – | – | 65.65 | 161.2 h | $53.20 |
Table 2. Pretraining progression and compute metrics of Asuka-FM on a single consumer NVIDIA RTX 4090 GPU (24GB). All stages use continuous aspect-normalized RoPE, bfloat16, and torch.compile. Pretraining uses an effective batch size of 256. Unabridged configurations and token processing rates are detailed in Table 8 (Appendix § 10). Evaluated at $0.33/GPU-hour.
Calibrated Continuous Aspect-Normalized RoPE
Standard discrete integer positional encodings fail when generalizing across variable aspect ratios, inducing artifacts. To resolve this, previous work [47] treats vertical and horizontal spatial frequencies as learnable parameters ( for patch-grid height and head dimension ) initialized via log-spaced heuristics. In contrast, Asuka-FM establishes a parameter-free continuous coordinate formulation. We mathematically diagnose why naive continuous RoPE undergoes spatial phase collapse and derive a closed-form, calibrated frequency spectrum that achieves isotropic decay without optimizer overhead.
Continuous Coordinate Mapping. For a latent patch grid of height and width , aspect-ratio scale factors and preserving unit canvas area satisfy and , yielding and . Continuous normalized coordinates for a patch centered at are bounded within via:
Continuous RoPE Phase Collapse. In 2D axial RoPE, the head dimension is partitioned equally across vertical and horizontal axes, yielding two-dimensional rotation channels per axis. Let denote the spatial frequency for channel . The normalized spatial attention kernel between patches at coordinates and evaluates to:
In standard discrete RoPE [39], adjacent vertical patch separation is fixed at . However, when mapped to continuous normalized coordinates on an patch canvas, the normalized inter-patch separation contracts to . Under standard geometric base frequencies ( across dimensions per axis), the adjacent phase shift contracts to . Evaluating the second-order Taylor expansion of the cosine kernel around between adjacent patches and (shown for a diagonal step ; an axial step yields prefactor ) gives:
Here, denotes Landau’s big- notation, characterizing the asymptotic upper bound on the Taylor expansion truncation error as grid resolution . Specifically, since and adjacent phase shifts scale inversely with resolution (), the remainder terms vanish at least as fast as . Summing over all frequency channels proves that the kernel’s deviation from identity decays asymptotically as .
Consequently, as canvas resolution scales, uniformly across the grid (Figure 5D). Adjacent tokens lose spatial discriminability (), causing phase collapse and anatomical tiling artifacts.
Figure 5. Continuous RoPE coordinate geometry and frequency calibration analysis. Top Row: (A) Adjacent patch phase step across channels : uncalibrated normalized coordinates collapse phase shifts to (), while calibrated frequencies () restore angular separation (–). (B) Center-row coordinate trajectories comparing discrete integer indices against continuous coordinates . Bottom Row: Spatial attention kernel heatmaps centered at probe patch (cyan star). (C) Discrete integer baseline exhibits natural radial decay. (D) Uncalibrated continuous RoPE undergoes phase collapse (). (E) Calibrated continuous RoPE restores sharp isotropic radial decay across arbitrary canvas dimensions without learned parameters.
Full-Spectrum Frequency Calibration. To guarantee isotropic radial decay without harmonic aliasing ( for horizontal coordinate separation and peak frequency ), spatial frequencies must satisfy two boundary constraints between upper bound and lower bound :
Local Dispersion Bound (): On the canonical square patch grid (, ), adjacent patches separated by must achieve angular dispersion to maintain local boundary discriminability without exceeding the Nyquist limit , yielding (Figure 5A).
Global Monotonicity Bound (): Across the maximum canvas separation , the slowest channel must not complete a half-cycle (), preventing periodic harmonic resonance across the canvas and setting .
Frequencies are interpolated log-linearly from to across channels :
This closed-form formulation enforces sharp radial decay across arbitrary aspect ratios without requiring learned parameters or optimizer tracking (Figure 5E vs. discrete integer baseline in Figure 5C). These frequencies are registered as static buffers (register_buffer), avoiding optimizer tracking entirely.
Shifted Square Crop and Camera Dynamics
Variable aspect-ratio training typically requires padding or dynamic batching, triggering continuous JIT graph recompilations under torch.compile. Following Patch Diffusion [44] and HDM [47], we train strictly on fixed square crops while conditioning on continuous coordinates:
An input image with native resolution is isotropically scaled to such that its shorter side matches training dimension (e.g., or ), i.e., .
Continuous 2D coordinates are calculated across the scaled patch grid (, , with effective patch stride ) using Eq. (5).
A random window is extracted, yielding the aligned pair of cropped image and cropped position map . Because tensor shapes remain strictly static (), GPU execution graphs execute without recompilations or CUDA graph invalidations.
Zero-Shot Virtual Camera Inversion. Because the model is conditioned on continuous coordinates rather than discrete token indices, affine transformations parameterized by an optical magnification factor and normalized pan offsets can be applied to the evaluation coordinate mesh at inference time to steer the virtual camera viewport without parameter fine-tuning:
Mathematically, contracts the continuous coordinate query bounds from to . Because the transformer outputs tokens over a fixed -patch grid, querying this contracted coordinate neighborhood forces the model to synthesize a smaller spatial sub-region across the full canvas, creating continuous optical focal magnification (zoom in). Conversely, expands the coordinate field to incorporate wide-angle scene context. Concurrently, applies a linear translation vector to the coordinate center, shifting the camera frustum horizontally and vertically across identical noise seeds (Appendix § 13).
Latent-to-Pixel Decoupling via DiP Head
While latent diffusion provides rapid semantic pretraining, the frozen VAE decoder imposes latency bottlenecks during few-step inference and online reinforcement learning rollouts (e.g., Flow-GRPO [28]). We bridge this trade-off by pretraining in latent space and transferring to raw pixel space via a lightweight convolutional Patch Detailer Head (PDH) based on DiP [10].
PDH Architecture. Patch-level visual tokens () emitted by the DiT decoder are mapped into raw RGB patches via a 4-stage convolutional U-Net adding only 10.09M parameters ( smaller than standard autoencoder decoders). To prevent high-frequency aliasing from transposed convolutions, Asuka-FM employs nearest-neighbor upsampling coupled with convolutions (Upsample + Conv2d). Reconstructed patches are folded back into the full RGB pixel canvas of shape (for batch size and 3 RGB channels).
Direct Velocity Regression vs. Clean Data Prediction. Previous pixel diffusion studies [24, 26] advocated clean data prediction (-prediction). However, converting predicted clean states into velocity targets from noisy pixel states at timestep via Eq. (1), , introduces mathematical instability. As , this conversion suffers from division-by-zero singularities under bfloat16, requiring artificial clamping and dynamic percentile clipping that flattens dynamic range and induces chromatic blowout. In contrast, our ablations (corroborated by Iris-3B [5]) demonstrate that direct velocity regression (-prediction) is superior. Retaining native -prediction preserves continuous probability paths, stabilizes optimization, and eliminates the need for dynamic activation clipping.
Progressive Training and Systems Engineering
Curated Multi-Source Dataset and Ingestion Pipeline
Open-Domain General Corpus (625K). Following empirical findings from the [49], training on balanced mixtures with 100k–400k high-quality samples per source yields competitive generative performance without massive web scraping. We curate an open-domain dataset of 625,000 samples by extracting exactly 125,000 samples across five sources including synthetic and real samples: (i) FLUX-Reason-6M [17] (complex composition); (ii) GPT-Edit-Simpler (paired instructional edits); (iii) Midjourney-v6-Recap (artistic lighting and textures); (iv) Pexels (natural photography and anatomy); and (v) TextAtlas5M [41] (text-rich scenes). All images satisfy px.
VLM Multi-Captioning. We maintain five synthetic captions per image generated via Qwen3-VL-30B-A3B from [49] dataset. At each epoch, a caption is sampled dynamically, providing continuous semantic regularization that prevents text memorization. Training context is partitioned into three discrete tiers ([77, 152, 227] tokens), reducing padding overhead by up to 66% and triggering TorchInductor recompilation exactly twice during warmup.
Curated Danbooru Corpus (337K) & Tag-Rarity Weighting. To investigate domain specialization and high-density concept binding, we curate 337,000 illustrations5 from Danbooru2023 [2]. Because community tags often lack structural details, each image is processed through a SwinV2 tagger [30] to upsample descriptive metadata. Prompts are formatted into a canonical prefix structure with dynamically shuffled general tags:
[character count], [character name], [series], [artist], [rating], [aesthetic tags], [year], [shuffled general tags]
To prevent frequent tags (e.g., 1girl, solo) from dominating gradient updates while rare artist styles remain undertrained, we scale the flow matching loss by an inverse tag-frequency prompt weight to obtain the weighted loss . Given the active tag set of cardinality , dataset size , and corpus frequency of tag , we define:
where is the active tag set, is the dataset size, and is the corpus frequency of tag .
We use this dataset to train an anime-specialized Asuka-FM model6 in order to validate our Open-Domain dataset results.
Progressive Pretraining Recipe
Pretraining advances through two core resolution tiers (Table 2; see Table 8 in Appendix § 10 for the complete progression matrix including batch sizes, epoch schedules, and average token rates): base semantic anchoring at 256 × 256 and texture synthesis at 512 × 512. All stages employ 8-bit AdamW (, , weight decay ) via bitsandbytes, mixed-precision bfloat16 with TF32 tensor cores, and FlashAttention-2 inside torch.compile. Classifier-Free Guidance dropout is set to .
Table 8: Comprehensive pretraining progression and compute metrics of Asuka-FM on a single consumer NVIDIA RTX 4090 GPU (24GB). Unabridged reference for Table 2. Sequence lengths account for visual image tokens plus context (227 text + 1 time = 228 tokens). MFU evaluates actual tensor operations on silicon under selective SwiGLU activation checkpointing ( work factor, where denotes active model parameters and denotes tokens per sample); MFU denotes standard theoretical algorithmic utilization ( factor). Hardware costs evaluated at $0.33/GPU-hour.
| Stage / Phase | Res. () | Batch Size | Drop () | Epochs | Samples | Avg Tokens | Throughput (img/s) | Token Rate (tok/s) | MFU (%) Theo / Act† | Compute (EFLOPs)∗ | Wall Time | Cost ($) |
| Stage 1: Base Semantic Pretraining (256 × 256, Latent Patch Grid 16 × 16 = 256 Image Tokens) | ||||||||||||
| Offline VAE Encoding‡ | 64 | – | – | 0.625M | – | 45.2 | – | – | – | 3.8 h | $1.27 | |
| 1A: SPRINT Sparse | 256 | 0.75 | 75 | 46.88M | 351.1 tok | 282.6 | 99.2k | 74.4% / 86.8% | 23.74 | 47.0 h | $15.51 | |
| 1B: Dense Refine | 256 | 0.00 | 25 | 12.50M | 484.0 tok | 172.5 | 83.5k | 62.9% / 73.4% | 8.73 | 21.0 h | $6.93 | |
| Stage 2: High-Resolution Adaptation (512 × 512, Latent Patch Grid 32 × 32 = 1,024 Image Tokens) | ||||||||||||
| Offline VAE Encoding‡ | 32 | – | – | 0.625M | – | 20.6 | – | – | – | 8.4 h | $2.78 | |
| 2A: High-Res Sparse | 256 | 0.75 | 25 | 15.63M | 720.3 tok | 108.5 | 78.1k | 58.5% / 68.3% | 16.24 | 40.0 h | $13.20 | |
| 2B: Dense Refine | 256 | 0.00 | 15 | 9.38M | 1,252.0 tok | 64.8 | 81.1k | 61.0% / 71.2% | 16.94 | 41.0 h | $13.53 | |
| Cumulative DiT Training Total (Stages 1 + 2) | 140 | 84.38M | – | – | – | 64.8% / 75.6% | 65.65 | 149.0 h | $49.17 | |||
| Cumulative Grand Total (Including Offline VAE Pre-Encoding) | – | – | – | – | – | – | – | – | 65.65 | 161.2 h | $53.20 |
Table 8.
Execution Breakdown. Cumulatively, 149.0 hours of DiT latent pretraining ($49.17 cost) executed 65.65 ExaFLOPs (0.0657 ZFLOPs) at an average sustained utilization of . Including one-time offline VAE pre-encoding (12.2h, $4.03), the entire latent pretraining pipeline converges in 161.2 total GPU hours ($53.20).
Memory Hierarchy Optimizations
Pretraining experiments were executed on a single workstation: NVIDIA RTX 4090 GPU (24GB GDDR6X, 165.2 TFLOPS peak), AMD Ryzen 9 7950X CPU, and 128GB DDR5 RAM under Ubuntu 24.04 and PyTorch 2.8.
(i) In-RAM Latent Caching (): Pre-encoded latents for all 625k samples at require GB. Caching latents directly in system RAM eliminated disk I/O and autoencoder latency, sustaining 282.6 img/s (86.8% MFU). (ii) Network Latent Streaming (): At , latents expand to GB, exceeding system RAM. Latents were pre-encoded offline (8.4h) and streamed asynchronously via dynamic FIFO queues, sustaining 108.5 img/s (68.3% MFU). (iii) Selective Activation Checkpointing: Checkpointing only the SwiGLU blocks while retaining joint self-attention tensors reduced peak activation memory by , enabling micro-batch size 64 at within 24 GB VRAM. Recomputing MLP projections adds work factor ( on silicon vs. theoretical, where and here denote active model parameters and tokens per sample).
Experiments and Empirical Analysis
We ablated our architectural components and the SPRINT algorithm on a controlled toy T2I benchmark (details in Appendix § 11); here, we focus on open-domain foundation benchmarks and micro-architectural systems profiling.
Open-Domain Benchmark Evaluation
We evaluate Asuka-FM across two benchmarks: GenEval [21] and DPG-Bench [23] (Table 3; the complete GenEval breakdown is provided in Table 9 of Appendix § 10). At , Asuka-FM achieves GenEval of 0.588 and DPG-Bench of 78.96. When scaling to under 75% SPRINT token dropping (Phase 2A), performance temporarily drops to 0.480 on GenEval and 68.40 on DPG-Bench due to fine spatial binding degradation (color attribution drops from 41.75% to 16.25%). However, full-sequence dense refinement (Phase 2B) rapidly restores high-frequency fidelity: within 20,000 steps, DPG-Bench surges to 79.49, converging at Epoch 139 to 0.609 GenEval and 78.51 DPG-Bench.
| Model | Params | Hardware Budget | Training Data | GenEval Total ↑ | DPG-Bench Total ↑ |
| Industrial Foundation Models (Multi-Billion Parameters, Datacenter Compute) | |||||
| SDXL Base [35] | 2.6B | Hundreds of A100s | Millions (Web) | 0.406* | 72.40* |
| SD3 Medium [16] | 2.0B | Massive Cluster | Millions (Web) | 0.620* | 84.08* |
| FLUX.1 [Dev] [3] | 12.0B | Massive Cluster | Web-Scale | 0.660* | 83.84* |
| Janus-Pro-7B [9] | 7.0B | Large Cluster | Multimodal Web | 0.800* | 84.19* |
| i1 (Open Foundation) [49] | 3.0B | TPU v5p-128 Pod | Curated Open | 0.840 | 86.73 |
| Accessible Academic Baselines (Sub-2B Parameters, Finetuned on BLIP3o-60K†) | |||||
| PixNerd-XXL [42] | 1.2B | Multi-GPU Cluster | Pretrain + BLIP3o† | 0.730* | 80.90* |
| PixelGen-XXL [32] | 1.1B | 8 × H800 (6 Days) | 36M + BLIP3o† | 0.790* | 79.36 |
| DeCo-XXL [31] | 1.1B | Multi-GPU Cluster | Pretrain + BLIP3o† | 0.860* | 81.40* |
| Looped-DiT B/16 [12] | 0.26B | 8 × H100 (3 Days) | CC12M + BLIP3o† | 0.874* | 87.00* |
| MiniT2I-L/16 [43] | 0.91B | 8 × H100 (27 Days) | CC12M + BLIP3o† | 0.883* | 84.80* |
| Asuka-FM (Ours: Single Consumer RTX 4090, 625K Balanced Dataset, No BLIP3o Finetuning) | |||||
| Asuka-256 (Dense Final) | 0.21B | 1 × RTX 4090 (68h) | 625K Open Curated | 0.588 | 78.96 |
| Asuka-512 (75% Sparse) | 0.21B | 1 × RTX 4090 (40h) | 625K Open Curated | 0.480 | 68.40 |
| Asuka-512 (Dense FT 20k Steps) | 0.21B | 1 × RTX 4090 (60h) | 625K Open Curated | 0.575 | 79.49 |
| Asuka-512 (Dense Final, Ep 139) | 0.21B | 1 × RTX 4090 (81h) | 625K Open Curated | 0.609 | 78.51 |
Table 3. Quantitative comparison on representative text-to-image benchmarks (GenEval and DPG-Bench). Results marked with (* ) denote numbers reported in prior literature [12, 40, 43] [49]. Models marked with † indicate fine-tuning on the BLIP3o-60K dataset, which specifically targets GenEval metrics. Asuka-FM is trained from scratch on open data without benchmark-specific fine-tuning. The full GenEval breakdown is provided in Table 9 in Appendix § 10.
| Model | Params | Hardware Budget | Training Data | Single | Two-Obj | Count | Colors | Position | Color-Attr | GenEval Overall ↑ | DPG-Bench Total Score ↑ |
| Industrial Foundation Models (Multi-Billion Parameters, Datacenter Compute) | |||||||||||
| SDXL Base [35] | 2.6B | Hundreds of A100s | Millions (Web) | – | – | – | – | – | – | 0.406* | 72.40* |
| SD3 Medium [16] | 2.0B | Massive Cluster | Millions (Web) | 0.98 | 0.84 | 0.66 | 0.74 | 0.40 | 0.43 | 0.620* | 84.08* |
| FLUX.1 [Dev] [3] | 12.0B | Massive Cluster | Web-Scale | 0.99 | 0.81 | 0.79 | 0.74 | 0.20 | 0.47 | 0.660* | 83.84* |
| Janus-Pro-7B [9] | 7.0B | Large Cluster | Multimodal Web | – | – | – | – | – | – | 0.800* | 84.19* |
| i1 (Open Foundation) [49] | 3.0B | TPU v5p-128 Pod | Curated Open | – | – | – | – | – | – | 0.840 | 86.73 |
| Accessible Academic Baselines (Sub-2B Parameters, Finetuned on BLIP3o-60K†) | |||||||||||
| PixNerd-XXL [42] | 1.2B | Multi-GPU Cluster | Pretrain + BLIP3o† | 0.97 | 0.86 | 0.44 | 0.83 | 0.71 | 0.53 | 0.730* | 80.90* |
| PixelGen-XXL [32] | 1.1B | 8× H800 (6 Days) | 36M + BLIP3o† | 0.99 | 0.88 | 0.59 | 0.90 | 0.70 | 0.70 | 0.790* | 79.36 |
| DeCo-XXL [31] | 1.1B | Multi-GPU Cluster | Pretrain + BLIP3o† | – | – | – | – | – | – | 0.860* | 81.40* |
| Looped-DiT B/16 [12] | 0.26B | 8× H100 (3 Days) | CC12M + BLIP3o† | – | – | – | – | – | – | 0.874* | 87.00* |
| MiniT2I-L/16 [43] | 0.91B | 8× H100 (27 Days) | CC12M + BLIP3o† | – | – | – | – | – | – | 0.883* | 84.80* |
| Asuka-FM (Ours: Single Consumer RTX 4090, 625K Balanced Dataset, No BLIP3o Finetuning) | |||||||||||
| Asuka-256 (Dense Final) | 0.21B | 1× RTX 4090 (68h) | 625K Open Curated | 0.950 | 0.750 | 0.319 | 0.782 | 0.308 | 0.418 | 0.588 | 78.96 |
| Asuka-512 (75% Sparse) | 0.21B | 1× RTX 4090 (40h) | 625K Open Curated | 0.963 | 0.639 | 0.253 | 0.673 | 0.190 | 0.163 | 0.480 | 68.40 |
| Asuka-512 (Dense FT 20k Steps) | 0.21B | 1× RTX 4090 (60h) | 625K Open Curated | 0.953 | 0.788 | 0.331 | 0.715 | 0.305 | 0.358 | 0.575 | 79.49 |
| Asuka-512 (Dense Final, Ep 139) | 0.21B | 1× RTX 4090 (81h) | 625K Open Curated | 0.950 | 0.811 | 0.347 | 0.817 | 0.278 | 0.450 | 0.609 | 78.51 |
Table 9. Table 9: Comprehensive quantitative comparison on representative text-to-image benchmarks (GenEval and DPG-Bench) with full task breakdown. Unabridged reference for Table 3. Results marked with (*) denote numbers reported in prior literature. Models marked with † indicate fine-tuning on the BLIP3o-60K instruction dataset. Asuka-FM is trained from scratch on open-source data without benchmark-specific fine-tuning.
Evaluation Protocol and Checkpoint selection. To ensure strict reproducibility, benchmarks are evaluated using a 100-step Euler ODE solver. Classifier-Free Guidance (CFG) scale is set to at and reduced to at to prevent contrast blowout. We explicitly select Epoch 139 over the 20k-step checkpoint despite the slight 0.98-point variance in DPG: extended training improves fine-grained multi-object binding, with color attribution rising from 0.358 to 0.450 and two-object layout rising from 0.788 to 0.811 (Table 9).
Contextualizing Benchmark Fine-Tuning. As documented in prior evaluations [46] [49], several academic baselines (e.g., Looped-DiT, MiniT2I, DeCo) report scores following fine-tuning on the BLIP3o-60K prompt dataset [7], which directly aligns models to GenEval syntactic patterns. In contrast, Asuka-FM is evaluated strictly as an open-domain foundation model trained on general multi-source data without benchmark-specific fine-tuning. Even without specialized prompt tuning, Asuka-FM’s DPG-Bench score (78.51–79.49) is highly competitive with models trained on massive clusters (DeCo at 81.40, SDXL at 72.40), while outperforming SDXL on GenEval (0.609 vs. 0.406).
Compute Budget Comparison
Table 4 compares Asuka-FM against modern foundation models. While sub-2B models require multi-GPU nodes costing $2,000–$5,000, Asuka-FM completes its latent two-stage curriculum in 149.0 hours on a single RTX 4090 GPU ($49.17), establishing an order-of-magnitude cost reduction. On the Pareto frontier (Figure 1, Panel A), Asuka-FM executes only 0.0657 ZFLOPs (56.27 EFLOPs theoretical), sitting at the extreme low-compute anchor while maintaining high fidelity.

Figure 1. Asuka-FM framework overview. (a) Denoiser training compute vs. compositional alignment (DPG-Bench). (b) High-fidelity visual synthesis from our 206M dual-stream backbone. (c) Representative performance summary across industrial models, academic baselines, and Asuka-FM (†denotes finetuning on BLIP3o-60K).
| Model | Params | Hardware | Wall Time | GPU-h | Cost ($) |
| FLUX.1 [Dev] | 12.0B | H100 SXM5 Pod | Weeks | > 100k | > $175k |
| SD3 Medium | 2.0B | H100 SXM5 Pod | Weeks | > 50k | > $70k |
| i1. | 3.0B | TPU v5p-128 | 29.5 Days | 90.6k | ≈ $380k |
| PixelGen-XXL | 1.1B | 8 × H800 | 6.0 Days | 1,152 | ≈ $2,880 |
| MiniT2I-B/32 | 0.26B | 8 × H100 | 3.0 Days | 576 | ≈ $2,016 |
| Looped-DiT | 0.26B | 8 × H100 | 3.0 Days | 576 | ≈ $2,016 |
| Asuka-256 | 0.21B | 1 × RTX 4090 | 68.0 h | 68.0 | $22.44 |
| Asuka-512 | 0.21B | 1 × RTX 4090 | 81.0 h | 81.0 | $26.73 |
| Asuka-FM DiT Total | 0.21B | 1 × RTX 4090 | 149.0 h | 149.0 | $49.17 |
| Asuka-FM Full (Inc. VAE) | 0.21B | 1 × RTX 4090 | 161.2 h | 161.2 | $53.20 |
Table 4. Pretraining hardware and financial cost comparison across foundation T2I models. Cloud rates: $0.33/hr (RTX 4090), $1.50/hr (H800), $2.89/hr (H100), $537.60/hr (TPU v5p-128 pod).
Hardware Profiling: RTX 4090 vs. NVIDIA A40
We profile kernel execution across consumer and datacenter silicon: RTX 4090 (24GB GDDR6X, Ada Lovelace) and NVIDIA A40 (48GB GDDR6, Ampere) under identical software stacks (Table 5; see Table 10 in Appendix § 10 for complete resolution and token sequence specifications).
| Hardware Device | Peak VRAM Allocated | Data / IO | Online VAE | Text Enc. | Backbone (F+B) | Opt. & Clip | Total Step Latency | Backbone Throughput | MFU (%) (Isolated / Sys) |
| Production Latent Pretraining: Kernel-Fused Tensor Saturation (, In-RAM Latents) | |||||||||
| NVIDIA A40 | 13,254.5 MB | 1.44 ms | – | 99.98 ms | 469.88 ms | 36.33 ms | 607.65 ms | 44.43 TFLOPS | 26.90% / 31.71% |
| RTX 4090 | 15,249.5 MB | 0.27 ms | 0.28 ms | 38.94 ms | 247.91 ms | 20.87 ms | 308.27 ms | 84.22 TFLOPS | 50.98% / 63.25% |
| Online Autoencoder Wall (, FLUX.2 VAE On-the-Fly Encoding, Dense Tokens) | |||||||||
| NVIDIA A40 | 35,201.0 MB | 0.33 ms | 2,263.52 ms | 111.34 ms | 1,396.95 ms | 36.28 ms | 3,808.42 ms | 51.39 TFLOPS | 31.11% / 15.27% |
Table 5. Micro-architectural profiling across hardware architectures at and . Profiled across 30 active iterations via PyTorch FlopCounterMode (165.2 TFLOPS peak reference, micro-batch 64 with 16,384 tokens/step at and 65,536 tokens/step at ). Canvas dimensions and token lengths are reported in Table 10 (Appendix A).
| Hardware Device | Res. Canvas | Batch Size (Tokens/Step) | Peak VRAM Allocated | Data/IO | Online VAE | Text Enc. | Backbone (F+B) | Opt. & Clip | Total Step Latency | Backbone Throughput | MFU (%) (Iso. / Sys) |
| Production Latent Pretraining: Kernel-Fused Tensor Saturation (, In-RAM Latents) | |||||||||||
| NVIDIA A40 | 64 (16,384) | 13,254.5 MB | 1.44 ms | – | 99.98 ms | 469.88 ms | 36.33 ms | 607.65 ms | 44.43 TFLOPS | 26.90% / 31.71% | |
| RTX 4090 | 64 (16,384) | 15,249.5 MB | 0.27 ms | 0.28 ms | 38.94 ms | 247.91 ms | 20.87 ms | 308.27 ms | 84.22 TFLOPS | 50.98% / 63.25% | |
| Online Autoencoder Wall (, FLUX.2 VAE On-the-Fly Encoding, Dense Tokens) | |||||||||||
| NVIDIA A40 | 64 (65,536) | 35,201.0 MB | 0.33 ms | 2,263.52 ms | 111.34 ms | 1,396.95 ms | 36.28 ms | 3,808.42 ms | 51.39 TFLOPS | 31.11% / 15.27% |
Table 10. Comprehensive micro-architectural profiling across hardware architectures at and . Unabridged reference for Table 5. Profiled across 30 active iterations via PyTorch FlopCounterMode (165.2 TFLOPS peak reference). Reports explicit spatial dimensions, per-step token volumes, and isolated backbone (‘Iso.’) vs. end-to-end system (‘Sys’) MFU.
The VRAM and Peak TFLOPS Analysis. The A40 provides double the physical VRAM (48GB vs. 24GB) and comparable rated compute (~150 TFLOPS). However, at batch size 64, the RTX 4090 completes each compiled step in 308.27 ms (84.22 TFLOPS, ), whereas the A40 requires 607.65 ms (44.43 TFLOPS, ). The consumer GPU executes identical workloads 1.97× faster. Peak memory allocation reaches only 15.25 GB on the RTX 4090; the extra 34.75 GB on the A40 remains completely idle.
Micro-Architectural Drivers. This divergence is driven by two factors: (i) 12× L2 Cache Differential (72 MB vs. 6 MB): Under torch.compile, TorchInductor fuses projections, norms, and SwiGLU gating into compound Triton kernels. On Ada Lovelace, the 72 MB L2 cache keeps intermediate activations resident in on-chip SRAM across kernel invocations. On Ampere (6 MB L2), batched sequence activations ( tokens/step) overflow, thrashing memory buses. (ii) DRAM Bandwidth Starvation (1,008 GB/s vs. 696 GB/s): When cache overflows, the A40 falls back to GDDR6 memory (31% slower). Memory-bound phases suffer heavy penalties: text encoder forward is 2.57× slower on A40 (99.98 ms vs. 38.94 ms), and AdamW steps are 1.74× slower (36.33 ms vs. 20.87 ms).
The Online VAE Wall at 512px and 1MP. Profiling at on the A40 under online VAE encoding (Table 5) shows that in-place VAE encoding consumes 2,263.52 ms out of 3,808.42 ms per step (59.4% of wall-clock time; see Figure 7 in Appendix § 10 for resolution breakdown), dragging end-to-end system MFU down to 15.27%. This bottleneck intensifies at native 1-megapixel (, tokens): caching uncompressed 32-channel float16 latents for 337k samples requires over 1.2 TB of storage, exceeding consumer NVMe capacities and forcing on-the-fly encoding. Profiling at micro-batch size on a single RTX 4090 reveals that the frozen VAE encoder consumes 164.92 ms per step (62.0% of total step latency, 265.44 ms compiled), whereas the DiT forward and backward passes execute in 23.70 ms and 44.50 ms respectively ( ms combined). Because the frozen autoencoder runs 2.42× slower than the generative backbone, physical hardware MFU is capped at 14.5%–26.2%—establishing the primary systems motivation for our autoencoder-free latent-to-pixel adaptation strategy (preliminary 1MP SPRINT scaling diagnostics on the Danbooru corpus are deferred to Appendix § 16).
Latent-to-Pixel Adaptation at 512px Resolution
Following our pretraining recipe, we adapt the converged 512px latent model directly to raw RGB pixel space using the 10.09M convolutional Patch Detailer Head (PDH) across three progressive stages (Table 6):
| Stage / Phase | Res. () | Drop () | Samples | Throughput (img/s) | MFU (%) Theo / Act† | Compute (EFLOPs)* | Wall Time | Cost |
| Phase 0: Offline Synth Generation (A40)‡ | – | 0.10M | 1.85 img/s | – | – | 15.0 h | $6.00 | |
| Stage 1: Decoupled PDH Warmup (Synth) | 0.75 | 0.50M | 61.8 img/s | 62.4% / 72.8% | 0.52 | 2.25 h | $0.74 | |
| Stage 2A: Joint SPRINT Sparse (Synth) | 0.75 | 2.50M | 57.7 img/s | 58.1% / 67.8% | 2.60 | 12.0 h | $3.98 | |
| Stage 2B: Full-Token Dense Refine (Synth) | 0.00 | 1.50M | 44.4 img/s | 59.8% / 69.8% | 2.71 | 9.4 h | $3.10 | |
| Stage 3: Real-Data SFT Refinement (Real) | 0.00 | 5.16M | 42.1 img/s | 56.7% / 66.2% | 9.32 | 34.0 h | $11.22 | |
| Cumulative Pixel DiT Adaptation Total (Stages 1–3) | 9.66M | 46.5 img/s | 57.8% / 67.5% | 15.15 | 57.65 h | $19.04 | ||
| Cumulative Grand Total (Including Offline Synth Dataset Generation) | – | – | – | 15.15 | 72.65 h | $25.04 | ||
| Progressive Token Compression via Large Patch Adaptation () | ||||||||
| ps32-adapt16 (Large Patch Engine) | 0.00 | 9.92M | 38.3 img/s | 54.2% / 63.3% | 17.92 | 72.0 h |
Table 6. Latent-to-Pixel adaptation progression and compute metrics of Asuka-FM on a single consumer NVIDIA RTX 4090 GPU (24GB). Adapts the converged 512px latent foundation model to direct RGB pixel space using a 10.09M convolutional Patch Detailer Head (PDH) under native -prediction at effective batch size 128. Hardware costs evaluated at /GPU-hour for RTX 4090 and /GPU-hour for A40 (Phase 0). Unabridged telemetry and hyperparameter matrices are detailed in Table 13 in Appendix § 12.
Phase 0 (Offline Synthetic Dataset Generation): Using our converged Asuka-512 latent foundation checkpoint, we generate a support corpus of 100,000 synthetic images across diverse prompts. On an NVIDIA A40 GPU, this offline generation phase completed in 15.0 hours at a sustained rate of 1.85 img/s ($6.00 compute cost).
Stage 1 (Decoupled PDH Warmup): To suppress initial gradient shocks from randomly initialized convolutional filters, the 206M DiT backbone is frozen. Only the input patch embedder (
x_embedder) and the PDH convolutional U-Net are trained on 100k synthetic images for 5 epochs (0.50M samples total) under 75% SPRINT token dropping (, peak lr with 25% linear warmup). This phase converged in 2.25 hours (2h 14m 45s) on an RTX 4090, sustaining 61.8 img/s (MFU, $0.74 cost).
Stage 2A (Joint Synthetic SPRINT Adaptation): The full network is unfrozen. Training proceeds for 25 epochs (2.50M samples) on synthetic data under token dropping (DiT backbone learning-rate multiplier 1.25×, base lr with 10% warmup), completing in 12.0 hours (12h 02m 42s) at 57.7 img/s ($3.98 cost).
Stage 2B (Full-Token Dense Refinement): The network switches to dense tokens (, micro-batch size 8, effective batch size 128) for 15 epochs (1.50M samples) with a 10% WSD linear decay schedule. This phase converged in 9.4 hours (9h 23m 31s) at 44.4 img/s ($3.10 cost).
Stage 3 (Real-Data SFT Refinement): Synthetic data is discarded. The model is fine-tuned strictly on full tokens () over a curated high-aesthetic real corpus of 396,635 samples (px) extracted from the i1 dataset. Training for 13 epochs (5.16M samples) completed in 34.0 hours on a single RTX 4090 at 42.1 img/s ($11.22 cost).
Cumulatively, the three DiT pixel adaptation stages executed 15.15 EFLOPs over 57.65 wall-clock hours ($19.04 compute cost) at an average sustained utilization of . Including offline synthetic dataset generation, the entire latent-to-pixel adaptation pipeline converges in 72.65 GPU hours ($25.04).
Empirical Analysis of Latent-to-Pixel Mechanics
Quantitative Convergence and Benchmark Recovery. As documented in Table 7, the final converged Pixel DiT (, Stage 3 SFT) achieves 75.31 DPG-Bench and 0.568 GenEval, recovering over 95% of the generative compositional fidelity of the source latent foundation model (78.51 DPG / 0.609 GenEval) while permanently bypassing the autoencoder decoding wall.
| Configuration / Stage | Patch Size () | Target Objective | Training Schedule | GenEval Total | DPG-Bench Total | Sampling Latency / Img (20-step / 30-step) |
| Baseline Latent Foundation Model (Pretraining Anchor) | ||||||
| Asuka-512 (Base Latent) | -pred | 139 Epochs (81.0h) | 0.609 | 78.51 | 0.250 s / 0.374 s | |
| Latent-to-Pixel Adaptation Progression (DiP Architecture, Zero VAE Decode) | ||||||
| Stage 2A: Joint SPRINT Sparse () | -pred | 25 Epochs (12.0h) | 0.465 | 69.63 | 0.329 s / 0.493 s | |
| Stage 2B: Full-Token Refine (Synth, -pred) | -pred | 15 Epochs (9.4h) | 0.505 | 72.35 | 0.329 s / 0.493 s | |
| Stage 2B: Full-Token Refine (Synth, -pred) | -pred | 15 Epochs (9.4h) | 0.477 | 71.49 | 0.329 s / 0.493 s | |
| Stage 3 SFT (-pred Ablation) | -pred | 13 Epochs (34.0h) | 0.545 | 76.32 | 0.329 s / 0.493 s | |
| Stage 3 SFT (-pred, Final Pixel DiT) | -pred | 13 Epochs (34.0h) | 0.568 | 75.31 | 0.329 s / 0.493 s | |
| Progressive Token Compression via Large Patch Adaptation () | ||||||
| ps32-adapt16 (Large Patch Pixel Engine) | -pred | +25 Epochs (72.0h) | 0.504 | 69.87 | 0.187 s / 0.277 s |
Table 7. Quantitative benchmark summary of Latent-to-Pixel adaptation on GenEval and DPG-Bench. Evaluated at canvas resolution using a 100-step Euler ODE solver with CFG scale . Sampling throughput and per-image latency profiled on a single NVIDIA A40 GPU at batch size 32. Full task-by-task GenEval breakdowns are provided in Table 14 in Appendix § 12.
-Prediction vs. -Prediction. To rigorously evaluate prediction objectives under identical training compute, we benchmarked -prediction against -prediction across Stages 2B and 3. As detailed in Table 7, while -prediction reached a marginally higher overall DPG score (76.32 vs. 75.31), -prediction delivered strictly superior performance on GenEval (0.568 vs. 0.545) and fine-grained color attribution (0.420 vs. 0.348; Table 14).
Table 14: Comprehensive benchmark evaluation of latent-to-pixel adaptation configurations on GenEval and DPG-Bench across training stages. All models evaluate at canvas resolution using a 100-step Euler ODE solver with CFG scale . Sampling throughput and per-image latency profiled on a single NVIDIA A40 GPU at batch size 32. Intermediate stages demonstrate that -prediction achieves faster initial convergence and prevents color desaturation.
| Configuration / Stage | Patch Size () | Target Obj. | Steps / Epochs | Single | Two-Obj | Count | Colors | Position | Color-Attr | GenEval Overall | DPG-Bench Total | Latency / Img (20-step / 30-step) |
| Baseline Latent Foundation Model (Pretraining Anchor) | ||||||||||||
| Asuka-512 (Base Latent) | v-pred | 139 Ep | 0.950 | 0.811 | 0.347 | 0.817 | 0.278 | 0.450 | 0.609 | 78.51 | 0.250 s / 0.374 s | |
| Stage 2: Synthetic Data SPRINT Adaptation Progression (100k Synthetic Images) | ||||||||||||
| Stage 2A: SPRINT Sparse () | v-pred | 25 Ep (12.0h) | 0.928 | 0.543 | 0.259 | 0.662 | 0.183 | 0.215 | 0.465 | 69.63 | 0.329 s / 0.493 s | |
| Stage 2B: Dense Refine () | -pred | 15 Ep (9.4h) | 0.928 | 0.586 | 0.291 | 0.628 | 0.195 | 0.233 | 0.477 | 71.49 | 0.329 s / 0.493 s | |
| Stage 2B: Dense Refine () | -pred | 15 Ep (9.4h) | 0.941 | 0.614 | 0.284 | 0.726 | 0.193 | 0.273 | 0.505 | 72.35 | 0.329 s / 0.493 s | |
| Stage 3: Real-Data SFT Convergence (396k Curated Images, ) | ||||||||||||
| Stage 3 SFT (-pred Ablation) | -pred | 70k (13 Ep) | 0.950 | 0.699 | 0.297 | 0.707 | 0.270 | 0.348 | 0.545 | 76.32 | 0.329 s / 0.493 s | |
| Stage 3 SFT (v-pred, Final Ours) | v-pred | 70k (13 Ep) | 0.950 | 0.780 | 0.335 | 0.750 | 0.285 | 0.420 | 0.568 | 75.31 | 0.329 s / 0.493 s | |
| Progressive Token Compression via Large Patch Adaptation () | ||||||||||||
| ps32-adapt16 (Large Patch) | v-pred | +75k (25 Ep) | 0.938 | 0.576 | 0.325 | 0.739 | 0.195 | 0.250 | 0.504 | 69.87 | 0.187 s / 0.277 s |
Table 14.
Critically, converting predicted clean states into velocity targets introduces a fatal singularity as in bfloat16, requiring dynamic percentile clipping that flattens dynamic range and induces chromatic blowout. In contrast, native -prediction maintained smooth optimization trajectories with zero activation clipping, completely avoiding color desaturation.
Progressive Token Compression (). To accelerate multi-step pixel inference, we evaluated progressive patch widening to (4 token compression, sequence length at ; 25 epochs, 72.0h). While compositional binding declines under the larger patch stride (DPG-Bench drops to 69.87, GenEval to 0.504), sampling latency at on an NVIDIA A40 drops to 0.187 s (20 steps) and 0.277 s (30 steps, a 1.78 speedup over ), establishing an ultra-fast sampling engine for compute-critical applications.
Inference Profiling: The Amortization Asymmetry
A common assumption in pixel diffusion literature [10] is that eliminating the VAE decoder accelerates inference. Profiling across sampling steps on an NVIDIA A40 (Table 11) reveals an Amortization Asymmetry: for standard multi-step sampling (), pixel diffusion is actually 9%–31% slower than latent diffusion. At (batch size ), generating 30-step trajectories requires 46.43 s in latent mode versus 53.28 s in pixel mode (14.8% slower).
| Resolution | Mode | Steps | Total Time | Latency / Image | Step Cost () | Fixed Overhead () | Peak VRAM | Rate |
| Latent + VAE | 30 | 46.43 s | 2.902 s / img | 1.387 s / step | 4.82 s (VAE Decode) | 6.75 GB | 0.34 img/s | |
| Latent + VAE | 20 | 32.56 s | 2.035 s / img | 1.387 s / step | 4.82 s (VAE Decode) | 6.75 GB | 0.49 img/s | |
| Pixel (DiP) | 30 | 53.28 s | 3.330 s / img | 1.773 s / step | 0.09 s (Zero VAE) | 13.22 GB | 0.30 img/s | |
| Pixel (DiP) | 20 | 35.55 s | 2.222 s / img | 1.773 s / step | 0.09 s (Zero VAE) | 13.22 GB | 0.45 img/s | |
| Latent + VAE | 30 | 5.99 s | 0.374 s / img | 0.181 s / step | 0.56 s (VAE Decode) | 2.78 GB | 2.67 img/s | |
| Pixel (DiP) | 30 | 7.89 s | 0.493 s / img | 0.260 s / step | 0.09 s (Zero VAE) | 4.40 GB | 2.03 img/s |
Table 11. Sampling execution profile across latent and pixel-space adaptation on NVIDIA A40 ( samples, TorchInductor compilation, FlashAttention-2, bfloat16). Latent mode evaluates 206M DiT followed by a terminal VAE decode. Pixel mode evaluates 216M DiT + PDH without an autoencoder.
Deconstructing the Asymmetry. Total sampling latency over ODE steps decomposes into per-step evaluation latency and fixed terminal decoding overhead : . In latent sampling, terminal VAE decoding incurs a one-time fixed penalty ( s at ), while per-step latency remains low ( s per batch step). In pixel sampling, while the VAE overhead is eliminated ( s ), the convolutional PDH head must evaluate recurrently at every step (and twice per step under CFG), increasing per-step latency to s (+27.8%). Across 30 steps, this recurrent overhead accumulates to 11.58 s, completely eclipsing the 4.82 s VAE decode savings.
Crossover Dynamics and Operational Regimes. Equating both linear trajectories yields an empirical break-even crossover threshold of sampling steps (formal derivation in Appendix § 10.§ 4), cleanly partitioning operational utility (Figure 6). For multi-step regimes ( steps), latent sampling is strictly faster because the VAE decode is amortized across many steps. Conversely, in few-step regimes ( steps), pixel diffusion is decisively superior: under 4-step distilled sampling (DMD2 [48]), pixel latency is 7.18 s (0.449 s/img) versus 10.37 s (0.648 s/img) for latent mode (a 1.44 speedup). At step, pixel mode achieves a 3.34 acceleration, establishing pixel models as natural engines for distilled few-step rollout.
Figure 6. Sampling latency trade-off at 1MP: Latent DiT vs. DiP Pixel Engine on NVIDIA A40. Pixel sampling is faster in few-step regimes (, 3.34× speedup at 1 step), while latent sampling is faster for multi-step ancestral sampling ().
Manifold Support: Square Synthetics vs. Multi-Aspect Corpora
An empirical discovery is the divergence in camera controllability across training distributions. Models trained on synthetic open-domain datasets (e.g., FLUX-Reason-6M, Midjourney-v6) exhibit geometric warping and background collapse on some prompts during coordinate manipulation. Synthetic datasets are mainly generated as centered squares. Extracting shifted square crops from square-native images forcibly clips through canonical focal points, teaching the model that off-center coordinate queries correlate with truncated subjects.
In contrast, our curated Danbooru corpus naturally spans non-square aspect ratios (1 : 2 portrait pin-ups to 21 : 9 panoramic spreads). Performing shifted cropping on raw rectangular illustrations forces the continuous RoPE manifold to encounter the entire coordinate spectrum over diverse, naturally framed focal points and perspective depths. Consequently, the anime-specialized model establishes genuine spatial invariance, unlocking robust zero-shot virtual camera manipulation without auxiliary conditioning tokens.
Discussion, Limitations, and Future Work
Limitations. While Asuka-FM achieves competitive synthesis on a single consumer GPU, several limitations remain: (i) Pretraining was conducted on 625,000 samples; multi-billion parameter models trained on hundreds of millions of images remain superior on complex crowd scenes and long-tail concepts. (ii) Although our corpus includes TextAtlas5M [41], the 206M backbone does not consistently render multi-word text strings. (iii) In pixel space, multi-step sampling ( steps) without patch enlargement () exhibits higher latency than latent diffusion due to recurrent PDH evaluation.
Future Work. Immediate research will expand along two directions: (i) Native 1MP Scaling Curriculum: Building on our preliminary 1MP profiling (Appendix § 16). (ii) Zero-Decode Downstream Post-Training (Flow-GRPO): In Flow-GRPO reinforcement learning [28], generating rollouts per prompt requires 12 independent VAE decodes in latent mode ( of GPU memory). By emitting direct RGB tensors, our pixel model eliminates this decoding bottleneck, enabling scalable online RL on consumer workstations.
Conclusion
Asuka-FM demonstrates that high-resolution text-to-image foundation models do not fundamentally require datacenter-scale compute clusters. By synthesizing an AdaLN-free dual-stream backbone with prepended temporal conditioning, structured sequence sparsification via SPRINT, continuous aspect-normalized RoPE with shifted square crop training, and decoupled latent-to-pixel adaptation via a lightweight Patch Detailer Head, Asuka-FM successfully trains a 206M latent generative foundation model from scratch on a single consumer NVIDIA RTX 4090 GPU in 161.2 wall-clock hours, executing 65.65 ExaFLOPs for only $53.2 in compute.
Operating on the Pareto frontier of denoiser compute (0.0657 ZFLOPs), Asuka-FM achieves a GenEval score of 0.609 and DPG-Bench score of 78.51 at 512px without benchmark-specific fine-tuning. Micro-architectural profiling reveals an Amortization Asymmetry: while pixel adaptation eliminates the online VAE wall during pretraining and delivers up to speedups in few-step regimes ( sampling steps), latent sampling remains faster for multi-step trajectories ( steps). Asuka-FM provides an empirical and systems foundation for accessible, low-budget generative foundation modeling across academic and independent research laboratories.
Acknowledgments
We gratefully acknowledge the Department of Signal Theory and Communications at Universidad Carlos III de Madrid (UC3M) for providing the dedicated GPU workstations and computing infrastructure that made these extensive pretraining experiments possible.
Víctor Rodrigo González and Alejandro Lancho have received funding from the Ministerio de Ciencia, Innovación y Universidades, Spain, under Grant Agreement No. PID2023-148856OA-I00 (funded by MICIU/AEI/10.13039/501100011033 and by ERDF/UE). Alejandro Lancho and Pablo M. Olmos have received funding from the Comunidad de Madrid through the IDEA-CM project (Grant Agreement No. TEC-2024/COM-89).
Additionally, Alejandro Lancho has received funding from the Comunidad de Madrid through the CARE-CM project (Grant Agreement No. SYG-2024/COM-870). Ignacio Peis acknowledges support from the Danish Data Science Academy, which is funded by the Novo Nordisk Foundation (NNF21SA0069429), and from the “la Caixa” Foundation (ID 100010434) through the Junior Leader Incoming fellowship LCF/BQ/PI26/12120010. Pablo M. Olmos has additionally been supported by the Ministerio de Ciencia, Innovación y Universidades, Spain, under Grant PID2024-157856NB-I00 (CARTESIAN), funded by MCIN/AEI/10.13039/501100011033 and, as appropriate, by “ERDF A way of making Europe”; by the ELLIS Unit Madrid (European Laboratory for Learning and Intelligent Systems); and by the Comunidad de Madrid under grant IND2024/TIC-34728.
Our work stands on the shoulders of the open machine learning research community. In particular, we express our sincere appreciation to the authors of:
HDM ([47]), for pioneering consumer-scale anime foundation pretraining and demonstrating the viability of shifted square cropping and continuous RoPE.
SPRINT ([33]), for the sparse-dense residual fusion paradigm that fundamentally slashed our attention sequence complexity.
i1 ([49]), for establishing an exemplary open recipe for multi-modal diffusion transformers and demonstrating that explicit AdaLN modulation can be omitted, and for publicly releasing the synthetic captions used for training.
DiP ([10]) and the empirical pixel diffusion studies by [11, 24], for illuminating the dynamics of the Patch Detailer Head and pixel-space velocity trajectories.
We note that while we did not have direct communication or collaboration with the authors of these prior works, our system-architectural framework was made possible through their transparent public papers, released checkpoints, and open codebases.
We also extend our gratitude to the broader anime and open-source community, particularly the crowdsourced annotators and maintainers of the Danbooru platform [2] whose decades of detailed image tagging provided the rich semantic metadata necessary for concept learning, as well as the maintainers of bitsandbytes, FlashAttention [13], and PyTorch.
Finally, the project Asuka-FM is dedicated to Asuka Soryuu Langley from Neon Genesis Evangelion, whose unyielding determination inspired our resolve through countless errors and pretraining milestones.
†MFU evaluates actual tensor ops ( per sample, for active parameter count and tokens per sample ) under SwiGLU recomputation; MFU denotes standard theoretical utilization ().
∗1 EFLOP = FLOPs = PFLOPs = ZFLOPs. Cumulative theoretical algorithmic compute is 56.27 EFLOPs.
‡One-time offline encoding using FLUX.2 VAE (, ) permanently removing autoencoder overhead from generative pretraining.
Extended Benchmark Breakdowns and Systems Telemetry
This section provides the complete, unabridged numerical tables corresponding to the condensed performance summaries in the main text.
Full Pretraining Lifecycle and Token Telemetry
Table 8 provides the unabridged pretraining lifecycle metrics of Asuka-FM across both square training resolutions ( and ) and SPRINT token drop ratios , detailing per-stage batch sizes, epoch allocations, cumulative token sequences, and physical hardware throughputs. As discussed in § 5, the token processing rate demonstrates the arithmetic efficiency of SPRINT: by processing only 25% of visual tokens in the middle blocks (), the system achieves 99.2k tok/s in Phase 1A and 78.1k tok/s in Phase 2A, substantially mitigating memory bandwidth saturation on consumer-grade GDDR6X buses.
Fine-Grained GenEval Task Breakdown
Table 9 presents the complete compositional task breakdown on GenEval [21] alongside DPG-Bench [23]. The individual columns evaluate basic object synthesis (Single Object), relational spatial semantics (Two Objects, Position), numerical consistency (Counting), and chromatic attribute binding (Colors, Color Attribution).
Crucially, the breakdown contextualizes the dynamics of SPRINT during pretraining. In Phase 2A (75% token drop), the model maintains strong single-object recognition (0.963) and two-object layout (0.639), but suffers a temporary drop in color-attribute binding (0.163). Once terminal dense refinement is initiated in Stage 2B, full spatial cross-attention across all 1,024 tokens recovers color attribution to 0.450, directly proving that macro-semantics can be learned sparsely while fine attribute binding requires dense token interaction.
Full Hardware Profiling and Micro-Architectural Telemetry
Table 10 details the step latency, VRAM allocation, and tensor core utilization—reported both for the isolated DiT backbone (‘Iso.’) and end-to-end training step (‘Sys’)—across the RTX 4090 and NVIDIA A40 GPUs, retaining the explicit token sequence dimensions per step.
This full view illustrates the hardware architectural difference: at (16,384 tokens/step), the RTX 4090’s large 72MB L2 cache keeps the entire fused Inductor kernel resident on-chip, sustaining
84.22 TFLOPS and cutting total step latency to 308.27 ms. In contrast, the A40 overflows its 6MB L2 cache, incurring frequent spills to GDDR6 DRAM (696 GB/s) and doubling total step time to 607.65 ms. Figure 7 further contrasts In-RAM latent caching at against online VAE encoding at and , highlighting why high-resolution online autoencoding stalls the generative backbone.
Analytical Derivation of the Amortization Asymmetry
To formally model the sampling latency trade-off between latent diffusion (with a terminal autoencoder) and pixel diffusion (with a per-step convolutional detailer head), let denote the number of ODE sampling steps, the per-step latent DiT evaluation latency, the one-time terminal VAE decoding latency, the marginal per-step latency added by evaluating the Patch Detailer Head (PDH), and the negligible fixed pixel tensor formatting overhead. Total generation latencies and for a batch of images over sampling steps are then expressed as:
Defining the per-step pixel evaluation latency as , setting yields the analytical break-even step threshold :
Substituting the empirical latency parameters benchmarked on an NVIDIA A40 at resolution with batch size (Table 11):
Evaluating Eq. (13) yields:
When , the accumulated per-step penalty exceeds the one-time VAE decoding cost , proving that pixel-space diffusion with recurrent convolutional heads is mathematically sub-optimal for multi-step trajectories without architectural modifications such as patch enlargement (effective pixel patch size ).
Toy-Diffusion Architectural Ablation Study
Prior to full-scale foundation pretraining on the 625K dataset, we established the controlled toy-diffusion benchmark on 87,000 image crops (256 256) from the Anime Faces dataset7 to isolate structural components (Table 12).8
| Backbone / Architecture Variant | Params | BS | Epochs | FID |
| Part I: Latent Architecture & Conditioning Ablations | ||||
| Single-Stream DiT (, ) | 218M | 256 | 600 | 3.89 |
| Efficient U-Net Baseline | 185M | 256 | 400 | 3.45 |
| Dual-Stream DiT + MLP Text Adapter | 200M | 256 | 600 | 3.42 |
| Dual-Stream DiT + LLM (Frozen LFM2.5) | 206M | 512 | 1000 | 3.12 |
| Dual-Stream Baseline (Constant LR) | 206M | 512 | 600 | 3.06 |
| Dual-Stream Baseline (Large Batch) | 206M | 1024 | 1400 | 2.42 |
| Dual-Stream Wide (, ) | 352M | 512 | 750 | 2.85 |
| SPRINT Dual-Stream (, Sparse Only) | 206M | 256 | 700 | 5.56 |
| SPRINT Dual-Stream ( Dense FT) | 206M | 256 | 1000 | 3.96 |
| Part II: Latent-to-Pixel DiP Adaptation (Toy Scale) | ||||
| Stage 1: PDH Only (Frozen DiT Backbone) | +10M | 256 | 15 | 28.40 |
| Stage 2: Joint FT (, Hard Clamp) | 216M | 256 | 50 | 19.85 |
| Stage 2: Joint FT (, Dynamic Clip) | 216M | 256 | 20 | 18.20 |
| Stage 2: Joint FT (, Dynamic Clip) | 216M | 256 | 50 | 16.00 |
| Stage 2: Joint FT (, Dynamic Clip) | 216M | 256 | 85 | 14.39 |
| Stage 2: Staged Patch Widening () | 216M | 256 | 50 | 33.80 |
Table 12. Architectural and conditioning ablations on the toy-diffusion framework (evaluated on 87,000 facial portraits at , patch size ).
Dual-Stream vs. Single-Stream Efficiency. Dual-stream cross-attention decisively outperformed single-stream concatenation. Concatenating disparate text and visual embeddings into shared attention matrices induced modality interference, capping convergence at FID = 3.89. Decoupling modalities into dedicated projections achieved higher sustained hardware MFU (70.4% vs. 49.8%) and improved distributional fidelity to FID = 3.06. Furthermore, eliminating per-block AdaLN modulation in favor of prepending continuous sinusoidal time tokens directly to the context stream reduced backbone parameters by 28% without degrading generative quality.
Bidirectional Text Adapters. Replacing a static linear/MLP projection with a 2-layer bidirectional Transformer Text Adapter (6.09M parameters) reduced FID from 3.42 to 3.12. The adapter aligns unidirectional causal representations emitted by the frozen LFM2.5-230M [1] LLM into an expressive spatial conditioning subspace.
SPRINT Token-Dropping Dynamics. Under structured patch dropping (), SPRINT reduced training iteration latency by 52%. While sparse pretraining alone plateaued at FID = 5.56, executing a terminal dense refinement phase () dropped the metric to 3.96, matching dense baselines within 0.5 FID while saving over half the cumulative pretraining FLOPs (Figure 8).

Figure 8. Quantitative convergence on the toy-diffusion benchmark. Evaluation of SPRINT token drop schedules () on FID.
Parameter Cooldown via WSD Scheduling. Diffusion transformers are predominantly trained with static learning rates. We evaluated Warmup-Stable-Decay (WSD), transitioning from a flat learning rate to linear decay in the terminal 20% of epochs. For the backbone, parameter cooldown dropped FID from 3.27 to 2.85; on the canonical model, FID dropped from 3.27 to 3.06, proving that scheduled learning rate decay is essential for optimal flow trajectory convergence.
Comprehensive Latent-to-Pixel Adaptation Telemetry and Breakdowns
This section provides the complete, unabridged numerical tables corresponding to the condensed Latent-to-Pixel adaptation summaries in § 6.4.
Complete Pixel Adaptation Lifecycle and Hyperparameter Matrix
Table 13 details the complete optimization progression across all pixel adaptation stages, specifying the trained modules, effective pixel patch size , SPRINT token drop ratio , per-GPU micro-batch size (‘Micro-BS’), peak learning rate , warmup/decay schedule, epoch and sample counts, and physical silicon throughputs.
Table 13: Complete pretraining and adaptation progression for Latent-to-Pixel transfer at on NVIDIA RTX 4090 (24GB). Hardware evaluated at $0.33/GPU-hour ($0.40/h for A40 in Phase 0). Effective batch size is maintained at 128 across all stages using gradient accumulation.
| Stage / Phase | Modules Trained | Patch Size () | Drop () | Micro-BS (Per GPU) | Peak LR () | Warmup / Schedule | Epochs | Samples | Throughput (img/s) | MFU (%) Theo / Act | Wall Time | Cost |
| Phase 0: Synth Gen (A40) | – (Sampling) | – | – | 32 | – | – | – | 0.10M | 1.85 | – | 15.0 h | $6.00 |
| Stage 1: PDH Warmup | Embed + PDH | 16 | 0.75 | 16 | 25% Const | 5 | 0.50M | 61.8 | 62.4% / 72.8% | 2.25 h | $0.74 | |
| Stage 2A: Joint SPRINT | Full Model | 16 | 0.75 | 16 | 10% Const | 25 | 2.50M | 57.7 | 58.1% / 67.8% | 12.0 h | $3.98 | |
| Stage 2B: Dense Refine | Full Model | 16 | 0.00 | 8 | 10% WSD | 15 | 1.50M | 44.4 | 59.8% / 69.8% | 9.4 h | $3.10 | |
| Stage 3: Real Data SFT | Full Model | 16 | 0.00 | 8 | 10% WSD | 13 | 5.16M | 42.1 | 56.7% / 66.2% | 34.0 h | $11.22 | |
| Cumulative Pixel DiT Adaptation Total (Stages 1–3) | 58 | 9.66M | 46.5 | 57.8% / 67.5% | 57.65 h | $19.04 | ||||||
| Cumulative Grand Total (Including Offline Synth Generation) | – | – | – | – | 72.65 h | $25.04 | ||||||
| Progressive Token Compression via Large Patch Adaptation () | ||||||||||||
| ps32-adapt16 (Large Patch) | Full Model | 32 | 0.00 | 16 | 10% WSD | 25 | 9.92M | 38.3 | 54.2% / 63.3% | 72.0 h | $23.76 |
Table 13.
Fine-Grained GenEval and DPG-Bench Task Breakdowns
Table 14 isolates the specific compositional sub-metrics across training checkpoints, comparing native velocity regression (-prediction) against clean-data prediction (-prediction) under identical compute durations across Stage 2 (synthetic anchor adaptation) and Stage 3 (real-data SFT refinement).
Across Stage 2B, switching from sparse () to dense () tokens under -prediction improves two-object layout (0.543 0.614), color accuracy (0.662 0.726), and color attribution (0.215 0.273), consistently outperforming -prediction (0.505 vs. 0.477 overall GenEval). As illustrated qualitatively in Figure 9, -prediction requires dynamic percentile clipping to suppress singularities in bfloat16 when recovering velocities , which flattens dynamic range and desaturates colors, whereas native -prediction preserves vibrant chromatic fidelity with zero activation clipping.

(a) Stage 3 SFT (-Prediction Ablation, ): Converting into velocity targets triggers singularities in bfloat16, forcing dynamic percentile clipping that flattens dynamic range and desaturates colors.

(b) Stage 3 SFT (Native -Prediction, Final Pixel DiT): Direct velocity regression maintains smooth probability paths with zero activation clipping, recovering of the latent anchor while boosting Two-Object (0.780 vs. 0.699) and Color Attribution.
Figure 9. Qualitative and quantitative comparison of -prediction vs. native -prediction in Latent-to-Pixel adaptation under identical random seeds. Native -prediction eliminates dynamic clipping artifacts and color blowout.
Camera Dynamics, Spatial Disentanglement, and Failure Modes
Our continuous aspect-normalized RoPE coordinate manifold (Eq. (5)) and shifted square crop pretraining confer zero-shot virtual camera controllability at inference time without requiring auxiliary camera tokens, fine-tuning, or structural modifications.
Zero-Shot Virtual Viewport Manipulation
Applying affine transformations (Eq. (9)) directly to the continuous evaluation coordinate mesh () parameterized by normalized horizontal and vertical pan offsets () and an optical magnification factor enables three independent, continuous spatial degrees of freedom under identical random seeds:
Horizontal Panning (-Shift): Figure 10 and Figure 13 (Row 1) illustrate horizontal camera panning. Applying non-zero offsets translates the virtual viewport horizontally across the scene manifold relative to the centered baseline (), shifting subject framing and revealing peripheral background context while preserving fine anatomical structures, eye catchlights, and woven fabric textures without artifacting.

(a) (a)

(b) (b)

(c) (c)
Figure 10. Horizontal (-axis) viewport panning under identical noise seeds. Modulating the horizontal offset of the continuous coordinate mesh translates the virtual viewport across the manifold, shifting the dachshund horizontally relative to the centered baseline () while preserving knit cap details and snow texture geometry.

Figure 13. Zero-shot continuous virtual camera manipulation on anime illustrations under identical random seeds. Manipulating the continuous coordinate mesh allows continuous horizontal panning (Row 1), vertical elevation adjustment (Row 2), and optical zoom (Row 3) purely via position map manipulation, without retraining, camera adapters, or prompt modifications.
Vertical Panning (-Shift): Figure 11 and Figure 13 (Row 2) demonstrate vertical framing control. Injecting a positive offset () lowers the virtual camera viewpoint, grounding the

Figure 11. Vertical (-axis) viewport framing under identical noise seeds. Adjusting the vertical offset alters the virtual camera elevation: elevates the framing to resolve upper floral tiers and the ballroom chandelier, whereas lowers the viewpoint to ground the ornate metallic pedestal stand.
composition by resolving the ornate metallic pedestal stand, whereas a negative offset () elevates the viewpoint to capture the upper floral tiers and the ballroom chandelier.
Continuous Optical Zoom: Figure 12 and Figure 13 (Row 3) demonstrate continuous focal magnification. Compressing the evaluation grid via concentrates the DiT’s spatial attention on the central wreath cavity, magnifying individual porcelain teacup rims and petal folds without geometric distortion, while expands the field of view for wide-angle framing.

Figure 12. Continuous focal zoom scaling via RoPE coordinate grid compression. Rescaling the continuous evaluation mesh by optical magnification factor simulates continuous wide-angle expansion and focal zoom under identical latent noise initializations. At , transformer attention capacity concentrates on local topological features, resolving fine rose petal contours and porcelain teacup glazes while strictly maintaining global structural alignment.
Failure Modes and Generation Artifacts
While Asuka-FM achieves competitive open-domain synthesis (0.609 GenEval, 78.51 DPG-Bench) within a limited compute envelope ( GPU hours), the model exhibits localized failure modes when subjected to out-of-distribution prompts. Figure 14 provides an explicit comparative ablation.

Figure 14. Failure mode comparison under continuous RoPE zoom on a complex organic composition. (a) At canonical scale (), the 206M dual-stream backbone synthesizes a coherent scene with sharp lighting, structured perspective, and clean floral silhouettes. (b) Applying continuous RoPE focal zoom () compresses the evaluation grid, inducing: (i) painterly high-frequency texture smearing on the stone pathway, (ii) semantic attribute bleeding (pink pigment leaking into ground cobblestones), and (iii) spatial non-rigidity and warping along linear wooden bench slats.
A systematic analysis reveals that these artifacts are not structural flaws of the base generator, but rather emerge from the physical limits of continuous coordinate scaling:
High-Frequency Texture Smearing via Semantic Dilation. As demonstrated in Figure 14(a), the base model readily generates coherent cobblestones, realistic foliage distributions, and consistent depth-of-field. However, when focal zoom () is applied (Figure 14(b)), the coordinate grid contracts, forcing each patch of effective pixel stride to cover a smaller physical subregion of the scene. Because the compact 206M backbone lacks multi-scale texture detailers or sub-patch supervision, it cannot hallucinate micro-surface details (e.g., individual stone crevices or fine wood grain). The model consequently defaults to smooth velocity trajectories, producing the waxy, painterly swirls visible in the
foreground path.
2. Spatial Non-Rigidity via Rotary Phase Dampening. Recall that denotes the aspect-ratio width scale factor, the horizontal patch-grid count, and the spatial frequency for rotation channel . While the calibrated spectrum guarantees isotropic decay at unit scale (), compressing the coordinate mesh by zoom factor reduces the effective inter-patch phase difference to . This phase dampening slightly weakens relative positional discriminability between adjacent visual tokens in the spatial attention kernel (, Eq. (6)), resulting in geometric relaxation. While organic structures (foliage, clouds) tolerate this relaxation, man-made rigid objects (such as the linear wooden slats of the bench) exhibit noticeable curvature and structural warping.
3. Semantic Attribute Bleeding under Compressed Receptive Fields. In dense compositional scenes containing strong complementary colors (vibrant pink petals against neutral grey stones and brown wood), joint self-attention in the shallow encoder tends to disperse dominant color tokens. Under focal zoom, where coordinate density is artificially compressed, the effective receptive field of self-attention spans a broader semantic neighborhood in normalized space. This causes the pink floral pigment to bleed into adjacent latents, tinting the cobblestone path and staining the bench wood.
Positional Representation Robustness
During early exploratory pretraining on the 337K Danbooru corpus, Asuka-FM was initially optimized with discrete integer position IDs (, where is the image token count). Deviating from the square aspect ratio during inference exposed an architectural failure: because inter-patch indices outside the training grid are unseen, the model interpreted the expanded canvas as additional spatial instances, replicating heads and torsos (Figure 15a).
Rather than restarting pretraining from scratch, we investigated positional representation plasticity: whether a partially trained diffusion transformer could adapt mid-flight to continuous aspect-normalized RoPE without catastrophic forgetting of visual and character priors.

(a) Discrete RoPE Collapse: Unseen integer positions outside the square grid induce harmonic repetition, causing duplicated heads and fragmented torsos along the vertical canvas.

(b) Continuous Calibrated RoPE: Aspect-normalized continuous coordinates bound the manifold within , yielding single-subject anatomical consistency.
Figure 15. Ablation of RoPE positional formulations across non-square aspect ratios under identical random seeds. Switching to continuous aspect-normalized coordinates with frequency spectrum completely resolves anatomical tiling without requiring pretraining restarts.
As detailed in Table 15, the RoPE adaptation phase (Phase 1B of the swapped run) realigned the model to continuous coordinates within only 20 epochs (5.6 GPU hours). The attention query-key projection matrices smoothly adapted to continuous phase rotations without loss of text-image alignment or character identities, and subsequent dense fine-tuning (Phases 1C and 1D) completely eradicated replication artifacts across arbitrary aspect ratios (Figure 15b).
| Phase | Epochs | Drop | RoPE Mode | Samples | Patches | PFLOPs | Time |
| 1A: Base Sparse | 0–100 | 0.75 | Discrete | 33.70M | 8.63B | 7.2 | 28.0 h |
| 1B: RoPE Adaptation | 100–120 | 0.75 | Continuous | 6.74M | 1.73B | 1.4 | 5.6 h |
| 1C: Dense FT | 120–145 | 0.00 | Continuous | 8.43M | 2.16B | 4.2 | 10.0 h |
| 1D: Cosine Decay | 145–170 | 0.00 | Continuous | 8.43M | 2.16B | 4.2 | 10.0 h |
| Total Stage 1 (Swapped Run) | 57.30M | 14.68B | 17.0 | 53.6 h |
Table 15. Stage 1 RoPE swap progression on the 337K Danbooru corpus: Successful coordinate adaptation from discrete integer grid to continuous aspect-normalized RoPE within 20 epochs (≈6.74M samples) on a single RTX 4090.
Semantic Attribute Disentanglement
A hallmark of a well-conditioned multi-modal transformer is the mathematical decoupling of semantic attributes (e.g., color, apparel) from spatial and geometric priors (e.g., body pose, lighting, perspective). To evaluate whether Asuka-FM achieves clean semantic disentanglement, we sample images under identical standard Gaussian prior noise initializations while altering a single descriptive attribute in the prompt context. As demonstrated in Figure 16, toggling the tag from pink_hair to blue_hair selectively alters only the chromatic values of the hair strands. The complex finger pose, ribbon folds, facial morphology, and specular lighting remain invariant down to the pixel level. This confirms that the Dual-Stream DiT architecture effectively prevents cross-attention attribute leakage across distinct token subspaces.

Figure 16. (a) Prompt conditioning: pink_hair, cat_ears, maid_outfit (b) Prompt conditioning: blue_hair, cat_ears, maid_outfit. Strict semantic color disentanglement under identical latent initializations. Modifying exclusively the hair color prompt token updates the pigment channels without perturbing facial geometry, eye catchlights, clothing ruffles, hand gestures, or background sparkle lighting.
Visual Dynamics of SPRINT: Sparse Attenuation and Dense Recovery
To understand how sequence sparsification affects visual representation learning, we examine the qualitative representations produced during the 75% sparse phase (token drop ratio ) versus the final dense refinement phase ().
Structured Group-Wise vs. Uniform Random Token Dropping. Uniform random token dropping produces irregular spatial holes that destabilize continuous RoPE phase shifts, as adjacent patch distances vary stochastically across the sequence. For an input canvas of resolution and effective patch size , SPRINT avoids irregular holes by partitioning the patch grid into non-overlapping windows (), deterministically sampling active token per group. This preserves a regular, stride-2 downsampled grid that acts as a coherent, coarser canvas for the sparse deep middle path .

(a) SPRINT Sparse Phase (): Macro-composition, character silhouette, and color blocking converge rapidly, but high-frequency lineart, eye highlights, and fringe hair details exhibit transient attenuation.

(b) Dense Refined Phase (): Full token interaction restores high-frequency lineart, resolves complex eye iris stars, sharpens finger contours, and stabilizes intricate stage lighting.
Figure 17. Visual impact of SPRINT sequence sparsification during training under identical noise seeds. Sparse pretraining establishes robust macro-semantic layout at compute savings, after which dense refinement recovers razor-sharp textural boundaries.
High-Frequency Attenuation in Sparse Regimes. Because 75% of visual tokens are replaced with the learnable mask vector inside the middle blocks , the model relies on the shallow dense encoder features (produced by ) and the linear skip fusion (where denotes the mask-padded deep representation, and , are the fusion projection parameters) to reconstruct fine edges. As seen in Figure 17a, sparse checkpoints capture overall pose and vibrant coloring but lack micro-structural crispness. Transitioning to dense fine-tuning (Figure 17b) allows attention weights across all 1,024 patches (at ) to re-engage, completely recovering intricate lineart.
Multi-Character Scaling at 1MP Resolution. At native 1-megapixel resolution (1024 1024, total image token count ), 75% token dropping discards 3,072 patches per step. In complex multi-character compositions, this creates spatial blind spots wider than individual faces or hands. As illustrated in Figure 18(a), early sparse epochs on multi-character compositions induced anatomical fractures and color leakage across character boundaries. Executing subsequent dense fine-tuning (Phase 3B in Table 16) re-established global semantic coordination (Figure 18(b)), proving that dense refinement is mandatory when scaling sequence sparsification to megapixel regimes.

Figure 18. (a) 1MP SPRINT Sparse (, 3 Epochs): Spatial stride of 32px induces multi-character anatomical fracturing and blurred limb interactions. (b) 1MP Dense Finetuned (, 4 Epochs): Restores complete multi-subject anatomical separation, fine ribbon details, and environmental clarity.
| Stage / Phase | Res. () | Drop () | Epochs | Samples | Throughput (img/s) | MFU (%) Theo / Act† | Compute (EFLOPs)* | Wall Time | Cost ($) |
| Stage 1: Base Semantic Pretraining (256 × 256, Latent Patch Grid 16 × 16 = 256 Image Tokens) | |||||||||
| 1A: SPRINT Sparse | 0.75 | 100 | 33.70M | 334.3 | 69.1% / 80.6% | 13.42 | 28.0 h | $9.24 | |
| 1B: Dense Refine | 0.00 | 50 | 16.85M | 234.0 | 71.7% / 83.6% | 9.94 | 20.0 h | $6.60 | |
| 1C: WSD Cooldown | 0.00 | 25 | 8.43M | 234.0 | 71.7% / 83.6% | 4.97 | 10.0 h | $3.30 | |
| Stage 2: High-Resolution Adaptation (512 × 512, Latent Patch Grid 32 × 32 = 1,024 Image Tokens) | |||||||||
| 2A: High-Res Sparse | 0.75 | 25 | 8.43M | 70.9 | 34.3% / 40.0% | 7.84 | 33.0 h | $10.89 | |
| 2B: Dense Finetune | 0.00 | 10 | 3.37M | 72.0 | 63.4% / 74.0% | 5.72 | 13.0 h | $4.29 | |
| 2C: WSD Cooldown | 0.00 | 5 | 1.69M | 78.0 | 68.7% / 80.2% | 2.86 | 6.0 h | $1.98 | |
| Stage 3: Native Megapixel Scaling (1024 × 1024, Latent Patch Grid 64 × 64 = 4,096 Image Tokens) | |||||||||
| 3A: SPRINT Sparse | 0.75 | 3 | 1.01M | 7.80 | 12.4% / 14.5% | 3.10 | 36.0 h | $11.88 | |
| 3B: Dense Finetune | 0.00 | 4 | 1.35M | 7.07 | 22.5% / 26.2% | 8.26 | 53.0 h | $17.49 | |
| Cumulative Subtotal (Stages 1 + 2) | 215 | 72.46M | – | 58.7% / 68.4% | 44.77 | 110.0 h | $36.30 | ||
| Cumulative Grand Total (Stages 1 + 2 + 3) | 222 | 74.81M | – | 40.6% / 47.4% | 56.13 | 199.0 h | $65.67 |
Table 16. Pretraining progression and compute metrics of Asuka-FM on the curated anime dataset () on a single consumer NVIDIA RTX 4090 GPU (24GB). Formatted symmetrically with Table 2. Pretraining executed using an effective batch size of 256 for Stages 1 and 2 (micro-batch 64 with 4 gradient accumulation steps), and effective batch size 128 for Stage 3 (micro-batch 16 with 8 accumulation steps). Throughput, compute, and MFU are derived directly from per-stage epochs and wall-clock hours using the medium context tier (152 text + 1 time = 153 tokens). Hardware costs evaluated at $0.33/GPU-hour. †MFU evaluates actual tensor ops ( per sample, for active parameter count M and tokens per sample ) under SwiGLU recomputation; MFU denotes standard algorithmic utilization (). *1 EFLOP = FLOPs = PFLOPs = ZFLOPs. Cumulative theoretical algorithmic compute () is 38.37 EFLOPs (Stages 1–2) and 48.11 EFLOPs (Stages 1–3).
Curated Anime Dataset and Multi-Stage Pretraining Progression
To evaluate domain specialization, extreme aspect-ratio scaling, and continuous coordinate geometry, we established the curated-danbooru-20269 testbed. This dataset comprises high-density anime-style illustrations sampled from Danbooru2023 [2].
Table 16 documents the complete computational expenditure, physical throughput, and financial accounting across the three resolution tiers () under the same accounting as Table 2 (for active parameters and tokens per sample). Reflecting the bimodal prompt-length distribution of Danbooru tags, we evaluate sequence compute at the representative medium context tier (152 text + 1 time = 153 context tokens, vs. 228 tokens in the 625K corpus). Consequently, Stages 1 and 2B/2C achieve higher image throughput (234.0–334.3 img/s at and 72.0–78.0 img/s at dense) while sustaining 68.4% cumulative across Stages 1–2. Note that Stage 2A (70.9 img/s, ) did not realize the full SPRINT speedup achieved in the 625K pipeline (108.5 img/s) due to network speed during latent streaming from HuggingFace, while Stage 3 () drops to 14.5%–26.2% due to the online VAE bottleneck (Section G).
SwinV2 Prompt Upsampling and Dynamic Prefix Formatting. Because crowdsourced tags frequently suffer from omissions, every training image is tagged via a SwinV2-Transformer tagger [30], extracting fine-grained descriptors across apparel, hair styling, accessories, and atmospheric lighting. As introduced in § 5, tags are assembled into the canonical prefix format:
[character count], [character name], [series], [artist], [rating], [aesthetic tags], [year], [shuffled general tags]
General tags are shuffled per epoch to eliminate sequence order biases.
Tag-Rarity Loss Weighting. To prevent high-frequency tags (1girl, solo) from drowning out gradient signals for rare characters and artist signatures, we apply the inverse tag-frequency weighting of Eq. (10). Given the active prompt tag set of cardinality , dataset size , and corpus frequency of tag , we compute the per-sample prompt weight and scale the base flow-matching objective to obtain the weighted loss :
where represents the prompt tag set, , and is the corpus tag frequency.
Hierarchical Aesthetic Tiering. Samples are classified into four quality tiers based on community scores and favorite counts: masterpiece, good score, bad score, and worse score. To preserve distinctive low-visibility styles, curated artist works are capped against falling into lower aesthetic classes.
Native 1MP Scaling and Hardware Bottleneck Profiling
While prior evaluations of SPRINT were restricted to ImageNet-1K at , we investigated extending Asuka-FM directly to native 1-megapixel (, total image token count ) generation on single GPUs. Pretraining under 75% SPRINT token dropping (drop ratio , Phase 3A) followed by dense fine-tuning (, Phase 3B) revealed severe systems barriers:
The Online VAE Wall at Scale. At , storing uncompressed 32-channel float16 latents for hundreds of thousands of samples requires TB of high-speed NVMe storage, exceeding consumer workstation capacity. Consequently, latents must be encoded from raw pixels on-the-fly.
As detailed in Table 17, profiling on the NVIDIA A40 demonstrates that across both micro-batch regimes, the frozen in-place VAE forward pass consumes an overwhelming 72.5%–72.6% of total step latency (2,548.10 ms at micro-batch 16, 4,906.91 ms at micro-batch 32). In contrast, the entire 206M generative backbone forward-backward pass executes in only 888.96 ms at micro-batch 16 and 1,756.68 ms at micro-batch 32. Consistent with our RTX 4090 profiling at micro-batch 8 (§ 6.3, where the frozen VAE accounts for 62.0% of step latency and runs 2.42 slower than the DiT backbone), the frozen autoencoder on the A40 runs nearly 3 slower than the trainable generative backbone, causing tensor cores to idle for almost three-quarters of execution time and collapsing end-to-end system MFU to 3.66%–3.79%. This finding reinforces that consumer-scale generative modeling cannot rely on online autoencoding at high resolutions, providing decisive systems justification for our decoupled latent-to-pixel adaptation strategy.
| Pipeline Component | Micro-BS 16 (Accum 8) | Micro-BS 32 (Accum 4) | ||
| Latency (ms) | Share (%) | Latency (ms) | Share (%) | |
| Data Loading / IO (CPU) | 0.16 ms | 0.0% | 0.18 ms | 0.0% |
| Online Frozen VAE Encoding | 2,548.10 ms | 72.6% | 4,906.91 ms | 72.5% |
| Text Encoder Forward Pass | 31.84 ms | 0.9% | 63.23 ms | 0.9% |
| Backbone Forward Pass (Sparse) | 229.31 ms | 6.5% | 457.20 ms | 6.8% |
| Backbone Backward Pass (Sparse) | 659.65 ms | 18.8% | 1,299.48 ms | 19.2% |
| Optimizer Step & Norm Clip | 38.49 ms | 1.1% | 37.83 ms | 0.6% |
| Total Iteration Latency | 3,507.55 ms | 100.0% | 6,764.83 ms | 100.0% |
| Backbone Isolated Throughput | 46.42 TFLOPS (MFU = 28.10%) | 46.98 TFLOPS (MFU = 28.44%) | ||
| End-to-End System MFU | 3.66% | 3.79% |
Table 17. Execution profiling breakdown of Asuka-FM at native resolution on a single NVIDIA A40 GPU across variable micro-batch sizes (FLUX.2 VAE, tokens, TorchInductor compilation, bfloat16 precision).
The frozen autoencoder runs nearly 3 slower than the trainable generative backbone, causing tensor cores to idle for almost three-quarters of execution time and collapsing end-to-end system MFU to an unacceptable 3.66%–3.79%. This finding reinforces that consumer-scale generative modeling cannot rely on online autoencoding at high resolutions, providing decisive systems justification for our decoupled latent-to-pixel adaptation strategy.
Shallow Receptive-Field Starvation in 1MP SPRINT. SPRINT extracts dense spatial representations in the initial shallow encoder ( layers) before subsampling. At , two self-attention layers with effective patch size span a local receptive field covering 12.5% of the linear canvas span (capturing full facial landmarks). At , however, that same two-layer receptive field covers only 3.1% of the linear canvas span.
Subsampling 75% of tokens () immediately after Layer 2 forces the sparse middle blocks to route global semantic composition from ungrounded micro-edges with an effective spatial stride of 32 px. During Phase 3A, this receptive-field starvation induced anatomical fracturing in complex multi-character scenes (Figure 18(a)). While subsequent dense fine-tuning (Phase 3B, Figure 18(b)) restored anatomical coherence, this degradation demonstrates that sequence sparsification at 1MP could require a density curriculum (starting at ) or a deeper shallow encoder (). These empirical findings validate our design decision to focus consumer-scale latent-to-pixel adaptation within the regime, where the autoencoder throughput wall is permanently bypassed.
Curated Anime Qualitative Generation Gallery
In Figure 19, we provide high-resolution zero-shot samples synthesized by our domain-specialized 206M Asuka-FM model pretrained from scratch on the curated 337K Danbooru corpus on a single consumer RTX 4090 GPU, demonstrating sharp lineart, anatomical rendering, and character-concept binding.
References
- [1]Liquid AI. “LFM2.5-230M: Built to Run Anywhere”. In: Liquid AI Blog (2026). www.liquid.ai/blog/lfm2-5-230m (cit. on pp. 7, 27).
- [2]Anonymous and Danbooru Community. Danbooru2023: A large-scale crowdsourced & tagged anime illustration dataset. 2023. URL: https://huggingface.co/datasets/nyanko7/danbooru2023 (cit. on pp. 12, 23, 37).
- [3]Black Forest Labs. FLUX.1: Text-to-Image Generation Models. https://github.com/black-forest-labs/flux. 2024 (cit. on pp. 3, 4, 13, 25).
- [4]Black Forest Labs. FLUX.2: Analyzing and Enhancing the Latent Space of FLUX – Representation Comparison. 2025. URL: https://bfl.ai/research/representation-comparison (cit. on p. 3).
- [5]Hanqiu Li Cai and Chema Garabito. Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning. 2026. arXiv: 2610.09450 [cs.CV]. URL: https://arxiv.org/abs/2610.09450 (cit. on pp. 5, 11).
- [6]Chen Chen et al. DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture Design in Text to Image Generation. 2025. arXiv: 2503.10618 [cs.CV]. URL: https://arxiv.org/abs/2503.10618 (cit. on p. 4).
- [7]Jiuhai Chen et al. BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset. 2025. arXiv: 2505.09568 [cs.CV]. URL: https://arxiv.org/abs/2505.09568 (cit. on p. 14).
- [8]Shoufa Chen, Chongjian Ge, Shilong Zhang, Peize Sun, and Ping Luo. PixelFlow: Pixel-Space Generative Models with Flow. 2025. arXiv: 2504.07963 [cs.CV]. URL: https://arxiv.org/abs/2504.07963 (cit. on p. 5).
- [9]Xiaokang Chen et al. Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling. 2025. arXiv: 2501.17811 [cs.AI]. URL: https://arxiv.org/abs/2501.17811 (cit. on pp. 13, 25).
- [10]Zhennan Chen et al. DiP: Taming Diffusion Models in Pixel Space. 2026. arXiv: 2511.18822 [cs.CV]. URL: https://arxiv.org/abs/2511.18822 (cit. on pp. 4, 5, 11, 17, 23).
- [11]Zhennan Chen et al. L2P: Unlocking Latent Potential for Pixel Generation. 2026. arXiv: 2605.12013 [cs.CV]. URL: https://arxiv.org/abs/2605.12013 (cit. on p. 23).
- [12]Yong Xieng Cheng et al. Looped Diffusion Transformer. 2026. arXiv: 2609.40305 [cs.CV]. URL: https://arxiv.org/abs/2609.40305 (cit. on pp. 5, 13, 25).
- [13]Tri Dao. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. 2023. arXiv: 2307.08691 [cs.LG]. URL: https://arxiv.org/abs/2307.08691 (cit. on p. 23).
- [14]Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal Transformers. 2019. arXiv: 1807.03819 [cs.CL]. URL: https://arxiv.org/abs/1807.03819 (cit. on p. 5).
- [15]Ming Ding et al. CogView: Mastering Text-to-Image Generation via Transformers. 2021. arXiv: 2105.13290 [cs.CV]. URL: https://arxiv.org/abs/2105.13290 (cit. on pp. 4, 7).
- [16]Patrick Esser et al. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. 2024. arXiv: 2403.03206 [cs.CV]. URL: https://arxiv.org/abs/2403.03206 (cit. on pp. 1, 3, 4, 6, 13, 25).
- [17]Rongyao Fang et al. FLUX-Reason-6M and PRISM-Bench: A Million-Scale Text-to-Image Reasoning Dataset and Comprehensive Benchmark. 2025. arXiv: 2509.09680 [cs.CV]. URL: https://arxiv.org/abs/2509.09680 (cit. on p. 12).
- [18]Peng Gao et al. Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers. 2024. arXiv: 2405.05945 [cs.CV]. URL: https://arxiv.org/abs/2405.05945 (cit. on p. 4).
- [19]Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer. 2024. arXiv: 2303.14389 [cs.CV]. URL: https://arxiv.org/abs/2303.14389 (cit. on p. 5).
- [20]Jonas Geiping et al. Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. 2025. arXiv: 2502.05171 [cs.LG]. URL: https://arxiv.org/abs/2502.05171 (cit. on p. 5).
- [21]Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. “GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment”. In: Advances in Neural Information Processing Systems (NeurIPS). Vol. 36. 2023 (cit. on pp. 13, 24).arxiv.org/abs/2310.11513
- [22]Jonathan Ho, Ajay Jain, and Pieter Abbeel. “Denoising Diffusion Probabilistic Models”. In: Advances in Neural Information Processing Systems. Vol. 33. 2020, pp. 6840–6851. URL: https://proceedings.neurips.cc/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf (cit. on p. 5).DOI
- [23]Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment. 2024. arXiv: 2403.05135 [cs.CV]. URL: https://arxiv.org/abs/2403.05135 (cit. on pp. 13, 24).
- [24]Dengyang Jiang et al. An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models. 2026. arXiv: 2608.16887 [cs.CV]. URL: https://arxiv.org/abs/2608.16887 (cit. on pp. 5, 11, 23).
- [25]Felix Krause, Timy Phan, Ming Gui, Stefan Andreas Baumann, Vincent Tao Hu, and Björn Ommer. TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Training. 2025. arXiv: 2501.04765 [cs.CV]. URL: https://arxiv.org/abs/2501.04765 (cit. on p. 5).
- [26]Tianhong Li and Kaiming He. Back to Basics: Let Denoising Generative Models Denoise. 2026. arXiv: 2511.13720 [cs.CV]. URL: https://arxiv.org/abs/2511.13720 (cit. on pp. 5, 11).
- [27]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. “Flow Matching for Generative Modeling”. In: The Eleventh International Conference on Learning Representations. 2023. URL: https://openreview.net/forum?id=PqvMRDCJT9t (cit. on pp. 3, 5).DOI
- [28]Jie Liu et al. Flow-GRPO: Training Flow Matching Models via Online RL. 2025. arXiv: 2505.05470 [cs.CV]. URL: https://arxiv.org/abs/2505.05470 (cit. on pp. 3, 11, 18).
- [29]Xingchao Liu, Chengyue Gong, and Qiang Liu. “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow”. In: The Eleventh International Conference on Learning Representations. 2023. URL: https://openreview.net/forum?id=XVjTT1nw5z (cit. on p. 5).arxiv.org/abs/2209.03003
- [30]Ze Liu et al. “Swin Transformer V2: Scaling Up Capacity and Resolution”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022, pp. 12009–12019 (cit. on pp. 12, 37).arxiv.org/abs/2111.09883
- [31]Zehong Ma, Longhui Wei, Shuai Wang, Shiliang Zhang, and Qi Tian. “DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2026 (cit. on pp. 13, 25).arxiv.org/abs/2511.19365
- [32]Zehong Ma, Ruihan Xu, and Shiliang Zhang. PixelGen: Improving Pixel Diffusion with Perceptual Supervision. 2026. arXiv: 2602.02493 [cs.CV]. URL: https://arxiv.org/abs/2602.02493 (cit. on pp. 13, 25).
- [33]Dogyun Park et al. Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers. 2026. arXiv: 2510.21986 [cs.CV]. URL: https://arxiv.org/abs/2510.21986 (cit. on pp. 3, 5, 7, 23).
- [34]William Peebles and Saining Xie. Scalable Diffusion Models with Transformers. 2023. arXiv: 2212.09748 [cs.CV]. URL: https://arxiv.org/abs/2212.09748 (cit. on pp. 3–5).
- [35]Dustin Podell et al. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. 2023. arXiv: 2307.01952 [cs.CV]. URL: https://arxiv.org/abs/2307.01952 (cit. on pp. 13, 25).
- [36]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. “High-Resolution Image Synthesis with Latent Diffusion Models”. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2022, pp. 10674–10685 (cit. on pp. 3, 5).arxiv.org/abs/2112.10752
- [37]Vikash Sehweg, Xianghao Kong, Jingtao Li, and Michael Spranger. “Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2025 (cit. on p. 5).
- [38]Noam Shazeer. GLU Variants Improve Transformer. 2020. arXiv: 2002.05202 [cs.LG]. URL: https://arxiv.org/abs/2002.05202 (cit. on pp. 4, 7).
- [39]Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. RoFormer: Enhanced Transformer with Rotary Position Embedding. 2023. arXiv: 2104.09864 [cs.CL]. URL: https://arxiv.org/abs/2104.09864 (cit. on pp. 6, 9).
- [40]Image Team et al. Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer. 2026. arXiv: 2511.22699 [cs.CV]. URL: https://arxiv.org/abs/2511.22699 (cit. on pp. 4, 13).
- [41]Alex Jinpeng Wang et al. TextAtlas5M: A Large-scale Dataset for Dense Text Image Generation. 2025. arXiv: 2502.07870 [cs.CV]. URL: https://arxiv.org/abs/2502.07870 (cit. on pp. 12, 18).
- [42]Shuai Wang, Ziteng Gao, Chenhui Zhu, Weilin Huang, and Limin Wang. “PixNerd: Pixel Neural Field Diffusion”. In: International Conference on Learning Representations (ICLR). 2026 (cit. on pp. 13, 25).arxiv.org/abs/2507.23268
- [43]Xianbang Wang, Hanhong Zhao, Yiyang Lu, Kangyang Zhou, Linrui Ma, and Kaiming He. MiniT2I: A Minimalist Baseline for Text-to-Image Generation. 2026. URL: https://peppaking8.github.io/minit2i/ (cit. on pp. 1, 13, 25).
- [44]Zhendong Wang et al. Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models. 2023. arXiv: 2304.12526 [cs.CV]. URL: https://arxiv.org/abs/2304.12526 (cit. on pp. 4, 6, 10).
- [45]Chenfei Wu et al. Qwen-Image Technical Report. 2025. arXiv: 2508.02324 [cs.CV]. URL: https://arxiv.org/abs/2508.02324 (cit. on p. 3).
- [46]Size Wu et al. OpenUni: A Simple Baseline for Unified Multimodal Understanding and Generation. 2025. arXiv: 2505.23661 [cs.CV]. URL: https://arxiv.org/abs/2505.23661 (cit. on p. 14).
- [47]Shih-Ying Yeh. Home-made Diffusion Model from Scratch to Hatch. 2025. arXiv: 2509.06068 [cs.CV]. URL: https://arxiv.org/abs/2509.06068 (cit. on pp. 4, 5, 8, 10, 23).


