PixelBench: Can LLMs Understand and Generate Images Pixel by Pixel?

Introduction

Large language models (LLMs) were originally developed to predict the next token in natural-language sequences. The same autoregressive training paradigm has since produced strong code-generation capabilities. Some generated code, including SVG code and CAD programs, can be rendered into images, providing an interface for probing how LLMs express visual information [49, 43, 11, 46, 64, 27]. However, these formats describe images using semantically meaningful, high-level primitives and are therefore one step away from actual visual appearance: a renderer expands each primitive into many pixels. By contrast, RGB rasters provide a more direct and ubiquitous representation of digital images, with each spatial location in a two-dimensional grid storing a color value. Most modern image generators map learned visual tokens or latent representations to pixels through dedicated decoders [6, 56]. Prior work has also trained specialized autoregressive image models to generate images directly in raster space (van den Oord et al., 2016a;b; Parmar et al., 2018; Chen et al., 2020). These studies demonstrate the feasibility of direct pixel generation, but rely on models specifically designed and trained for image modeling. To better investigate pixel-based visual understanding and generation, we ask the following question:

Can LLMs directly generate and understand visual content pixel by pixel?

To address this research question, we introduce PixelBench, a benchmark for evaluating LLMs on direct pixel generation and understanding. PixelBench uses a palette-indexed raster representation: the model first defines a palette of up to eight RGB colors and then generates a grid of palette indices, with each grid position specifying the color of one pixel. This interface needs no learned tokenizer, image decoder, or architectural change. PixelBench evaluates three tasks: text-to-pixel generation (T2P), where a model generates a recognizable pixel image from an object label, image-to-pixel generation (I2P), where the model reconstructs a given image in pixel format, and pixel understanding, where the model identifies the underlying concept from a textual palette-and-grid representation generated by either an algorithmic or a model-based pixel oracle.

Our evaluation shows that frontier LLMs already exhibit substantial pixel-generation capabilities, with leading models producing recognizable T2P outputs and competitive I2P reconstructions. Their behavior, however, depends strongly on target resolution. For many models, both visual quality and quantitative performance peak at an intermediate resolution, with 64 × 64 often performing worse than 32 × 32, while GPT-6 Astra exhibits notably better robustness. We therefore conduct a fine-grained resolution-scaling study and find that, except for Astra’s consistently strong performance, most evaluated models initially improve but eventually decline as resolution increases. Viewing resolution as a measure of visual abstraction, these model-specific performance peaks suggest that different models are most effective at different levels of abstraction. Pixel understanding exhibits a similar ranking across models and correlates strongly with generation performance, suggesting that the connection between visual understanding and generation persists for pixel-based interface.

The observed connection between pixel generation and visual understanding raises a natural question of whether reconstructing an input image can serve as a training objective to improve a model’s visual understanding. Reconstruction Alignment (RecA) [63] shows that reconstruction-aligned visual representations can improve image generation, but whether reconstruction can also enhance visual understanding remains underexplored. Motivated by this, we propose Pixel Reconstruction Tuning (PRT), which augments post-training with trajectories that describe an image, explain an automatically constructed color palette, and reconstruct the image as a pixel grid. In a controlled experiment with Molmo2-4B, we start from its pretraining checkpoint and post-train it on 500k LLaVA-OneVision-1.5 instructions mixed with PRT examples. Incorporating PRT data improves average performance across 16 visual-centric benchmarks. Ablations varying the number of PRT examples from 1k to 8k show consistent improvements across all tested data scales. A complementary study with Qwen3-VL further shows that Pixel data can regularize post-training when incorporated into a general instruction mixture. Our contributions are summarized below:

• A unified interface for evaluating pixel capabilities. We introduce PixelBench, a shared text-based pixel interface for evaluating frontier LLMs on both pixel generation and understanding.

• Capability limits at increasing resolutions. We characterize model-specific performance peaks and failure modes. GPT-6 Astra remains robust as resolution increases, while other evaluated models generally improve and then decline.

• Pixels as training supervision. We propose Pixel Reconstruction Tuning (PRT), which uses pixel reconstruction as a post-training objective. Starting from the Molmo2-4B pretraining checkpoint, incorporating PRT data into general instruction tuning consistently improves performance.

PixelBench: Evaluating Pixel Understanding and Generation

Pixels as a Visual Representation

Pixel art is a compact visual output format. Its restricted palette and raster structure can preserve recognizable colors and shapes while limiting the number of symbols that a model must generate. Related representations are commonly used in icons, pictograms, low-resolution displays, and digital graphics. In this work, pixel art refers specifically to a palette-indexed raster with two components: a palette of RGB colors and an $S \times S$ grid of palette indices. Each grid entry selects the color of one pixel. The representation is human-interpretable and deterministic to render, and can be expressed using the model’s existing text vocabulary. We serialize the palette and index grid as JSON. The palette field contains two to eight opaque RGB colors in #RRGGBB format, and the pixels field contains the $S$ rows of palette indices. Rendering maps each index to its corresponding RGB value. Figure 2 shows rendered examples and their palettes.

Benchmark Introduction and Design

We introduce PixelBench to probe how much image-generation capability a language model can express through its native token interface. Unlike SVG and CAD, which describe images through high-level geometric primitives, our palette-indexed grids specify raster values directly: each generated grid entry corresponds to one rendered pixel, and rendering requires only a deterministic palette lookup. PixelBench therefore evaluates visual generation through a representation closer to the native form of raster images, without relying on a learned image tokenizer or decoder.

§ 2.2 defines the generation and understanding tasks, models, baselines, and protocol; § 2.3 specifies the metrics; and § 2.4 presents the main results. We find that current models already exhibit substantial pixel-image generation ability through their text interface, while their pixel-understanding performance closely tracks their generation performance. Generation quality does not consistently improve with output resolution, as further analyzed in § 4.

Tasks and datasets. In text-to-pixel generation (T2P), a model maps an English object label to a recognizable icon without a reference image. We use 100 concrete concepts spanning five categories (Appendix § 15). In image-to-pixel generation (I2P), a vision encoder supplies a natural image and the model reconstructs it while preserving subject identity, pose, composition, and dominant colors. We sample one validation image from each of 100 ImageNet-1K classes ([17]) and center-crop it to $256 \times256$ (Appendix § 16); the category is withheld. Every item in both generation tasks is evaluated at $16^2$, $32^2$, and $64^2$, yielding 300 instances per task and model. In pixel understanding, a model receives only the textual serialization of an RGB palette and index grid and selects the concept from ten candidates. Inputs come from deterministic PIA-reduced reconstructions and GPT-6 Astra I2P generations (Astra-I2P), each with 100 prompts at each resolution.

Models and baselines. We evaluate Gemini 3.6 Flash ([22]), GPT-5.6 Sol ([38]), GPT-6 Astra ([39]), Claude Opus 5 ([4]), Seed-2.1 Pro ([5]), and Kimi K3 ([28]) on both generation tasks and pixel understanding in a zero-shot setting. DeepSeek V4 Pro ([15]) is included as a supplementary model for T2P and pixel understanding, but not for I2P generation. Detailed task-specific inference settings, including temperature, reasoning configuration, output-token limits, and samples per item, are reported in Appendix E. For I2P, we additionally compare against three representative palette-matched algorithms: nearest-neighbor sampling followed by median-cut quantization [25]; SSIM-optimal downscaling followed by median cut [41]; and a reduced Pixelated Image Abstraction pipeline [20] that combines SLIC regions [1] with an eight-color weighted $k$-means palette. The complete ten-method comparison and implementation details are in Appendix L.

Generation protocol. Both generation tasks use the palette-indexed representation in Figure 2: two to eight opaque RGB colors and an exactly sized index grid, with task-specific prompts. Successfully returned outputs with invalid JSON, palette references, or grid sizes receive zero; rate limits and token-limit truncations are retried, raising the limit when needed. Pixel understanding directly presents the serialized grid and ten choices, with no rendered image. Prompts are in Appendix G.

Evaluation Metrics

Text-to-Pixel metrics. Using the paired SigLIP-SO400M text and vision encoders [65], we retrieve text candidates represented as “a pixel art icon depicting {label}.” The main table reports 500-way Top-1 and a complementary VLM-as-judge score from Gemini 3.5 Flash [21], whose 0-100 rubric prompt is in Appendix G.2. The 100-way Top-1 and MRR diagnostics are in Appendix K; retrieval definitions and vocabulary are in Appendices F and J.

Image-to-Pixel metrics. We report 100-way Top-1 by embedding each generated output and the 100 candidate source images with the same SigLIP vision encoder. We additionally compute paired cosine similarity using pooled SigLIP features and DINOv2-Large CLS features [40], which measure semantic and appearance-level similarity to the source, respectively. For both T2P and I2P, invalid outputs count as retrieval misses or zero similarity. We also report the Valid and Flat rates. Valid is the percentage of outputs with well-formed JSON, valid palette references, and the required grid size; Flat is the percentage of outputs that are valid but render to a single color.

Pixel-understanding metric. We report 10-way recognition accuracy from the serialized palette-and-grid inputs. Each resolution contains 100 questions, so each source average covers 300 questions and the overall average covers all 600. Random-choice accuracy is 10%.

Experimental Results

Text-to-Pixel Generation. Table 1 summarizes the main results for T2P generation. GPT-6 Astra leads throughout: its 500-way Top-1 rises from 87% to 94% across resolutions and its Agg. score reaches 90.8. GPT-5.6 Sol and Claude Opus 5 follow at 81.0 and 79.4, with Kimi K3 at 76.0. The stricter judge spans 90.66–51.45 overall and leaves Astra 14.01 points ahead of Sol, providing clearer discrimination while preserving the same leader. Figure 3 and Appendix C further show that model-generated pixelizations can preserve semantic structure; together with Astra’s leading I2P Aggregated score, this motivates Astra-I2P as one understanding oracle. Full 100-way Top-1 and MRR results are in Appendix K. Most models decline from $32^{2}$ to $64^{2}$, motivating § 4.

Image-to-Pixel Generation. Table 2 compares six multimodal language models with three palette-matched algorithms. Five models are fully valid; Kimi K3 reaches 96.3%, and only Gemini 3.6 Flash produces flat fills (1.0%). GPT-5.6 Sol leads model Top-1 Avg (57.0%), while GPT-6 Astra leads model SigLIP (.654) and DINO (.175) averages. The best model exceeds all algorithms at $16^{2}$ and $32^{2}$, but algorithms lead at $64^{2}$: PIA reaches 78% Top-1 and .697 SigLIP, and SSIM-optimized pixelization reaches .543 DINO. Four models also decline from $32^{2}$ to $64^{2}$. GPT-6 Astra achieves the highest Aggregated score (80.45), ahead of PIA (77.50). See Appendix L for all algorithms.

Model / method	Valid	Flat	16	32	64	Avg	16	32	64	Avg	16	32	64	Avg	Agg.
Multimodal language models
GPT-6 Astra	100.0	0.0	18	50	76	48.0	.615	.652	.695	.654	.023	.177	.326	.175	80.45
GPT-5.6 Sol	100.0	0.0	44	61	63	56.0	.569	.597	.599	.588	.022	.112	.147	.094	57.27
Claude Opus 5	100.0	0.0	26	49	47	40.7	.535	.567	.582	.561	.021	.062	.098	.060	23.16
Kimi K3	96.3	0.0	34	46	30	36.7	.558	.562	.501	.541	.020	.064	.038	.040	8.48
Seed-2.1 Pro	100.0	0.0	33	42	30	35.0	.536	.553	.553	.547	.020	.056	.054	.043	8.52
Gemini 3.6 Flash	100.0	1.0	28	37	27	30.7	.535	.546	.536	.539	.018	.065	.070	.051	1.98
Algorithmic baselines
Nearest + median cut	100.0	0.0	9	28	62	33.0	.523	.568	.665	.585	.012	.090	.480	.194	44.11
SSIM-opt. + median cut	100.0	0.0	9	34	75	39.3	.524	.579	.686	.596	.013	.119	.543	.225	61.19
PIA (reduced)	100.0	0.0	18	50	78	48.7	.546	.608	.697	.617	.014	.135	.490	.213	77.50

Pixel Understanding

We construct understanding inputs from two complementary sources. PIA is deterministic but not uniformly optimal. GPT-6 Astra achieves the highest I2P Aggregated score (80.45), ahead of PIA (77.50), and at $32^{2}$ also leads PIA on SigLIP cosine (.652 vs. .608) and DINO cosine (.177 vs. .135). Figure 3 and Appendix § 10 further show cases in which model-generated outputs match or surpass deterministic pixelization, so we also use Astra-I2P as a model-generated oracle.

Model	Overall	16	32	64	Avg	16	32	64	Avg
	PIA source					Astra-I2P source
GPT-6 Astra	67.5	48	57	67	57.3	72	76	85	77.7
GPT-5.6 Sol	39.2	34	29	34	32.3	45	42	51	46.0
Claude Opus 5	35.3	22	26	24	24.0	40	50	50	46.7
Kimi K3	27.3	20	19	16	18.3	39	34	36	36.3
Gemini 3.6 Flash	21.8	21	11	17	16.3	28	24	30	27.3
Seed-2.1 Pro	16.0	15	14	12	13.7	20	15	20	18.3
DeepSeek V4 Pro	12.5	13	10	8	10.3	15	14	15	14.7

Table 3 reports recognition accuracy for both sources. GPT-6 Astra leads overall accuracy (67.5%), followed by GPT-5.6 Sol (39.2%). On PIA alone it already leads by a wide margin (57.3% vs. 32.3% for GPT-5.6 Sol), and the two sources largely preserve the same ranking. Switching to Astra-I2P raises every model’s source average, with Claude Opus 5 gaining more than Astra (+22.7 vs. +20.4), so the model-based oracle helps broadly rather than rewarding Astra for recognizing its own generations. Overall accuracy correlates strongly with generation performance: the Pearson correlation is 0.871 for T2P and 0.930 for I2P. Table 4 also reports Spearman and Kendall coefficients, which support the same strong relationship. The T2P comparison uses all evaluated models, whereas the I2P comparison uses those with available generation results. Since the comparison covers only a small set of models, we treat these correlations as descriptive rather than general and use them to motivate the training experiment in § 5.

Comparison	$n$	Pearson $r$	Spearman $\rho$	Kendall $\tau$
Understanding–T2P	7	.871	1.000	1.000
Understanding–I2P (w/o DeepSeek)	6	.930	.829	.733

Resolution Scaling and Failure Modes

The main results suggest that higher resolution is not uniformly better: for many models in both T2P and I2P, $64^{2}$ underperforms $32^{2}$ (Table 1 and Table 2). To examine this behavior more closely, we sweep GPT-6 Astra, GPT-5.6 Sol, Claude Opus 5, and Gemini 3.6 Flash over ten resolutions from $16^{2}$ to $128^{2}$. The number of predicted positions grows quadratically with side length: relative to $16^{2}$, $64^{2}$ and $128^{2}$ require $16\times$ and $64\times$ as many pixel indices. Resolution scaling therefore increases both representational capacity and the burden of coordinating a long spatial output.

Model	Valid	Flat	500-way Top-1 16	500-way Top-1 32	500-way Top-1 64	500-way Top-1 Avg	VLM Judge 16	VLM Judge 32	VLM Judge 64	VLM Judge Avg	Agg.
GPT-6 Astra	100.0	0.0	87	92	94	91.0	90.8	91.6	89.7	90.7	90.8
Claude Opus 5	100.0	0.0	81	87	89	85.7	75.1	74.1	70.1	73.1	79.4
GPT-5.6 Sol	100.0	0.0	84	89	83	85.3	78.1	78.9	73.0	76.7	81.0
Kimi K3	99.7	0.0	81	87	77	81.7	74.5	74.7	61.5	70.3	76.0
Gemini 3.6 Flash	100.0	1.3	87	78	72	79.0	77.8	71.2	60.4	69.8	74.4
Seed-2.1 Pro	100.0	0.0	70	72	74	72.0	68.6	64.6	59.3	64.2	68.1
DeepSeek V4 Pro	98.7	1.3	49	59	51	53.0	53.7	55.0	45.7	51.5	52.2

Text-to-Pixel Generation. The left two panels of Figure 4 report 500-way retrieval and VLM-as-judge quality, showing that increasing resolution does not consistently improve performance under either metric. GPT-6 Astra maintains strong performance across resolutions, with its judge score remaining near its peak through $96^2$ and declining only modestly at the two highest resolutions. GPT-5.6 Sol fluctuates after an early peak, while Claude Opus 5 exhibits oscillating retrieval performance but a clearer downward trend under VLM judging. Gemini 3.6 Flash shows the sharpest degradation at high resolutions, particularly beyond $64^2$. The consistent agreement between retrieval and judge-based evaluations also suggests that the lack of monotonic improvement with resolution is not an artifact of the retrieval metric.

Vector (PDF)

Vector (PDF)

Image-to-Pixel Generation. The right two panels reveal a clearer scaling pattern. GPT-6 Astra increasingly benefits from larger grids, with both retrieval accuracy and DINO similarity remaining strong at the highest resolutions. In contrast, the other models generally peak at intermediate resolutions and then decline. These results suggest that larger grids offer additional expressive capacity, but only GPT-6 Astra can translate this capacity into more faithful reconstructions.

To inspect these differences directly, Figure 5 shows two I2P examples across the same resolution sweep. GPT-6 Astra progressively adds texture, sharper boundaries, and architectural detail while preserving the subject and overall layout. GPT-5.6 Sol instead exhibits an effective-resolution collapse: despite the increasing nominal grid size, its outputs become dominated by large color blocks. This behavior can preserve a coarse semantic layout, but it underuses the available grid and reduces fidelity. Claude Opus 5 shows different failures across the two inputs. In the upper example, it mechanically repeats details from the central region, sacrificing the fidelity of the global reconstruction. In the lower indoor-architecture example, it preserves only a coarse layout and fails to recover fine structural details. Gemini 3.6 Flash degrades most severely, with the subject or key structures gradually vanishing at high resolutions.

Together, these examples expose three high-resolution failure modes: effective-resolution collapse, repetitive local patterns that disrupt global structure, and omission of the subject or other key structures. They also reveal a broader consistency gap across resolutions. Beyond how effectively models use additional grid positions, other models can produce substantially different reconstructions as resolution increases, whereas Astra preserves subject identity, composition, and overall appearance across scales. Astra’s advantage is therefore not merely its ability to produce longer outputs, but its ability to organize additional grid positions into coherent details while maintaining a stable reconstruction. Representative T2P cases in Appendix § 9 exhibit the same failure modes.

Pixel Reconstruction Tuning

The strong correlation between pixel generation and pixel understanding in PixelBench raises a training question: can pixel generation itself serve as an objective that improves a VLM’s general visual understanding? RecA (Xie et al., [63]) explores a related idea for unified multimodal models, training them to reconstruct input images from their visual representations and thereby improving general image generation and editing. For visual understanding, however, conventional image-text and VQA supervision provides sparse, non-unique textual targets rather than dense supervision over visual content. Pixel generation provides such a target by requiring a VLM to reconstruct its input as a palette-indexed pixel grid, turning the image itself into deterministic, spatially aligned training supervision. We term this approach Pixel Reconstruction Tuning (PRT).

Data and Training

Pixel Reconstruction Tuning Data Construction. We sample images from a 20k-image PixMo-Cap pool (Deitke et al., [16]) after removing exact matches with evaluation images. To avoid distortion from square reconstruction targets, we discard images with aspect ratios above 1.33. We also exclude text-heavy and multi-panel images that cannot be faithfully represented at resolutions of $16^{2}$ to $32^{2}$. We render both resolutions with the PIA-reduced procedure (Appendix § 19) and compute the SigLIP2 cosine similarity between each rendered pixel image and its source image. We retain the top 40% at $16^2$ and the top 80% at $32^2$. For each retained image, we query Gemini 3.8 Flash to write an image description and summarize the colors in the algorithmically constructed palette. We package these with the grid using <CAPTION>, <PALETTE>, and <GRID>, while keeping the palette and grid values deterministic. The retained examples are our PRT data (see Appendix § 11).

Post-training setup. Starting from the Molmo2-4B ([13]) step-32k pretraining checkpoint, we post-train all modules for one pass on 500k LLaVA-OneVision-1.5 instructions ([3]), adding PRT data only in treatment runs. The main 2k experiment (0.4%; Table 6) uses four matched seed pairs. The quantity ablation tests 1k, 2k, 4k, and 8k examples (Table 7) on two matched seeds shared across amounts; 2k includes two additional seeds and is reported over four pairs. Within each pair, OV-only and OV+Pixel share the same random seed, isolating the effect of PRT data.

Benchmark	$N$	OV	+Pixel	$\Delta$
BlindTest	3,434	58.3	59.7	+1.4
VLM Bias	2,784	26.1	27.2	+1.1
CV-Bench	2,638	75.8	76.7	+0.9
ColorBench-R.	493	61.8	62.6	+0.8
BLINK	1,901	52.8	53.6	+0.8
MMStar	1,500	55.3	56.0	+0.7
MathVista	1,000	56.2	56.7	+0.5
ColorBench	955	32.1	32.5	+0.4
Q-Bench	1,495	71.9	72.2	+0.3
Hallusion	1,129	59.6	59.9	+0.2
MMVP	150	58.7	58.8	+0.2
ChartQA	2,500	77.9	77.9	+0.0
Perception	3,000	25.9	25.9	+0.0
MMBench	1,292	78.6	78.6	+0.0
BabyVision	388	10.2	10.1	−0.1
V-STAR	191	65.2	65.1	−0.1
Example avg.	24,850	54.2	54.8	+0.6
Benchmark avg.	16	54.1	54.6	+0.5

Avg.	$\Delta_{1k}$	$\Delta_{2k}$	$\Delta_{4k}$	$\Delta_{8k}$
Example	+0.40	+0.61	+0.36	+0.50
Benchmark	+0.56	+0.45	+0.39	+0.51

Evaluation suite. We evaluate 16 benchmarks across four groups (Table 5): general perception and reasoning, fine-grained visual and spatial detail, color, and charts and mathematics. All use their full evaluation sets except BlindTest, for which we sample a category-balanced subset.

Benchmark	Task content	$N$
General perception and reasoning
MMStar ([8])	Vision-essential perception and reasoning across six capability areas	1,500
MMBench ([35])	Multiple-choice perception and reasoning over diverse visual content	1,292
BLINK ([19])	Fourteen classic computer-vision tasks recast as visual question answering	1,901
BabyVision ([7])	Foundational visual perception and reasoning over simple scenes	388
PerceptionBench ([32])	Atomic perception of existence, count, position, color, and relations	3,000
Fine-grained visual and spatial detail
MMVP ([51])	Fine-grained visual details missed by CLIP-style vision encoders	150
Q-Bench ([59])	Low-level visual attributes, image quality, and visual perception	1,495
HallusionBench ([23])	Image-context reasoning that exposes visual illusion and hallucination	1,129
V-STAR Bench ([60])	High-resolution visual search and grounding for small or hidden targets	191
CV-Bench ([50])	2D spatial relations and counting; 3D depth and distance	2,638
BlindTest ((Rahmanzadehgervi et al., 2024))	Geometric primitives, spatial relations, and precise attribute reading	3,434
VLMs Are Biased ((Vo et al., 2026))	Counting in counterfactual images that conflict with learned priors	2,784
Color
ColorBench ([31])	Color recognition, extraction, comparison, counting, and reasoning	955
ColorBench-Robust ([31])	Answer consistency across each original image and nine recolored variants	493
Charts and mathematics
ChartQA ([37])	Visual extraction and logical or arithmetic reasoning over real-world charts	2,500
MathVista ([36])	Mathematical reasoning over diagrams, plots, puzzles, and scientific figures	1,000

Main Results

PRT Improves VLM Performance. In the controlled 2k comparison (Table 6), Pixel trajectories raise Example averaging from 54.2 to 54.8 (+0.6) and Benchmark averaging from 54.1 to 54.6 (+0.5); the seed-paired Benchmark gain is positive in three of four pairs (+0.8 / +0.5 / +0.7 / −0.1). Across quantities (Table 7), both aggregate deltas stay positive from 1k to 8k (+0.36–+0.61) without a monotonic scaling trend; 2k is the only four-seed setting. At the benchmark level, the largest gains are on BlindTest (+1.4), VLMs Are Biased (+1.1), and CV-Bench (+0.9). ChartQA, PerceptionBench, and MMBench are unchanged, while BabyVision and V-STAR decrease by only 0.1 point. Example averaging peaks at 2k (+0.61), whereas

Benchmark averaging peaks at 1k (+0.56), further indicating that the benefit does not scale monotonically with quantity. Full results and caveats are in Appendix § 11.

Pixel Reconstruction Serves as Regularization

We study Qwen3-VL-Instruct at 2B, 4B, and 8B. FV uses 64k FineVision examples; FV+Pixel adds 3k examples that directly pair an input image with its deterministic palette-indexed reconstruction. Here, the target contains only the reconstructed pixels, with no accompanying image description or palette explanation. Table 8 shows positive gains at every scale: 2B rises from 48.1/47.3 to 50.1/50.3 (+2.0/ + 3.0), 4B from 53.9/53.6 to 56.8/57.3 (+2.9/ + 3.7), and 8B from 55.1/56.3 to 57.2/57.9 (+2.1/ + 1.6); 4B gains the most. Resolution preferences vary by scale (Table 9): 2B peaks at $32^2$ (+2.50/ + 3.37), 4B at $16^2$ (+3.14/ + 4.07), and 8B at Mix for Example averaging (+2.07) but $64^2$ for Benchmark averaging (+1.73). Mix nevertheless ranks first or second in four of six rows, making it a robust choice without committing to a single resolution.

Model	Avg.	FV	FV+Pixel	$\Delta$
2B	Example	48.1	50.1	+2.0
	Benchmark	47.3	50.3	+3.0
4B	Example	53.9	56.8	+2.9
	Benchmark	53.6	57.3	+3.7
8B	Example	55.1	57.2	+2.1
	Benchmark	56.3	57.9	+1.6

Model	Avg.	$\Delta_{16}$	$\Delta_{32}$	$\Delta_{64}$	$\Delta_{\mathrm{Mix}}$
2B	Example	+2.14	+2.50	+2.27	+2.01
	Benchmark	+2.66	+3.37	+2.99	+3.02
4B	Example	+3.14	+2.87	+2.18	+2.94
	Benchmark	+4.07	+3.90	+3.48	+3.70
8B	Example	+1.88	+1.92	+1.90	+2.07
	Benchmark	+1.38	+1.23	+1.73	+1.60

Concluding Remarks

We introduce PixelBench, a text-based interface for direct pixel generation and understanding without a learned image tokenizer or decoder. Across T2P, I2P, and understanding tasks, frontier models exhibit substantial yet uneven capabilities, with model-specific resolution peaks. GPT-6 Astra remains robust as grid resolution increases, whereas most other models initially improve before declining. Beyond evaluation, we introduce PRT, which transforms deterministic palettes and grids into instruction trajectories. By combining general instruction data with curated PRT data, we post-train Molmo2-4B from its pretraining checkpoint, improving performance across 16 visual-centric benchmarks. The performance gain can also be observed in Qwen3-VL across different scales. These findings highlight the dual role of explicit pixels as a behavioral probe for evaluating visual capabilities and a practical training signal for improving them.

Acknowledgments

HZ thanks Ji Xie for a careful review of the paper and helpful suggestions. HZ also thanks Yangyi Huang, Sihan Yang, Zhou Jiang, Yi Xie, and Hongrui Zhang for useful comments.

Related Work

Symbolic Visual Representations

A growing line of work represents visual content as a discrete symbol sequence that a renderer executes, rather than as a dense array of pixels. On the understanding side, SGP-Bench evaluates whether language models can infer the semantics of SVG and CAD programs without seeing their renderings and improves this ability through Symbolic Instruction Tuning ([43]), while VGBench and SVGenius broaden evaluation to editing and generation across many formats, domains, and complexity levels ([67, 10]). On the generation side, models emit SVG code or executable CAD programs directly ([11, 46, 64, 61, 27, 48, 66]), and Sharma et al. [49] use rendering code as a stand-in for pixels, demonstrating that text-only models encode substantial visual knowledge. Together, these studies show that general-purpose models encode usable geometric knowledge in symbolic form, although their performance degrades when precise spatial and structural fidelity is required.

Two properties separate that setting from ours. A program mediates between model and image, so the renderer supplies geometry and resolution independence and each primitive covers many pixels; token-based generative models instead keep the raster form but substitute a learned codebook whose vocabulary is opaque and whose decoding requires the paired detokenizer ([9, 6, 56]). PixelBench asks for an explicit palette and per-pixel indices, so rendering is a table lookup and every emitted symbol is one spatial location. The closest prior evaluation of this format is Pixel Art Bench, a web leaderboard that ranks $24 \times24$ grids for seven subjects by LLM-judged Elo ([18]). Color-index grids also appear in abstract reasoning ([12]). PixelBench broadens this to text-to-pixel and image-to-pixel generation over 100 concepts and 100 natural images at three resolutions, adds a complementary pixel-understanding evaluation, replaces judge-based ranking with deterministic retrieval and fidelity metrics, and compares against palette-matched pixelization and downscaling algorithms ([41, 57, 62]).

Vision-Language Training and Visual Representations

Contemporary VLMs pair a pretrained visual encoder with a language model through a connector, aligned by image-text pretraining and visual instruction tuning ([44, 65, 2, 30, 33, 14, 34]). Controlled ablations attribute much of the reported progress to the unimodal backbones and to training data rather than to the connector ([26, 29]), and a parallel body of work stresses the composition, quality, and cleanliness of that data: vision-centric mixtures sharpen fine-grained perception ([50]), curated non-distilled and decontaminated corpora improve generalization ([16, 58]), and benchmark audits show that apparent gains can stem from leakage or from questions answerable without the image ([8]). Across these pipelines the supervision target is language, whatever the visual input.

That choice has consequences: distinctions lost by the encoder propagate into the assembled model ([51]), which remains unreliable on low-level spatial queries ([45, 19]). Objectives that require reproducing visual detail instead yield transferable representations ([24, 9]), and MetaMorph finds understanding and visual generation mutually beneficial ([52]). Reconstruction Alignment (RecA) conditions unified multimodal models on their own visual encoder representations and trains them to reconstruct the input image, thereby connecting visual understanding and generation ([63]). PRT studies an analogous objective for VLMs without a dedicated image generator: predicting the palette-index grid itself through the existing vocabulary, with no learned image tokenizer or decoder. Our primary protocol starts from a Molmo2 pretraining checkpoint and adds caption-palette-grid trajectories to a fixed 500k-example instruction mixture. Appendix § 11 additionally reports the earlier Qwen3-VL experiments, where pixel targets are added to FineVision post-training as a regularizer.

Additional Resolution-Scaling Examples

These cases instantiate the same three failure modes observed in the qualitative I2P sweep. GPT-5.6 Sol exhibits effective-resolution underutilization: its $64^2$ skateboard and fire are dominated by blocks roughly one-eighth of the canvas width, giving them a granularity closer to an $8 \times8$ grid. The outputs remain recognizable, but the additional positions contribute little detail. Gemini 3.6 Flash uses finer boundaries and local variation, but these local patterns can disrupt global structure: at $64^2$, its skateboard collapses to a diagonal stroke, its star loses the five-point structure, and its fire loses a coherent silhouette. The latter cases also omit identity-bearing structures. Claude Opus 5 varies by prompt: it preserves the depicted objects in these examples while using the higher-resolution grid, but its aggregate T2P performance fluctuates across resolutions and does not sustain its intermediate-resolution peak.

Additional Qualitative PixelBench Cases

The following pages expand Figure 3 with 20 additional T2P cases and 20 additional I2P cases. Each T2P case compares all seven evaluated models at $16^2$, $32^2$, and $64^2$. Each I2P case compares the six evaluated models and three algorithmic baselines at the same resolutions. The I2P panels provide additional qualitative evidence that some model-based reconstructions preserve semantic structure more clearly than the algorithmic baselines at low resolutions, complementing the aggregate results in Table 2. Pixel images use nearest-neighbor enlargement for display.

Additional Pixel Reconstruction Tuning Results

Molmo2 Pixel-Trajectory Construction

Image pool and filtering. Pixel trajectories use PixMo-Cap images with human-written dense captions. We first construct a 20k-image pool and expand it with previously unused images for the 4k and 8k sets. Exact hash matches to evaluation images are removed. We reject examples whose captions indicate screenshots, webpages, user interfaces, documents, posters, text, charts, diagrams, infographics, collages, or multi-panel layouts because text-heavy and composite content cannot be faithfully represented at $16^{2}$–$32^{2}$. Because PIA-reduced converts every source to a square grid, we also reject images with aspect ratio above 1.33 to avoid substantial geometric distortion.

Recognizability and grid construction. For each candidate, we construct $16^{2}$ and $32^{2}$ eight-color grids using the PIA-reduced procedure (Appendix § 19). We nearest-neighbor upscale each grid and compute its paired cosine similarity with the source image using SigLIP2-base-patch16-224. At each resolution, candidates are ranked by this similarity; we retain the top 40% at $16^{2}$ and the top 80% at $32^{2}$. Images are unique within every set. The 2k set is balanced across $16^{2}$ and $32^{2}$; 4k and 8k are nested extensions whose new rows use previously unused images.

Natural-language trajectory. Gemini 3.8 Flash receives the original image, upscaled ground-truth grid, one mask visualization per palette letter, the PixMo caption, and measured palette/layout hints. At temperature 0.4 and thinking budget 2,048, it writes only the caption and palette explanation; the palette and grid remain deterministic. The response must discuss every palette letter exactly once and in order, otherwise it is retried up to three times. Three paraphrased user prompts are assigned deterministically. Median target lengths are 1,693 characters at $16^{2}$ and 2,739 at $32^{2}$.

Molmo2 Training and Evaluation

Item	Setting
Initialization	Molmo2-4B-Pretrain, step 32,000; Qwen3-4B language model
OV mixture	500,000 single-image rows from LLaVA-OneVision-1.5-Instruct; answers over 2,048 tokens removed; image long side capped at 1,024 px
Pixel mixture	OV 500k plus 1k, 2k, 4k, or 8k unique Pixel trajectories
Schedule	One pass; OV 1,190 steps; OV+Nk uses 1,190 + $4N$ steps
Batching	128 packed sequences × 4,096 tokens; one image per sample; maximum answer length 1,800 tokens
Optimization	AdamW; LLM LR $10^{-5}$; ViT/connector LR $5 \times10^{-6}$; 60-step warmup; final LR multiplier 0.1; gradient clip 1.0; bf16
Hardware	Four nodes × eight NVIDIA A100-80GB GPUs per run
Seeds	Four matched seed pairs for OV and +2k; two seeds for 1k, 4k, and 8k

The primary suite comprises MMVP ([51]), BlindTest ((Rahmanzadehgerveri et al., 2024)), V-STAR ([60]), BabyVision ([7]), HallusionBench ([23]), PerceptionBench ([32]), CV-Bench ([50]), MMStar ([8]), Q-Bench ([59]), MMMench ([35]), VLMs Are Biased ((Vo et al., 2026)), BLINK ([19]), ColorBench and ColorBench-Robust ([31]), ChartQA ([37]), and MathVista ([36]).

All runs use fixed question subsets, identical prompts, and greedy decoding. Rule-based scoring is used except for the free-form portion of BabyVision and PerceptionBench, which use their benchmark-specific LLM judging protocols.

Run	Example avg.	Benchmark avg.
OV-only s0	54.1	54.3
OV-only s1	54.7	54.4
OV-only s2	53.8	53.6
OV-only s3	54.2	54.2
OV+2k s0	55.1	55.1
OV+2k s1	55.0	54.9
OV+2k s2	54.9	54.3
OV+2k s3	54.3	54.0

Benchmark	$\Delta_{1\mathrm{k}}$	$\Delta_{2\mathrm{k}}$	$\Delta_{4\mathrm{k}}$	$\Delta_{8\mathrm{k}}$	All Pixel
CV-Bench	+1.8	+0.9	+1.5	+1.0	+1.2
ColorBench-Robust	+1.3	+0.8	+1.5	+0.4	+1.0
VLMs Are Biased	+0.6	+1.1	+0.2	+1.7	+1.0
MathVista	+0.2	+0.5	+1.5	+1.3	+0.8
MMStar	+0.5	+0.8	+1.0	+0.8	+0.8
HallusionBench	+0.1	+0.2	+2.2	+0.5	+0.7
BabyVision	+1.3	−0.1	+0.3	+1.0	+0.5
MMVP	+2.0	+0.2	−0.7	+0.7	+0.5
BlindTest	+0.2	+1.4	−0.5	−0.3	+0.5
Q-Bench	+0.6	+0.3	+0.2	+0.4	+0.4
ColorBench	+0.4	+0.4	+0.3	+0.2	+0.3
BLINK	−1.3	+0.8	+0.3	+0.5	+0.2
ChartQA	+0.5	+0.0	−0.1	+0.2	+0.1
MMBench	+0.2	+0.0	−0.2	+0.3	+0.1
PerceptionBench	−0.0	+0.0	−0.1	−0.1	−0.0
V-STAR Bench	+0.5	−0.1	−1.0	−0.5	−0.3
Example avg.	+0.4	+0.6	+0.4	+0.5	+0.5
Benchmark avg.	+0.6	+0.5	+0.4	+0.5	+0.5

Qwen3-VL Training Setup

Data composition. All PRT experiments initialize from Qwen3-VL-Instruct checkpoints at the 2B, 4B, and 8B scales; tables abbreviate these scales as 2B, 4B, and 8B. These experiments use an additive design. The FV-only baseline uses 64,000 FineVision occurrences. Each FV+Pixel configuration retains those occurrences and adds 3,000 pixel-reconstruction occurrences mixed approximately evenly across $16^2$, $32^2$, and $64^2$, for 67,000 total training examples. All examples are jointly shuffled. FineVision responses are capped at 2,048 tokens, and pixel targets use the deterministic eight-color PIA-reduced conversion. Pixel source images are sampled without replacement. In the Pixel-Mix run, each of the 3,000 unique source images is assigned to exactly one of $16^2$, $32^2$, or $64^2$. The single-resolution ablations deliberately reuse the same 3,000 source-image IDs across separate runs, converting each image once at the corresponding run’s target resolution; no run therefore repeats a source image because of the resolution construction.

Optimization. We fully fine-tune all parameters, including the vision transformer, using AdamW ($\beta_1 = 0.9$, $\beta_2 = 0.999$, $\epsilon= 10^{-8}$), zero weight decay, a peak learning rate of $10^{-5}$ with cosine decay and 3% warmup, gradient clipping at 1.0, bf16 precision, and training seed 42. The effective batch size is 64. FV-only trains for 1,000 optimizer steps, whereas FV+Pixel trains for 1,047 steps. Consequently, the additive delta measures the combined effect of adding Grid data and training for 47 additional steps.

Table 13: Qwen3-VL post-training as a regularization study. Base is the released Instruct checkpoint; FV is FineVision-only post-training; FV+Pixel retains all 64k FV examples and adds 3k reconstruction examples. $\Delta$ is FV+Pixel minus FV.

Model	Avg.	Base	FV	FV+Pixel	$\Delta$
2B	Example	43.0	48.1	50.1	+2.0
	Benchmark	45.2	47.3	50.3	+3.0
4B	Example	48.8	53.9	56.8	+2.9
	Benchmark	51.5	53.6	57.3	+3.7
8B	Example	48.8	55.1	57.2	+2.1
	Benchmark	53.6	56.3	57.9	+1.6

Additive FV + Pixel Results

Table 14 summarizes benchmark-level target-resolution deltas across model scales. Each panel is independently ordered by the mixed-resolution delta, from the largest gain to the largest drop.

Table 14: Benchmark-level target-resolution deltas across model scales. Each entry is the accuracy change, in percentage points, relative to the corresponding FineVision-only baseline under the additive protocol. Mix distributes Grid examples approximately evenly across the three resolutions. Panels are independently sorted by $\Delta_{\mathrm{Mix}}$. Example averaging assigns equal weight to every official evaluation unit; Benchmark averaging assigns equal weight to every benchmark. * denotes a randomly sampled subset.

Model	$N$	$\Delta_{16}$	$\Delta_{32}$	$\Delta_{64}$	$\Delta_{\mathrm{Mix}}$
2B — MMVP	150	+7.3	+12.0	+9.3	+11.3
2B — BabyVision	388	+4.6	+2.8	+7.0	+4.9
2B — MMBench	1292	+5.0	+4.8	+4.2	+4.8
2B — CV-Bench	2638	+3.1	+3.4	+3.5	+3.0
2B — TallyQA*	3000	+3.3	+2.8	+2.4	+2.9
2B — Hallusion	1129	+5.1	+1.2	+3.8	+2.8
2B — VStar	191	+1.0	+3.1	0.0	+2.6
2B — Q-Bench	1495	+1.8	+3.7	+3.1	+2.1
2B — Perception	3000	+2.0	+2.0	+1.2	+1.9
2B — BlindTest	3434	+3.6	+1.3	+3.9	+1.7
2B — BLINK	1901	-0.7	+1.5	+0.3	+1.6
2B — MMStar	1500	-0.5	+3.3	+0.9	+0.3
2B — VLMBias	2784	-1.1	+1.9	-0.6	-0.8
2B — Example	22902	+2.14	+2.50	+2.27	+2.01
2B — Benchmark	13	+2.66	+3.37	+2.99	+3.02
4B — MMVP	150	+18.0	+22.7	+23.3	+20.7
4B — BlindTest	3434	+10.7	+10.6	+7.3	+9.6
4B — BabyVision	388	+2.6	+3.9	+3.6	+5.2
4B — CV-Bench	2638	+3.4	+3.2	+2.9	+3.0
4B — VLMBias	2784	-0.9	+1.1	+0.2	+2.7
4B — MMStar	1500	+1.6	+3.4	+0.4	+2.3
4B — MMBench	1292	+3.6	+3.3	+2.6	+2.2
4B — Perception	3000	+2.0	+1.2	+1.6	+1.5
4B — BLINK	1901	-0.6	-0.1	+1.4	+1.3
4B — Q-Bench	1495	+0.4	-1.1	+0.3	+0.8
4B — TallyQA*	3000	+2.0	+0.8	+0.3	+0.6
4B — VStar	191	+5.2	+2.6	+2.6	+0.5
4B — Hallusion	1129	+4.9	-0.8	-1.5	-2.1
4B — Example	22902	+3.14	+2.87	+2.18	+2.94
4B — Benchmark	13	+4.07	+3.90	+3.48	+3.70
8B — BlindTest	3434	+4.6	+4.3	+5.2	+7.8
8B — MMVP	150	+0.7	+0.7	+8.0	+4.0
8B — Q-Bench	1495	+3.0	+4.3	+2.5	+2.6
8B — TallyQA*	3000	+2.8	+3.3	+2.5	+2.6
8B — MMStar	1500	+4.5	+2.3	+4.0	+2.3
8B — BabyVision	388	-1.8	+1.5	0.0	+2.1
8B — BLINK	1901	+2.9	+0.8	+1.6	+1.7
8B — MMBench	1292	+3.4	+1.6	+1.5	+1.6
8B — Perception	3000	+1.4	+2.0	+1.9	+1.3
8B — CV-Bench	2638	+0.3	+0.8	+0.2	+1.0
8B — VStar	191	+1.6	-0.5	0.0	-1.0
8B — VLMBias	2784	+0.1	+1.8	+0.9	-1.1
8B — Hallusion	1129	-4.9	-7.0	-6.1	-4.1
8B — Example	22902	+1.88	+1.92	+1.90	+2.07
8B — Benchmark	13	+1.38	+1.23	+1.73	+1.60

Table 15: Qwen3-VL Grid-quantity sweep. Values are accuracy deltas from the corresponding FV-only baseline. Example pools official evaluation units and Benchmark weights benchmarks equally. Bold and underline mark the best and second-best distinct values within each row.

Model	Avg.	$\Delta_{1k}$	$\Delta_{2k}$	$\Delta_{3k}$	$\Delta_{4k}$	$\Delta_{5k}$
2B	Example	+2.43	+2.68	+2.01	+2.31	+2.18
	Benchmark	+3.30	+3.20	+3.02	+3.31	+3.91
4B	Example	+2.73	+2.08	+2.94	+1.92	+2.66
	Benchmark	+3.33	+3.01	+3.70	+2.77	+3.61
8B	Example	+1.59	+1.21	+2.07	+0.59	+0.98
	Benchmark	+1.67	+1.04	+1.60	+0.90	+0.55

Fixed Data Amount and Training Steps

To compare reconstruction tuning under matched training conditions, we fix both the amount of training data and the number of updates. FV-only contains 64,000 FineVision occurrences. Each Pixel configuration replaces 3,000 of those occurrences with pixel-reconstruction examples, so every fine-tuning run uses exactly 64,000 occurrences and stops at 1,000 optimizer steps. The Pixel configurations therefore contain less FineVision supervision.

The single-resolution variants use eight-color PIA-reduced targets at $16^2$, $32^2$, or $64^2$ in separate runs, reusing the same 3,000 unique source-image IDs across runs. The mixed variant assigns each image from that batch to exactly one resolution, approximately evenly, using sampling seed 0. Source-image sampling is without replacement, so no image is repeated within a run. All fine-tuning runs use training seed 42, four GPUs with DeepSpeed ZeRO-2, per-GPU batch size 4, and four gradient accumulation steps; the 4B runs use NVIDIA A100 80GB GPUs. These settings match the number of training occurrences and optimizer steps; target sequence length still grows with grid size.

Training-seed sensitivity. For the 4B setting with 4,000 Pixel occurrences, all three seeds yield positive, tightly clustered deltas (Table 17; SD 0.150 Example and 0.197 Benchmark). This auxiliary study uses the original MMVP scoring protocol, so the values differ from the main PRT tables; all runs and baselines here use the same evaluation protocol.

Run	$\Delta$ Example	$\Delta$ Benchmark
Seed 42	+1.455	+1.789
Seed 43	+1.746	+2.053
Seed 44	+1.534	+1.668
Mean ± SD	+1.578 ± 0.150	+1.837 ± 0.197
SEM	0.087	0.114
95% CI	[1.205, 1.952]	[1.348, 2.326]

Model	Setting	$\Delta$ Example	$\Delta$ Benchmark
2B	Pixel-16	+3.152	+4.278
2B	Pixel-32	+1.534	+2.481
2B	Pixel-64	+2.833	+3.744
2B	Pixel-Mix	+2.966	+3.577
4B	Pixel-16	+2.270	+3.129
4B	Pixel-32	+1.335	+1.828
4B	Pixel-64	+1.171	+2.306
4B	Pixel-Mix	+1.134	+1.993

Under the official aggregation, Pixel-16 yields the largest gains at both model scales, and every setting—including 4B Pixel-Mix—improves both averages.

Limitations

The main Molmo comparison adds Pixel trajectories to OV rather than replacing OV rows, so the 2k condition receives eight additional updates. We also do not compare against alternative auxiliary objectives on the same images, such as caption-only trajectories. We therefore cannot fully separate the palette-index grid from additional image exposure or its accompanying language supervision. Pixel trajectories are longer than typical responses, so their token share exceeds their 0.4% share of examples.

The 2k Benchmark-average effect is small (+0.5 points), has a standardized difference of $z = 1.5$, and is positive in three of four matched seed pairs. The 1k, 4k, and 8k settings have only two seeds each, and the observed gains do not increase with quantity. These results support a small, consistent aggregate benefit, not a data-scaling claim.

The secondary Qwen controls compare PRT against FineVision-only tuning and do not isolate pixel targets from additional image exposure. Their additive runs also contain 47 extra updates. The fixed-data, equal-step experiment above addresses update count but replaces some FineVision supervision, leaving the auxiliary-objective comparison unresolved.

Model Inference Settings

Table 18 reports the task-specific inference settings used for PixelBench generation. We retain one completed output per item. A completed response that violates the output constraints is a model error and receives zero, whereas provider-side failures and token-limit truncations are retried; for the latter, we raise the output-token limit when needed. Seed-2.1 Pro is the only model for which we impose an explicit reasoning budget. We cap it at 2,500 tokens because pilot runs frequently produced long, repetitive deliberation without improving the final pixel output.

Model	Temperature	Reasoning configuration	Max output tokens	Samples/item
Text-to-Pixel (T2P)
GPT-6 Astra	1.0	Default	32,768	1
Claude Opus 5	1.0	Default (high effort)	32,768	1
GPT-5.6 Sol	1.0	High effort	32,768	1
Kimi K3	1.0	High effort	32,768	1
Gemini 3.6 Flash	0.7	Default	32,768	1
Seed-2.1 Pro	0.7	Reasoning budget 2,500	32,768	1
DeepSeek V4 Pro	0.7	Default	32,768	1
Image-to-Pixel (I2P)
GPT-6 Astra	1.0	Default	32,768	1
Claude Opus 5	1.0	Default (high effort)	32,768	1
GPT-5.6 Sol	1.0	High effort	32,768	1
Kimi K3	1.0	High effort	32,768	1
Gemini 3.6 Flash	0.7	Default	32,768	1
Seed-2.1 Pro	0.7	Reasoning budget 2,500	32,768	1

SigLIP Retrieval Scores

Let $\hat{x}$ denote a rendered pixel image and let $\mathbf{q}$ be its SigLIP embedding. Given a candidate set $C = \{\mathbf{c}_i\}_{i=1}^{K}$ of the same embedding space, we score each candidate by cosine similarity,

The rank of the intended target $\mathbf{c}^{\star}$ is

so $r(\hat{x}) = 1$ if and only if the target is the nearest neighbor of the query. Let $\mathcal{V}$ index the valid outputs among $N$ attempted generations. The reported 100-way Top-1 or 500-way Top-1 score counts invalid outputs as misses:

The appendix MRR diagnostic is averaged over valid outputs:

For T2P, $\mathbf{q}$ comes from the SigLIP-SO400M image encoder and each $\mathbf{c}_i$ is the corresponding text embedding of “a pixel art icon depicting {label}”; we report $K=100$ and $K=500$. For I2P, both the query and candidates are image embeddings from the same SigLIP-SO400M model, and $K=100$ is the set of benchmark source images.

Full Evaluation Prompts

The fields label (T2P) and sample_id (I2P) are included only to simplify automated parsing and pairing during evaluation; they are not part of the visual generation objective. For pixel understanding, {compact_pixel_json} is the serialized palette-and-grid input, and {choice_1}–{choice_10} are the ten candidate labels. For VLM judging, {label} and {size} are replaced with the target concept and native grid size, and the rendered image is supplied.

Text-to-Pixel Prompt

Create exactly N recognizable SxS pixel-art icons, one for each of these English labels in
this exact order:
{labels}

Return JSON only, with no Markdown fence or commentary, using:
{"images": [{"label": "{label}",
"palette": ["#FFFFFF", "#D62828", "#2A9D3F"],
"pixels": ["000...", "... exactly S strings"]}]}

Requirements:
- Every icon must have exactly S rows and S columns.
- Use 2 to 8 opaque RGB colors, including a background color.
- Encode each pixel as one digit from 0 to 7.
- Favor simple silhouettes, high contrast, and centered subjects.
- Make each object recognizable at SxS resolution; use no text.

Text-to-Pixel VLM-as-Judge Prompt

Judge inference and stability. We use Gemini 3.5 Flash with temperature 0, thinking budget 0, a maximum of 1,536 output tokens, and one sample per item. As a stability check, we re-score a 10% subset after the initial evaluation; the mean absolute change in total score is below 2 points on the 100-point scale, indicating that the reported VLM Judge scores are robust to repeated evaluation.

Act as a strict evaluator of a text-to-pixel generation. The target concept is
"{label}". The supplied image is a nearest-neighbor enlargement of a
native {size}x{size} pixel-art image. Large square pixels are intentional.

Score the image using all six independent dimensions below. Do not reward
complexity by itself, do not infer the generator identity, and do not excuse a
wrong or unrecognizable subject merely because the image is attractive.

1. subject_identity (0-20): Does the visible subject clearly depict the target
concept rather than a related, ambiguous, or incorrect object?
2. defining_attributes (0-25): Are the target's identity-bearing parts, shapes,
and relations present and correct?
3. structure_proportion (0-20): Is the silhouette coherent, proportionate,
well framed, and structurally correct rather than cropped or merged?
4. detail_quality (0-20): Are internal details coherent, informative, and
consistent with the target at the native resolution?
5. pixel_art_execution (0-10): Are pixel clusters and edges deliberate
and appropriate for the native resolution, without random noise or
fragmented pseudo-texture?
6. palette_contrast (0-5): Is the limited palette coherent and does contrast
preserve the subject and its important parts?

Return JSON only, without Markdown:
{
  "observed_subject": "short literal description of what is visibly depicted",
  "subject_identity": {"score": 0, "reason": "brief evidence"},
  "defining_attributes": {"score": 0, "reason": "brief evidence"},
  "structure_proportion": {"score": 0, "reason": "brief evidence"},
  "detail_quality": {"score": 0, "reason": "brief evidence"},
  "pixel_art_execution": {"score": 0, "reason": "brief evidence"},
  "palette_contrast": {"score": 0, "reason": "brief evidence"},
  "confidence": "low|medium|high",
  "failure_flags": []
}

Image-to-Pixel Prompt

Convert the supplied reference image into a faithful SxS pixel-art image. Preserve the main
subject's identity, silhouette, pose, composition, and dominant colors. Simplify fine
detail only as required by the pixel resolution.
Do not use any object name or text.

Return JSON only:
{"sample_id": "{sample_id}",
"palette": ["#FFFFFF", "#111111"],
"pixels": ["000...", "... exactly S strings"]}

Use 2 to 8 opaque #RRGGBB colors.

Pixel Understanding Prompt

Candidate construction. For each question, we exclude the correct concept from the benchmark’s 100 object concepts, deterministically shuffle the remaining 99, and select the first nine as distractors. We then deterministically reshuffle the correct answer together with these nine distractors to assign the ten choices to A-J. Each item therefore contains ten distinct candidates with exactly one correct answer, and the construction is reproducible.

This is a {size}x{size} pixel-art icon encoded as JSON.
"palette" is 2-8 hex colors. "pixels" are {size} row strings; each character is a palette
index 0-7.
Identify the depicted object. Reply with ONE letter only (A, B, C, D, E, F, G, H, I, J).

{compact_pixel_json}

A. {choice_1}
B. {choice_2}
C. {choice_3}
D. {choice_4}
E. {choice_5}
F. {choice_6}
G. {choice_7}
H. {choice_8}
I. {choice_9}
J. {choice_10}

Text-to-Pixel Concept List

Table 19 lists the 100 T2P concepts in the order used by the benchmark. The set contains 18 animals, 20 food items, 30 household objects, 15 means of transport, and 17 nature-related objects or symbols.

4. cat	8. fish	17. dog	18. bird	19. rabbit
20. elephant	21. turtle	22. butterfly	51. cow	52. sheep
53. pig	54. chicken	55. owl	56. penguin	57. frog
58. snake	59. bee	60. snail	1. apple	3. banana
23. orange	24. strawberry	25. watermelon	26. carrot	27. pizza
28. cake	29. bread	30. ice cream	61. pear	62. pineapple
63. grapes	64. cherry	65. lemon	66. corn	67. donut
68. hamburger	69. sandwich	70. cookie	2. stool	5. house
10. cup	14. key	15. umbrella	16. clock	31. lamp
32. spoon	33. scissors	34. toothbrush	35. bottle	36. camera
37. phone	38. book	39. hammer	71. fork	72. knife
73. plate	74. bowl	75. bed	76. table	77. sofa
78. television	79. computer	80. backpack	82. hat	89. wrench
90. screwdriver	91. axe	92. shovel	7. car	40. bus
41. bicycle	42. airplane	43. boat	44. train	45. rocket
46. truck	81. shoe	83. motorcycle	84. helicopter	85. submarine
86. tractor	87. skateboard	88. sailboat	6. tree	9. flower
11. star	12. heart	13. mushroom	47. sun	48. moon
49. cloud	50. mountain	93. rainbow	94. snowflake	95. lightning
96. fire	97. leaf	98. cactus	99. palm tree	100. castle

Image-to-Pixel Source Images

Figure 7 shows the 100 ImageNet validation crops used as I2P inputs. Each class contributes one center-cropped $256 \times256$ image; labels are shown for reference and are not provided to the model during generation.

Vector (PDF)

500-Way Retrieval Vocabulary

Table 20 and Table 21 list the 500 text candidates used for T2P 500-way Top-1 retrieval. Numbers are evaluation indices: the first 100 indices are the benchmark concepts in Table 19, and indices 101–500 are additional distractors. For layout, we sort entries by label length and split at 9 characters (271 shorter labels; 229 longer ones). Within each table, entries fill columns from top to bottom so shorter labels occupy narrower columns.

500. ox	269. harp	281. flute	162. buckle	286. teapot	93. rainbow	168. lorikeet	421. drumstick
91. axe	31. lamp	327. goose	98. cactus	375. throne	448. Samoyed	163. Malinois	165. golf cart
75. bed	97. leaf	12. heart	36. camera	344. tights	154. sawmill	446. marmoset	362. grey wolf
59. bee	333. mask	5. house	26. carrot	386. tripod	257. snorkel	50. mountain	189. guacamole
204. bra	275. maze	172. junco	100. castle	21. turtle	424. spatula	13. mushroom	334. gyromitra
40. bus	161. mink	72. knife	64. cherry	216. tusker	140. sundial	221. odometer	355. hair clip
7. car	48. moon	65. lemon	498. collie	191. valley	128. T-shirt	229. payphone	68. hamburger
4. cat	61. pear	252. llama	70. cookie	434. Vizsla	483. thimble	235. platypus	237. harmonica
51. cow	337. safe	202. macaw	155. coucal	124. wallet	86. tractor	402. plectrum	195. hourglass
10. cup	81. shoe	147. moped	132. cougar	180. weasel	268. viaduct	433. popsicle	30. ice cream
17. dog	473. sock	37. phone	497. coyote	89. wrench	330. volcano	293. rapeseed	309. lakeshore
484. fig	77. sofa	27. pizza	425. dugong	371. bassoon	137. warthog	88. sailboat	95. lightning
122. fly	11. star	73. plate	491. eggnog	273. bathtub	192. website	69. sandwich	233. monastery
82. hat	489. suit	118. purse	450. eraser	41. bicycle	42. airplane	33. scissors	416. motorboat
116. hay	414. tank	432. quail	9. flower	54. chicken	80. backpack	389. scorpion	99. palm tree
328. jay	396. tent	266. quill	179. geyser	135. doormat	350. baseball	205. sea lion	435. partridge
14. key	404. tram	488. quilt	63. grapes	243. echidna	340. chainsaw	138. stingray	62. pineapple
361. mop	6. tree	392. rifle	39. hammer	254. go-kart	79. computer	479. sturgeon	181. plant pot
55. owl	303. vase	208. scarf	377. isopod	478. gondola	284. consomme	236. terrapin	321. porcupine
53. pig	318. acorn	52. sheep	324. jaguar	199. gorilla	198. crayfish	15. umbrella	265. projector
47. sun	381. agama	60. snail	245. Kuvasz	347. hamster	206. dumbbell	459. vestment	113. red panda
212. wok	1. apple	58. snake	169. langur	406. hatchet	20. elephant	280. armadillo	94. snowflake
18. bird	130. bagel	32. spoon	315. maraca	464. hot dog	436. envelope	193. barometer	150. spoonbill
43. boat	360. beach	2. stool	261. muzzle	395. hot pot	171. fireboat	102. bobsleigh	480. stopwatch
38. book	262. bison	76. table	23. orange	209. lady	74. bowl	29. bread	44. train	111. pillow	119. steam locomotive
28. cake	339. canoe	46. truck	240. poncho	196. computer keyboard	66. corn	219. chain	454. vase	194. prison	186. impala
443. dome	447. cloak	354. zebra	19. rabbit	364. marimba	177. fucoat	401. chickadee	152. trash can
96. fire	16. clock	481. baboon	410. racket	305. minibus	167. goldfish	141. cockroach	115. tree frog
8. fish	456. clogs	3. banana	45. rocket	210. monitor	299. hair wig	282. corkscrew	190. turnstile
71. fork	49. cloud	387. beaker	431. sandal	108. padlock	238. hornbill	475. crossword	153. water jug
57. frog	311. dingo	290. beaver	160. shield	384. pelican	278. knee pad	157. Dalmatian	366. wild boar
419. gown	67. donut	35. bottle	92. shovel	56. penguin	246. lacewing	291. dishcloth

260. Arctic fox	499. coffeemaker	249. Cairn Terrier	119. steam locomotive
363. bald eagle	442. dung beetle	365. china cabinet	196. computer keyboard
397. black swan	382. fire screen	490. digital clock	186. impala (antelope)
477. Bloodhound	264. freight car	336. garbage truck	482. little blue heron
383. box turtle	207. grand piano	166. grocery store	295. monarch butterfly
461. bighorn	452. great egret	158. flower monkey	373. Redbone Coonhound
213. can opener	234. green mamba	427. Irish Terrier	285. rhinoceros beetle
156. car mirror	139. hermit crab	106. Japanese Chin	356. Shetland Sheepdog
439. chiffonnier	495. hummingbird	105. manhole cover	107. sulphur butterfly
277. chimpanzee	259. hunting bow	201. measuring cup	405. threshing machine
296. coral reef	385. leaf beetle	148. messenger bag	426. Bluebonnet Coonhound
463. crane bird	272. mixing bowl	465. movie theater	469. Christmas stocking
451. dishwasher	185. pencil case	487. red king crab	485. hand-held computer
370. fire truck	357. Persian cat	331. reflex camera	493. Labrador Retriever
121. frying pan	230. pirate ship	378. rocking chair	453. Scottish Deerhound
242. greenhouse	223. planetarium	467. silver salmon	367. semi-trailer truck
255. guinea pig	173. rain barrel	174. upright piano	444. smooth green snake
462. hair dryer	250. salt shaker	215. velvet fabric	149. Alaskan tundra wolf
144. hair spray	90. screwdriver	412. American robin	248. black-footed ferret
274. hammerhead	496. sea anemone	126. Carolina anole	267. electric locomotive
84. helicopter	241. Siamese cat	353. cliff dwelling	297. Irish Water Spaniel
317. hoop skirt	300. tape player	253. Cocker Spaniel	486. kite (bird of prey)
428. lawn mower	326. tennis ball	117. Dungeness crab	307. through arch bridge
142. leafhopper	363. tiger shark	103. filling cabinet	476. Cardigan Welsh Corgi
146. lighthouse	214. tiki monkey	379. French Bulldog	359. entertainment center
83. motorcycle	332. toilet seat	449. graduation cap	306. Old English Sheepdog
415. mud turtle	164. trench coat	420. great grey owl	289. southern black widow
349. neck brace	322. triceratops	372. ping-pong ball	125. swim trunks / shorts
423. Otterhound	298. wheelbarrow	182. shower curtain	400. European green lizard
352. paintbrush	256. Windsor tie	380. soap dispenser	348. red admiral butterfly
325. park bench	178. wine bottle	101. spiral or coil	492. rotary dial telephone
251. piggy bank	224. wolf spider	310. tandem bicycle	323. small white butterfly
329. pipe organ	351. Afghan Hound	127. American dipper	188. yellow lady’s slipper
239. Pomeranian	109. barber chair	104. baseball player	374. Curly-coated Retriever
418. pool table	292. black grouse	225. chocolate syrup	222. loupe magnifying glass
151. promontory	403. Brittany dog	358. cocktail shaker	258. ram (adult male sheep)
455. pufferfish	466. carton	393. common redshank	302. red wolf or maned wolf
312. rotisserie	440. Egyptian Mau	314. electric guitar	228. sidewinder rattlesnake
369. safety pin	399. fox squirrel	226. fire salamander	429. Welsh Springer Spaniel
197. sea urchin	316. green iguana	422. football helmet	304. black-and-white colobus
200. shoe store	131. handkerchief	458. four-poster bed	136. Australian Silky Terrier
460. shower cap	368. home theater	211. Groenendael dog	417. automated teller machine
87. skateboard	277. killer whale	471. laptop computer	170. Bouviers des Flandres dog
391. snoek fish	335. mosquito net	145. longhorn beetle	301. Chesapeake Bay Retriever
407. snowmobile	231. pats monkey	270. paddle potatoes	283. English Springer Spaniel
110. spider web	438. picket fence	346. Norfolk Terrier	232. sulphur-crested cockatoo
345. stone wall	287. punching bag	376. Standard Poodle	279. German Shorthaired Pointer
24. strawberry	247. railroad car	472. aircraft carrier	288. product packet / packaging
474. table lamp	494. refrigerator	184. bulletproof vest	112. Staffordshire Bull Terrier
468. teddy bear	183. sleeping bag	430. butternut squash	470. shoji screen / room divider
78. television	445. sliding door	341. chain-link fence	129. Soft-coated Wheaten Terrier
34. toothbrush	342. slot machine	159. desktop computer	320. American Staffordshire Terrier
25. watermelon	187. snow leopard	120. English foxhound	413. desert grassland whiptail lizard
203. worm snake	134. swimming cap	457. high-speed train	437. military hat (bearskin or shako)
133. barn spider	411. toilet paper	390. magnetic compass	343. Mexican hairless dog (xoloitzcuintli)
409. bell pepper	341. Affenpinscher	217. military uniform
175. block plane	308. baby pacifier	388. Miniature Poodle
319. brain coral	408. boron sheep	176. proboscis monkey

Technical Report

Full Text-to-Pixel Results

Tables 22 and 23 report the per-resolution Top-1 and MRR scores corresponding to Table 1. Table 24 decomposes the VLM Judge score into its six rubric dimensions.

Model	ID/20	Parts/25	Struct./20	Detail/20	Pixel/10	Color/5	Total/100
$16^2$
GPT-6 Astra	18.83	22.89	18.16	16.95	8.94	4.98	90.75
GPT-5.6 Sol	16.11	19.31	15.77	13.98	8.21	4.75	78.13
Claude Opus 5	15.77	18.52	15.61	12.60	7.88	4.75	75.13
Kimi K3	15.48	18.21	15.27	12.90	7.99	4.69	74.54
Gemini 3.6 Flash	16.22	19.23	15.79	13.66	8.17	4.72	77.79
Seed-2.1 Pro	14.12	16.57	14.07	11.61	7.64	4.56	68.57
DeepSeek V4 Pro	10.99	12.34	11.47	8.17	6.68	4.00	53.65
$32^2$
GPT-6 Astra	19.16	23.18	18.36	16.94	8.95	4.99	91.58
GPT-5.6 Sol	16.57	19.52	16.06	13.80	8.11	4.80	78.86
Claude Opus 5	16.04	18.39	15.49	11.92	7.59	4.69	74.12
Kimi K3	15.80	18.27	15.35	12.68	7.92	4.65	74.67
Gemini 3.6 Flash	14.77	17.05	14.47	12.33	7.99	4.60	71.21
Seed-2.1 Pro	13.56	15.55	13.22	10.77	7.10	4.36	64.56
DeepSeek V4 Pro	11.76	13.13	11.50	8.51	6.17	3.95	55.02
$64^2$
GPT-6 Astra	18.74	22.63	18.04	16.38	8.92	4.95	89.66
GPT-5.6 Sol	15.14	17.95	15.05	12.23	7.87	4.72	72.96
Claude Opus 5	15.55	17.54	14.58	10.95	6.97	4.48	70.07
Kimi K3	12.95	14.69	12.82	9.68	7.05	4.34	61.53
Gemini 3.6 Flash	12.67	14.30	12.30	9.87	7.05	4.16	60.35
Seed-2.1 Pro	12.89	14.24	12.27	9.32	6.45	4.16	59.33
DeepSeek V4 Pro	10.16	10.77	9.58	6.54	5.20	3.42	45.67

Resolution	Pool	Gem 3.6 F	GPT-5.6 Sol	GPT-6 Astra	Opus 5	Kimi K3	DeepSeek V4 Pro	Seed-2.1 Pro
$16^2$	100-way	.910	.910	.950	.910	.900	.580	.810
$16^2$	500-way	.870	.840	.870	.810	.810	.490	.700
$32^2$	100-way	.830	.920	.980	.950	.930	.680	.820
$32^2$	500-way	.780	.890	.920	.870	.870	.590	.720
$64^2$	100-way	.810	.890	.980	.970	.860	.630	.830
$64^2$	500-way	.720	.830	.940	.890	.770	.510	.740

Resolution	Pool	Gem3.6 F	GPT-5.6 Sol	GPT-6 Astra	Opus 5	Kimi K3	DeepSeek V4 Pro	Seed-2.1 Pro
$16^2$	100-way	.945	.945	.972	.949	.941	.697	.874
$16^2$	500-way	.913	.899	.927	.884	.881	.599	.792
$32^2$	100-way	.887	.956	.990	.972	.965	.769	.892
$32^2$	500-way	.837	.931	.960	.925	.932	.681	.815
$64^2$	100-way	.843	.933	.990	.985	.896{sup}†	.718	.898
$64^2$	500-way	.785	.893	.970	.945	.826{sup}†	.614	.832

Algorithmic Image-to-Pixel Baselines in Full

Each method either combines a spatial downscaling stage with a subsequent color quantization stage or jointly determines the output regions and palette. We set the palette size to eight for all constrained methods and evaluate them on the same 100 ImageNet validation crops used for the VLMs.

Tier 0: simple resampling. Nearest-neighbor sampling selects the source-cell center, while box filtering averages the corresponding source patch. Both are followed by median-cut quantization [25].

Tier 1: linear filters. Bicubic and Lanczos interpolation use standard fixed resampling kernels before median-cut quantization. These methods allocate the output resolution uniformly across the image.

Tier 2: content-adaptive downscaling. DPID [57] weights input pixels according to their deviation from a local guidance color; we evaluate $\lambda\in\{0.5, 1.0\}$. SSIM-optimal downscaling (Öztireli & Gross, 2015) minimizes structural dissimilarity using local image statistics. Both methods are followed by median-cut quantization.

Tier 3: pixel-art-specific methods. Pyxelate¹ uses Sobel-gradient information to merge local cells and estimates an eight-color palette with a Bayesian Gaussian mixture. We use version 2.1 or later with sobel=3, depth=1, and dithering disabled. Pixelated Image Abstraction (PIA) [20] jointly constructs output regions and a reduced palette. The reference implementation uses modified SLIC mapping, mass-constrained deterministic annealing [47], bilateral superpixel smoothing, and a saturation factor of 1.1. PIA (reduced) retains SLIC with compactness $m = 45$, replaces annealing with weighted eight-means over superpixel colors, and omits bilateral smoothing and saturation adjustment.

Representative Implementations

The following procedures specify the three representatives used in the main comparison. All receive a $256 \times256$ RGB crop $I$ and target side length $N \in\{16, 32, 64\}$. Median cut uses $K = 8$ without dithering. SLIC and $k$-means in PIA (reduced) operate in CIELAB; the other operations use RGB.

Nearest + median cut

Input: image $I$ and target side length $N$.

1. For each output cell $(y, x)$, sample $I$ at the center of the corresponding source cell to obtain $Y[y, x]$.

2. Apply MedianCut$(Y, K = 8)$ and return the quantized grid.

SSIM-optimal + median cut

Input: image $I$ of side length $H$ and target side length $N$.

1. Set $s = H/N$. For each output cell, compute the aligned $s \times s$ block mean $L[y, x]$ and squared-value mean $Q[y, x]$ for each RGB channel.

2. For each $2 \times2$ output neighborhood, compute its mean $\mu$, the variance $\sigma_L^2$ of $L$, and $\sigma_H^2 = \operatorname{mean}(Q) - \mu^2$.

3. Set $r = \sqrt{\sigma_H^2/\sigma_L^2}$ when $\sigma_L^2 \ge\varepsilon$, and $r = 0$ otherwise. Form the local proposal $\hat{Y} = \mu+ r(L - \mu)$.

4. Average overlapping proposals, clip values to $[0, 1]$, and return MedianCut$(\hat{Y}, K = 8)$.

PIA (reduced)

Input: image $I$ and target side length $N$.

1. Compute $S = \operatorname{SLIC}(I; n = N^2, m = 45)$ in CIELAB.

2. For each superpixel $u$, compute its mean color $c_u$ and size $w_u = \lvert u\rvert$.

3. Fit eight palette colors $\{p_k\}_{k=1}^{8}$ and assignments $a$ using weighted $k$-means on $\{c_u\}$ with weights $\{w_u\}$.

4. For each output cell, identify the superpixel at the corresponding source-cell center and assign the RGB conversion of $p_{a(u)}$.

Complete Algorithmic Results

Method	DINO CLS cosine			SigLIP cosine			100-way Top-1 (%)
	$16^2$	$32^2$	$64^2$	$16^2$	$32^2$	$64^2$	$16^2$	$32^2$	$64^2$
Tier 0 Simple resampling
Nearest + median cut	.012	.090	.480	.523	.568	.665	9	28	62
Box + median cut	.012	.099	.475	.523	.577	.673	10	27	65
Tier 1 Linear filters
Bicubic + median cut	.013	.093	.459	.524	.573	.672	9	26	64
Lanczos + median cut	.013	.099	.478	.524	.576	.673	10	29	64
Tier 2 Content-adaptive downscaling
DPID $\lambda= 0.5$ + median cut	.011	.103	.509	.523	.578	.679	10	27	68
DPID $\lambda= 1.0$ + median cut	.011	.104	.527	.522	.580	.678	9	32	65
SSIM-optimal + median cut	.013	.119	.543	.524	.579	.686	9	34	75
Tier 3 Pixel-art-specific methods
Pyxelate	.013	.085	.377	.525	.569	.655	12	32	65
PIA (reduced)	.014	.135	.490	.546	.608	.697	18	50	78
PIA (reference)	.015	.123	.485	.541	.599	.692	17	40	71

Table 25 shows that standard resampling filters and the two DPID settings yield similar results, while the strongest method depends on the metric. SSIM-optimal obtains the highest DINOv2 fidelity at $64^2$, whereas PIA (reduced) leads both SigLIP metrics across resolutions. We therefore use one representative for simple resampling, fidelity-oriented downscaling, and joint geometry-palette optimization.