WorksComputer Science
LEDGE: Separating Critic and Editor Failures in Text-to-Image Self-Correction
Abstract
Text-to-image (T2I) self-correction combines a vision-language model (VLM) critic that diagnoses errors and proposes instructions with an image editor that executes them. End-to-end evaluation obscures whether failures arise from incorrect diagnoses or an editor unable to execute valid instructions. We introduce LEDGE (Learning Error Diagnosis in Generation and Editing), a framework that separates these bottlenecks. LEDGE compiles oracle edit instructions from GenEval while holding the generated images and the editor fixed, isolating the editor's correction capability from the critic's diagnostic accuracy. On Stable Diffusion 3.5 Large, oracle instructions raise GenEval from 72.0% to 81.2%, versus 74.1% for the baseline VLM critic. Yet 435 of 644 failed images remain unrepaired, exposing a substantial editor gap. We also use oracle labels from disjoint training prompts to supervise the critic. Oracle supervision raises exact diagnosis from 33.3% (the same VLM with our structured output, untrained) to 70.8%. With the same editor, the resulting critic improves GenEval from 42.2% (uncorrected) to 59.4% on Stable Diffusion 1.5, against 50.9% for the baseline critic, and from 72.0% to 76.0% on Stable Diffusion 3.5 Large, despite training only on Stable Diffusion 1.5. Improvements transfer to unseen object classes, a second editor, and independent T2I-CompBench judges and prompts. A small human evaluation confirms the ranking in estimated net corrections but suggests a smaller gain than detector-based evaluation. LEDGE thus distinguishes two limits of self-correction: knowing what to fix and being able to fix it.
Introduction
Text-to-image (T2I) models still fail on compositional prompts: they drop objects, miscount, color the wrong object and ignore spatial relations. On GenEval [15], Stable Diffusion 1.5 [42] places objects correctly in 4.5% of position prompts and binds colors correctly in 7.8% of color-attribution prompts. A common remedy is self-correction: a vision-language model (VLM) critic inspects the generated image and writes an editing instruction, which an image editor then executes [8, 23, 36, 45, 50].
When such a loop fails, there are two possible causes. The critic may misdiagnose the image or phrase an instruction that the editor cannot use. Alternatively, the editor may fail to carry out a valid correction. Prior work has established that VLM guidance improves editing and that critic-driven refinement improves generation [14, 23], but it evaluates the correction process as a whole, so these two failures cannot be told apart. This distinction is crucial: improving the editor cannot compensate for incorrect instructions, just as improving the critic cannot help if the editor fails to execute them.
In this paper, we introduce LEDGE (Learning Error Diagnosis in Generation and Editing), which separates the two causes using a simple observation. Benchmarks such as GenEval specify each prompt’s requirements in structured form and report which requirement a failed image violates. From these we compile oracle instructions, i.e., the correct edit for every failure, phrased with the same templates used to render the critic’s instructions (Figure 1). With the editor and the images held fixed, the oracle measures how much the instruction alone limits correction (the instruction gap) and how much remains even with correct instructions (the editor gap).

Figure 1. Overview of LEDGE. The critic lists requirement violations; a fixed renderer turns them into an instruction for a fixed editor. The oracle compiler produces the correct edits from the benchmark’s requirements and the detector’s failure reasons. On test images it is a reference; on training prompts it supervises the critic (blue).
The oracle cannot be used at test time, since it reads the evaluator’s verdict, but the diagnosis it enables suggests a direct use: it provides free supervision. To this end, we generate images on training prompts whose specifications are disjoint from the benchmark’s, label them with the oracle, and fine-tune the baseline model. The resulting oracle-supervised critic then lists every requirement violation in a structured form, from which editing instructions are rendered deterministically. On SD1.5 with the VAREdit editor [38], the oracle-supervised critic closes most of the instruction gap with fewer regressions than the baseline critic, and, although trained only on SD1.5 images, it also improves correction of SD3.5-Large images, so the learned diagnosis transfers across generators. The ranking of critics also holds when the same images are re-scored by the independent judges of T2I-CompBench [20]. Our contributions are summarized as follows:
We introduce LEDGE, which holds the images and the editor fixed and compiles oracle instructions from GenEval’s verdicts. With either of two editors, oracle instructions repair 2.3× more SD1.5 failures than the baseline critic (Qwen3-VL-8B) [1], yet 61% of SD1.5 and 68% of SD3.5 failures remain unrepaired (§ 6.2).
We analyze why the baseline critic fails: it identifies the right error in 39% of failures, defers 35% to review and misses 12%, and on counting it often asks to remove objects that are missing (§ 6.3).
We propose oracle-supervised critics, trained by LoRA on labels compiled automatically from prompts disjoint from the benchmark. They raise exact diagnosis from 33.3% to 70.8% and GenEval from 50.9% to 59.4% with the same editor, and the same supervision improves a Molmo critic from 52.4% to 56.5% (§ 6.4).
We show that the critic, trained only on SD1.5 images, transfers to SD3.5-Large (72.0% to 76.0%), to object classes absent from training, and to a second editor, FLUX.1 Kontext. The ranking of critics also holds under T2I-CompBench’s independent judges, where the oracle-supervised critic recovers 97% of the oracle’s gain (§ 6.5).
Related Work
VLM-guided correction. Multimodal LLMs already guide instruction-based editing: MGIE [14] learns expressive instructions that steer a diffusion editor end to end, and SmartEdit [21] uses an MLLM to interpret complex instructions. For T2I generation, a VLM critic drives iterative refinement of the generator’s output [23], rewrites the prompt between rounds [11, 27], decomposes long instructions into verified, reversible edits [8], or localizes distortions for repair [43]. EditRefiner [46] refines edited images with an agentic loop, and Inline Critic [26] predicts errors inside the editor’s forward pass. R3-Bench [45] scores a VLM’s verdict, critique and rectification of T2I outputs, and self-corrected image generation [36] closes the loop at generation time. These works establish that critic-guided correction helps, and we do not claim the loop itself as a contribution. They evaluate it as a whole; LEDGE instead holds the images and the editor fixed to separate diagnosis from execution, and uses the resulting oracle to supervise the critic.
Improving editing instructions. EditThinker [28] trains a Qwen3-VL-8B model that critiques an editor’s output, rewrites the instruction and retries, targeting user editing requests on editing benchmarks. Complex-Edit [47] studies how instruction complexity affects editors. Our setting differs in two ways: the target is fidelity to the original T2I prompt, and our supervision is compiled from a verifier rather than generated as reasoning data.
Instruction-guided editors. Editors are trained on synthetic or curated pairs [6, 22, 48], with human feedback [49] and, more recently, with online reinforcement learning against learned reward models [16, 29, 34, 35]; training-free methods regulate the editing strength during denoising [12]. Editing benchmarks evaluate editors on curated instructions [3, 24], while MLLMs can detect and assess the changes an edit makes [2]. In contrast, we do not train the editor. We hold it fixed (VAREdit [38]) and, as a second editor, FLUX.1 Kontext [5] and evaluate it on the instructions required to correct a generator’s actual failures, thereby measuring the editor gap.
Training generators from corrections and rewards. RPO [50] turns VLM critiques into edits and trains SD1.5 and SDXL with Diffusion-DPO [44] on (original, edited) pairs, and region-aware bimodal DPO [33] builds edit-based preference pairs for compositional generation. Online reinforcement learning with verifiable GenEval-style rewards [32, 37] reaches high GenEval scores on prompts generated from the benchmark’s own templates, and Diffusion-SDPO [13] addresses objectives that improve by degrading the losing sample. Unlike these methods, we train the critic rather than the generator and keep the editor fixed, so that gains can be attributed to the instruction alone.
Evaluation of compositional generation. GenEval [15] checks object presence, counts, colors and relations with a Mask2Former detector [7] and a CLIP [41] color classifier. TIFA [19] and VQAScore [31] score faithfulness with visual question answering. Other benchmarks use different judges, including T2I-CompBench [20] (unrelated to the editing benchmark CompBench), GenEval 2 [25], RichHF-18K [30] and EvalMuse-40K [17]. We build the oracle on GenEval and re-score its images with T2I-CompBench’s judges as a check on its detector (§ 6.5).
Problem Setup
We first define the correction loop and the quantities that LEDGE measures. A generator produces an image for prompt . A critic returns a decision and, for EDIT, an instruction . An editor produces , while KEEP and REVIEW retain . A verifier returns (Figure 1).
Repairs and regressions. For each image, a repair is and a regression is . We report both and test paired differences with an exact McNemar test [10, 39] on the discordant images, i.e., a two-sided binomial test of repairs against regressions.
Scores. GenEval’s overall score is the macro average of its six task accuracies (single object, two objects, counting, colors, position and color attribution); the micro score is the fraction of all images that pass. Repairs and regressions convert directly to micro, whereas macro weights each task equally (task sizes range from 320 to 400 images; see Appendix § 10).
Instruction and editor gaps. Let denote the score after editing with instructions , the critic’s instructions and the oracle’s. The instruction gap is , and the editor gap is the fraction of failures that remain unrepaired under . For an improved critic , the share of the gap closed is .
Oracle Instructions
GenEval stores each prompt’s requirements (objects, counts, colors and relations to other objects) and, for every image, the detector’s reasons for failure. We parse five reason types: an object is missing (fewer instances than required), excess (more than allowed, which applies to counting prompts only), wrong color, wrong relation, or no target (the reference object of a relation is absent). Every failure on the four generators we tested (3,802 images) parses without exception.
From diagnosis to instruction. A correct diagnosis is not yet an executable instruction. GenEval’s reason “expected handbag , found 2” says what is wrong, but an editor also needs to know how many objects to add, where to place them and which total to keep. The compiler therefore maps each reason to a structured edit whose arguments come from the prompt’s requirements. A missing object becomes an add with the number to add and the total to leave (“Add two more handbags . . . , leaving exactly four handbags”), placed on the correct side of its relation partner (“Add one computer keyboard above the zebra”) and added together with the partner when both are missing. Excess becomes a remove with the total to keep, wrong color a recolor of every instance, wrong relation a move of the object that carries the relation, and no target is resolved by adding the missing reference. Excess counts are first checked for duplicate detections: if overlapping boxes (IoU ) explain the excess, the image is sent to REVIEW rather than told to delete a real object (3 of the 3,802 failures). Edits are emitted in a fixed order (additions, removals, recolors, moves) and rendered with the same templates as the critic (Appendix § 11), so the oracle and the structured critic differ only in their diagnoses. The oracle’s instruction is one valid edit among several (moving the reference object would also fix a relation), so it is a reference for this instruction format rather than a bound over all possible instructions.
Note that the oracle reads the evaluator’s verdict. It is thus a reference for analysis and never a test-time method. It also inherits the evaluator’s errors, which we examine with human inspection and with independent judges in § 6.5.
Oracle-Supervised Critics
This section describes the critic’s output format, its training data, and how we train and select it. We refer to the original Qwen3-VL-8B-Instruct model as the baseline critic, its zero-shot variant with structured output as the structured zero-shot critic, and the LoRA-fine-tuned variant as the oracle-supervised critic.
Structured critique. The baseline critic is prompted to return one actionable error from four types, or KEEP/REVIEW. However, more than a quarter of our training failures (367 of 1,339) require two edits, including about half of the counting failures (Table 9). A single-error format cannot capture these cases. Both structured critics instead return {"decision": "keep|edit", "edits": [...]}, where each edit is an add, remove, recolor or move record with its object, count, color, and relation fields (Appendix § 12). The editing instructions are rendered from the structured edits using fixed templates. Applying the same rendering process to oracle edits reproduces their instructions exactly for all 3,799 failures that receive edit lists (the remaining 3 of the 3,802 are sent to REVIEW).
| Pilot tag | Images | Keep | Edit | Two edits |
| Position | 448 | 43 | 405 | 0 |
| Counting | 376 | 37 | 339 | 175 |
| Color attribution | 322 | 28 | 294 | 103 |
| Color + position | 272 | 4 | 268 | 89 |
| Colors | 74 | 62 | 12 | 0 |
| Two objects | 38 | 17 | 21 | 0 |
| Total | 1,530 | 191 | 1,339 | 367 |
Table 9. Oracle labels on pilot training images (SD1.5, corrected metadata).
Training data. We use a pilot set of GenEval-style prompts covering 20 COCO classes. It contains 765 training prompts and 188 development (DEV) prompts, with two SD1.5 samples each (1,530 and 376 images). The 20 classes appear in GenEval’s vocabulary, but 65% of the official test images (1,428 of 2,212) involve none of them. Before labeling, we correct the pilot metadata to the official conventions. The original export placed an upper-count constraint on every prompt and stored relations with forward indices; we restrict upper counts to counting prompts (affecting 577 of 765 training prompts) and re-score the existing images without regeneration. The oracle compiler then labels these training images: 1,339 failures receive edit lists and 191 correct images receive KEEP.
Train/test separation. Oracle labels supervise the critic only on the pilot prompts. No training or DEV prompt shares a test prompt’s specification (task, objects, counts, colors and relation), the pilot images are generated separately from the test images, and checkpoints are selected on DEV, so no test prompt, image or verdict is used for training or selection. Because GenEval prompts follow fixed templates, disjoint specifications do not rule out shared templates; we therefore also report results on the test images whose objects never appear in training (§ 6.5).
Fine-tuning. We train LoRA adapters [18] on the language model of Qwen3-VL-8B, with the vision encoder frozen. The model is trained to emit the target JSON given the image, the original prompt, and a rules prompt. Correct images are oversampled 3× to counter the 12.5% keep rate. Hyperparameters are given in Table 10; checkpoints are saved every 50 optimizer steps (717 in total).
| Base model | Qwen3-VL-8B-Instruct (bf16), vision encoder frozen |
| LoRA | rank 16, , dropout 0.05; q, k, v, o, gate, up, down of the language model |
| Optimization | AdamW, lr , no weight decay, 5% warmup then cosine, grad-norm clip 1.0 |
| Batch | 1 example, 8 accumulation steps |
| Data | 1,530 labeled images; correct images repeated 3× (1,912 examples) |
| Schedule | 3 epochs, 717 optimizer steps; adapters saved every 50 steps |
| Selection | Best DEV exact match with of correct DEV images edited → step 400 |
| Decoding | Greedy, up to 512 new tokens |
Table 10. Critic fine-tuning hyperparameters.
Checkpoint selection. The selection rule is fixed in advance. On DEV, we choose the checkpoint with the highest exact-diagnosis rate, subject to editing at most 10% of correct DEV images. Ties are resolved in favor of the earlier checkpoint. The official benchmark is used once, with the selected checkpoint (Appendix § 14).
Experiments
We use LEDGE to evaluate three claims. First, both an instruction gap and an editor gap exist and can be measured separately (§ 6.2). Second, oracle supervision closes much of the instruction gap (§ 6.3 and § 6.4). Third, the oracle-supervised critic generalizes beyond its training conditions (§ 6.5).
Experimental Setup
Generators and images. SD1.5 at and SD3.5-Large at generate four samples for each of GenEval’s 553 prompts (2,212 images). SDXL [40] and FLUX.1-dev [4] serve as reference points (Table 8). We verify that the SD1.5 images used in every comparison match their generation-time SHA-256 checksums (2,212 of 2,212) and that the run manifest matches the hash recorded when the baseline was scored.
Editors. VAREdit-8B at 512px [38] (CFG 3.0, , seed 42) is the main editor. FLUX.1 Kontext [dev] [5] (28 steps, guidance 2.5, seed 42) is the second editor (§ 6.5). Every edited output is stored at the source resolution.
Resolution control. VAREdit-8B outputs images. For 1024px generators we thus create a Lanczos-downscaled copy of the run. We use this copy for the baseline, all critics, the oracle, and the editor, ensuring that kept and edited images share the same resolution. The downscaled SD3.5 copy scores 72.0%, compared with 71.0% at native resolution.
Critics. We compare (i) the baseline critic (Qwen3-VL-8B with single-error prompting); (ii) the structured zero-shot critic, the same model with our structured format and no training; (iii) the oracle-supervised critic, i.e., (ii) after LoRA training; (iv) Molmo + rules, in which Molmo [9] is prompted zero-shot to report the observed objects, counts, colors and relations and a rule script converts them into an instruction; (v) generic, which edits every image with a fixed instruction that quotes the original prompt, asks the editor to make the image satisfy it, and asks it to leave a correct image unchanged; and (vi) the oracle.
Statistics. Paired differences use exact McNemar tests. To estimate variability, we bootstrap over prompts, stratified by task. The SD1.5 baseline has a standard error of 1.00 points macro (0.98 points micro) and a 95% confidence interval of 40.3–44.2%.
The Instruction Gap
As shown in Table 1 and Figure 2, oracle instructions repair 512 of 1,324 failures with the editor held fixed and raise GenEval from 42.2% to 64.3%. The baseline critic repairs 226 failures and reaches 50.9%, capturing 39% of the oracle’s gain over the original images. The instruction gap is therefore 13.4 points.
| Instructions from | Macro | Micro | Rep. | Reg. | |
| None (original) | 42.2 | 40.1 | – | – | – |
| Generic instruction | 49.9 | 48.4 | 246 | 63 | 0.33 |
| Baseline (Qwen) | 50.9 | 49.3 | 226 | 23 | – |
| Molmo + rules | 52.4 | 50.9 | 301 | 64 | 0.09 |
| Molmo, oracle-sup. | 56.5 | 55.2 | 345 | 12 | |
| Ours (oracle-sup.) | 59.4 | 58.3 | 420 | 18 | |
| Oracle (reference) | 64.3 | 63.3 | 512 | 0 |
Table 1. GenEval on SD1.5 images with the same VAREdit editor and the same 2,212 images. Rep./Reg.: repairs and regressions against the original images; : exact McNemar test against the baseline critic on the same images. Best non-oracle results in bold.
Figure 2. GenEval overall score on SD1.5 images by instruction source, same editor. Dashed lines mark larger generators without correction. With the oracle-supervised critic, SD1.5 plus correction exceeds SDXL; whether this holds at matched compute is not established.
A generic instruction repairs more failures than the baseline critic (246 vs. 226) but breaks nearly three times as many correct images (63 vs. 23). This suggests that the baseline critic’s advantage over the generic prompt comes from abstaining on correct images rather than from more useful instructions; paired on the same images, the two do not differ significantly (177 vs. 197 images; ).
The editor gap. Even with oracle instructions, 812 of 1,324 SD1.5 failures (61.3%) and 435 of 644 SD3.5 failures (67.5%) remain unrepaired (Figure 3). Counting is the hardest task, with 38 of 201 failures (19%) repaired, followed by position (29%) and color attribution (32%); on SD3.5, the repair rate for position falls to 16%. To see how the editor fails, we compare GenEval’s failure reasons before and after each oracle edit. Among the unrepaired SD1.5 images, an object the instruction asks to add is still missing in 56.3% (failed insertion), a recolored object keeps or receives the wrong color in 17.5% (incorrect attribute modification), a moved or added object is still misplaced in 18%, and a requirement that held before editing is now violated in 9.5% (unintended change); an image can fall into several categories (Appendix § 17). On a 600-image sample, regenerating each failure once with a new seed repairs 16% of failures against 37% for oracle-instructed editing, but as many on counting (15 vs. 13 of 58), where the editor is thus the bottleneck (Appendix § 16). Out of the 420 repairs of the oracle-supervised critic, 30 (7%) are not repaired by the oracle, against 56 of 226 (25%) for the baseline critic and 81 of 246 (33%) for the generic instruction.
Figure 3. Editor gap of VAREdit. Bars give the share of failures repaired when VAREdit receives oracle instructions; labels give repairs / failures per task.
Why the Baseline Critic Fails
Table 2 shows that the baseline critic is a good gate but a weak diagnoser. It keeps 89% of correct images but edits only 53% of failures, correctly identifying the error and object in 39% of failed images. Position is the hardest category to diagnose, with 58% of position failures sent to review. All 23 regressions come from the 65 correct images that the baseline critic chose to edit, a regression rate of 35% on those images. On counting failures where the detector found too few objects, the baseline critic asks to remove extras 84 times and to add objects 33 times. Part of this may reflect detector undercounts at GenEval’s 0.9 counting threshold.
| Qwen decision | Correct (888) | Failed (1,324) |
| KEEP | 789 (88.9%) | 164 (12.4%) |
| EDIT | 65 (7.3%) | 698 (52.7%) |
| REVIEW | 34 (3.8%) | 462 (34.9%) |
| Outcome on 1,324 failures | Share | |
| Right error type and object | 39.4% | |
| Sent to review | 34.9% | |
| Kept (error missed) | 12.4% | |
| Wrong error type | 10.4% | |
| Right type, wrong object | 2.9% |
Table 2. Baseline critic decisions against GenEval’s verdict on the original SD1.5 images (left) and its diagnoses of failures against the oracle (right).
Oracle-Supervised Critic Results
Diagnosis. Fine-tuning raises exact diagnosis from 33.3% to 70.8% (Table 3) and lowers edits on correct images from 22.0% to 6.5%, below the baseline critic’s 7.3%. The structured format alone brings no clear gain: zero-shot, it edits 22.0% of correct images, three times the baseline critic’s rate, and its any-match accuracy (43.6%) is close to the baseline’s 42.3%, a looser type-level match. The gain thus comes from training.
| Critic | Correct images | Failed images | |||
| Keep | Edit ↓ | Edit | Exact | Any | |
| Baseline (single error) | 88.9 | 7.3 | 52.7 | – | 42.3† |
| Structured zero-shot | 77.5 | 22.0 | 91.4 | 33.3 | 43.6 |
| Ours (oracle-sup.) | 93.5 | 6.5 | 89.0 | 70.8 | 79.3 |
| Prompt-only rule | 0.0 | 100.0 | 100.0 | 49.1 | 57.8 |
Table 3. Diagnosis on official SD1.5 images (888 correct, 1,324 failed) against the oracle, without editing. Exact: same edit set as the oracle; Any: at least one matching edit (†type-level only, not directly comparable). The gray row ignores the image and is tuned on the test failures, an upper bound for prompt-only critics. Best critic results in bold.
Does the critic use the image? GenEval prompts follow fixed templates, so a critic might predict errors from the prompt alone. A prompt-only rule that outputs each task’s most common edit pattern, chosen on the test failures, reaches 49.1% exact diagnosis (Table 3) but must edit every correct image; over all 2,212 images it is right on 29.4%, against 79.9% for the oracle-supervised critic and 51.0% for the structured zero-shot critic. Our critic exceeds the rule on position (84% vs. 44%), color attribution (58% vs. 29%) and two objects (78% vs. 59%), where the image decides the error, and trails it on counting (73% vs. 76%) and colors (51% vs. 60%), where one error type dominates.
By task (Figure 8 in the supplement), exact diagnosis rises from 40% to 84% on position, 13% to 58% on color attribution and 36% to 78% on two objects, although training contains only 19 two-object prompts. Colors barely improves (49% to 51%); the critic keeps 39 of its 99 failures.

Figure 8. Exact diagnosis by task on official SD1.5 failures. Training most improves color attribution, position and two-object diagnosis. Colors improves little; it is the task least represented in training (37 of 765 pilot prompts).
End-to-end correction. With the same editor, the oracle-supervised critic raises GenEval to 59.4% macro and 58.3% micro (Table 1), with 420 repairs and 18 regressions. It recovers 63% of the instruction gap (8.5 of 13.4 points) and 82% of the oracle’s repairs. Compared with the baseline, 273 images pass only under our critic, versus 74 passing
only under the baseline (), for a gain of 8.5 points macro.
Per task (Table 4), the oracle-supervised critic closes 87% of the gap on position and 71% on color attribution. Colors (22%) and counting (41%) lag behind. We hypothesize that color performance lags due to limited training data, and counting because the editor rarely succeeds (Figure 3).
| Task | Orig. | Qwen | Molmo | Ours | Oracle | Closed |
| Single object | 95.9 | 96.6 | 95.0 | 97.8 | 99.4 | 44% |
| Two objects | 34.3 | 58.1 | 53.0 | 69.7 | 76.5 | 63% |
| Counting | 37.2 | 40.0 | 45.0 | 43.8 | 49.1 | 41% |
| Colors | 73.7 | 81.9 | 78.7 | 84.0 | 91.5 | 22% |
| Position | 4.5 | 10.8 | 28.5 | 29.8 | 32.5 | 87% |
| Color attr. | 7.8 | 18.3 | 14.3 | 31.5 | 37.0 | 71% |
| Macro | 42.2 | 50.9 | 52.4 | 59.4 | 64.3 | 63% |
Table 4. Per-task accuracy on SD1.5 after editing. Qwen: baseline critic; Molmo: Molmo + rules; closed: share of the baseline-to-oracle gap closed by the oracle-supervised critic. Bold marks the best non-oracle result for counting and for the macro average.
A different-family critic. Molmo reaches 52.4% by reporting visual observations that a rule-based script converts into instructions. It makes more repairs than the baseline critic (301 vs. 226) but nearly three times the regressions (64), and paired against it the difference is not significant (201 vs. 167 images, ). It is the best zero-shot critic on counting (45.0%), consistent with Molmo’s pointing and counting training, and nearly matches the oracle-supervised Qwen critic on position (28.5%). Trained on the same oracle labels with QLoRA, Molmo reaches 56.5% with 345 repairs and 12 regressions (Table 1), 4.1 points above its zero-shot version with one fifth of the regressions. It recovers 65% of the oracle’s gain, against 78% for the oracle-supervised Qwen critic, so oracle supervision improves critics from both model families.
Generalization
Cross-generator transfer. SD3.5-Large also exhibits an instruction gap: oracle instructions improve GenEval by 9.1 points (from 72.0% to 81.2%), repairing 209 of 644 failed images (Table 5). The oracle-supervised critic, trained only on SD1.5 images, transfers with a gain of 4.0 points, 109 repairs and 16 regressions (1.0% of correct images). This is 44% of the oracle’s gain, compared with 78% on SD1.5. Position transfers best (62% of the oracle’s gain) and counting worst (28%; Appendix § 14). Paired against the baseline critic, the oracle-supervised critic repairs 71 images that the baseline critic does not, against 26 in the other direction (), and it closes 27% of the instruction gap on SD3.5 (1.9 of 7.1 points), compared with 63% on SD1.5.
| Instructions from | Macro | Micro | Rep. | Reg. | |
| None (SD3.5-L) | 72.0 | 70.9 | – | – | – |
| Baseline (Qwen) | 74.1 | 73.1 | 59 | 11 | |
| Ours (oracle-sup.) | 76.0 | 75.1 | 109 | 16 | |
| Oracle (reference) | 81.2 | 80.3 | 209 | 0 |
Table 5. Results on SD3.5-Large (downscaled to ) with the same VAREdit editor. The oracle-supervised critic was trained only on SD1.5 images. : exact McNemar test against the original images. Rep./Reg.: repairs and regressions. Best non-oracle results in bold.

Figure 4. Examples where the baseline critic’s correction fails and the oracle-supervised critic succeeds (same editor). Each row is one example; columns show the original image, the baseline critic’s result and ours, with GenEval’s verdict as the frame color (green: pass, red: fail) and the instruction below each edited image. Examples are hand-selected from the images whose original fails GenEval, whose baseline-critic correction still fails and whose oracle-supervised correction passes (258 on SD1.5, 63 on SD3.5).
SD1.5. Because the critic never saw an SD3.5 image during training or selection, this indicates that the error diagnosis learned from SD1.5 oracle labels transfers across generators, although by a smaller margin.
Unseen objects. On the 1,428 official images whose objects never appear in the critic’s training prompts, the oracle-supervised critic closes 63% of the instruction gap (59.1% against 50.3% for the baseline critic and 64.4% for the oracle; 173 vs. 53 paired repairs, ), the same share as on all images; per-split results are in Appendix § 19.
Human verification. Because the critic is trained on and evaluated by the same detector, one annotator, blind to the method, evaluates randomly sampled repairs for prompt satisfaction and visual artifacts. On SD1.5, 72 of 100 repairs (50 oracle, 50 ours) are confirmed (95% Wilson interval 62–80%); on SD3.5, 58 of 60 (89–99%). A second SD1.5 sample of 40 repairs per method confirms 80% of the baseline critic’s, 60% of ours and 70% of the oracle’s (, ours vs. baseline, Fisher’s exact test); 12 of our 16 rejected repairs are color-attribution or position prompts, where our critic gains most. Scaling repairs by human confirmation rates, minus regressions, yields 158 estimated net corrections for the baseline, 234 for ours, and 358 for the oracle. Our gain is 3.4 micro points versus GenEval’s 9.0, though small samples limit confidence. Nearly all sampled SD3.5 repairs are confirmed.
A second editor. With FLUX.1 Kontext [dev] and the same instruction files (Table 7), the instruction gap is unchanged: oracle instructions reach 64.9% and the baseline critic 51.6%, a gap of 13.3 points (13.4 with VAREdit). Our critic reaches 61.5% and closes 74% of it (63% with VAREdit); paired against the baseline critic, 294 images pass only under ours and 65 only under the baseline (). The editor gap is also nearly unchanged (60.6% of failures unrepaired with oracle instructions, against 61.3%), so both gaps persist across editors.
Table 7. A second editor on SD1.5 (GenEval, same 2,212 images and instruction files). Rep./Reg.: repairs and regressions against the original images.
| Editor | Instructions | Macro | Micro | Rep. | Reg. |
| VAREdit | Baseline | 50.9 | 49.3 | 226 | 23 |
| VAREdit | Ours | 59.4 | 58.3 | 420 | 18 |
| VAREdit | Oracle | 64.3 | 63.3 | 512 | 0 |
| FLUX.1 Kontext | Baseline | 51.6 | 49.9 | 231 | 15 |
| FLUX.1 Kontext | Ours | 61.5 | 60.3 | 450 | 5 |
| FLUX.1 Kontext | Oracle | 64.9 | 63.7 | 522 | 0 |
Table 7.
Independent judges. GenEval’s detector defines our labels, oracle, checkpoint selection and main evaluation. We therefore re-score the same SD1.5 images with the judges of T2I-CompBench [20], which play no part in training or selection: BLIP-VQA for colors, color attribution and object presence, and UniDet for position and counting. Prompts are rephrased into the judges’ format (e.g., “on the left of”); counting covers the 31 prompts whose objects the judge knows, for 2,016 images in total. As scores are continuous, we compare images whose score rises or falls with an exact sign test.
The ranking holds (Table 6): our critic raises the mean from 47.45 to 61.17, against 53.89 for the baseline critic (527 vs. 257 images, ), and recovers 97% of the oracle’s gain (78% under GenEval). Against the original images, the baseline critic improves 382 images and worsens 101, ours 650 and 87, and the oracle 716 and 88 (all ). The oracle’s lead shrinks from 4.89 points to 0.37, which suggests that part of it comes from fixes specific to GenEval’s detector. Spatial scores are low for every method because UniDet needs exact object names. On T2I-CompBench’s own 900 prompts, our critic also raises the judges’ mean from 31.3 to 45.8, against 35.7 for the baseline critic (, paired; Appendix § 15). These are new images from prompts outside GenEval’s templates.
Table 6. Re-scoring GenEval images with T2I-CompBench’s judges (SD1.5, VAREdit, 2,016 images; scores ×100). Color: GenEval’s colors and color attribution; object: single and two objects (BLIP-VQA); spatial: position; numer.: counting (UniDet). Mean: unweighted mean of the four columns. Oracle instructions come from GenEval, not from these judges. Best results in bold.
| Instructions | Color | Object | Spatial | Numer. | Mean |
| None (original) | 50.75 | 68.63 | 1.46 | 68.95 | 47.45 |
| Baseline critic | 61.68 | 79.80 | 3.53 | 70.56 | 53.89 |
| Ours | 75.81 | 85.52 | 10.36 | 72.98 | 61.17 |
| Oracle (GenEval) | 77.07 | 86.58 | 10.32 | 72.18 | 61.54 |
Table 6.
Conclusion
In this paper, we introduced LEDGE, which separates the two bottlenecks of text-to-image self-correction by holding the images and the editor fixed and compiling oracle instructions from the benchmark’s requirements. Oracle instructions repair far more failures than the baseline critic, yet 61% of SD1.5 and 68% of SD3.5 failures remain unrepaired under oracle instructions. The same oracle that exposes the instruction gap also helps close it. Using its labels as free supervision, we train a critic that improves exact diagnosis and recovers most of the instruction gap with fewer regressions. Trained only on SD1.5 images, the oracle-supervised critic also transfers to SD3.5-Large and to a second editor.
Given structured requirements and a verifier that identifies failures, LEDGE can evaluate different generators, critics, and editors. Correction methods should report repairs and regressions next to an oracle reference: on SD1.5, a generic instruction repairs more failures than the baseline critic, whose advantage comes from abstaining on correct images. The two gaps also need different remedies: supervision compiled from the verifier narrows the instruction gap, whereas the editor gap needs better editors.
Our labels, checkpoint selection and main evaluation share GenEval’s detector. The ranking of critics holds under T2I-CompBench’s judges and on its free-form prompts, and the gain holds on the 65% of images whose objects never appear in training, but the critic is trained with a single seed. Correction also adds an 8B critic and an 8B editor, so we make no matched-compute claim against larger generators. Future work could train editors for counting and position, where the editor gap is largest, and distill verified corrections into the generator.
LLM Usage Statement
OpenAI’s ChatGPT was used to assist with manuscript drafting, language refinement, grammar checking, and presentation clarity. Anthropic’s Claude was used to assist with writing experiment and analysis scripts (data preparation, evaluation, statistical tests, and figure generation). All experiments were carried out by the authors, who reviewed and verified all text, code, and results and remain responsible for scientific claims, experimental results, and conclusions.
Generator Baselines
Table 8 reports official GenEval scores for every generator at native resolution and for the SD3.5-Large copy used for correction.
Macro and Micro Scores
Repairs and regressions convert directly to micro: for the oracle-supervised critic on SD1.5, correct images, or . Macro averages the per-task changes. The net 402 images split as across single object, two objects, counting, colors, position and color attribution (task sizes 320, 396, 320, 376, 400, 400), changing those accuracies by points; their mean, , gives .
Macro rises less than micro because most repairs fall in the large tasks, whose images carry slightly less weight in the macro average.
Instruction Templates
Every instruction ends with “Preserve all unrelated objects, their attributes, and the background.” Sentences are emitted in the order additions, removals, recolors, moves.
| Edit | Template (example) |
| add (none present) | Add one blue pizza in a clear, visible area of the scene. |
| add (some present) | Add two more handbags in a clear, visible area of the scene, leaving exactly four handbags. |
| add (relational) | Add one baseball glove to the right of the bear. |
| add (both missing) | Add one baseball glove and one bear, with the baseball glove to the right of the bear. |
| remove | Remove two extra handbags, leaving exactly four handbags. |
| recolor | Change the color of the pizza to blue. |
| move | Move the baseball glove to the right of the bear[, leaving a clear gap between them]. |
Table 8.
Structured Critique Schema
The structured critic receives the image, the rules below and the line “Original prompt: p”, and returns JSON.
Unparseable or invalid outputs become REVIEW (original image kept), the same policy as the baseline critic. On official SD1.5 images the oracle-supervised critic produced no invalid outputs.
Training Data and Hyperparameters
Table 9 breaks down the oracle labels on the pilot training images by task, and Table 10 lists the fine-tuning hyperparameters.
Checkpoint Selection and Per-Task Figures
Figure 6 shows the trade-off that motivates the pre-declared rule. Exact diagnosis on DEV rises from 50.6% at step 50 to 68.2% at step 717, but the share of correct images edited rises to 20% after step 400, since 87.5% of the training labels are edits. The rule selects step 400 (65.5% exact, 4 of 40 correct images edited). Figure 7, 8 and 9 plot the per-task results discussed in § 6.4 and § 6.5.
Figure 6. Checkpoint selection on pilot DEV. Diagnosis improves steadily with training (top), but after step 400 the oracle-supervised critic edits more correct images (bottom). The rule fixed in advance selects step 400. With 40 correct DEV images, differences of one or two images are within noise.
Figure 7. Per-task GenEval accuracy on SD1.5 after editing, same editor (the data of Table 4).
T2I-CompBench Prompts
We also generate two SD1.5 images for each of the 900 validation prompts of T2I-CompBench’s color, spatial and numeracy categories and correct them with the same critics and editor (VAREdit). These free-form prompts are written in a different style from GenEval’s templates, and no oracle label or GenEval verdict is involved: the critics are applied as trained, and the images are scored only by T2I-CompBench’s judges (Table 11).
| Instructions | Color | Numer. | Spatial | Mean |
| None (original) | 36.87 | 45.50 | 11.43 | 31.27 |
| Baseline critic | 48.88 | 45.89 | 12.27 | 35.68 |
| Ours | 66.38 | 52.73 | 18.40 | 45.83 |
Table 11.
Resampling Instead of Editing
We select 150 GenEval prompts at random, stratified by task, and keep all their samples (600 SD1.5 images, 374 failures). Each image is regenerated once with SD1.5 and a new seed and scored with GenEval. A resample variant replaces an image by its regeneration wherever a method would edit it: every failure for the oracle-flagged variant, and every EDIT decision for the critics. Edited results are the VAREdit outputs of the main experiments on the same images (Table 12). Regeneration alone, even when the oracle selects which images to regenerate, stays below editing with the oracle-supervised critic, except on counting.
| Method | Macro | Micro | Rep. | Reg. | Single (6) | Two obj. (75) | Count. (58) | Colors (30) | Pos. (106) | C. attr. (99) |
| None (original) | 39.7 | 37.7 | – | – | – | – | – | – | – | – |
| Oracle-flagged resample | 49.7 | 47.7 | 60 | 0 | 66.7 | 26.7 | 25.9 | 40.0 | 2.8 | 6.1 |
| Baseline critic, resample | 43.6 | 41.5 | 33 | 10 | 0.0 | 13.3 | 22.4 | 20.0 | 0.9 | 3.0 |
| Ours, resample | 45.8 | 43.8 | 43 | 6 | 33.3 | 22.7 | 22.4 | 13.3 | 2.8 | 4.0 |
| Baseline critic, edit | 49.0 | 47.5 | 67 | 8 | 33.3 | 42.7 | 12.1 | 40.0 | 5.7 | 8.1 |
| Ours, edit | 57.6 | 56.7 | 118 | 4 | 50.0 | 57.3 | 10.3 | 40.0 | 32.1 | 20.2 |
| Oracle, edit | 61.8 | 60.7 | 138 | 0 | 66.7 | 61.3 | 22.4 | 63.3 | 34.9 | 19.2 |
Table 12. Resampling vs. editing on a 600-image SD1.5 sample (374 failures). Rep./Reg.: repairs and regressions against the original images. Right: share of failures repaired per task (number of failures in parentheses).
Editor Failures
For every failure that receives an oracle edit, we parse GenEval’s failure reasons before and after editing with the oracle’s own parser and compare them per object. A requirement is not executed if the same reason remains (the object is still missing, extra, miscolored or misplaced) and
| Generator | Macro | Micro | Single | Two obj. | Count | Colors | Position | Color attr. |
| SD1.5 | 42.23 | 40.14 | 95.94 | 34.34 | 37.19 | 73.67 | 4.50 | 7.75 |
| SDXL | 54.36 | 52.94 | 99.38 | 74.24 | 39.06 | 83.24 | 11.00 | 19.25 |
| FLUX.1-dev | 66.75 | 65.19 | 98.75 | 81.06 | 74.06 | 77.39 | 21.25 | 48.00 |
| SD3.5-Large | 70.99 | 69.85 | 99.06 | 89.65 | 70.00 | 83.24 | 26.00 | 58.00 |
| SD3.5-Large, copy | 72.02 | 70.89 | 99.06 | 89.90 | 71.88 | 84.04 | 27.25 | 60.00 |
Table 8 (PDF p. 13). Official GenEval ( prompts samples) for each generator at native resolution.
{"decision": "keep" | "edit", "edits": [ ... ]} # keep <=> edits is empty
{"type": "add", "object", "color"|null, "count_to_add", "total",
"relation"|null, "reference"|null, "reference_color"|null, "reference_missing"}
{"type": "remove", "object", "count_to_remove", "total"}
{"type": "recolor", "object", "color", "total"}
{"type": "move", "object", "relation", "reference", "gap"}
Figure 5. Structured critique schema used by the structured zero-shot and oracle-supervised critics.
mis-executed if the object now fails differently (e.g., added on the wrong side or in the wrong color). An unintended change is a new failure on an object that satisfied the prompt before editing. GenEval sees only the prompt’s objects, counts, colors and relations, so other changes to the scene are not counted.
Figure 9. Per-task accuracy on SD3.5-Large at 512px. The oracle-supervised critic transfers from SD1.5 without retraining, recovering 44% of the oracle’s gain; position transfers best (62%).
Table 11. T2I-CompBench’s own prompts (SD1.5, VAREdit, 1,800 images; scores ). Against the original images, the baseline critic improves 325 images and worsens 142 (, exact sign test) and ours 636 and 159 (); paired, ours improves 577 images over the baseline critic and worsens 266 (). Best results in bold.
Reproducibility
All scripts are in the project repository: oracle compilation (oracle_instructions.py), critic schema and rendering (critic_schema.py), critic training and inference (train_critic.py, critic_infer.py), diagnosis scoring (score_critic.py), paired comparison (compare_runs.py), pilot metadata correction and re-scoring (fix_pilot_metadata.py, rescore_tree.py), resolution control (downscale_run.py), splits (split_by_objects.py, subset_by_cases.py), the VAREdit runner (benchmark.py), the second-editor runner (edit_diffusers.py), the image controls (image_controls.py, the resample baseline (small_sample.py, run_resample.sh), the editor-failure analysis (editor_failures.py) and the T2I-CompBench re-scoring (export_geneval_for_compbench.py, compare_compbench.py). Generator runs record per-image SHA-256 checksums; the SD1.5 run manifest hash is 2ebbcbd7.... Model revisions are pinned in model_lock.json; the Qwen3-VL commit used for training is recorded in the adapter configuration.
Results by Object Split
Images are split by whether their prompt’s objects appear among the 20 classes of the training prompts: none (unseen, 1,428 images), some (mixed, 464) or all (seen, 320).
| Category | SD1.5 | SD3.5 |
| Unrepaired / failures | 812 / 1,324 | 435 / 644 |
| Failed insertion | 56.3% | 29.0% |
| Incorrect attribute modification | 17.5% | 14.7% |
| Failed move | 18.0% | 48.7% |
| Unintended change | 9.5% | 7.6% |
Table 13. Unrepaired images under oracle instructions (VAREdit), by failure category. Categories overlap; percentages are of unrepaired images.
| Split | Orig. | Base. | Molmo | Ours | Oracle | Closed |
| Unseen (1,428) | 41.7 | 50.3 | 55.3 | 59.1 | 64.4 | 63% |
| Mixed (464) | 15.5 | 30.2 | 40.4 | 43.2 | 47.8 | 74% |
| Seen (320) | 42.1 | 49.8 | 55.2 | 60.6 | 65.8 | 67% |
| All (2,212) | 42.2 | 50.9 | 56.5 | 59.4 | 64.3 | 63% |
Table 14. GenEval macro accuracy by object split (SD1.5, VAREdit). Base.: baseline critic; Molmo: oracle-supervised Molmo (QLoRA); closed: share of the baseline-to-oracle gap closed by the oracle-supervised critic.
References
- [1]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, et al. Qwen3-VL technical report. arXiv preprint arXiv:2511.21631, 2025. 2arxiv.org/abs/2511.21631
- [2]Lorenzo Baraldi, Davide Bucciarelli, Federico Betti, Marcella Cornia, Nicu Sebe, and Rita Cucchiara. What changed? detecting and evaluating instruction-guided image edits with multimodal large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16217–16226, 2025. 2
- [3]Samyadeep Basu, Mehrdad Saberi, Shweta Bhardwaj, Atoosa Malemir Chegini, Daniela Massiceti, Maziar Sanjabi, Shell Xu Hu, and Soheil Feizi. EditVal: Benchmarking diffusion based text-guided image editing methods. arXiv preprint arXiv:2310.02426, 2023. 2arxiv.org/abs/2310.02426
- [4]Black Forest Labs. FLUX.1 [dev]. https://huggingface.co/black-forest-labs/FLUX.1-dev, 2024. Model release. 4
- [5]Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, et al. FLUX.1 Kontext: Flow matching for in-context image generation and editing in latent space. arXiv preprint arXiv:2506.15742, 2025. 2, 4arxiv.org/abs/2506.15742
- [6]Tim Brooks, Aleksander Holynski, and Alexei A. Efros. InstructPix2Pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 2arxiv.org/abs/2211.09800
- [7]Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. 2arxiv.org/abs/2112.01527
- [8]Meng Chu, Senqiao Yang, Haoxuan Che, Suiyun Zhang, Xichen Zhang, Shaozou Yu, Haokun Gui, Zhefan Rao, Dandan Tu, Rui Liu, and Jiaya Jia. VisionDirector: Vision-language guided closed-loop refinement for generative image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9203–9212, 2026. 1, 2arxiv.org/abs/2512.19243
- [9]Christopher Clark, Jieyu Zhang, Zixian Ma, Jae Sung Park, Mohammadreza Salehi, et al. Molmo2: Open weights and data for vision-language models with video understanding and grounding. arXiv preprint arXiv:2601.10611, 2026. 5arxiv.org/abs/2601.10611
- [10]Thomas G. Dietterich. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7):1895–1923, 1998. 2
- [11]Xiaoqi Dong, Xiangyu Zhou, Nicholas Evans, and Yujia Lin. LumiGen: An LVLM-enhanced iterative framework for fine-grained text-to-image generation, 2025. 2arxiv.org/abs/2508.04732
- [12]Fengyi Fu, Lei Zhang, Mengqi Huang, and Zhendong Mao. FeedEdit: Text-based image editing with dynamic feedback regulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2661–2670, 2025. 2DOI
- [13]Minghao Fu, Guo-Hua Wang, Tianyu Cui, Qing-Guo Chen, Zhao Xu, Weihua Luo, and Kaifu Zhang. Diffusion-SDPO: Safeguarded direct preference optimization for diffusion models, 2025. arXiv:2511.03317. 2arxiv.org/abs/2511.03317
- [14]Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinf ei Yang, and Zhe Gan. Guiding instruction-based image editing via multimodal large language models. In International Conference on Learning Representations, 2024. 1, 2arxiv.org/abs/2309.17102
- [15]Dhruba Ghosh, Hannah Hajishirzi, and Ludwig Schmidt. GenEval: An object-focused framework for evaluating text-to-image alignment. In Advances in Neural Information Processing Systems, 2023. arXiv:2310.11513. 1, 2arxiv.org/abs/2310.11513
- [16]Hanzhong Guo, Jie Wu, Jie Liu, Yu Gao, Zilyu Ye, Linxiao Yuan, Xiongwei Wang, Yizhou Yu, and Weilin Huang. Leveraging verifier-based reinforcement learning in image editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. arXiv:2604.27505. 2arxiv.org/abs/2604.27505
- [17]Shuhao Han, Haotian Fan, Jiachen Fu, Liang Li, Tao Li, Junhui Cui, Yunqiu Wang, Yang Tai, Jingwei Sun, Chunle Guo, and Chongyi Li. EvalMuse-40K: A reliable and fine-grained benchmark with comprehensive human annotations for text-to-image generation model evaluation, 2024. arXiv:2412.18150. 2arxiv.org/abs/2412.18150
- [18]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. 4arxiv.org/abs/2106.09685
- [19]Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A. Smith. TIFA: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20406–20417, 2023. 2arxiv.org/abs/2303.11897
- [20]Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2I-CompBench: A comprehensive benchmark for open-world compositional text-to-image generation. In Advances in Neural Information Processing Systems, 2023. 1, 2, 8DOI
- [21]Yuzhou Huang, Jiangbin Xie, Xintao Wang, Ziyang Yuan, Xiaodong Cun, Yixiao Ge, Jiantao Zhou, Chao Dong, Rui Huang, Ruimao Zhang, and Ying Shan. SmartEdit: Exploring complex instruction-based image editing with multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8362–8371, 2024. 2arxiv.org/abs/2312.06739
- [22]Mude Hui, Siwei Yang, Bingchen Zhao, Yichun Shi, Heng Wang, Peng Wang, Chuang Xie, and Yuyin Zhou. HQ-Edit: A high-quality dataset for instruction-based image editing. In International Conference on Learning Representations, 2025. arXiv:2404.09990. 2arxiv.org/abs/2404.09990
- [23]Shantanu Jaiswal, Mihir Prabhudesai, Nikash Bhardwaj, Zheyang Qin, Amir Zadeh, Chuan Li, Katerina Fragiadaki, and Deepak Pathak. Iterative refinement improves compositional image generation, 2026. 1, 2arxiv.org/abs/2601.15286
- [24]Bohan Jia, Wenxuan Huang, Yuntian Tang, Junbo Qiao, Jincheng Liao, Shaosheng Cao, Fei Zhao, Zhaopeng Feng, Zhouhong Gu, Zhenfei Yin, Lei Bai, Wanli Ouyang, Lin Chen, Fei Zhao, Zihan Wang, Yuan Xie, and Shaohui Lin. CompBench: Benchmarking complex instruction-guided image editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1112–1122, 2026. 2arxiv.org/abs/2505.12200
- [25]Amita Kamath, Kai-Wei Chang, Ranjay Krishna, Luke Zettlemoyer, Yushi Hu, and Marjan Ghazvininejad. GenEval 2: Addressing benchmark drift in text-to-image evaluation, 2025. arXiv:2512.16853. 2, 3arxiv.org/abs/2512.16853
- [26]Weitai Kang, Xiaohang Zhan, Yizhou Wang, Mang Tik Chiu, Jason Kuen, Kangning Liu, and Yan Yan. Inline critic steers image editing, 2026. 2arxiv.org/abs/2605.12724
- [27]Mohammed Abdul Hafeez Khan, Yash Jain, Siddhartha Bhattacharyya, and Vibhav Vineet. Test-time prompt refinement for text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 6565–6575, 2025. 2arxiv.org/abs/2507.22076
- [28]Hongyu Li, Manyuan Zhang, Dian Zheng, Ziyu Guo, Yimeng Jia, Kaituo Feng, Hao Yu, Yexin Liu, Yan Feng, Peng Pei, Xunliang Cai, Linjiang Huang, Hongsheng Li, and Si Liu. EditThinker: Unlocking iterative reasoning for any image editor, 2025. arXiv:2512.05965. 2arxiv.org/abs/2512.05965
- [29]Zongjian Li, Zheyuan Liu, Qihui Zhang, Bin Lin, Feize Wu, Shenghai Yuan, Zhiyuan Yan, Yang Ye, Wangbo Yu, Yuwei Niu, Shaodong Wang, Xinhua Cheng, and Li Yuan. Uniworld-V2: Reinforce image editing with diffusion negative-aware finetuning and MLLM implicit feedback, 2025. arXiv:2510.16888. 2arxiv.org/abs/2510.16888
- [30]Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, Junjie Ke, Krishnamurthy Dj Dvijotham, Katherine M. Collins, Yiwen Luo, Yang Li, Kai J Kohlhoff, Deepak Ramachandran, and Vidhya Naval-pakkam. Rich human feedback for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2arxiv.org/abs/2312.10240
- [31]Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. In European Conference on Computer Vision, 2024. 2arxiv.org/abs/2404.01291
- [32]Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiacheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-GRPO: Training flow matching models via online RL, 2025. arXiv:2505.05470. 2arxiv.org/abs/2505.05470
- [33]Zhuohan Liu, Wujian Peng, Yitong Chen, and Zuxuan Wu. Compositional text-to-image generation via region-aware bimodal direct preference optimization. In Proceedings ofarxiv.org/abs/2605.28615
- [34]Yancheng Long, Yankai Yang, Hongyang Wei, Wei Chen, Tianke Zhang, Haonan Fan, Changyi Liu, Kaiyu Jiang, Jiankang Chen, Kaiyu Tang, Bin Wen, Fan Yang, Tingting Gao, Han Li, and Shuo Yang. SpatialReward: Bridging the perception gap in online RL for image editing via explicit spatial reasoning. In Proceedings of the 43rd International Conference on Machine Learning, pages 80476–80505, 2026. 2arxiv.org/abs/2602.07458
- [35]Xin Luo, Jiahao Wang, Chenyuan Wu, Shitao Xiao, Xiyan Jiang, Defu Lian, Jiajun Zhang, Dong Liu, and Zheng Liu. EditScore: Unlocking online RL for image editing via high-fidelity reward modeling. In International Conference on Learning Representations, 2026. arXiv:2509.23909. 2arxiv.org/abs/2509.23909
- [36]Yinyi Luo, Hrishikesh Gokhale, Marios Savvides, Jindong Wang, and Shengfeng He. Self-corrected image generation with explainable latent rewards. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. 1, 2arxiv.org/abs/2603.24965
- [37]Yihong Luo, Tianyang Hu, and Jing Tang. Reinforcing diffusion models by direct group preference optimization. In International Conference on Learning Representations, 2026. arXiv:2510.08425. 2arxiv.org/abs/2510.08425
- [38]Qingyang Mao, Qi Cai, Yehao Li, Yingwei Pan, Mingyue Cheng, Ting Yao, Qi Liu, and Tao Mei. Visual autoregressive modeling for instruction-guided image editing (VAREdit), 2025. arXiv:2508.15772. 1, 2, 4arxiv.org/abs/2508.15772
- [39]Quinn McNemar. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157, 1947. 2
- [40]Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 4arxiv.org/abs/2307.01952
- [41]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021. 2arxiv.org/abs/2103.00020
- [42]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. 1arxiv.org/abs/2112.10752
- [43]Shaocheng Shen, Jianfeng Liang, Chunlei Cai, Cong Geng, Huiyu Duan, Xiaoyun Zhang, Qiang Hu, and Guangtao Zhai. Agentic retoucher for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 29114–29125, 2026. 2arxiv.org/abs/2601.02046
- [44]Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2arxiv.org/abs/2311.12908
- [45]Junjie Wang, Xinghua Lou, Xiangtai Li, Ye Tian, Keyu Chen, Yulin Li, Bin Kang, Guangcan Mai, Yanwei Li, Zhuotao Tian, and Liqiang Nie. Benchmarking and evolving reason-reflect-rectify for reflective visual generation. In Proceedings of the 43rd International Conference on Machine Learning, pages 128684–128713, 2026. 1, 2arxiv.org/abs/2605.19639
- [46]Zitong Xu, Huiyu Duan, Yifei Nie, Mingda Du, Sijing Wu, Xionguo Min, Tianyi Zheng, Jian Zhang, Shusong Xu, Jinwei Chen, Bo Li, and Guangtao Zhai. EditRefiner: A human-aligned agentic framework for image editing refinement, 2026. 2arxiv.org/abs/2605.07457
- [47]Siwei Yang, Mude Hui, Bingchen Zhao, Yuyin Zhou, Nataniel Ruiz, and Cihang Xie. Complex-Edit: Cot-like instruction generation for complexity-controllable image editing benchmark, 2025. arXiv:2504.13143. 2arxiv.org/abs/2504.13143
- [48]Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. MagicBrush: A manually annotated dataset for instruction-guided image editing. In Advances in Neural Information Processing Systems, 2023. 2arxiv.org/abs/2306.10012
- [49]Shu Zhang, Xinyi Yang, Yihao Feng, Can Qin, Chia-Chih Chen, Ning Yu, Zeyuan Chen, Huan Wang, Silvio Savarese, Stefano Ermon, Caiming Xiong, and Ran Xu. HIVE: Harnessing human feedback for instructional visual editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2arxiv.org/abs/2303.09618
- [50]Hanyang Zhao, Haoxian Chen, Yucheng Guo, Genta Indra Winata, Tingting Ou, Ziyu Huang, David D. Yao, and Wenpin Tang. Fine-tuning diffusion models via rich preference optimization, 2025. arXiv:2503.11720. 1, 2




