WorksComputer Science
Can Image Agents Complete Visual Commissions? A Benchmark and Modular Framework
Abstract
Visual commissions require image agents to integrate heterogeneous source materials, preserve requirements across dependent stages, and produce coordinated deliverables. Existing evaluations examine specific capabilities and execution settings, leaving end-to-end commission completion less understood. We introduce Visual Commission Bench (VCB), comprising 600 tasks across 12 categories and three complementary tiers, from capability-focused tasks to complete commissions. VCB uses 5,039 requirement checks and separate visual quality ratings to assess content correctness, visual realization, constraint preservation, and delivery completeness without prescribing an internal workflow. We present V-Protocol, a reference agent that dynamically composes reusable task, cognitive, and expression modules at inference time. Task modules organize delivery, cognitive modules derive content, and expression modules guide visual realization. The modules exchange structured intermediate artifacts, while a shared execution harness tracks artifact revisions and dependencies, manages tool execution, and checks completion. With the same controller and image generator as the agent baselines, V-Protocol achieves 69.0% strict task success on VCB, compared with 42.0% for SCOPE. On Mind-Bench, strict accuracy increases from 39.2% to 45.4%.

Figure 1. Gallery for V-protocol.
Introduction
Modern image models follow detailed instructions Wu et al. [21] to produce high-quality visuals, with growing use in visual production tasks. A visual commission, however, may begin with incomplete requirements and heterogeneous source materials and involve multiple dependent stages before final delivery. For example, preparing promotional materials for a tea house may require identifying valid content, deciding how to present it across different assets, preserving relevant constraints through generation and revision, and producing a complete and coherent set of deliverables. Much of this process is still coordinated manually, while text-to-image generation serves as only one part of the workflow. We study whether image agents can carry out such visual commissions, from interpreting source materials through to producing the required deliverables, within a given execution budget. Figure 2 illustrates representative tasks and deliverables.
Figure 2. From image generation to visual commission completion. (a). An explicit prompt guides image synthesis. (b). A specified generation goal is supplemented with missing information through reasoning, search, and planning. (c). A commission and its source materials require the agent to determine content and design, preserve active constraints, and coordinate the requested deliverables. VCB evaluates content correctness, visual realization, constraint preservation, and delivery completeness. The panels illustrate task settings rather than mutually exclusive method categories.
Existing evaluations have extended image generation beyond direct prompt following by introducing tasks that require reasoning, search, or planning Zhang et al. [24]; Ren et al. [17]; Tang et al. [19]; Ki et al. [8]. These settings provide valuable tests of capabilities and execution demands involved in visual production. A further question is whether success in these settings translates into reliable completion of visual commissions, where agents must integrate requirements and evidence distributed across source materials, carry intermediate decisions and constraints through dependent subtasks, and coordinate multiple deliverables. Evaluating these demands together allows us to examine how constituent capabilities support completion of the overall commission.
To address this question, we introduce Visual Commission Bench (VCB), comprising 600 tasks across 12 categories and three complementary tiers: Specific-Capability (S), Multi-Stage (M), and Commission-Level (C). S tasks isolate capabilities relevant to visual task completion, such as reasoning, planning, information extraction, and retrieval. M tasks examine how intermediate results, decisions, and constraints are carried across dependent stages. C tasks evaluate complete visual commissions from source materials to coordinated delivery. The three tiers distinguish evaluation scope and task dependencies rather than intrinsic difficulty or tool-call counts, allowing us to study both individual capabilities and how they contribute to end-to-end task completion. All tasks are evaluated through their delivered artifacts without prescribing an internal workflow. 5,039 carefully designed requirement checks are used to measure requirement-level fulfillment and derive task-level success.
VCB further reveals execution challenges that become salient at the commission level. Some trajectories make little observable task progress despite substantial interaction, with agents repeatedly revisiting guidance and source materials, checking unchanged states, or recovering from similar interface errors. These behaviors suggest that commission-level execution depends not only on the capabilities available to the agent, but also on how appropriate task procedures are selected, reused, and coordinated throughout execution Li et al. [11]; Liu et al. [13]; Dong et al. [4]; Tan et al. [18]. This motivates us to examine structured support for both task guidance and execution management. Preliminary runs on VCB reveal execution difficulties in carrying visual commissions through to completion.
We present V-Protocol as a reference agent for end-to-end visual commission completion. V-Protocol organizes reusable task knowledge into task, cognitive, and expression modules, and dynamically selects compatible combinations based on the request. Task modules specify deliverables and execution dependencies, cognitive modules derive content from the inputs, and expression modules produce layout and rendering specifications. Selected modules exchange structured intermediate artifacts, while a shared harness Pan et al. [16]; Dantanarayana et al. [3]; Lu et al. [14]; Xia et al. [22] manages stage transitions, tool execution, and completion checks. Together, these components support controlled study of visual commission completion while keeping the underlying foundation models fixed.
Related Work
Benchmarks for image agents. WISE evaluates world knowledge integration Niu et al. [15], while Mind-Bench and KnowGen assess reasoning and search in image generation He et al. [6]; Feng et al. [5]. Gen-Arena measures entity and constraint satisfaction Ren et al. [17], and IA-Bench covers planning, reasoning, search, and memory Zhang et al. [24]. GraphicBench and IGenBench examine graphic design and infographic reliability, respectively Ki et al. [8]; Tang et al. [19]. VCB evaluates complete visual commissions that require agents to integrate distributed source materials, preserve requirements across dependent stages, and coordinate multiple deliverables. It assesses the delivered artifacts without prescribing an internal workflow.
Agentic image generation. Gen-Searcher and Mind-Brush integrate retrieval and reasoning into generation Feng et al. [5]; He et al. [6]. GenEvolve learns tool use from visual experience Chen et al. [2], and GEMS integrates memory and on-demand skills He et al. [7]. SCOPE maintains evolving semantic specifications for conditional skill orchestration Ren et al. [17], while Qwen-Image-Agent constructs generation context through planning and grounding Zhang et al. [24]. Other approaches use executable sketches Ye et al. [23], adaptive workflow routing Chen et al. [1], or evolving skill harnesses Li et al. [12]. V-Protocol coordinates reusable task, cognitive, and expression modules through structured intermediate artifacts, linking content derived from source materials to its use across visual deliverables.
Skills and execution management. AgentSkillOS and SkillSelect-Serve address skill organization and budget-aware selection Li et al. [9]; Zheng et al. [25]. SkillSmith models complementarity and conflict among skills Wei et al. [20]. GraSP composes skills into typed dependency graphs with verification and local repair Xia et al. [22]. Natural-language harnesses expose reusable execution policies Pan et al. [16], while SIGIL and ContractSkill make procedural guidance executable and checkable Dantanarayana et al. [3]; Lu et al. [14]. V-Protocol builds on these mechanisms to track artifact dependencies and completion conditions, leaving content and design decisions to the model. Visual checks assess requirement satisfaction alongside the harness’s checks on execution state.
Visual Commission Benchmark
Visual Commission Benchmark (VCB) evaluates the capabilities required for visual commission completion at different levels. Tasks require agents to identify valid content from source materials, decide how to present it across different assets, preserve relevant constraints through generation and revision, and produce a complete and coherent set of deliverables. Evaluation is based on the submitted artifacts, covering content correctness, visual realization, constraint preservation, and
| Benchmark | # Tasks | Checks / Ratings | Primary focus |
| WISE | 1,000 | 3-dim ratings | World-knowledge reasoning |
| Mind-Bench | 500 | 1,633 checks | Reasoning and search |
| KnowGen | 630 | 4-dim ratings | Search-grounded generation |
| Gen-Arena | 300 | 2,533 checks | Entity and constraint satisfaction |
| IA-Bench1 | 730 | 1,801 checks | Planning, reasoning, search, and memory |
| VCB (Ours) | 600 | 5,039 checks<br>+ 4-dim ratings | Distributed-source integration, dependent subtask execution, and coordinated multi-artifact delivery |
caption: Comparison of benchmark scale and evaluation focus. Checks denote task-specific verification items, while rating dimensions denote graded evaluation criteria. VCB broadens the evaluation toward end-to-end visual commission completion.
:::
delivery completeness without prescribing an internal workflow. Table 1 summarizes how VCB relates to existing evaluation settings. Figure 5 in the appendix provides an overview of V-Protocol, including adaptive module composition, protocol execution, and artifact revision management.
Figure 5. Overview of V-Protocol. Public requests and source materials define a task specification, from which the agent composes an executable program using task (T), cognitive (C), and expression (E) modules. Colored tags indicate module assignments; the dashed branch illustrates a revisable future operation. The model determines content, design, and tool arguments, while the harness checks runtime conditions and records artifacts. Verification feedback informs edits, regeneration, or program revision. The bottom panels illustrate how changes to declared sources make dependency snapshots stale, while changes to an image or relevant criteria invalidate its verification reports. Private benchmark annotations are used only for external evaluation.
Task Scope and Public Inputs
As shown in Figure 3, VCB comprises 600 tasks across 12 categories, with 50 tasks per category, organized into three complementary tiers: Specific-Capability (S), Multi-Stage (M), and Commission-Level (C). S tasks emphasize a focal capability: reasoning, planning, long-context information extraction, or retrieval. M tasks examine how capabilities work together through iterative editing, visual goal interpretation, diagnosis and repair, and evidence synthesis. C tasks involve visual argumentation, example-driven transfer, scientific visualization, and workspace-based design, requiring agents to make content and presentation decisions and organize the work needed for a complete delivery. The tiers provide a progression in evaluation scope and capability coordination without prescribing a fixed workflow or number of model and tool calls.
Figure 3. Overview of Visual Commission Benchmark (VCB). VCB contains 600 tasks across three tiers, 12 categories, and 45 subcategories. The S tier focuses on reasoning, planning, long-context extraction, and retrieval. The M tier combines capabilities through iterative editing, visual goal interpretation, diagnosis and repair, and evidence synthesis. The C tier evaluates integrated visual commissions through visual argumentation, example transfer, research visualization, and workspace design. Each panel presents an example input and output, with darker labels indicating the represented subcategories. Documents and workspaces are shown schematically, and the displayed outputs are illustrative rather than experimental results.
Each task provides a user request, access to its source materials, and designated output locations. Materials may include documents, tables, reference images, demonstrations, interaction histories, and workspace files. Task-relevant information and constraints may be distributed across the source materials, requiring agents to identify and integrate them. A task may require a single image, a sequence of edited images, or a coordinated set of visual artifacts. Private evaluation annotations specify how to assess requirements grounded in the request and materials, without adding undisclosed task goals or prescribing a unique valid solution. All tasks are self-contained evaluation packages and do not require additional user clarification during execution.
Construction and Annotation
Task sources and diversity. VCB draws on hard cases from existing text-to-image Li et al. [10] and image-agent benchmarks Ni et al. (2026), He et al. [6], Feng et al. [5], newly constructed tasks, everyday user requests, and completed design commissions. Real-world cases are anonymized and adapted into evaluation tasks while preserving their information dependencies, requirement changes, and delivery constraints. Diversity spans both content domains and task structures, including information selection, dependent operations, constraint preservation, and multi-artifact coordination. The resulting benchmark contains 600 tasks across 12 task categories and 45 subcategories, with 50 tasks per category and an equal split of 300 Chinese and 300 English tasks.
Task construction. Source cases are organized into evaluation tasks comprising a user request, source materials, and delivery requirements. Relevant information may be distributed across documents, images, demonstrations, and interaction histories, requiring agents to identify and integrate it. In total, VCB provides 4,288 source files containing approximately 9.81 million tokens of machine-readable text across task-specific materials and shared corpora. Token counts use cl100k_base and exclude text embedded in images. Existing design outcomes can inform task construction and review. For tasks that permit alternative designs, they serve as references rather than the only acceptable solutions.
Checklist construction and review. For each task, multiple models independently construct candidate checklists from its request and source materials. The participating models include GPT5.6-Sol, GPT6-Astra, GLM-5.3-Flash, and DeepSeek-v4.1-Flash. The candidate checklists are consolidated by merging overlapping checks, retaining complementary requirements, and resolving conflicting criteria. Each resulting check expresses a concrete judgment question and specifies the supporting evidence and images needed for evaluation. The models then assess the consolidated checks individually for correctness, grounding, and clarity, and examine the checklists for unsupported requirements, redundancy, and omissions. Human reviewers examine the proposed checks and model judgments to identify necessary revisions. The checklists are refined through repeated rounds of model assessment, human review, and revision.
External Evaluation
ReqPass and TaskPass. Evaluation is based on the artifacts saved at the designated output locations. Let denote the applicable checklist items for task , and let indicate whether item passes. Let denote the task-level delivery check, which verifies that all required artifacts are saved as specified and satisfy the applicable file-level requirements. We report requirement pass rate and strict task pass rate:
ReqPass pools applicable checklist items and one delivery check per task, while TaskPass requires all of them to pass. Checks that depend on a missing or invalid artifact fail; checks on other available artifacts remain eligible for partial credit. Conditional checks are included only when applicable, and revision checks follow the requirements active at the corresponding checkpoint.
Visual quality. Visual quality (VQ) assesses readability, layout, visual coherence, and aesthetic appeal separately from factual correctness. Each assessable output image receives a rating from 1 to 5 on each dimension. Let be the set of required output images for task , and let be the rating of image on dimension . We report
Missing or undecodable required images receive zero on all four dimensions. Ratings are averaged within each task before being averaged across tasks, giving each task equal weight.
Judgment protocol. File-level properties, such as existence, decodability, and specified dimensions, are checked programmatically. Qwen3.8-Flash-Next serves as the evaluator for both requirement satisfaction and visual quality, with separate inputs and criteria. For requirement evaluation, it receives the checklist question, its supporting criteria, and the images needed for that judgment. Source facts are incorporated into the criteria, while reference images are supplied when visual comparison is necessary. For VQ evaluation, it receives only the output image and the fixed quality rubric. Both evaluations are blinded to the identity of the evaluated system and exclude its execution trace. Evaluator failures or unresolved judgments are recorded separately and retried rather than treated as task failures or assigned zero VQ. Final scores are reported once all required judgments have been resolved.
V-Protocol
V-Protocol composes reusable modules into executable programs for visual commissions. The model selects procedures and determines content and design; a shared harness manages artifact dependencies, tool execution, and completion checks.
Module Representation
Task modules specify commission procedures and optional program templates; cognitive modules guide material interpretation, retrieval, reasoning, and synthesis; expression modules provide layout, style, and rendering guidance. Each module declares its family, capabilities, input and output types, applicability conditions, and procedural guidance. Registry metadata supports selection, with full guidance loaded on activation. Declared interfaces identify possible handoffs, while the model assesses their suitability for the current task.
Adaptive Module Composition
The agent derives a task specification from the public request and inspected materials, recording the goal, inputs, active requirements, and deliverables. A suitable task template initializes the program; otherwise, the model constructs it directly. At step , the program is
where nodes represent operations, edges record declared artifact dependencies, and records module assignments. Each operation specifies its objective, input and output artifacts, capability needs, and permitted actions. Source materials can supply inputs directly. Operations may use several modules, and modules may be reused across operations.
For each operation, the registry retrieves modules whose capabilities overlap its needs and orders candidates by configured priority and capability coverage. The model selects applicable guidance for the current objective and inputs, revising the program as inspection and generation reveal additional needs. Simple requests can use a minimal program. M
Protocol Execution
The harness validates the operation graph and artifact references. Before each action, it checks input availability, dependency freshness, permissions, and remaining budget. The model supplies content and tool arguments; the harness mediates calls and registers outputs. Node completion requires recorded artifacts and valid tool results. Program updates retain execution history and valid completed outputs while allowing future operations to change. Retries and repairs consume the same task budget.
Artifacts carry identifiers, revisions, and declared source dependencies. Dependency snapshots record the source versions and content hashes used by downstream operations. A change to a declared source makes the corresponding snapshot stale, preventing affected artifacts from being reused as current evidence until the dependency is resolved. Image edits record their source revision, and replacing an image at the same path creates a new revision.
Online verification compares the current image with active requirements and content checkpoints. Reports bind to the image identity, revision, content hash, and verification criteria; changes to the image or relevant criteria invalidate prior reports. The model uses feedback to choose edits, regeneration, or program revisions. Before delivery, the harness checks that required artifacts exist at their designated locations and satisfy runtime completion conditions. These checks concern recorded requirements and dependencies; semantic judgments remain the model’s responsibility. Private benchmark annotations are unavailable during planning, verification, and repair, and final artifacts are assessed through the external evaluation protocol.
Experiments
Experimental Setup
We evaluate V-Protocol on VCB, WISE-Verified, and Mind-Bench. We follow each benchmark’s evaluation protocol. For VCB, we report ReqPass, TaskPass, and visual quality (VQ), both overall and across the S, M, and C tiers. WISE-Verified uses binary WiScore, and Mind-Bench uses checklist-based strict accuracy (CSA).
We compare against direct Qwen-Image2.1 generation, Mind-Brush, GEMS, and SCOPE. Agent-based methods share a Qwen3.8-27B controller with medium thinking and a Qwen-Image2.1 image backend. Where web search is permitted, they use DuckDuckGo. Each method retains its own planning, generation, and revision procedure. Qwen3.8-Flash-Next evaluates VCB requirement satisfaction and visual quality using separate prompts. Published reference results are presented separately from experiments with shared backends. Additional adaptation, evaluation, and recovery details are provided in Appendix § 9.
Results and Analysis
VCB. Table 2 shows that V-Protocol achieves the highest overall scores, with 94.48% ReqPass, 69.00% TaskPass, and 4.200 VQ. Compared with SCOPE, the strongest baseline overall, V-Protocol improves ReqPass by 12.70 percentage points and TaskPass by 27.00 percentage points. Direct generation, Mind-Brush, and GEMS complete substantially fewer tasks.
| System | S RP↑ | S TP↑ | S VQ↑ | M RP↑ | M TP↑ | M VQ↑ | C RP↑ | C TP↑ | C VQ↑ | Overall RP↑ | Overall TP↑ | Overall VQ↑ |
| Direct | 8.33 | 0.50 | 2.396 | 12.15 | 0.00 | 2.039 | 12.81 | 0.00 | 2.091 | 11.14 | 0.17 | 2.175 |
| Mind-Brush | 13.83 | 1.00 | 2.794 | 6.71 | 0.00 | 2.010 | 15.32 | 0.00 | 2.121 | 12.69 | 0.33 | 2.308 |
| GEMS | 41.20 | 16.00 | 3.639 | 10.49 | 0.00 | 2.236 | 34.69 | 8.00 | 2.337 | 30.92 | 8.00 | 2.737 |
| SCOPE | 84.36 | 55.00 | 3.658 | 77.74 | 39.50 | 4.097 | 82.10 | 31.50 | 4.192 | 81.78 | 42.00 | 3.982 |
| V-Protocol | 96.95 | 82.00 | 4.250 | 91.78 | 67.00 | 4.157 | 94.15 | 58.00 | 4.195 | 94.48 | 69.00 | 4.200 |
Table 2. Commission completion on VCB. Each tier reports REQPASS (RP), TASKPASS (TP), and visual quality (VQ). RP and TP are percentages. VQ averages 1–5 ratings, with missing or undecodable required images assigned zero. Overall reports benchmark-wide results. V-Protocol achieves the highest scores, while complete commissions remain challenging even when visual quality is high.
Completion remains challenging as task scope expands. V-Protocol’s TaskPass decreases from 82.00% on S to 67.00% on M and 58.00% on C. SCOPE follows the same trend. V-Protocol’s C-tier ReqPass of 94.15% also contrasts with its 58.00% TaskPass, highlighting the remaining challenge of satisfying all requirements within a commission. Figure 4 complements the quantitative results with a qualitative comparison of visual outputs on selected VCB tasks.
Figure 4. Qualitative comparison. V-Protocol achieves better visual commissions completion.
Public benchmark for image agents. Table 3 shows that V-Protocol also achieves the highest overall scores among the compared systems on WISE-Verified and Mind-Bench. WISE-Verified has a more focused task scope than VCB, and leading systems already perform well. V-Protocol scores 0.9090, narrowly ahead of SCOPE’s 0.9010.
| System | Culture | Time | Space | Biology | Physics | Chemistry | Overall | Knowledge | Reasoning | Overall |
| GPT-Image-1.5 | 0.8900 | 0.6917 | 0.8833 | 0.8000 | 0.7583 | 0.7750 | 0.8250 | 0.220 | 0.180 | 0.210 |
| Nano Banana Pro | 0.8975 | 0.8167 | 0.9333 | 0.8167 | 0.8667 | 0.8750 | 0.8760 | 0.400 | 0.440 | 0.410 |
| Qwen-Image-Agent | 0.9200 | 0.9167 | 0.9333 | 0.8333 | 0.8667 | 0.9000 | 0.9020 | 0.404 | 0.436 | 0.420 |
| Direct | 0.4800 | 0.4667 | 0.5583 | 0.3417 | 0.5417 | 0.3083 | 0.4580 | 0.032 | 0.052 | 0.042 |
| Mind-Brush | 0.6550 | 0.6417 | 0.6750 | 0.3750 | 0.5583 | 0.3500 | 0.5740 | 0.136 | 0.060 | 0.098 |
| GEMS | 0.8625 | 0.7833 | 0.9000 | 0.7667 | 0.7917 | 0.7167 | 0.8200 | 0.228 | 0.336 | 0.282 |
| SCOPE | 0.9175 | 0.9333 | 0.9083 | 0.8583 | 0.8500 | 0.9000 | 0.9010 | 0.368 | 0.416 | 0.392 |
| V-Protocol | 0.9100 | 0.9083 | 0.9000 | 0.8750 | 0.9417 | 0.9167 | 0.9090 | 0.368 | 0.540 | 0.454 |
Table 3. Results on WISE-Verified and Mind-Bench. V-Protocol achieves the highest overall scores among the compared systems, with a larger advantage over SCOPE on reasoning-driven Mind-Bench tasks. WISE-Verified reports binary WiScore, and Mind-Bench reports CSA. Knowledge and Reasoning denote the corresponding category averages. The first three rows report published references; the remaining rows use shared backends. Bold marks the best results within the shared-backend group.
The difference is larger on Mind-Bench, where V-Protocol achieves 0.454 CSA compared with 0.392 for SCOPE. Their knowledge-driven averages are identical at 0.368, while V-Protocol improves the reasoning-driven average from 0.416 to 0.540. Its advantage on Mind-Bench is therefore Component analysis. We examine runtime enforcement and adaptive module selection on VCB. The runtime variant disables execution gates while retaining module selection and tool access. The selection variant replaces task-adaptive module choices with a predefined module set.
As shown in Table 4, the two variants achieve 60.17% TASKPASS and 60.83% TASKPASS, respectively, compared with 69.00% for the Full reference. Both also obtain lower REQPASS and VQ. These comparisons are exploratory because Full reuses historical outputs, including recovery runs, whereas the variants use a later frozen runtime. Detailed controls and additional results are provided in Appendix § 10.
| Configuration | RP↑ | TP↑ | VQ↑ |
| w/o Runtime Enforcement | 82.14 | 60.17 | 3.800 |
| w/o Adaptive Module Selection | 82.20 | 60.83 | 3.862 |
| Full V-Protocol | 94.48 | 69.00 | 4.200 |
Table 4. Component analysis on VCB.
concentrated in tasks requiring reasoning to determine the visual content. Together with the larger completion gap on VCB, these results show that comparable performance on knowledge-oriented generation does not necessarily imply comparable ability to complete visual commissions.
Conclusion
We introduced Visual Commission Bench (VCB) to evaluate image agents on complete visual commissions, from interpreting distributed source materials to delivering coordinated artifacts. V-Protocol supports these tasks by composing reusable task, cognitive, and expression modules through structured intermediate artifacts, while an execution harness tracks dependencies, revisions, and completion. With shared model backbones, V-Protocol achieves higher task success than the evaluated baselines. The commission-level results reveal substantial differences in completion despite comparable visual quality, demonstrating the importance of evaluating fulfillment of the entire request. Complete commissions remain challenging even with explicit execution support. Reliable use of intermediate results and preservation of requirements across revisions and deliverables remain priorities for further work.
Supplementary Material
V-Protocol
V-Protocol composes reusable modules into executable programs for visual commissions (Figure 5). The model selects applicable guidance, determines content and design, and revises future operations using inspected materials and verification feedback. A shared harness mediates tool execution, tracks declared artifact dependencies and revisions, and checks runtime completion conditions. Semantic judgments remain with the model.
Experimental Setup and Evaluation Metrics
Benchmarks and Evaluation Metrics
VCB contains 600 tasks across three tiers, with 200 tasks per tier and 50 tasks in each of the 12 categories. ReQPass pools applicable checklist items and one delivery check per task. TaskPass measures the fraction of tasks satisfying all applicable checks and the delivery check. VQ averages four ratings within each required image, then averages images within a task and tasks within the benchmark. Thus, TaskPass and VQ weight tasks equally, whereas ReQPass weights applicable checks equally. Overall ReQPass is computed from pooled counts rather than the arithmetic mean of the tier scores.
WISE-Verified contains 1,000 requests across Culture, Time, Space, Biology, Physics, and Chemistry. Its overall binary WiScore weights Culture by 0.40 and each remaining category by 0.12. Mind-Bench contains 500 requests across ten equally sized categories. A task passes checklist-based strict accuracy (CSA) only when all applicable checklist conditions pass. Overall CSA is computed across the 500 requests, while the knowledge-driven and reasoning-driven averages each cover five categories. Results are reported separately for the three benchmarks.
Backends and Execution Settings
The agent controller is Qwen3.8-27B with medium thinking, and the image backend is Qwen-Image2.1. Web-search calls use DuckDuckGo where permitted. VCB retrieval uses the benchmark-provided local corpora. Direct generation uses the image backend without an agent controller. The agent methods retain their own procedures for planning, tool use, generation, and revision. Shared backends do not imply equal numbers of model calls, image generations, or execution time. Published reference results retain their original configurations and are distinguished from the shared-backend comparisons. For dated requests, the target is the time specified by the benchmark.
Component Analysis and Module Budgets
Runtime enforcement. The runtime variant retains module selection, generation and editing tools, artifact registration, input validation, access restrictions, and resource limits. It disables specified execution gates, including node-action authorization, generation prerequisites, dependency-freshness checks, node-completion enforcement, and final-delivery enforcement. The intervention therefore removes selected execution constraints rather than the entire runtime. The agent may declare completion, but task success is determined independently from its delivered artifacts.
Adaptive module selection. The fixed-selection variant uses the same predefined module set for every request: task-deliverable-bundle, cognitive-evidence-synthesis, expression-basic, and expression-hybrid-composition.
Execution and evaluation. The new variants use the same controller and image backend, with limits of 256 steps, eight generation calls, and eight editing calls per task. Their source code and module definitions are frozen. Each task receives one execution, which may include generation and revision steps within these limits. No output-driven recovery run or task replacement is added afterward. Evaluation-service failures are retried separately. Missing or invalid deliverables remain subject to the benchmark’s failure and zero-VQ rules.
Module-budget setting. We examine restricted module budgets with . The budget counts distinct module identifiers loaded throughout a task across the task, cognitive, and expression axes, including default expression guidance. Reusing a module does not consume another slot, whereas loading a new module at a later stage does. The budget is an upper bound, not a requirement to load exactly modules, and no minimum allocation is imposed on any axis.
When the budget leaves no room for an expression module, the agent may use the base instructions and tool schemas without satisfying the expression-module prerequisite. Other artifact and execution constraints remain active. Table 5 compares these selected budget configurations with the historical Full reference.
| Configuration | RP↑ | TP↑ | VQ↑ |
| Adaptive selection, | 82.35 | 60.17 | 3.769 |
| Adaptive selection, | 84.24 | 59.83 | 3.853 |
| Adaptive selection, | 84.83 | 59.83 | 3.872 |
| Full V-Protocol (reference) | 94.48 | 69.00 | 4.200 |
Table 5. Selected module-budget results. caps the number of distinct modules loaded throughout a task across all three axes. RP and TP are percentages; VQ is measured on a 0–5 scale.
Execution Observations from SCOPE and V-Protocol
We examine retained execution traces of SCOPE Ren et al. [17] and V-Protocol on VCB (600 tasks), WISE-Verified (1,000 tasks), and Mind-Bench (500 tasks). The analysis documents cases in which repeated controller actions do not directly inspect task state or advance a visual artifact. It provides evidence for execution difficulties in visual tasks rather than isolating algorithmic causes.
Coverage and deployment. Shell-event counts use one retained log per task and method, rather than a complete history of all attempts. Earlier overwritten attempts and later manual recovery are not included in these trace statistics. The observations characterize the evaluated executions and their controller–tool interfaces; they do not by themselves identify limitations intrinsic to the original SCOPE algorithm.
Controller-issued waiting and repetition. Table 6 reports explicit shell waiting and repeated shell commands, using one retained log per task for each method. SCOPE’s VCB logs contain 5,386 sleep-containing shell records across 361 tasks and 2,652 repeated command records. V-Protocol’s corresponding logs contain no explicit shell sleep and 31 repeated commands across 28 tasks. Neither method is assumed to have zero backend waiting: the table captures controller-issued shell operations, not waits within tools. Both methods use the same sleep-detection pattern, \bsleep\s+\d, on recorded shell command strings. This lexical rule does not capture every possible waiting construct. Repeated commands are counted within each task after whitespace normalization, starting with the second occurrence. In particular, V-Protocol’s WISE-Verified and Mind-Bench traces contain no shell calls; their zeros describe the execution interface, not an absence of service latency.
| Benchmark | Method | Tasks | Tasks with sleep | Sleep records | Repeated commands |
| VCB | SCOPE | 600 | 361 | 5,386 | 2,652 |
| VCB | V-Protocol | 600 | 0 | 0 | 31 |
| WISE-Verified | SCOPE | 1,000 | 519 | 5,136 | 1,325 |
| WISE-Verified | V-Protocol | 1,000 | 0 | 0 | 0 |
| Mind-Bench | SCOPE | 500 | 358 | 8,970 | 3,829 |
| Mind-Bench | V-Protocol | 500 | 0 | 0 | 0 |
Table 6. Observed controller-issued shell operations. Each method has one retained log per task. Sleep counts records containing an explicit shell sleep; repeated commands count occurrences beyond the first identical command within a task after whitespace normalization. These are execution-pattern statistics, not counts of all wasted work. Tool-internal waits are excluded.
Repeated polling can be useful, so repetition alone is not evidence of waste. A clearer example is VCB task 0378_M4: the SCOPE controller submits sleep 1; echo ok 80 times, with 78 successful executions and two argument-parsing failures. This command neither checks task state nor produces a deliverable. The retained session contains 344 tool requests over 63.91 minutes. In 0120_S3, 212 of 453 tool requests contain sleep, and two wait-and-check commands each recur 12 times. These examples establish the existence of low-progress interaction, not its prevalence across all repeated calls. The first session’s duration is not attributed entirely to the repeated command.
Runtime intervention. V-Protocol’s retained VCB logs include 82 rejections tagged as no-progress submissions and four consecutive-identical call guard rejections. These events show that progress checks intervene during execution. They do not establish that every rejected action is followed by successful recovery. Argument and format errors also occur in both systems, but their different controller–tool interfaces do not provide a common basis for comparing raw error counts.
Controller usage and session duration. Table 7 summarizes recorded controller usage and session duration on VCB. The analysis uses tasks with recoverable controller usage and session timestamps for both methods, paired by task identifier without filtering by task success. Tasks without complete paired records are excluded from this analysis but remain in the benchmark evaluation.
| Method | Input (k) | Output (k) | Total (k) | Median time (min) | P90 time (min) |
| SCOPE | 4,895.7 | 54.7 | 4,950.4 | 18.45 | 57.01 |
| V-Protocol | 594.7 | 22.1 | 616.8 | 11.01 | 18.88 |
Table 7. Recorded controller usage on VCB. Token counts are per-task means in thousands, with cached input included. Time measures the retained session’s event span. V-Protocol records lower token volumes and shorter median and P90 session durations in these historical runs.
Token counts accumulate input and output across calls within the retained primary-controller session, including repeated context and cached input. Session duration measures the interval between the first and last recorded events, including tool waits. It does not include elapsed time across earlier attempts or subsequent manual recovery.
Cached input accounts for 96.2% of SCOPE’s recorded input tokens, while V-Protocol’s retained logs do not preserve the cache split. Token volumes therefore do not directly measure differences in uncached computation or monetary cost. The accounting covers the primary controller and excludes separate image-model computation, external evaluation, and auxiliary model calls not recorded in its usage. Because concurrency, service load, and recovery history were not matched, session durations are descriptive observations rather than a controlled latency comparison.
Implication. The concrete cases show that substantial controller interaction can occur without directly advancing a visual artifact. These observations motivate support for execution progress and artifact completion alongside reasoning about image content. They do not establish that the repeated shell operations alone explain the observed resource differences.
Figure 6. S1: Reasoning. The example compares liquid depth, base pressure, and total force for equal liquid volumes in containers with different base areas.
Figure 7. S2: Planning. The example translates a request for a seaside studio into a spatial arrangement supporting both work and relaxation.
Qualitative Examples
Task Inputs and Outputs
The following figures illustrate inputs and visual outputs for the twelve task categories. Long materials are summarized by excerpts or file representations for presentation. These illustrations explain the task settings rather than estimate benchmark performance.
Figure 8. S3: Long-Context Extraction. The example extracts the requested record from a long dossier and presents the relevant fields in a visual summary.
Figure 9. S4: Retrieval. The example requires locating the correct product image and current specifications before constructing the visual output.
Vector (PDF)M1 Iterative Editing
0243 · Four edits, one selective undo
Subject Appearance | Scene Editing Graphic Content Editing | Object & Style Editing
Input | Output
Tests: Preserve state while undoing one earlier change | Also: 0242 / 0248
Figure 10. M1: Iterative Editing. Successive edits modify the subject and scene. A later instruction reverses one earlier change while retaining the other active edits.
Vector (PDF)M2 Visual Goals
0297 · Read and solve a visual puzzle
Spatial Transformations | Symbolic Constraints | Path Reasoning
Input | Output
Tests: Infer the task from the image, then solve it | Also: 0267 / 0264
Figure 11. M2: Visual Goals. The task instructions and spatial constraints are embedded in the input image. The output visualizes the inferred solution.

Figure 12. M3: Diagnosis and Repair. The example concerns an inconsistent mirror reflection. Repair must address the reflected object while preserving the surrounding scene.

Figure 13. M4: Evidence Synthesis. The example combines product specifications, prices, and validity records to construct an evidence-grounded comparison.

Figure 14. C1: Visual Argumentation. The example uses a constructed case and a comic narrative to explain why compression can prevent recovery of the original information.

Figure 15. C2: Example Transfer. Three paired demonstrations define coloring, ceramic material conversion, and miniature-worker composition. The illustrated sequence applies these operations to a new object.
Figure 16. C3: Scientific Visualization. The example integrates supplied papers into a visual comparison of FCN, U-Net, and DeepLab, including their approaches to recovering spatial detail.
Figure 17. C4: Workspace Design. The example combines conversation history and source files into a coordinated coffee-shop identity, poster, and banner.
Comparison with Existing Systems
Figure 18–Figure 21 present selected outputs from GEMS, SCOPE, and V-Protocol on the same tasks. These qualitative cases illustrate differences in transformation realization, evidence coverage, and coordinated delivery.
Figure 18. Open workspace: earbuds. Inputs are shown above; outputs from GEMS, SCOPE, and V-Protocol (Ours) are shown below. Empty slots indicate missing deliverables.
Figure 21. Paper evidence: video understanding. Comparison of evidence coverage and information organization across the supplied papers.
Figure 19. Example transfer: bus-depot renewal. Comparison of topic adaptation and material transformation.
Figure 20. Diagnosis and repair: floating table. Comparison of physical-error detection and repair.

Figure 22. Qualitative gallery: visual identity. Typography, material, and composition in the V-Protocol identity illustration.

Figure 23. Qualitative gallery: brand and layout adaptation. Coordinated brand presentation across different page formats.
Additional Visual Gallery
The gallery presents selected visual examples grouped by theme, including identity, layout, illustration, and scene composition. These examples provide visual context and are mainly from the quantitative benchmark comparisons.

Figure 24. Qualitative gallery: character design. Character appearance, multiple views, and related presentation materials.

Figure 25. Qualitative gallery: poster design. Examples combining imagery, typography, and promotional composition.

Figure 26. Qualitative gallery: visual information. Examples organizing explanatory and promotional content through text and imagery.

Figure 27. Qualitative gallery: illustration and visual storytelling. Examples of stylized depiction and narrative composition.

Figure 28. Qualitative gallery: fashion and portraits. Examples of portrait treatment, styling, and editorial composition.

Figure 29. Qualitative gallery: landscapes and atmosphere. Examples varying scene composition, lighting, and environmental detail.

Figure 30. Qualitative gallery: imagined scenes. Examples combining familiar subjects with constructed visual settings.

Figure 31. Qualitative gallery: material and detail. Examples emphasizing surface appearance, texture, and object composition.



Figure 32. Qualitative gallery: motion and fantasy. Examples of dynamic poses and imaginative visual composition.

Figure 33. Qualitative gallery: people and characters. Examples of human and character depiction across different visual settings.

Figure 34. Qualitative gallery: culture and events. Examples drawing on cultural themes, seasonal imagery, and public events.
References
- [1]Harold Haodong Chen, Zhiyu Hou, Wen-Jie Shu, Weilin Ruan, Yingjie Xu, Litao Guo, and Ying-Cong Chen. Genrouter: Unified workflow routing for agentic image generation. arXiv preprint arXiv:2608.16721, 2026a.arxiv.org/abs/2608.16721
- [2]Sixiang Chen, Zhaohu Xing, Tian Ye, Xinyu Geng, Yunlong Lin, Jianyu Lai, Xuanhua He, Fuxiang Zhai, Jialin Gao, and Lei Zhu. Genevolve: Self-evolving image generation agents via tool-orchestrated visual experience distillation. arXiv preprint arXiv:2605.21605, 2026b.arxiv.org/abs/2605.21605
- [3]Jayanaka Dantanarayana, Savini Kashmira, Lingjia Tang, and Jason Mars. Sigil: Compiling agent skills into typed harnesses. arXiv preprint arXiv:2607.27309, 2026.
- [4]Gen Dong, Yanjie Gao, Liqun Li, Tianyin Xu, Yu Hua, and Fan Yang. Agent skills can be harmful: An empirical study of skill-induced failures in llm agents. arXiv preprint arXiv:2608.11888, 2026.arxiv.org/abs/2608.11888
- [5]Kaituo Feng, Manyuan Zhang, Shuang Chen, Yunlong Lin, Kaixuan Fan, Yilei Jiang, Hongyu Li, Dian Zheng, Chenyang Wang, and Xiangyu Yue. Gen-searcher: Reinforcing agentic search for image generation. arXiv preprint arXiv:2603.28767, 2026.arxiv.org/abs/2603.28767
- [6]Jun He, Junyan Ye, Zilong Huang, Dongzhi Jiang, Chenjue Zhang, Leqi Zhu, Renrui Zhang, Xiang Zhang, and Weijia Li. Mind-brush: Integrating agentic cognitive search and reasoning into image generation. arXiv preprint arXiv:2602.01756, 2026a.arxiv.org/abs/2602.01756
- [7]Zefeng He, Siyuan Huang, Xiaoye Qu, Yafu Li, Tong Zhu, Yu Cheng, and Yang Yang. Gems: Agent-native multimodal generation with memory and skills. arXiv preprint arXiv:2603.28088, 2026b.arxiv.org/abs/2603.28088
- [8]Dayeon Ki, Tianyi Zhou, Marine Carpuat, Gang Wu, Puneet Mathur, and Viswanathan Swaminathan. Graphicbench: A planning benchmark for graphic design with language agents. arXiv preprint arXiv:2504.11571, 2025.arxiv.org/abs/2504.11571
- [9]Hao Li, Chunjiiang Mu, Jianhao Chen, Siyue Ren, Zhiyao Cui, Yiqun Zhang, Lei Bai, and Shuyue Hu. Organizing, orchestrating, and benchmarking agent skills at ecosystem scale. arXiv preprint arXiv:2603.02176, 2026a.arxiv.org/abs/2603.02176
- [10]Niantong Li, Guangzheng Hu, Weixu Qiao, Ying Ba, Qichen Hong, Shijun Shen, Jinlin Wang, Fan Zhou, Jianye Kang, Xin Shang, Ziyi He, Wei Wang, Dalin Li, Jiahao Li, Jie Zhang, Kaiyuan Gao, Kun Yan, Lihan Jiang, Ningyuan Tang, Shengming Yin, Tianhe Wu, Xiao Xu, Xiaoyue Chen, Yuxiang Chen, Yan Shu, Yanran Zhang, Yilei Chen, Yixian Xu, Zekai Zhang, Zhendong Wang, Zihao Liu, Zikai Zhou, Hongzhu Shi, Yi Wang, Bing Zhao, Hu Wei, Lin Qu, and Chenfei Wu. Qwen-image-bench: From generation to creation in text-to-image evaluation. arXiv preprint arXiv:2605.28091, 2026b.arxiv.org/abs/2605.28091
- [11]Xiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, Zonglin Di, Yifeng He, Shenghan Zheng, Kyuoung Whan Choe, Jiankai Sun, Shuyi Wang, et al. Skillsbench: Benchmarking how well agent skills work across diverse tasks. arXiv preprint arXiv:2602.12670, 2026c.arxiv.org/abs/2602.12670
- [12]Zongxia Li, Dawei Liu, Fuxiao Liu, Yuhang Zhou, Xiyang Wu, Jingxi Chen, Jing Xie, Xiaomin Wu, and Lichao Sun. Comfyclaw: Self-evolving skill harnesses for image generation workflows. arXiv preprint arXiv:2607.01709, 2026d.arxiv.org/abs/2607.01709
- [13]Yujian Liu, Jiabao Ji, Li An, Tommi Jaakkola, Yang Zhang, and Shiyu Chang. How well do agentic skills work in the wild: Benchmarking llm skill usage in realistic settings. arXiv preprint arXiv:2604.04323, 2026.arxiv.org/abs/2604.04323
- [14]Zijian Lu, Yiping Zuo, Yupeng Nie, Xin He, Weibei Fan, Lianyong Qi, and Shi Jin. Contractskill: Repairable contract-based skills for multimodal web agents. arXiv preprint arXiv:2603.20340, 2026.arxiv.org/abs/2603.20340
- [15]Yuwei Niu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Fanqing Meng, Kun-Peng Ning, Bin Zhu, and Li Yuan. WISE: World knowledge-informed semantic evaluation for text-to-image generation. 2026.arxiv.org/abs/2503.07265
- [16]Linyue Pan, Lexiao Zou, Shuo Guo, Jingchen Ni, and Hai-Tao Zheng. Natural-language agent harnesses. arXiv preprint arXiv:2603.25723, 2026.arxiv.org/abs/2603.25723
- [17]Tianfei Ren, Zhipeng Yan, Yiming Zhao, Zhen Fang, Yu Zeng, Guohui Zhang, Hang Xu, Xiaoxiao Ma, Shiting Huang, Ke Xu, et al. Scope: Structured decomposition and conditional skill orchestration for complex image generation. arXiv preprint arXiv:2605.08043, 2026.arxiv.org/abs/2605.08043
- [18]Boyin Tan, Xiaowei Huang, and Youcheng Sun. Skill coverage: A test adequacy metric for agent skills. arXiv preprint arXiv:2606.20659, 2026.arxiv.org/abs/2606.20659
- [19]Yinghao Tang, Xueding Liu, Boyuan Zhang, Tingfeng Lan, Yupeng Xie, Jiale Lao, Yiyao Wang, Haoxuan Li, Tingting Gao, Bo Pan, et al. Ingenbench: Benchmarking the reliability of text-to-infographic generation. arXiv preprint arXiv:2601.04498, 2026.arxiv.org/abs/2601.04498
- [20]Yangbo Wei, Zhen Huang, Shaoqiang Lu, Junhong Qian, Qifan Wang, Chen Wu, and Lei He. Skillsmith: Co-evolving skills and tools for self-improving agent systems. arXiv preprint arXiv:2606.01314, 2026.arxiv.org/abs/2606.01314
- [21]Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report. arXiv preprint arXiv:2508.02324, 2025.arxiv.org/abs/2508.02324
- [22]Tianle Xia, Lingxiang Hu, Yiding Sun, Ming Xu, Lan Xu, Siying Wang, Wei Xu, and Jie Jiang. Grasp: Graph-structured skill compositions for llm agents. arXiv preprint arXiv:2604.17870, 2026.arxiv.org/abs/2604.17870
- [23]Junyan Ye, Jun He, Zilong Huang, Dongzhi Jiang, Xuan Yang, Rui Chen, and Weijia Li. Genclaw: Code-driven agentic image generation. arXiv preprint arXiv:2605.30248, 2026.arxiv.org/abs/2605.30248
- [24]Zekai Zhang, Jiahao Li, Jie Zhang, Kaiyuan Gao, Kun Yan, Lihan Jiang, Ningyuan Tang, Shengming Yin, Tianhe Wu, Xiaoyu Chen, et al. Qwen-image-agent: Bridging the context gap in real-world image generation. arXiv preprint arXiv:2606.26907, 2026.arxiv.org/abs/2606.26907
- [25]Jingyuan Zheng, Dongjing Wang, Xin Zhang, Butian Huang, Haiping Zhang, Dongjin Yu, and Shuguang Deng. Skillselect-serve: Budget-controllable and qos-aware skill service recommendation and composition for small llm agents. arXiv preprint arXiv:2607.00011, 2026.arxiv.org/abs/2607.00011













