WorksComputer Science
Do Transaction Prices Reflect Visual Preference? A Diagnostic Study with Multimodal Language Models
Abstract
Existing visual preference evaluation relies on costly human annotations or preference reward models, whereas online marketplace transaction prices are readily available and reflect real purchasing behavior, offering potential low-cost supervision. However, transaction prices conflate visual preferences with non-visual factors such as provenance, demand, and competition, making aggregate accuracy insufficient to distinguish model errors, unreliable price signals, and protocol confounds. To address this identifiability challenge, we introduce MarketImage, a diagnostic framework that identifies when transaction prices reflect visual preferences, where they fail, and whether multimodal language models can recover human-supported visual preference signals. MarketImage comprises three components: (i) MarketImageArt, 266 six-image comparison sets with transaction prices and price-blind human rankings; (ii) MarketImageLens, a tie-preserving protocol that partitions 3,990 within-set image pairs into price-human agreement, conflict, and price-tie regions while separately auditing presentation-order effects; and (iii) MarketValuator, six elicitation methods spanning direct comparison, independent scoring, budget constraints, and multi-agent aggregation to evaluate signal recovery by static visual scorers and three multimodal language models. Experiments show that only 24.2% of image pairs have a non-tied price direction, of which 34.7% agree with the human majority. Within this region, HPSv2 leads at 66.4% Recoverable Accuracy; Qwen's budgeted and aggregated variants trail baselines, while GPT interface gains remain statistically unresolved. Thus, transaction prices encode recoverable visual preferences only under limited conditions and should be treated as uncertain proxies rather than ground truth.
CCS Concepts • Computing methodologies → Artificial intelligence; • Human-centered computing → Human computer interaction .
WWW ’27, Dublin, Ireland 2027.
ACM Reference Format: Fengxian Ji, Jingpu Yang, Fan Zhang, Junhong Liang, Qian Jiang, Yuke Li, Jinri Guo, Rongxin Zhu, and Zhuohan Xie. 2027. Do Transaction Prices Reflect Visual Preference? A Diagnostic Study with Multimodal Language Models. In Proceedings of the ACM Web Conference 2027 (WWW ’27). ACM, New York, NY, USA, 14 pages.
With the growing adoption of generative vision models in practical applications, evaluating whether generated images meet human needs and preferences has become an important research problem [8, 18]. Existing methods rely primarily on human annotations [22], while using automated metrics [7, 11, 31], preference-trained reward models, and vision–language model judges to assess image quality and human preferences [15, 20, 28, 29]. Meanwhile, commercial recommendation research models preferences from implicit user feedback [6], and visual commerce research has examined images in real-world commercial settings, including their relationships with social signals and art market value [13, 16, 21]. These studies suggest that online commercial behavior provides observable feedback beyond human annotations; however, whether transaction prices can serve as reference signals for human visual preferences remains unverified.
As shown in Figure 1(a), existing visual preference evaluation typically uses human annotations as a reference and employs preference-based reward models and vision–language model (VLM) judges to approximate human judgments [15, 20, 22, 28, 29]; automated alignment metrics provide complementary signals for image evaluation [7, 11, 31]. Meanwhile, related work combines implicit user feedback with visual features for personalized preference ranking [6], or examines the relationships between visual and social attributes and art transaction prices [16, 21], as shown in Figure 1(b). However, using market information to support modeling or explain market outcomes does not establish that transaction prices represent human visual preferences. We therefore test whether
Figure 1. Conceptual progression from (a) LLM judgments evaluated against price-blind human rankings, to (b) prior price-assisted prediction with human preferences as targets, and (c) our test of transaction prices as candidate preference labels. Panel (c) measures price coverage and agreement with human rankings while preserving ties and conflicts. Prices are treated as a protocol-dependent proxy, not assumed to be ground truth.
Related Work
Visual Preference Evaluation and Market Signals. Existing visual evaluation methods use alignment metrics, aesthetic predictors, and preference or reward models trained from human feedback. CLIPScore [7] measures text–image alignment, while Pick-a-Pic [15] supports learned preference scoring. Related studies examine self-supervised and multi-task aesthetic assessment, personalized aesthetic preferences, and art-preference prediction from visual features [9, 24, 26, 30]. Other work suggests that aesthetic judgments can be partly dissociated from semantic content [23], while recent reward models incorporate richer human feedback [2, 20]. Preference-based VLM rewards have also been used to improve visual text rendering [4]. Fine-grained multimodal benchmarks diagnose visual-grounding failures through stage-wise metrics and expose VLM–human gaps on visual-text tasks [14, 32]. Beyond explicit annotations, visual recommendation models use implicit user feedback to learn personalized rankings [6], and visual-commerce research relates image properties and social signals to commercial outcomes, including human payment decisions and art-market value [13, 16, 21]. However, whether observed transaction-price ordering can directly serve as a reference for human visual preference remains unclear. Unlike price prediction, our work examines the availability, human corroboration, and evaluator recoverability of price-implied preference directions.
Market Mechanisms and Evaluation Uncertainty. Market mechanisms derive prices, allocations, or rankings through interactions involving agents, budgets, and competition. Prior studies investigate auction-based LLM agents [3], fixed-budget voting for multi-issue decisions [5], and token-curated registries for incentive-aware content curation [10]. Auction-inspired mechanisms have also been incorporated into generative modeling, including auction-driven diffusion and multi-player GAN training [25, 27]. Such interactions may produce strategic behavior, including collusion, that alters market outcomes [1]; transaction prices may therefore contain visually relevant information while also reflecting market demand, competition, and allocation. Related LLM-based financial reporting workflows use evidence-grounded agents to normalize disclosures across jurisdictions [33]; that setting differs from using realized market prices as candidate visual-preference signals. Complementary benchmarks evaluate multi-agent debate across domains and modalities [17]. Human and model judgments can also vary with evaluation protocols, including grading scales and stylistic presentation [12, 19, 22]. Motivated by these uncertainties, we separately diagnose price-reference ambiguity, presentation-order confounds, model recovery, and elicitation effects rather than interpreting aggregate agreement as visual-preference accuracy.
MarketImageArt
Task Formulation
For comparison set , let denote its candidate images, where and . These are benchmark evaluation sets formed within a price stratum; they are not claimed to be six lots from one observed auction. Panel (1) of Figure 2 illustrates one such set alongside its transaction-price signal and human-preference ranking. Artwork has transaction price and unobserved non-visual market factors , such as provenance, timing, demand, and competition. We represent price as
Figure 2. Overview of the MarketImage workflow. (1) MarketImageArt pairs each constructed six-image set with transaction prices and price-blind human rankings; sets are not claimed to be single observed auctions. (2) MarketImageLens compares visual-evaluator rankings with price rankings and MarketValuator rankings with human rankings, preserving ties. (3) MarketValuator illustrates image-based, multi-agent elicitation with and without budget and auction constraints; the evaluated interfaces do not replay the original transactions. Here, , , , and denote price, human, visual-evaluator, and MarketValuator rankings; BA and VA are normalized rank agreements for the two comparisons in (2). Source order is audited separately as a protocol confound.
where denotes latent visually grounded factors and is an unknown price-formation process. This is a conceptual decomposition: we do not estimate or causally separate its inputs. Instead, we audit whether pairwise directions derived from the observed vector are available and corroborated by independently elicited visual judgments.
Benchmark Construction
MarketImageArt is constructed from real emerging-art auction records through data cleaning and comparison-set organization. Data cleaning: We initially restrict prices to EUR 10–1,000 and retain art-print listings with valid normalized prices and retrievable images. We remove inconsistent records and evaluator-visible auction metadata. The final sample contains 1,596 artworks priced from EUR 10 to EUR 395 over 2020–2024.
Comparison-set construction: As shown in Table 1, the artworks are partitioned into low-, mid-, and high-price strata and organized into 266 six-image sets. Low-, mid-, and high-price subsets are disjoint views of these sets. Near-tie-low and Near-tie-high each select 53 sets from Overall by the smallest and largest within-set relative price gap. They are derived stress tests rather than independent samples. A post-construction audit found that every canonical set is stored in non-decreasing price order; we retain this historical order, disclose it, and test model-side order sensitivity instead of silently rewriting completed experiments.
| Overall Price Statistics (EUR) | ||||||
| Subset | Min | Q25 | Median | Mean | Q75 | Max |
| Overall | 10 | 40 | 70 | 96.9 | 121 | 395 |
| Normal Scenarios | ||||||
| Subset | #Items | #Groups | Min | Max | Mean | Median |
| Overall | 1596 | 266 | 10 | 395 | 96.9 | 70 |
| Low-price | 858 | 143 | 10 | 75 | 43.2 | 45 |
| Mid-price | 504 | 84 | 75 | 170 | 112.2 | 110 |
| High-price | 234 | 39 | 170 | 395 | 260.5 | 250 |
| Stress-Test Scenarios | ||||||
| Near-tie-low | 318 | 53 | 10 | 45 | 30.1 | 30 |
| Near-tie-high | 318 | 53 | 12 | 390 | 121.5 | 78 |
Table 1. MarketImageArt price statistics. Low-, mid-, and high-price sets partition the 1,596 items; Near-tie-low and Near-tie-high are derived 20% set views, not additional artworks.
Visual preference annotation: The released reference uses three of four complete annotator files; the exclusion rationale is undocumented. We use this three-annotator consensus for the primary analysis and report the all-four sensitivity analysis in Appendix A (Table 7). Each included annotator completed all 266 Overall sets. Annotators saw the six images, folder IDs, and numeric source-position labels, but not prices, estimates, or artwork metadata. Images remained in canonical source order, so the annotations are price-blind but not order-randomized. The primary human reference is the annotator-averaged pairwise preference probability; a Kemeny consensus order is retained only for visualization. Pairwise annotator agreements are 65.8%, 61.8%, and 67.6% (Kendall’s , 0.236, and 0.352); leave-one-out agreement with the other two annotators’ pairwise consensus is 70.4%, 77.0%, and 72.3%. Reference-signal construction: MarketImageArt thus provides aligned transaction-price and price-blind human-preference signals.
| Measure | Selected 3 | All 4 |
| Price-ordered pairs | 966 | 966 |
| Human ties | 0 | 213 |
| Decisive human pairs | 966 | 753 |
| Price–human supported pairs | 335 | 257 |
| Price–human conflicts | 631 | 496 |
| Price–human agreement (%) | 34.7 | 34.1 |
| Recoverable Accuracy (%) | ||
| HPSv2 | 66.4 | 66.5 |
| Qwen Pair | 8.4 | 8.2 |
| GPT-5.5 Pair | 60.3 | 61.5 |
| GPT-5.6 Pair | 62.1 | 62.3 |
Table 7. Post-hoc annotator-count sensitivity, comparing the released three-file consensus with all four complete files. The all-four analysis retains 2:2 votes as ties. Price–human agreement conditions on price-ordered pairs with decisive human votes (966 for three annotators; 753 for four); recovery uses each condition’s supported subset and the same saved model outputs.
aligned transaction-price and price-blind human-preference signals. Section 3.1 defines how MarketImageLens compares them pairwise while preserving price ties.
Benchmark Validation
Appendix Figure 5 summarizes the data-quality and set-structure audit. The sample spans EUR 10–395 (median EUR 70) over 2020–2024; the image payload excludes explicit prices, estimates, lot identifiers, URLs, and auction metadata. Only 2 of 1,596 titles retain an artist-name mention, and titles are not supplied to evaluated models. The 266 six-image sets are item-disjoint and concentrated in an outcome-near-tie regime: 87.2% have zero top-two price gap and 57.9% have all six prices tied. Canonical order is price-sorted, making source position an implicit target cue. Appendix A reports the full reference audit and seeded model-side order intervention. Thus, explicit metadata leakage is controlled, but presentation-order exposure remains part of the benchmark.
MarketImage Framework
Figure 2 summarizes the three-stage workflow: MarketImageArt provides aligned transaction-price and human-preference signals, MarketImageLens audits their pairwise relation and evaluator recovery, and MarketValuator tests whether alternative elicitation interfaces change that recovery. The framework keeps reference validity, evaluator performance, and interface effects analytically separate.
MarketImageLens
MarketImageLens analyzes heterogeneous value signals in a common pairwise space while preserving ties. For a set-level signal , the direction of a within-set comparison is
where is transaction price, is a visual-evaluator output, and is a signal elicited by MarketValuator. The human direction is obtained from the price-blind pairwise majority under the recorded fixed presentation order. Because the human reference preserves that source order, these comparisons are protocol-conditional; Appendix A details the reference audit and seeded order intervention.
The lens partitions comparisons into three observable reference regimes: price–human agreement, price–human conflict, and price ties. It first audits whether price provides a direction and whether that direction is corroborated by the recorded human majority. Evaluator recovery is then assessed on the protocol-supported subset, while conflict and tied-price pairs remain separate diagnostic regions. Finally, paired changes between direct, budgeted, and multi-profile interfaces measure system-level elicitation contrasts within a fixed base model. This decomposition prevents a model error, an uninformative price tie, and a visually unsupported market outcome from receiving the same interpretation. Table 2 summarizes the diagnostics and conclusions licensed at each stage.
| Stage | Observable condition | Primary diagnostic | Licensed conclusion |
| Reference | or | Coverage; price–human agreement | Price direction is absent or lacks human support |
| Protocol | Position correlates with or | Source-order audit; permutation | Reference or prediction is order-confounded |
| Recovery | , but | Conditional accuracy; ties and margin | Evaluator disagrees or ties on a supported reference |
| Elicitation | Mechanism changes direct baseline | Paired contrast; correction/harm | Interface repairs, preserves, or damages judgments |
Table 2. Stage-wise decision map. , , and denote price, human-majority, and evaluator pair directions; downstream scores cannot validate an unsupported reference.
MarketValuator
As illustrated in Figure 3, MarketValuator varies how image-based relative values are elicited; the tested interfaces do not replay the original auctions. Given comparison set , bidder profile , and budget , the simultaneous interface returns
Figure 3. Conceptual comparison between direct visual judgment and budget-constrained multi-agent valuation. The implementations evaluated in § 3.2 are elicitation interfaces, not reproductions of the original auctions.
All candidates are therefore judged jointly under a shared resource constraint. With three bidder profiles, the reported API-native Valuator signal is the per-item second-highest bid,
Valuator Judge is the non-monetary ablation: each profile returns a 0–100 vector and is their mean. Budgeted Single-agent uses one constrained vector without profile aggregation.
The legacy Qwen Valuator follows a different call graph. It presents items sequentially, retains bidding history and remaining-budget context, and uses the terminal accepted bid as its score. The API-native replication instead elicits three independent simultaneous allocations and applies (4); the bidders do not observe one another’s responses. We preserve both completed implementations but compare an interface only with baselines using the same base model. These are system-level contrasts, not token-matched estimates of a pure competition effect. MarketImageLens then maps every resulting vector into the common tie-preserving pairwise space. Appendix B reports the complementary audit of budget use, bidder disagreement, and aggregation behavior.
Experiments
Diagnostic Questions
The experiments do not assume that transaction prices are visual-preference labels. Instead, they separate three questions that were conflated by the set-level evaluation in the earlier design: RQ1 asks when transaction-price order is identifiable and agrees with price-blind human visual preference under the observed presentation protocol; RQ2 asks whether an evaluator recovers price order on that protocol-supported subset; and RQ3 asks whether budgeted and multi-profile interfaces improve or damage this recovery relative to matched comparison methods. This ordering is important: mechanism performance cannot establish label validity, and agreement with a price tie cannot establish directional recovery.
Experimental Setup
Evaluators. We re-score the eight static visual evaluators used in the original experiment (CLIPScore, NIMA, HPSv2, PickScore, ImageReward, RichHF, Q-Align, and LLaVA-Reward) and six Qwen3-VL-4B elicitation variants: Single-agent (Pair), Single-agent, Single-agent (LLM Score), Budgeted Single-agent, Valuator Judge, and Valuator. We additionally replicate all six interfaces with GPT-5.5 and GPT-5.6 Sol (runner identifiers gpt-5.5 and gpt-5.6-sol) through ephemeral Codex-hosted sessions with medium reasoning effort. Every row covers the same 266 comparison sets. Joint interfaces use the price-sorted canonical source order; outputs are aligned by the unordered set of internal item identifiers and scored in canonical orientation. Appendix A audits this exposure, and Appendix A reruns the Pair interface under seeded permutations for the two GPT models. Higher scores indicate stronger preference or value.
API interface replication. The API runs preserve the information and decision constraints of each interface while using structured outputs. Single-agent independently predicts one price per image, and Single-agent (LLM Score) independently assigns one 0–100 score per image. Single-agent (Pair) ranks all six images jointly. Budgeted Single-agent allocates bids under a EUR 600 set-level budget cap. Valuator Judge averages scores from three independent judge profiles, whereas Valuator uses three independently budgeted bidder profiles, each with the same cap, and the per-item second-highest bid as its aggregation statistic. The controlled user instruction contains no transaction price, human preference, or artwork metadata and explicitly prohibits using file names. Images are attached by the runner, and joint calls retain canonical position. These runs are API-native replications of the six elicitation interfaces rather than token-identical replays of the local Qwen implementation; raw outputs and allocation traces are retained for audit.
Canonical pair index. Each six-image set contributes within-set pairs, giving
We never compare images across sets because human references are collected within-set and joint budgeted outputs are not calibrated across sets. For pair , the price direction is
where denotes an exact transaction-price tie. From the three complete human rankings, we compute
Thus and has no tie. We retain both 2:1 majority and 3:0 unanimous judgments and report them separately for RQ1.
Reference reliability (RQ1). Price Coverage is the fraction of pairs for which . Let and if , otherwise . Conditional on price-ordered pairs, hard and soft price–human agreement are
Price-tied pairs are excluded from both denominators rather than being treated as chance-level agreement.
Evaluator recovery (RQ2). For evaluator scores , the predicted direction is . We preserve exact prediction ties and never jitter scores or randomly break rankings. The primary recovery set is
which contains pairs for which the observed price direction agrees with the price-blind human majority under the recorded presentation protocol. We call this subset protocol-supported; membership is observational and does not identify a causal visual component of price. Define the directional score
Recoverable Accuracy is
where exact prediction ties receive half credit, and we report their frequency separately to expose methods that collapse to ties. For any pair subset and target vector , define . Human Accuracy is ; Price Accuracy is ; Conflict→H is ; and Price-tie→H is . Thus, the latter two columns measure each evaluator’s agreement with the human majority within those reference strata, not inter-annotator agreement.
Uncertainty estimates and completeness. All confidence intervals use 10,000 percentile-bootstrap samples over the 266 sets, not independent pair resampling. This preserves dependence among the 15 pairs in a set. A method is included in the main table only when all 266 canonical sets align successfully. The API-native runs additionally retain parse, fallback, and trace status. Historical static and Qwen files that expose only final scores are treated as aligned legacy artifacts, not as proof that no internal fallback occurred.
RQ1: When Is Price Direction Identifiable and Human-Corroborated?
To determine whether transaction prices provide a usable visual-preference reference before evaluating models, we measure price-direction coverage and agreement with price-blind human judgments over the 3,990 within-set pairs (definitions above; Table 3). Price direction is sparse and weakly corroborated: only 966 pairs have a non-tied price order (24.2% coverage), with human agreement on 335 and conflict on 631. This gives 34.7% hard agreement (95% set-cluster CI: 30.2–39.0) and 37.1% soft agreement.
| Pair class | 2:1 | 3:0 | Total |
| Price–human agree | 228 | 107 | 335 |
| Price–human conflict | 298 | 333 | 631 |
| Price tie | 1565 | 1459 | 3024 |
| Total | 2091 | 1899 | 3990 |
Table 3. Reference decomposition over all 3,990 within-set pairs. The 2:1 and 3:0 columns denote majority and unanimous human judgments. Price ties are not assigned a price direction.
Clearer human consensus does not yield stronger price agreement: it is 24.3% for unanimous pairs versus 43.3% for 2:1 majorities. Further, every non-tied price direction favors the later canonical item, while the human majority favors the earlier item in 65.9% of pairs (Table 4); all three annotators show this positional tendency. Thus, the observed agreement mixes reference disagreement with fixed-order exposure and cannot identify the visual contribution to price. We do not treat raw transaction order as visual ground truth and use the 335 concordant pairs only for conditional recovery.
Preference source | Earlier (%) | Later (%) Annotator A | 56.9 | 43.1 Annotator B | 63.1 | 36.9 Annotator C | 70.8 | 29.2 Human majority | 65.9 | 34.1
Table 4. Human preference by initial source position over all 3,990 pairs.
RQ2: Can Models Recover the Protocol-Supported Price Order?
To test whether evaluators recover price directions corroborated by humans, we score all static models and elicitation interfaces on the same 335-pair protocol-supported set. Table 5 also reports complementary measures on the broader reference strata. Overall, recovery is partial and varies across systems; no method establishes transaction price as visual ground truth.
| Method | Sets/Rec. | Rec. Acc. [95% CI] | Human Acc. | Price Acc. | Conflict→H | Price-tie→H | Pred. tie |
| Static visual preference models | |||||||
| CLIPScore | 266/335 | 46.0 [38.6, 53.3] | 48.1 | 47.8 | 51.2 | 47.7 | 0.4 |
| NIMA | 266/335 | 63.6 [56.9, 70.2] | 56.6 | 51.4 | 55.0 | 56.2 | 0.0 |
| HPSv2 | 266/335 | 66.4 [59.1, 73.2] | 61.8 | 49.5 | 59.5 | 61.8 | 0.3 |
| PickScore | 266/335 | 66.0 [60.0, 72.2] | 59.8 | 47.0 | 63.1 | 58.4 | 0.0 |
| ImageReward | 266/335 | 63.6 [56.9, 70.0] | 59.8 | 49.2 | 58.5 | 59.7 | 0.0 |
| RichHF | 266/335 | 40.0 [33.7, 46.5] | 43.6 | 50.6 | 43.7 | 44.0 | 0.0 |
| Q-Align | 266/335 | 43.6 [36.4, 50.8] | 47.2 | 46.9 | 51.3 | 46.7 | 0.2 |
| LLaVA-Reward | 266/335 | 49.7 [43.4, 56.2] | 48.6 | 49.1 | 51.3 | 47.9 | 0.9 |
| Qwen3-VL-4B elicitation methods | |||||||
| Single-agent (Pair) | 266/335 | 8.4 [4.6, 13.0] | 59.5 | 12.1 | 85.9 | 59.7 | 0.0 |
| Single-agent | 266/335 | 51.5 [46.5, 56.4] | 51.0 | 50.2 | 50.5 | 51.0 | 61.2 |
| Single-agent (LLM Score) | 266/335 | 42.7 [36.1, 49.7] | 47.7 | 47.4 | 50.2 | 47.7 | 19.1 |
| Budgeted Single-agent | 266/335 | 41.5 [33.1, 50.0] | 51.6 | 43.6 | 55.3 | 51.9 | 0.1 |
| Valuator Judge | 266/335 | 44.3 [37.7, 50.9] | 49.9 | 44.9 | 54.8 | 49.5 | 17.3 |
| Valuator | 266/335 | 24.9 [20.0, 29.9] | 55.2 | 27.8 | 70.6 | 55.3 | 5.4 |
| GPT-5.5 elicitation methods | |||||||
| Single-agent (Pair) | 266/335 | 60.3 [52.0, 68.2] | 55.6 | 50.5 | 54.7 | 55.2 | 0.0 |
| Single-agent | 266/335 | 45.8 [38.8, 53.1] | 47.2 | 51.4 | 45.6 | 47.7 | 8.3 |
| Single-agent (LLM Score) | 266/335 | 55.8 [47.9, 63.6] | 57.0 | 46.8 | 57.9 | 57.0 | 12.8 |
| Budgeted Single-agent | 266/335 | 48.8 [40.3, 57.4] | 55.2 | 42.9 | 60.3 | 54.9 | 0.3 |
| Valuator Judge | 266/335 | 52.7 [45.1, 60.0] | 55.3 | 44.5 | 59.8 | 54.7 | 0.6 |
| Valuator | 266/335 | 51.9 [43.7, 60.1] | 51.9 | 48.3 | 53.6 | 51.6 | 1.5 |
| GPT-5.6 Sol elicitation methods | |||||||
| Single-agent (Pair) | 266/335 | 62.1 [54.0, 69.9] | 58.9 | 48.9 | 58.2 | 58.8 | 0.0 |
| Single-agent | 266/335 | 52.8 [46.1, 59.4] | 47.3 | 57.2 | 40.5 | 48.1 | 7.9 |
| Single-agent (LLM Score) | 266/335 | 60.4 [52.8, 67.7] | 57.9 | 49.3 | 56.6 | 57.9 | 8.5 |
| Budgeted Single-agent | 266/335 | 58.8 [50.6, 66.7] | 58.2 | 46.1 | 60.7 | 57.6 | 0.1 |
| Valuator Judge | 266/335 | 55.2 [47.1, 62.9] | 56.3 | 45.9 | 59.1 | 55.8 | 0.8 |
| Valuator | 266/335 | 57.6 [50.0, 65.0] | 56.2 | 48.1 | 56.9 | 55.9 | 1.5 |
Table 5. Pairwise diagnosis on MarketImageArt under the recorded fixed presentation protocol. Rec. Acc. is evaluated only on the 335 pairs for which transaction-price order and the human majority agree. Human Acc. uses all 3,990 pairs; Price Acc. uses the 966 non-price-tied pairs. Conflict→H and Price-tie→H report evaluator agreement with the human majority on the 631 conflicting and 3,024 tied-price pairs, respectively. Exact prediction ties receive half credit in directional accuracies; Pred. tie reports their frequency over all 3,990 pairs. Confidence intervals use 10,000 set-cluster bootstrap samples.
Among static scorers, HPSv2 leads at 66.4% (95% CI: 59.1–73.2), followed by PickScore at 66.0%; NIMA and ImageReward each reach 63.6%, while RichHF is lowest at 40.0%. The leading static models therefore recover part of the price–human relation, but the selected subset retains the order confound. Across model families, GPT-5.5 and GPT-5.6 Sol Pair reach 60.3% and 62.1%, respectively, versus 8.4% for Qwen3-VL-4B Pair; their 0–100 visual-score interfaces also exceed the corresponding Qwen interface (55.8% and 60.4% vs. 42.7%). This is not uniform across interfaces: GPT-5.5 Single-agent scores 45.8%, below Qwen3-VL-4B Single-agent at 51.5%.
The low Qwen Pair recovery is especially notable: it has 59.5% Human Accuracy and 85.9% agreement on price–human conflicts, yet reaches only 8.4% and favors the earlier canonical item in 83.3% of decisive comparisons (Appendix A), consistent with order sensitivity. Qwen Single-agent’s 51.5% also coincides with a 61.2% prediction-tie rate, so it does not indicate reliable directional recovery. GPT-5.6 Sol lowers the direct price predictor’s tie rate to 7.9%. HPSv2 nevertheless remains the highest-scoring method, with a confidence interval overlapping both GPT Pair interfaces; the observed differences do not establish that agent elicitation outperforms static visual scoring.
RQ3: Do Budgeted and Multi-Profile Interfaces Help?
To test whether budget constraints and profile aggregation improve recovery, we compare interfaces within each model family on the same 335 protocol-supported pairs. Table 6 reports paired Recoverable Accuracy change , where each contrast row specifies the tested interface and comparator, along with correction and harm rates and set-cluster bootstrap intervals. Correction counts baseline errors fixed exactly by the tested interface; Harm counts baseline exact-correct predictions made non-correct. Ties receive half credit for accuracy but count as non-correct for these two rates.
Contrast | Rec. Acc. [95% CI] | Corr. | Harm Qwen3-VL-4B | | | Budgeted − Single | -10.0 [-18.9, -1.0] | 37.7 | 45.3 Judge − Single | -7.2 [-15.4, 1.2] | 33.5 | 64.0 Valuator − Judge | -19.4 [-25.3, -13.7] | 6.3 | 49.1 Valuator − Single | -26.6 [-32.7, -20.5] | 20.4 | 74.7 GPT-5.5 | | | Budgeted − Single | 3.0 [-6.7, 12.2] | 37.9 | 36.4 Judge − Single | 6.9 [-1.5, 15.0] | 43.1 | 34.3 Valuator − Judge | -0.7 [-4.5, 2.9] | 11.3 | 13.6 Valuator − Single | 6.1 [-3.2, 15.3] | 39.5 | 33.6 GPT-5.6 Sol | | | Budgeted − Single | 6.0 [-3.4, 14.9] | 48.2 | 30.8 Judge − Single | 2.4 [-5.9, 10.4] | 40.4 | 31.4 Valuator − Judge | 2.4 [-1.4, 6.3] | 15.1 | 9.8 Valuator − Single | 4.8 [-3.5, 13.0] | 44.0 | 32.0
Table 6. Paired contrasts on the 335 protocol-supported pairs. and confidence intervals are percentage-point changes. Corr. is the percentage of baseline non-correct decisions corrected exactly; Harm is the percentage of baseline exact-correct decisions made non-correct. Ties count as non-correct for these two rates. Intervals are 95% set-cluster bootstrap CIs.
The paired results show no general recovery gain from adding budget or multi-profile aggregation. The clearest negative effects occur for Qwen3-VL-4B: Budgeted Single-agent is 10.0 points below Single-agent (95% CI: −18.9 to −1.0), while Valuator is 19.4 points below Valuator Judge and 26.6 points below Single-agent. Valuator corrects 20.4% of Single-agent’s non-correct decisions but harms 74.7% of its exact-correct decisions, making it the most harmful tested contrast in this family.
The pattern differs for the GPT systems. All GPT confidence intervals include zero: changes relative to Single-agent range from +3.0 to +6.9 points for GPT-5.5 and +2.4 to +6.0 points for GPT-5.6 Sol, with the largest increase for GPT-5.5 Valuator Judge. Valuator versus Valuator Judge changes recovery by −0.7 and +2.4 points, respectively. Thus, these positive point estimates do not resolve into a statistically supported advantage, unlike the measured Qwen degradation. These are system-level contrasts rather than isolated competition effects because the interfaces are not compute-matched. All 3,192 API method–set outputs align without parse failure or fallback. Figure 4 illustrates selected examples of the distinct failure modes; they are diagnostic cases, not prevalence estimates. Additional appendix results assess annotator-count and relative-price-gap sensitivity (Table 7 and Table 8), alternative tie scoring and margin-based selection (Figure 6 and Figure 7), seeded input-order permutations (Table 9), budget and profile-aggregation traces (Table 10), and comparison-set-size effects (Figure 8 and Table 11 and Table 12).
Vector (PDF)
(a) (a) Price unidentifiable (set 27). All six prices tie; 13/15 human comparisons are unanimous. A: EUR 25; H#2. B: EUR 25; H#1. C: EUR 25; H#3. D: EUR 25; H#4. E: EUR 25; H#5. F: EUR 25; H#6.

(b) (b) Reference conflict (set 161). 11/13 ordered price pairs oppose the human majority. A: EUR 87; H#3. B: EUR 88; H#1. C: EUR 89; H#2. D: EUR 89; H#4. E: EUR 90; H#5. F: EUR 90; H#6.

(c) (c) Recoverable signal (set 250). GPT-5.6 Sol Pair order: F>E>D>C>A>B; correct on 10/10 supported pairs. A: EUR 270; H#2. B: EUR 270; H#6. C: EUR 274; H#5. D: EUR 279; H#4. E: EUR 280; H#3. F: EUR 280; H#1.

(d) (d) Elicitation harm (set 57). Qwen Single: D>B=E=F>A=C; Valuator: C>A=B>E>D>F. Valuator reverses 5 baseline-correct supported pairs. A: EUR 38; H#3. B: EUR 38; H#4. C: EUR 38; H#5. D: EUR 39; H#2. E: EUR 39; H#6. F: EUR 39; H#1.
Figure 4. Qualitative audit of the four-stage diagnosis. A–F denote canonical item identity, EUR is observed transaction price, and H# is Kemeny human rank (1 is best). Cases are selected deterministically: (a) maximizes unanimous human comparisons among all-price-tie sets; (b) maximizes unanimous price–human conflicts; (c) maximizes correctly recovered supported pairs for GPT-5.6 Sol Pair; and (d) maximizes Qwen Valuator harm minus correction relative to Single-agent. These diagnostic extremes are not representative performance estimates.
| Relative gap | Pairs | Supported | Hard agree [95% CI] | Soft agree |
| 225 | 63 | 28.0 [19.6, 36.3] | 31.1 | |
| 1–2% | 364 | 122 | 33.5 [27.5, 39.8] | 35.7 |
| 2–5% | 323 | 125 | 38.7 [30.4, 46.9] | 41.1 |
| >5% | 54 | 25 | 46.3 [30.6, 63.2] | 47.5 |
Table 8. Reference reliability by relative price gap among price-ordered pairs. Relative normalization reveals a condition hidden by absolute EUR bins, while the largest-gap bin remains small.
Figure 6. Recoverable Accuracy under strict, half-tie, and decisive scoring, alongside accuracy stratified by 2:1, 3:0, and consensus-weighted human judgments. The five methods are defined in the plot; values are percentages.
Figure 7. Within-method recovery for the highest-margin nominal quartile and all 335 supported pairs. Circles denote selected pairs and squares all pairs; boundary ties are retained, so realized coverage may exceed 25%. Short method names are defined in the plot; values are percentages.
Figure 8. Recoverable Accuracy in the matched comparison-set-size ablation. Left: GPT-5.5; right: GPT-5.6 Sol. Every condition contains 266 sets, and denotes recoverable pairs. Whiskers are 95% set-cluster bootstrap intervals; the dashed line marks 50%. The vertical axis spans 20–100%. Legend labels Pair, Single, Score, Budget, and Judge denote Single-agent (Pair), Single-agent, Single-agent (LLM Score), Budgeted Single-agent, and Valuator Judge, respectively; Valuator keeps its full name. The trajectories are descriptive and do not test differences between set sizes.
Table 9: Seeded input-order intervention on 50 sets. Recovery ranges span the three permutations; pair agreement and unanimity use the 49 sets with distinct orders. Earlier is the percentage of decisive predictions favoring the earlier attachment.
| Model | Rec. Acc. range | Pair agreement | 3-order unanimity | Earlier range |
| GPT-5.5 | 53.6–55.4 | 83.3 | 75.0 | 42.7–50.5 |
| GPT-5.6 Sol | 57.1–60.7 | 84.2 | 76.3 | 49.1–51.3 |
Table 9.
Table 10: Audit of API-native budget and multi-agent traces. Budget is mean budget utilization, Full is the fraction of vectors exactly exhausting the cap, and Top is the largest item’s mean allocation share. Agent agr. is exact pair-direction agreement across profile pairs. Unanim. is the share of recoverable pairs on which all three profiles agree; Acc. U/N gives aggregate accuracy on unanimous/non-unanimous pairs. Dashes denote inapplicable quantities.
| Model | Interface | Budget | Full | Top | Agent agr. | Unanim. | Acc. U/N |
| GPT-5.5 | Budgeted Single-agent | 99.9 | 99.2 | 27.3 | – | – | – |
| GPT-5.5 | Valuator Judge | – | – | – | 73.8 | 58.5 | 56.1/47.8 |
| GPT-5.5 | Valuator | 99.9 | 98.5 | 27.2 | 72.3 | 57.3 | 53.1/50.3 |
| GPT-5.6 Sol | Budgeted Single-agent | 100.0 | 100.0 | 26.7 | – | – | – |
| GPT-5.6 Sol | Valuator Judge | – | – | – | 78.8 | 66.0 | 57.9/50.0 |
| GPT-5.6 Sol | Valuator | 100.0 | 98.4 | 26.3 | 74.8 | 57.0 | 59.2/55.6 |
Table 10.
Table 12: Change in Human Acc. ( minus , percentage points) on the same 266 focal pairs; ties receive half credit. Method names are abbreviated as in Table 5. Per-image scores are reused; joint interface changes are descriptive, not significance tests.
| Method | GPT-5.5 | GPT-5.6 Sol |
| Pair | -5.6 | -1.9 |
| Single | 0.0 | 0.0 |
| Score | 0.0 | 0.0 |
| Budget | -7.1 | -4.1 |
| Judge | -13.0 | -4.7 |
| Valuator | -16.5 | -6.6 |
Table 12.
| Images | Pairs | Supported | Price-ord. | Conflict | Ties | |
| 2 | 532 | 266 | 21 | 65 | 44 | 201 |
| 3 | 798 | 798 | 62 | 198 | 136 | 600 |
| 4 | 1,064 | 1,596 | 133 | 410 | 277 | 1,186 |
| 5 | 1,330 | 2,660 | 225 | 660 | 435 | 2,000 |
| 6 | 1,596 | 3,990 | 335 | 966 | 631 | 3,024 |
Table 11. Matched set-size conditions (266 sets each). Pairs counts all within-set comparisons; Price-ord. counts non-tied pairs, split into Supported and Conflicts; Ties are price-tied pairs.
Conclusion
Transaction prices offer observable behavioral signals for visual preference, but non-visual market factors and evaluation protocols complicate their interpretation. MarketImage tests their suitability through MarketImageArt, a benchmark pairing prices with human rankings; MarketImageLens, a tie-preserving diagnostic; and MarketValuator, a comparison of direct, budgeted, and multi-agent interfaces. Only 24.2% of pairs have a non-tied price direction, and 34.7% of these agree with the human majority. On this protocol-supported subset, HPSv2 leads at 66.4% Recoverable Accuracy; GPT-5.5 Pair and GPT-5.6 Sol Pair reach 60.3% and 62.1%. Budgeted and aggregated interfaces lower Qwen recovery, while GPT gains remain statistically unresolved. Thus, price directions are sparse, weakly corroborated, and order-confounded; they are not validated ground truth for visual preference. Future work should test broader markets with randomized presentation.
References
A Reference-Signal and Evaluator Robustness Audit
We compare the released three-of-four consensus with all four complete rankings on the same 266 sets, retaining 2:2 votes as ties. We keep the released consensus as primary because its exclusion rationale was not recorded; selected-three agreement statistics appear in the main text.
The all-four analysis preserves the qualitative recovery ordering despite a smaller supported subset. As reported in the main text, 154 sets have all prices tied; among the remaining 112, 71 contain two distinct prices, 25 three, 14 four, and two five, with none containing six.
To test whether larger price differences are more human-corroborated, we stratify the 966 price-ordered pairs by absolute EUR gap and by relative gap . Absolute-gap bins show little change in hard agreement (34.2–37.2%); relative-gap results are shown below:
Agreement rises descriptively with relative gap, but the largest-gap bin has only 54 pairs; this trend does not establish price as a reliable visual reference. The fixed-order evidence is reported in Benchmark Validation and RQ1/RQ2. The following checks test whether evaluator conclusions depend on scoring choices, output margins, or presentation order.
On the 335 supported pairs, we vary prediction-tie scoring and human-consensus definitions (Figure 6). Most static and API results are stable; Qwen Single-agent is sensitive because it predicts ties on many pairs (22.4% strict versus 51.5% half-credit and 53.6% decisive-only). HPSv2 remains near 66% overall and reaches 76.6% on unanimous pairs, above the GPT Pair interfaces (64.5% and 67.3%). We also rank pairs within each method by and compare the highest-margin nominal quartile with all supported pairs; boundary ties are retained and realized coverage is reported (Figure 7). HPSv2 rises from 66.4% to 77.4% at 25.1% coverage, but margins are not calibrated confidence and gains vary by method.
To estimate model-side order sensitivity, we rerun GPT-5.5 and GPT-5.6 Sol Pair on a fixed subset of 50 canonical sets under three seeded permutations (22, 42, 168). Outputs are mapped back to item IDs before scoring, yielding 300 complete model–set runs.
Recovery varies little across permutations, and neither model shows a consistent attachment-position preference. Because only the two GPT Pair systems are rerun, this experiment does not resolve the human-order confound or test Qwen Pair.
B Market Elicitation and Mechanism Diagnostics
To diagnose the budget and aggregation results, we inspect budget compliance, agreement among individual profiles, and sensitivity to the aggregation reducer.
Budget Use, Agent Disagreement, and Aggregation
We analyze API-native allocations and three individual judge or bidder vectors for GPT-5.5 and GPT-5.6 Sol; legacy Qwen files lack aligned per-profile vectors. Table 10 summarizes budget use, allocation concentration, and profile agreement.
Budget use is near-complete (99.9–100.0% mean utilization), so cap under-spending does not explain the lack of a consistent mechanism gain. Profiles agree on 72.3–78.8% of pair directions, while unanimity covers 57.0–66.0% of recoverable pairs. Unanimous-profile accuracy is higher in all four model/interface rows, most clearly for GPT-5.5 Valuator Judge (56.1% versus 47.8%); this descriptive
split is not an independent significance test. Alternative reducers yield 51.3–52.5% recovery for GPT-5.5 and 54.6–57.6% for GPT-5.6 Sol, with no consistent post-hoc gain.

Figure 5. Benchmark validation for MarketImageArt. Left: leakage control confirms that evaluator inputs are free of explicit price-related cues. Middle: the retained emerging-art records span EUR 10–395 (median EUR 70). Right: the canonical comparison sets are dominated by zero top-two price gaps. The implicit price information in source order is audited separately.
Supplementary Analysis Notes
The reference decomposition and three diagnostic questions were fixed before the full API suites. Gap bins, alternative tie scoring, margin selection, aggregation variants, and the order intervention are descriptive robustness analyses, not multiple-comparison-corrected findings. Bootstrap intervals use seed 2027 with deterministic model–method offsets; the order intervention uses seeds 22, 42, and 168. Metric definitions, denominators, and run-completion checks are given in the main Experimental Setup.
Matched Comparison-Set Size Ablation
Question and Matched Data Construction
We test whether joint-prompt context size changes the six GPT elicitation interfaces. From each canonical six-image set, deterministic SHA-256 priorities (seed 2027, set ID, item ID) define nested subsets for . Selection uses no prices, human votes, or model outputs; selected items retain their canonical relative order. Thus, all conditions contain 266 sets but have different image and pair counts. For set :
The same subsets are used for both models and all six methods. Table 11 reports the resulting reference counts.
Human rankings are restricted from the original six-image annotations, not recollected at smaller ; they therefore represent six-image preferences. Retaining canonical order also preserves the price-order exposure audited in Appendix A.
Interfaces and Execution Accounting
GPT-5.5 and GPT-5.6 Sol use medium reasoning effort and the six interfaces in Table 5. Joint-interface prompts and output lengths are resized to , with personas and aggregation unchanged. Budgeted methods receive EUR 100K per image, a constant-per-image rather than fixed-total budget.
The four joint interfaces are rerun for each smaller . The two image-wise methods reuse saved scores projected onto selected items; their pair metrics can change with pair composition, but not with comparison context.
The matrix contains 48 files and 12,768 method–set outputs. Its 32 joint interface files add 8,512 set outputs and require
logical calls. Traces record 17,025 attempts, including one successful retry, with no final failure or parser fallback. The other 16 files contain 4,256 projected outputs and require no calls. All outputs pass coverage, identity, order, cardinality, and finite-value checks.
All Within-Set Pair Results
Across all within-set pairs, Figure 8 shows no uniform monotonic trend; changing pair identities and supported-pair counts prevent attributing differences to context size alone. On the same 266 focal pairs, joint-interface Human Accuracy falls by 1.9–16.5 points from to , while per-image methods are unchanged because their scores are reused (Table 12). This descriptive probe is not a significance test.
Fixed-Pair Comparison Across Context Sizes
To isolate context from pair composition, we track the same image pair within each of 266 sets (21 recoverable, 44 conflicting, 201 price-tied). Table 12 reports the change in Human Accuracy at relative to .
Limitations and Ethics.
(1) MarketImageArt covers one low-price emerging-art source and constructed comparison sets, not simultaneous auctions. (2) Human judgments used a fixed price-sorted order, and the released consensus selected three of four complete annotation files without a documented rationale; causal claims about the visual contribution to price are therefore unsupported. (3) GPT and Qwen interfaces differ in prompts, call graphs, and compute, so cross-family comparisons are system-level. (4) Human rankings are used only in aggregate; annotator demographics, recruitment, and compensation are undocumented. Generated prices and bids are not appraisals and should not guide purchases or claims about artists; any data release must respect image licenses and privacy and distinguish market outcomes from human judgments and model predictions.
References
- [1]Kushal Agrawal, Verona Teo, Juan J Vazquez, Sudarsh Kunnavakkam, Vishak Srikanth, and Andy Liu. 2025. Evaluating LLM Agent Collusion in Double Auctions. arXiv preprint arXiv:2507.01413.arxiv.org/abs/2507.01413
- [2]Ying Ba, Tianyu Zhang, Yalong Bai, Weny i Mo, Tao Liang, Bing Su, and Ji-Rong Wen. 2025. Enhancing reward models for high-quality image generation: Beyond text-image alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE/CVF, Honolulu, HI, USA, 19022–19031.arxiv.org/abs/2507.19002
- [3]Jiangjie Chen, Siyu Yuan, Rong Ye, Bodhisattva Prasad Majumder, and Kyle Richardson. 2023. Put your money where your mouth is: Evaluating strategic planning and execution of llm agents in an auction arena. arXiv preprint arXiv:2310.05746.arxiv.org/abs/2310.05746
- [4]Mingxuan Cui, Jingpu Yang, Fengxian Ji, Qian Jiang, Zhecheng Shi, Jiaming Wang, Zirui Song, Zhuohan Xie, Fajri Koto, and Xiuying Chen. 2026. Text-Align: Preference Alignment for Text Rendering with Hierarchical Rewards. arXiv:2605.19320 [cs.CV] doi:10.48550/arXiv.2605.19320DOI
- [5]Laura Georgeson, James Fox, Anna Gautier, and Michael Wooldridge. 2024. Fixed-budget and Multiple-issue Quadratic Voting. arXiv preprint arXiv:2409.06614.
- [6]Ruining He and Julian McAuley. 2016. VBPR: visual bayesian personalized ranking from implicit feedback. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30. AAAI Press, Phoenix, AZ, USA, 1442–1448.arxiv.org/abs/1510.01784
- [7]Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. Clipscore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 conference on empirical methods in natural language processing. Association for Computational Linguistics, Punta Cana, Dominican Republic, 7514–7528.
- [8]Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. 2025. T2i-compbench++: An enhanced and comprehensive benchmark for compositional text-to-image generation. IEEE Transactions on Pattern Analysis and Machine Intelligence.DOI
- [9]Kiyohito Iigaya, Sanghyun Yi, Iman A Wahle, Koranis Tanwisuth, and John P O’Doherty. 2021. Aesthetic preference for art can be predicted from a mixture of low-and high-level visual features. Nature Human Behaviour 5, 6 (2021), 743–755.
- [10]Kensuke Ito and Hideyuki Tanaka. 2019. Token-cured registry with citation graph. arXiv preprint arXiv:1906.03300.DOI
- [11]Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. 2024. Rethinking fid: Towards a better evaluation metric for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE/CVF, Seattle, WA, USA, 9307–9315.
- [12]Fengxian Ji, Yuke Li, Jingpu Yang, Juanfan Wu, Fan Zhang, Zhexuan Cui, Yu Xie, Min Peng, Qianqian Xie, Xiuying Chen, and Zhuohan Xie. 2026. Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generation. arXiv:2608.01666 [cs.CL] doi:10.48550/arXiv.2608.01666DOI
- [13]Fengxian Ji, Jingpu Yang, Zirui Song, Lang Gao, Junhong Liang, Zhenhao Chen, Jinghui Zhang, and Xiuying Chen. 2026. ServImage: An Image Generation and Editing Benchmark from Real-world Commercial Imaging Services. In Proceedings of the 6th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, San Diego, California, United States, 43504–43529. doi:10.18653/v1/2026.acl-long-2014
- [14]Fengxian Ji, Jingpu Yang, Zirui Song, Y uanxi Wang, Zhexuan Cui, Yuke Li, Qian Jiang, and Xiuying Chen. 2026. FineState-Bench: Benchmarking State-Conditioned Grounding for Fine-grained GUI State Setting. In Findings of the Association for Computational Linguistics: ACL 2026. Association for Computational Linguistics, San Diego, California, United States, 43073–43088. doi:10.18653/v1/2026.findings-acl-2136
- [15]Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. 2023. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in neural information processing systems 36 (2023), 36652–36663.
- [16]Kangsan Lee, Jaehyuk Park, Sam Goree, David Crandall, and Yong-Yeol Ahn. 2024. Social signals predict contemporary art prices better than visual features, particularly in emerging markets. Scientific reports 14, 1 (2024), 11615.DOI
- [17]Ao Li, Jinghui Zhang, Luyu Li, Yuxiang Duan, Lang Gao, Mingcai Chen, Weijun Qin, Shaopeng Ji, Fengxian Ji, Ning Liu, Lizhen Cui, Xiuying Chen, and Yuntao Du. 2026. M3MAD-Bench: Multi-Dimensional Evaluation of Multi-Agent Debate Across Domains and Modalities. arXiv:2601.02854 [cs.AI] doi:10.48550/arXiv.2601.02854 Accepted by ACM Multimedia 2026.arxiv.org/abs/2601.02854
- [18]Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Graham Neubig, et al. 2024. Genai-bench: Evaluating and improving compositional text-to-visual generation. arXiv preprint arXiv:2406.13743.
- [19]Weiyue Li, Minda Zhao, Weixuan Dong, Jiahui Cai, Yuze Wei, Michael Pocress, Yi Li, Wanyan Yuan, Xiaoyue Wang, Ruoyu Hou, et al. 2026. Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scale. arXiv preprint arXiv:2601.03444.arxiv.org/abs/2601.03444
- [20]Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, et al. 2024. Rich human feedback for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE/CVF, Seattle, WA, USA, 19401–19411.arxiv.org/abs/2312.10240
- [21]Marshall Xiaoyi Ma, Charles N Noussair, and Luc Renneboog. 2022. Colors, emotions, and the auction value of paintings. European Economic Review 142 (2022), 104004.
- [22]Mayu Otani, Riku Togashi, Yu Sawai, Ryosuke Ishigami, Yuta Nakashima, Esa Rahtu, Janne Heikkilä, and Shin’ichi Satoh. 2023. Toward verifiable and reproducible human evaluation for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE/CVF, Vancouver, Canada, 14277–14286.arxiv.org/abs/2304.01816
- [23]C Alejandro Parraga, Marcos Muñoz Gonzalez, Olivier Penacchio, Bogdan Raducanu, and Xavier Otazu. 2025. Aesthetics Without Semantics. SSRN 4817083.arxiv.org/abs/2505.05331
- [24]Kekai Sheng, Weiming Dong, Menglei Chai, Guohui Wang, Peng Zhou, Feiyue Huang, Bao-Gang Hu, Rongrong Ji, and Chongyang Ma. 2020. Revisiting image aesthetic assessment via self-supervised feature learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. AAAI Press, New York, NY, USA, 5709–5716.arxiv.org/abs/1911.11419
- [25]Joo Yong Shim, Jeon Seong Bjorn Choe, and Jong-Kook Kim. 2024. The Bid Picture: Auction-Inspired Multi-player Generative Adversarial Networks Training. arXiv preprint arXiv:2403.13866.arxiv.org/abs/2403.13866
- [26]Derya Soydaner and Johan Wagemans. 2024. Multi-task convolutional neural network for image aesthetic assessment. Ieee Access 12 (2024), 4716–4729.DOI
- [27]Lillian Sun, Henry Huang, Fucheng Warren Zhu, Giannis Daras, and Constantinos Costas Daskalakis. 2025. Bidding for Influence: Auction-Driven Diffusion Image Generation. ICML 2025 Workshop on Reliable and Responsible Foundation Models.
- [28]Xun Wu, Shaohan Huang, and Furu Wei. 2024. Multimodal large language model is a human-aligned annotator for text-to-image generation. arXiv preprint arXiv:2404.15100.arxiv.org/abs/2404.15100
- [29]Jiaheng Xu, Yu Huang, Jiale Cheng, Yuanning Yang, Jiajun Xu, Yuan Wang, Wenbo Duan, Shen Yang, Qunlin Jin, Shunrun Li, et al. 2024. Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation. arXiv preprint arXiv:2412.21059.arxiv.org/abs/2412.21059
- [30]Yuzhe Yang, Liwu Xu, Leida Li, Nan Qie, Yaqian Li, Peng Zhang, and Yandong Guo. 2022. Personalized image aesthetics assessment with rich attributes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE/CVF, New Orleans, LA, USA, 19861–19869.arxiv.org/abs/2203.16754
- [31]Michal Yarom, Yonatan Bitton, Soravit Changpinyo, Roee Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, and Idan Szpektor. 2023. What you see is what you read? improving text-image alignment evaluation. Advances in Neural Information Processing Systems 36 (2023), 1601–1619.arxiv.org/abs/2305.10400
- [32]Bojun Zhang, Junhong Liang, Feifei Zhai, Fengxian Ji, and Yu Zhou. 2026. ReViCo: Unveiling the Limitations of VLMs in Visual Text Understanding via Error Correction. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Budapest, Hungary, 19606–19622. https://aclanthology.org/2026.emnlp-main.845/arxiv.org/abs/2608.27154
- [33]Fan Zhang, Mingzi Song, Rania Elbadry, Yankai Chen, Shaobo Wang, Yixi Zhu, Xunwen Zheng, Yueru He, Yuyang Dai, Georgi Nenkov Georgiev, Ayesha Gull, Muhammad Usman Safder, Fan Wu, Liyuan Meng, Fengxian Ji, Junning Zhao, Xueqing Peng, Jimin Huang, YU Chen, Xue Liu, Preslav Nakov, and Zhuohan Xie. 2026. FineReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosure. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Greg Durrett and Ping Jian (Eds.). Association for Computational Linguistics, San Diego, California, United States, 728–735. doi:10.18653/v1/2026.acl-demo.71DOI





