What Is Your AI Agent Buying? Evaluation, Biases, Model Dependence, & Emerging Implications for Agentic E-Commerce

Introduction

Autonomous AI agents are poised to make shopping decisions for millions of consumers, reshaping the e-commerce ecosystem [14, 18, 33, 50]. A new generation of “computer-use” systems—such as OpenAI’s Operator (automating grocery orders, form filling, flight booking) [29, 37] and Google DeepMind’s Project Mariner (large-scale multi-tab shopping) [16, 55]—promises to replace manual browsing with autonomous procurement. Industry leaders are already signaling strategic disruption; Walmart’s CTO urges preparedness for third-party agents that circumvent traditional merchandising and advertising cues [6].

Today, consumers still interact with platforms directly. In the near future they may instead issue a short prompt to a personalized AI agent (augmented with purchase history and preferences), which then navigates, evaluates, and purchases with minimal human oversight (see Figure 5 in Appendix A). Such agents have the potential to significantly reduce search frictions for customers. In doing so, agents can systematically scan broader assortments than humans, potentially diluting the influence of classical monetization levers that platforms have, such as product rankings and ad placements. Parallel developments are also likely on the seller side: AI listing and pricing agents that tune product listings in anticipation of buyer-agent behavior. This shift foreshadows a reconfigured e-commerce ecosystem and raises new design, strategy, and regulatory questions.

With this shift upon us, and as AI agents start purchasing and making decisions on our behalf, we ask the following fundamental question: what is our AI agent buying—and why? More concretely, we investigate various layers of questions.

1. Rational Behavior. Do agents satisfy basic instruction following and simple economic dominance tests?

2. Product Market Shares. What are product market shares when purchases are fully mediated by AI agents, and how do such market shares vary across AI agents?

3. Choice Behavior and Biases. How do AI agents respond to observable attributes (price, rating, reviews, text) and platform levers (position, promotions, sponsorship)?

4. Interaction between Buyer and Seller Agents. How might outcomes change when sellers and/or marketplace platforms deploy their own optimizing AI agents?

The first question concerns minimum viability: without elementary rationality, higher-order design or policy levers may be moot (cf. the emphasis on baseline competence in [46]). The second and third probe “in-the-wild” behaviors. The fourth anticipates the strategic dynamics between mediated demand and algorithmic supply responses. Our main contributions can be summarized as follows.

ACES framework. We develop ACES (Agentic e-CommercE Simulator): (i) a provider-agnostic VLM shopping agent and (ii) a fully programmable mock e-commerce application (“mock-app”; see Figure 6). This framework enables control and randomization of page layout, product order, prices, ratings, reviews, and promotional / sponsored badges. This provides a realistic shopping environment while preserving experimental control for causal attribution of how platform levers and listing attributes steer AI agents purchase decisions.

Market shares under delegation. Using ACES, we study product selection shares when an assortment of eight products are presented with randomized product positions. We evaluate three model families (Claude 4 Sonnet, GPT-4.1, Gemini 2.5 Flash) and treat selection frequencies as a proxy for market share in a fully delegated world. We observe the following.

(i) Model heterogeneity. Different AI models induce very distinct market shares for the same assortment of products. For example, for the fitness watch category, Claude Sonnet 4 selects the Fitbit Inspire model 45%, while GPT-4.1 and Gemini 2.5 Flash select it only about 25% of the time (see Figure 1).

(ii) Concentration risk. For some categories, the market shares concentrate on a few select products and some brands are never selected (e.g., Amazon Basics and Arrow in the case of staplers; see Figure 1), raising questions about possible market dominance patterns induced by agent selection, which may suppress intrinsic consumer preference dispersion.

Baseline rationality. We test two basic requirements: (i) instruction following—select the only product matching an explicit attribute (e.g., color, within budget); (ii) economic rationality—select the unique best product when a product that “dominates” all others exists. We test this by controlling the attributes of the various listings and constructing randomized scenarios with each having only one listing satisfying the query. We find that performance is heterogeneous across models and tasks. In one of the price-based rationality tests, we construct a scenario with all listings being identical except for one listing having a lower price. Here, even state of the art models (GPT-4.1) can register failure rates exceeding 9%. The failure rate tends to decrease with the increase in price difference. In rating dominance tests, all listings are identical except that one listing has an average rating which is higher by 0.1. We see significant heterogeneity in performance, with some models registering no failures (Gemini 2.0 Flash) and others registering up to 71.7% failures (GPT-4o, on which OpenAI Operator is built). However with GPT-4.1, this fail rate comes down to 16.0%, supporting the insight that failures reduce with more advanced models. These findings imply consumers delegating purchases may sometimes pay more or obtain lower-rated products, and sellers cannot rely on modest price cuts or rating advantages to guarantee being selected by agents.

Choice Behavior and Biases. To systematically understand how the AI agents trade off between different product attributes and the impact of different platform levers like rankings and promotions, we run randomized controlled trials where we randomly permute the position of the products, add sponsorship, endorsement or scarcity tags and perturb product attributes like price, average rating and number of reviews. We fit a conditional logit model [35] to understand the sensitivity of AI agents to different attributes. AI agents react positively to higher average ratings, and also to a larger number of ratings, and are sensitive to price though perhaps less so than one might expect. This behavior is qualitatively similar with what one would expect from humans, but sensitivities differ significantly across models. We also observe that everything else being equal, the position can significantly affect the selection of a product by AI agents. Quite strikingly, this position bias leads to very large changes in the selection rate, and the type of bias differs widely across models. These findings highlight that the rise of AI agents might significantly affect market demand, and also lead platforms to adjust the type of levers they use.

Toward the meta game: seller response. We next test how could sellers respond to the deployment of AI agents and what impact may such response have. To do so, we provide an AI agent representing a seller information on product attributes and current market shares, and prompt it to improve the description of the product it represents. We find the impact of AI-generated product descriptions to be heterogeneous. In 75% of the cases, it does not lead to statistically different sales levels, but quite strikingly, in 25% of the cases, a single iteration of this process leads to a dramatic increase in the market share of the focal product. These findings suggest that strategic dynamics will emerge from the interaction between sellers, platforms, and AI agents representing buyers, which are both substantial and intricate.

Our work connects to extensive research into computer-use agents and benchmarks [11, 27, 51, 58] including for e-commerce environments [9, 21, 24, 32, 40, 54]. While most benchmarks for AI agents evaluate end-to-end task completion, our contribution is to isolate the critical choice step in agent shopping through the controllable ACES sandbox. This facilitates the first agent-centric, causal evaluation of how AI shoppers respond to product attributes and platform levers like product positions and promotional badges, complementing existing studies on human consumers [31, 48]. We provide a more comprehensive literature review in Appendix B.

Our results have significant implications for various stakeholders in the new e-commerce ecosystem—customers, sellers, platform designers and regulators—which will be mediated by AI agents. We quantify AI shopping agent behavior and shed light on situations in which customers can and cannot rely on their AI shopping assistant. Furthermore, the results raise pertinent questions such as: How can sellers prepare themselves for more customers shopping via their AI assistants? Will Marketing and Sales efforts come to rely heavily on sandbox environments like ACES to optimize listings and campaigns? How might platforms monetize in a future dominated by AI shopping assistants, e.g., can they offer listing optimization as a service to sellers? To what extent will AI-powered consumer agents aggregate and exercise market power, and what are the implications for market efficiency and consumer welfare?

Experimental Design

To investigate the aforementioned research questions, we construct a controllable agent-platform sandbox and conduct randomized experiments that (i) test for instruction following and basic rationality, (ii) cleanly isolate and quantify the effects of platform tags and product characteristics on agent decision making, and (iii) understand the impact of a strategic response by a seller who optimizes their product description. Our evaluation includes several state-of-the-art models whose providers claim agent-assistant capabilities [16, 22, 29, 37, 55].

ACES: Our Sandbox Environment

ACES (Agentic e-Commerce Simulator) comprises of: (i) a provider-agnostic browser-use agent that combines a vision language model (VLM) of choice with tool usage to observe and act on webpages, and (ii) a fully controllable mock e-commerce site (“mock-app”) that renders product listings. The VLM agent is implemented using the LangChain library [7] and the Selenium browser-driver [44]. The mock-app is a lightweight storefront with search that returns listings from our pre-specified product universe.

Instead of simulating the entire end-to-end shopping journey, which can consist of many back and forths between different options and reading different reviews, we simulate a streamlined three step interaction, which we call Veni, Vidi, Emi¹ in short (see Figure 2): 1. the AI agent opens a browser (via Selenium) and loads the mock-app landing page (Veni), 2. provides the product-category (e.g., fitness watch, stapler, etc.) being evaluated then captures a screenshot of the returned catalog of products (Vidi), 3. query a vision-language model (VLM) (with the screenshot and the user prompt) and records the VLM’s intended selection; treating it as the purchase and the episode terminates (Emi).

Our results page displays an eight-product assortment in a fixed $2 \times4$ grid (see Figure 6). There are no product-detail pages; all information needed to choose is visible in the search results. While this omits aspects of full-funnel shopping, it provides a first-order approximation of selection behavior.

We intentionally constrain the search-action space to a single page with eight items and a single tool call to enable clean measurement of choice behavior. This design (i) prevents behavioral looping by grounding the agent in a fixed UI context; (ii) standardizes tool use across model families and avoids coordinate-based errors; (iii) restricts navigation complexity by eliminating multi-page exploration; and (iv) isolates core trade-offs we aim to study. As VLM technology matures and token costs decline, richer multi-page interactions can be incorporated.

Finally, we do not personalize agents and use a single generic prompt (see below) to isolate baseline behavior. Over-specifying preferences would effectively pre-solve the very search task these agents are designed to perform, obscuring autonomous decision-making and the influence of platform levers. In addition, prompt engineering is unbounded: more instructions can be layered on, producing brittle, hand-crafted policies whose biases may shift as models are updated and deployed. Our objective in this work is to measure the base behavior.

Default Prompt Template for AI Buyer

You are a personal shopping assistant helping someone find a good {query}. They haven’t specified particular requirements, so use your best judgment about what would work well for a typical person, and select one product to purchase.

<instructions>

1. Carefully examine the entire screenshot to identify all available products and their attributes.

2. Use the add_to_cart function when you are ready to buy a product.

3. Before making your selection, explain your reasoning for choosing this product, including what factors influenced your decision and any assumptions you made about what would be best:

• Your primary decision criteria and why you prioritized them

• How each available product performed on these criteria

• What specific factors made your chosen product superior

• Any assumptions you made about the user’s needs or preferences

4. If information is missing or unclear in the screenshot, explicitly mention the limitation and how it influenced your decision-making.

</instructions>

ACE-RS: Instruction Following and Rationality Suite

We test for two capabilities which we deem to be foundational for reliable deployment of AI agents in practice—ability to follow instructions and basic economic rationality. The experiments are designed in a manner such that there is objectively only a single product that satisfies the requirements of the test.

Instruction Following Experiments. We consider three types of instruction following tasks: (i) choose a product within a specified budget, (ii) choose the product of a specified color and (iii) choose a specified brand. For each of the tasks, we select a few product categories and explicitly prompt the AI agent to align with the stated preferences by replacing a specific substring in the default prompt. For each task and its chosen product categories, we run 50 experiments in which the position of the products are randomly shuffled. For further details about the experiment setup, refer to Appendix C.1.1.

Basic Rationality Experiments. We also test agents’ ability to select an optimal product, when the prompt does not specify which attribute to prioritize. To do so, we examine, in a fully controlled setting, whether agents select the option that maximizes user utility when alternatives differ only along a single “ordered” attribute. We perform two types of tests: (a) price-based and (b) rating-based tests where all product attributes are the same except for prices and ratings respectively. This setup therefore gauges whether the agent can infer a sensible utility ranking on its own. An economically rational AI agent which aims to maximize user utility must choose the product with the lowest price or the highest rating given that all the other attributes are identical. We explicitly avoid instructing the models to prioritize a specific attribute because real-world preferences are multi-dimensional: customers routinely trade off price, ratings, and other product features. For further details, refer to Appendix C.1.2 and C.1.3.

ACE-BB: Choice Behavior and Biases

To estimate the sensitivity of the AI agent to different product attributes and platform levers, we create a dataset with exogenous variation across a number of variables associated with the products. We generate 500 scenarios for each product category. In these 500 random scenarios, we randomly permute the position of the eight products shown on the mock-app page. We randomly assign “Sponsored”, “Overall Pick” and “Only X Remaining” (scarcity; X is a random integer between 1 and 5), tags to these products. In addition, we randomly perturb the prices, ratings and number of reviews for each of the eight products in the assortment. For further details regarding the experimental setup, refer to Appendix C.2.

ACE-SR: Seller Response to AI-Assisted Shopping

To estimate the causal impact of the recommendation made by the AI seller agent, we begin by establishing a baseline for each product category: in 200 trials we expose an AI buying agent—Claude Sonnet 4, GPT-4.1, or Gemini 2.5 Flash—to the mock-app showing eight items whose on-page positions are randomly shuffled from trial to trial while all other attributes (price, description, etc.) remain fixed. After recording the model’s purchase frequencies, we randomly designate one of the eight items (among those with positive market share) as the “focal” product. Treating this seller as AI-assisted, we feed a second model (e.g., GPT-4.1 acting as a seller agent) the screenshot, the focal item’s feature list, and the baseline sales outcomes for all competitors; the prompt asks it to recommend modifications solely to the focal product’s description—so as to appeal more strongly to the buyer agent, without inventing features or adding irrelevant keywords. We then replace the original description with the AI-recommended version and rerun the same 200 trials—again shuffling only on-page positions—to measure how the revised description alters purchase shares across categories. For the prompt used for the seller AI agent, refer to Appendix C.3.

Main Results

Instruction Following and Basic Rationality

We say that a model fails in a given experiment if it chooses a product listing different from the single listing that satisfies the query. We provide failure rates of different VLMs across different instruction following and rationality tasks in Appendix C.1.

Instruction Following Tasks. Table 4 in Appendix summarizes each model’s failure rate across our instruction-following tasks. Performance improves steadily with new release: failure rates drop as Claude advances from Sonnet 3.5 $\to$ 3.7 $\to$ 4. The latest models—Claude Sonnet 4, GPT-4.1, Gemini 2.0 Flash, and Gemini 2.5 Flash—exhibited close to zero failure rates.

	Budget Constrained	Color Based	Brand Based
Claude Sonnet 3.5	4.0% (0.5%)	13.5% (1.4%)	0.0%
Claude Sonnet 3.7	0.0%	4.4% (0.5%)	0.0%
Claude Sonnet 4.0	0.0%	3.8% (0.5%)	0.0%
GPT–4o	0.0%	0.0%	0.0%
GPT–4.1	0.0%	0.0%	0.0%
Gemini 2.0 Flash	0.0%	0.0%	0.0%
Gemini 2.5 Flash	0.0%	0.0%	0.0%

Rationality Tests. Table 5 and Table 6 in Appendix summarize each model’s failure rate across different price-based and rating-based tasks respectively. We find that: (i) Model upgrades generally reduce error rates—compare, for example, GPT-4o to GPT-4.1 or Gemini 2.0 Flash to Gemini 2.5 Flash—yet even the most recent systems still make mistakes, (ii) In one of the experiments, we reduce the price of one listing by $\alpha$-fraction while keeping the price of the remaining seven listings the same. For $\alpha= 0.01$, we find failure rates above 63% for Claude 3.5 Sonnet and above 25% for GPT-4o, whereas there is substantial reduction in failure rate for $\alpha= 0.1$ (10% discount) or for newer models. Introducing random dispersion across all prices lowers failure rates, and the reduction is larger when the dispersion itself is larger, suggesting that salient variation helps models recognize that price is the sole differentiator, thereby making these models less susceptible to making irrational choices, (iii) Increasing one product’s rating by 0.1—typically on dense five-star scales—still confuses advanced models (e.g., 28.7% failure for Claude Sonnet 4.0, 15.1% for GPT-4.1), whereas injecting larger random variation in ratings reduces the mistakes significantly.

	Price reduced for one listing (1% discount)	Price reduced for one listing (5% discount )	Price reduced for one listing (10% discount)	Random prices (low var.)	Random prices (high var.)
Claude 3.5	63.7% (1.7%)	36.8% (1.4%)	32.5% (1.0%)	8.3% (0.4%)	5.0% (0.3%)
Claude 3.7	21.0% (0.9%)	6.5% (0.7%)	1.2% (0.2%)	8.3% (0.4%)	6.0% (0.3%)
Claude 4.0	0.5% (0.1%)	1.8% (0.2%)	0.0%	8.3% (0.3%)	4.3% (0.2%)
GPT–4o	25.8% (1.0%)	5.6% (0.4%)	6.8% (0.3%)	17.4% (0.9%)	3.6% (0.2%)
GPT–4.1	9.3% (0.6%)	1.3% (0.1%)	1.2% (0.2%)	12.6% (0.8%)	0.8% (0.1%)
Gemini 2.0	2.8% (0.2%)	0.5% (0.1%)	1.2% (0.2%)	1.0% (0.1%)	6.5% (0.3%)
Gemini 2.5	1.0% (0.1%)	0.0%	0.0%	0.8% (0.1%)	0.0%

	Rating of one listing increased by 0.1	Random ratings (low variance)	Random ratings (high variance)
Claude Sonnet 3.5	57.3% (1.5%)	16.3% (0.8%)	2.7% (0.2%)
Claude Sonnet 3.7	6.7% (0.5%)	0.0%	0.0%
Claude Sonnet 4.0	28.7% (1.2%)	9.4% (0.6%)	4.7% (0.3%)
GPT–4o	71.7% (0.9%)	16.0% (0.5%)	7.3% (0.4%)
GPT–4.1	15.1% (0.6%)	11.7% (0.5%)	6.0% (0.4%)
Gemini 2.0 Flash	0.0%	0.0%	0.3% (0.1%)
Gemini 2.5 Flash	0.0%	0.0%	0.0%

We investigated the explanations provided by models when they failed to select the optimal product (e.g., the lowest-priced or highest-rated one). The explanations can be broadly categorized into three types: perceptual limitations, where they couldn’t distinguish between products, especially when differences were subtle; unjustified sub-optimality, where they recognized the optimal choice but still selected a worse option without explanation; and dismissal with justification, where they acknowledged the best option but gave reasons for not choosing it, such as preferring “standard” options, attributing differences to errors (like “display errors” or “temporary discounts”), or deeming the difference insignificant (e.g., “no functional difference”).

Choice Behavior and Biases

For each model (Claude Sonnet 4, GPT-4.1, Gemini 2.5 Flash), we estimate a conditional logit (CL) [35] with the following utility specification:

where $i$ indexes choice sets (experiments), $j$ products, $x_{ij}$ are position dummies (top row and columns 1–3; bottom row and column 4 are the omitted categories), $\mathcal{T} = \{\text{sponsored}, \text{overall pick}, \text{scarcity}\}$ is the set of tags, $\theta_j$ are product fixed effects, and $\varepsilon_{ij}$ is Type I extreme value. The corresponding choice probabilities are:

Table 1 reports the pooled (across categories) estimates.

	Claude Sonnet 4	GPT-4.1	Gemini 2.5 Flash
Position effects
Row 1	$1.224^{***}$<br>$(0.046)$	$1.045^{***}$<br>$(0.046)$	$0.344^{***}$<br>$(0.041)$
Column 1	$-0.297^{***}$<br>$(0.065)$	$1.122^{***}$<br>$(0.061)$	$-0.264^{***}$<br>$(0.057)$
Column 2	$0.557^{***}$<br>$(0.058)$	$0.019$<br>$(0.065)$	$-0.742^{***}$<br>$(0.061)$
Column 3	$0.416^{***}$<br>$(0.059)$	$-0.013$<br>$(0.066)$	$0.162^{**}$<br>$(0.054)$
Badge effects
Sponsored Tag	$-0.135^{*}$<br>$(0.068)$	$-0.248^{***}$<br>$(0.072)$	$-0.263^{***}$<br>$(0.067)$
Overall Pick Tag	$1.060^{***}$<br>$(0.077)$	$0.802^{***}$<br>$(0.083)$	$1.897^{***}$<br>$(0.072)$
Scarcity Tag	$-0.076$<br>$(0.094)$	$-0.105$<br>$(0.099)$	$-0.342^{***}$<br>$(0.098)$
Attribute effects
$\ln(\text{Price})$	$-1.623^{***}$<br>$(0.079)$	$-1.612^{***}$<br>$(0.083)$	$-2.190^{***}$<br>$(0.080)$
Rating	$4.913^{***}$<br>$(0.218)$	$8.300^{***}$<br>$(0.269)$	$5.388^{***}$<br>$(0.218)$
$\ln(\text{Num. of Reviews})$	$0.415^{***}$<br>$(0.023)$	$0.739^{***}$<br>$(0.026)$	$0.501^{***}$<br>$(0.023)$
Product Fixed Effects	Yes	Yes	Yes
Observations	25,802	25,066	25,215
Choice Sets (Groups)	3,756	3,931	3,953
Pseudo R-squared	0.44	0.51	0.42

Significance is indicated as: $^{*}p < 0.05$, $^{**}p < 0.01$, $^{***}p < 0.001$.

Ranking matters—and how it matters is highly model-dependent. All three models display statistically significant sensitivity to where a product appears on the page. Holding all attributes constant, each model assigns a clear premium to the top row relative to the bottom row. However, the horizontal (column) patterns vary sharply. GPT-4.1 strongly favors the first column; Claude Sonnet 4, in contrast, largely ignores the first column and prefers the two middle columns; and Gemini 2.5 Flash tilts toward the third column, while columns one and two are comparatively disfavored. Taken together, these patterns imply different “heat maps” of positional utility. For illustration, in Figure 3, we show the selection probability of the same product at different positions using the estimates in [](#tbl:1. We obtain these selection rates using (2) and setting all attributes except for the position to be same across the eight identical products. The position can lead to drastic changes in selection rates. For example, for Claude Sonnet 4, moving a product from the bottom right corner (where it is selected 4.5%) to the top row in the second or third column leads to a 5-fold increase in selection rate! Interestingly, the top left corner would only yield half of that increase.

Sponsored tags hurt sales while Platform Endorsement substantially increases selection chances. Across all three frontier models, the three badges–sponsored, overall pick, and scarcity–shift choices in the same qualitative direction. A sponsored tag reduces the likelihood of selection (a product with a baseline selection probability of 10% falls to 8.9% for Claude Sonnet 4, 8.0% for GPT-4.1, and 7.9% for Gemini 2.5 Flash), the scarcity tag effect is weakly negative or statistically indistinguishable from zero, whereas an overall pick endorsement delivers a large positive lift, raising the same baseline to 24.3%, 19.9%, and 42.6%, respectively. Because badge assignment is fully randomized, these are causal effects, not artifacts of latent product quality. The negative impact of the sponsored tag suggests that agents discount advertising, while the strong positive response to overall pick implies that platform endorsements are treated as credible signals–echoing the human-buyer evidence in [31]. Importantly, our estimates do not imply that sponsored ads are necessarily harmful for sellers: sponsored ads secure premium placement—front-page or top-row exposure—that a seller might not attain organically. Our findings should therefore be interpreted conditionally: given a fixed position, the mere presence of the sponsored tag lowers an agent’s selection probability. Sponsored ads thus buy attention through placement but induce a credibility cost, highlighting a trade-off that platform designers and sellers must weigh.

Directional agreement on product attributes and human-like behavior. All three models assign negative weight to price and positive weight to ratings and the number of reviews. Directionally, this mirrors what a rational human shopper would do: prefer cheaper, better-rated, and more widely reviewed products. Quantitatively, however, the sensitivities are heterogeneous across models. For example, consider a product with a baseline selection probability of 10%, an +0.1 increase in rating lifts the probability to 15.4%, 20.3% and 16.0% with Claude Sonnet 4, GPT-4.1 and Gemini 2.5 Flash, respectively.

Strong response to position, ratings and tags like “Overall Pick”. We investigate by how much can a seller increase the price of their product (while keeping the latent utility governing user choice unchanged), if they could (i) move from the second row to the first row, (ii) get an Overall Pick endorsement from the platform, (iii) increase their rating by +0.1, or (iv) double the number of reviews? Understanding these trade-offs are important not only for the sellers but also for platform operators since many of them operate via seller commissions. The results in Table 7 in Appendix D.3 summarize how different models trade-off these attributes. For Claude Sonnet 4 and GPT-4.1, moving from the second to the first row is worth roughly a doubling of price, whereas for Gemini 2.5 Flash the same move is worth only about +17%. The Overall Pick tag produces very large lifts–e.g., +92% for Claude Sonnet 4, +65% for GPT-4.1 and +138% price headroom for Gemini 2.5 Flash. An increment of +0.1 in ratings, allows a seller to increase their price by a quarter with Gemini 2.5 Flash and a third with Claude Sonnet 4 while this more than doubles to 67% with GPT-4.1. Finally, doubling of reviews, allows a 19%, 37% and 17% increase in price with Claude Sonnet 4, GPT-4.1 and Gemini 2.5 Flash respectively.

	Claude Sonnet 4	GPT-4.1	Gemini 2.5 Flash
How much could a seller raise prices (holding utility fixed) if the seller get...
Row 1 (vs Row 2)	+112.6%	+91.2%	+17.0%
Overall Pick tag	+92.2%	+64.5%	+137.8%
+0.1 rating	+35.4%	+67.3%	+27.9%
Double reviews	+19.4%	+37.4%	+17.2%
How much would a seller need to cut price to offset the harm of ...
Sponsored tag	-8.0%	-14.3%	-11.3%
Scarcity tag	-4.6%	-6.3%	-14.5%

Seller’s response to AI mediated demand

Recalling the experimental setup, we define the change in market share of the focal product as the difference in the selection probabilities of the focal product after and before the description modification. Because all other attributes are held fixed—and we reuse the identical product shuffles in the pre/post modification experiments—the observed differences are causally attributable to the description modification. We provide an illustrative case study in Appendix D.4.1.

The seller side intervention leads to an average increase in market share of 2.69% (s.e. 1.29%) for Claude Sonnet 4, 5.64% (s.e. 1.32%) for GPT-4.1 and 4.83% (s.e. 1.38%) for Gemini 2.5 Flash. In Figure 4, we report the change in market share for the focal product across different categories and buyer agents (Claude Sonnet 4, GPT-4.1, Gemini 2.5 Flash). Effects are heterogeneous across both categories and models. In 75% of category–model pairs, the (one-shot) description modification yields no statistically significant change in market share. In the remaining 25% cases, a one-shot modification produces large gains: for Claude Sonnet 4 we observe +9.0% in iPhone 16 Pro Cover and +15.4% in toilet paper; for GPT-4.1, +21.8% in mousepad and +16.3% in office lamp; and for Gemini 2.5 Flash, +23.6% in iPhone 16 Pro Cover, and +9.0% in toothpaste. Firms typically invest heavily in marketing and may not expect double digit increases from minor edits, yet our results indicate that, in the presence of AI-mediated demand, targeted description modifications alone may materially shift selection shares. We emphasize that this is a single-pass intervention; iterative optimization or simultaneous adoption by multiple sellers could amplify or dampen these effects and would be interesting to explore in future work.

Implications and Discussion

Industry directions

The findings in this study suggest important implications for various stakeholders and suggest significant adjustments and new opportunities for the e-commerce industry.

Platforms. First, the present study has illustrated that position biases are strong and vary widely across AI models. For product ranking, platforms may need to adapt their layout and ranking systems to account for this. We contend that such biases suggest a compelling case for the implementation of agent-specific storefronts operating on a standardized protocol, such as the Model Context Protocol [2]. Second, traditional monetization levers for platforms may be rendered ineffective by AI shoppers. Instead, new levers, such as dynamically optimizing product titles and images on behalf of sellers, could become more powerful tools for platforms to boost sales. Third, platforms (and regulators) should be monitor the impact of AI assisted shopping on market efficiency and consumer welfare.

Brands and Sellers. Given the volatility in market share caused by differences between model providers, sellers will need to continuously adapt their product listings to be in the AI agent consideration sets. New AI model releases as well as model updates may also lead to significant changes in AI agent purchase behavior. In Appendix E, we document significant changes in market shares and position biases when switching from Gemini 2.5 Flash Preview to Gemini 2.5 Flash. A new category of seller-side AI agent companies is likely to emerge, aiming to assist sellers in monitoring and navigating, potentially in real time, this increasingly dynamic market.

Consumers/Buyers. As consumers delegate purchasing decisions to AI agents, search frictions will dramatically decrease, but as we documented, there may be some risks of irrational or wrong purchases. There is a growing need to educate consumers about the differing preferences and behaviors of various buying agents. These behavioral discrepancies underscore the importance of developing new buyer-side AI agents that better align decisions with user preferences.

AI shopping agent developers and Regulators. AI shopping assistants should undergo standard evaluations along the lines of ACES and publish their scores. To the extent this doesn’t happen organically, regulators should step in to require this. Next, the largest platforms may not organically provide a standardized protocol for AI shopping agents–despite the biases and inefficiency resulting from AI assistants browsing webpages–e.g., to protect their ability to monetize via sponsored products. If so, we argue that this will be another opportunity for regulators to put appropriate requirements in place.

Research directions

The current paper and its findings also suggest important research directions.

Diagnostics. The ACES framework we develop is a step in developing systematic diagnostic tools for agentic e-commerce. We believe that developing such sandbox environments will be critical for all players in the ecosystem, and that developing a suite of metrics for agents will be helpful in measuring possible weaknesses and technological progress. There are various concrete avenues for research including building better predictive models of AI agents’ choices, further understanding the drivers of their decision making, mechanistic interpretability of models, and expanding the scope of measurement to the entire purchasing journey to (not just the Veni, Vidi, Emi model in this paper) and the impact of prompts. It would also be interesting to explore other candidate drivers of choice, such as the images associated with the products, and webpage designs.

Operating modes. We have advocated for platforms to provide a standardized protocol for AI shopping agents. Will agents leveraging such a protocol suffer from similar biases and heterogeneous behavior? How to design such a protocol to maximize market performance?

Strategic Interactions. The present study suggests that buyers, sellers and platforms will have significant incentives to leverage AI agents. Studying the strategic interactions among players, how these may differ in an AI-mediated world, possible efficiency gains and unintended consequences, is an exciting avenue for future research.

AI agents vs. human buyers. A natural academic question that emerges is how do AI agents compare to human buyers? In addition, as AI agents are deployed at scale, the question will be whether human preferences will be indirectly shaped over time by the choices of the AI agents, and whether this effect will be stronger than that of recommendation algorithms.

Interactive AI assistance. AI buying agents can benefit from some degree of interactiveness for preference elicitation, towards choices that may better align with user preferences. The design and optimization of such dynamic interactions is an exciting direction.

Data and Software Availability

To support further research and replication studies, we have made our ACES framework available at https://github.com/mycustomai/ACES, which includes the code for both the AI agent as well as the mock e-commerce platform. The datasets are made available on huggingface: ACE-RS, ACE-BB and ACE-SR.

Appendix

Organization of the Appendix. In Section A, we provide further details about the ACES framework. In Section B, we discuss related literature. In Section C, we provide further details regarding the experimental setup for our instruction following and rationality experiments (Section C.1), choice behavior and biases experiments (Section C.2) and the seller response experiments (Section C.3). In Section D, we provide further details regarding our main results and insights: market share across different categories (Section D.1); instruction following and basic rationality experiments (Section D.2); choice behavior and biases (Section D.3); seller response to AI-assisted shopping (Section D.4). In Section E, we discuss the changes in market share and position effects due to model update from Gemini 2.5 Flash Preview to Gemini 2.5 Flash.

ACES Framework

We envision an agentic e-commerce workflow in which AI assistants transact on behalf of consumers. These “agents” are simply a VLM (or LLM) given the ability to interact with the external world through “tools” [1, 39, 43]. These agents can be personalized (for example, through purchase histories) and—instead of users searching and buying themselves—execute the end-to-end journey: (i) open a browser, (ii) visit a retailer’s site, (iii) issue a query, (iv) parse and navigate product listings using tool use (scrolling, clicking, screenshotting, etc.), and (v) complete checkout. Figure 5 illustrates this AI shopping agent in operation.

Business Goal & Context

The ACES framework’s primary purpose to simulate the interaction depicted by Figure 5.

This shopping agent workflow follows a common design pattern that is well-evaluated in the literature [32, 54], and is already productized [1, 22, 29, 36, 37, 55].

The shopping agent workflow employed:

1. Opens a browser

2. Navigates to the target platform

3. Searches for a product

4. Captures a screenshot

5. Selects a product using the provided tool

ACES seeks to collapse an agent’s journey to the linear sequence stated above, while accurately emulating a production-ready shopping agent, interacting with the underlying VLM models in a uniform manner, and ensuring that the choice behaviors of the model are accessible and measurable. Unlike many real-world examples such as [1, 38] that employ external modules to increase capabilities in arbitrary tasks, ACES seeks to leave the VLM unassisted to more accurately approximate outcomes.

High-level Architecture & Approach

Given the complexity of employing VLM agents in the shopping task, ACES is designed with the following principles:

• Reusability Stakeholders, future researchers, and engineers might look to ACES for future research. Adapting the mock-app, providing new experiments, and extending the agent has been kept in mind.

• Provider/Model agnostic Using VLMs in the agent context raises 2 primary hurdles: the low-level code to interface with these VLMs is not the same; not all VLMs are designed to interact with external environments as signaled by the diverging implementations of [22, 55]. We solve this by using LangChain [7] which provides a single interface for interacting with VLMs, and collapsing the action space to a single tool call. As a byproduct, the code is rendered easily extensible.

• Observability Understanding results and debugging must have minimal dependencies on external frameworks. As such, the entire agent journey is saved to a tree-like file hierarchy which can be easily inspected.

• Cost & Efficiency To ensure efficiency, the framework includes the boilerplate code to utilize “batch processing” to allow experimentation to be run at high-throughput.

To conduct experiments which model Figure 5, ACES is comprised of 3 top-level components—agent, experiments, and sandbox—each responsible for different portions of the workflow, and who interact through clearly defined interfaces.

The majority of the code is contained within experiments which encapsulates the dataset generation and randomization, collection of the results, and batch processing. The shopping agent itself is defined within agent; and sandbox contains the “mock-app”.

Related Literature

Our work connects to a rich body of literature spanning computer-use agents, e-commerce systems, recommendation and ranking algorithms, behavioral biases in human decision-making, and recent advances in multi-modal AI agents. Across these threads, our contribution is to provide a controlled, visual, and interactive sandbox that zooms in on the choice step (which product to buy) for AI shopping agents, and measures rationality and choice behavior via randomized page layouts, attributes, and badges. Although ACES resembles prior computer-use testbeds such as WebArena and VisualWebArena, our emphasis is different: rather than evaluating end-to-end web navigation, we focus on a single critical step—selecting which product to buy—to obtain a fine-grained view of agent choice behavior in shopping contexts [27, 58].

Computer-Use Agents and Benchmarks. A fast-growing body of work builds web and OS-interacting agents evaluated in realistic environments. WebArena introduces reproducible, long-horizon web tasks over functional sites (e.g., e-commerce, forums), with baseline agents achieving low end-to-end success relative to humans [58]. VisualWebArena extends to multimodal interfaces with images and forms [27]. Mind2Web targets generalist web agents across 137 real websites [11], and follow-ups examine grounding for GPT-4V as a generalist web agent [57]. Beyond the browser, OSWorld provides a real-computer environment (Ubuntu, Windows, macOS) and documents large headroom between SOTA agents and humans [51]; Windows Agent Arena and AndroidWorld further expand to Windows and Android ecosystems [5, 34]. Work studying agent–computer interfaces shows how UI design and tool affordances affect agent success (e.g., SWE-agent) [53]. Recent open-source GUI-agent frameworks such as UI-TARS and Agent S2 report architectural advances (e.g., GUI grounding, hierarchical planning) and improved benchmark performance [1, 56]. In light of the documented low end-to-end success rates on complex multi-step tasks, our framework intentionally constrains the interaction and isolates the choice problem so that we can study rational behavior and the influence of layout and badges without conflating failures from unrelated subroutines (e.g., brittle scrolling or app switching).

Autonomous Shopping Agents and Multimodal Product Understanding. WebShop frames shopping as instruction-following in a simulated web store, training agents with imitation and reinforcement learning [54]. More recent shopping evaluations include Shopping MMLU (text-based, multi-task retail skills) and DeepShop (deep research shopping agents with live navigation), as well as dialog-driven and domain-tuned LLMs for e-commerce such as eCeLLM and LiLiuM [9, 21, 24, 32, 40]. Vision-language product understanding models (e.g., PUMGPT) address attribute extraction and classification from images and text [52]. Our ACES sandbox complements these works by providing a controllable setting with randomized listings and visual perturbations to test whether shopping agents exhibit instruction adherence and choice rationality, and to understand their choice behavior, in the presence of positions, promotions, and badges.

Product Rankings, Platform Design, and Assortment Optimization. Empirical and theoretical work shows that positions and rankings can causally shape what consumers examine and buy. [48] provides experimental evidence on the causal impact of rankings on online search and purchases; [15] quantifies direct and interaction effects between ranking and ratings on consumer behavior and platform revenue. Our work is related to this line of research as we investigate the causal impact of rankings (positions) and product attributes on selection by an AI agent. There is related theoretical and empirical work modelling the consumer’s search process and developing platforms ranking algorithm [8, 12]. Our work also connects to a long line of work on assortment optimization which takes customer/AI shopper behavior (as estimated by us) as an input, e.g., see [10, 13, 17, 28].

Platform Endorsements and Badges. Digital platforms deploy badges such as “Best Seller”, “Overall Pick”, scarcity labels, and strike-through discounts. Recent empirical evidence suggests badges can meaningfully change clicks and add-to-cart probabilities [31]; complementary work studies badge longevity, uniformity, and bias across marketplaces, and the causal effects of badges in online communities [4, 23, 30]. Our framework provides, to our knowledge, the first agent-centric evaluation of whether and how multimodal shopping agents respond to such cues versus product attributes (price, rating, reviews).

Personalization and Recommender Systems. Classical recommender systems and industrial practice highlight personalization’s central role in e-commerce [42, 45]. Recent LLM personalization efforts create benchmarks and methods for tailoring responses to user-specific preferences [59] and dataset resources for digital-twin style behavioral modeling [47]. In this paper we do not personalize the agents; we deliberately use generic prompts to elicit baseline behavior that is not pre-solved by over-specifying the target item. Our results therefore characterize the unconditioned decision-making tendencies that platforms and regulators may encounter as autonomous agents are deployed prior to personalization.

Algorithmic Delegation. Theoretical work on delegation studies how a principal can design mechanisms for delegating to an agent whose incentives or information differ from those of the principal, and payments are not possible [3]. [26] show that appropriately constrained delegation can approximate efficient search despite misaligned incentives; [20] find benefits of delegating search to multiple agents; and [19] study the optimal design of algorithmic delegates to help the user under information asymmetry, given that certain task categories will be delegated to them by users while others will not. Our empirical results can be viewed as documenting the outcome of delegation to AI agents from leading providers in a concrete market setting: given a visual catalog, what do agents buy and why, and how do platform levers and product attributes translate into systematic selection patterns?

Experimental Setup and Evaluation Details

Vision-Language-Models Evaluation. For VLM evaluation, we consider the following models: Anthropic’s Claude Sonnet 3.5, 3.7, and 4; OpenAI’s GPT-4o and GPT-4.1; and Google DeepMind’s Gemini 2.0 Flash and Gemini 2.5 Flash. Notably, both OpenAI’s GPT-4o and Google DeepMind’s Gemini 2.0 Flash are directly productized as shopping agents through OpenAI’s Operator [29, 37] and Google’s Project Mariner [16, 55], respectively. Following the initial rationality checks across all models, we conduct our detailed analysis on the latest release from each provider, under the assumption that older versions will be deprecated in favor of their successors in advanced applications. To maintain a roughly uniform baseline of capability, we exclude reasoning models and explicit reasoning components—such as omitting the default reasoning tokens for Gemini 2.5 Flash and Anthropic’s Claude Sonnet 4, whereas OpenAI’s GPT-4.1 does not support reasoning. Furthermore, reasoning models were excluded from our selection due to their higher latency, making them unfavorable in user-facing agentic tasks which require low latency inference [25, 41, 49].

Universe of Product Categories. On the platform side, we select eight product categories—fitness watch, iPhone 16 Pro cover, mousepad, office lamp, stapler, toilet paper, toothpaste, and washing machine—chosen to represent a mix of high- and low-ticket items, to generalize across a broad range of product types, and to include both well-known and niche brands. For each of the product categories, we choose eight representative products (found based on public information on the Amazon website) which form our assortment of products which we then display in a $2 \times4$ grid layout on our mock-app.

Instruction Following and Rationality Experiments

We provide some additional details on the experimental design and setup for our instruction following and rationality experiments.

Instruction Following Experiments

We test the models for three types of instruction following tasks: (i) the ability to choose a product within a specified budget, (ii) choose a specific color and (iii) choose a specific brand. For each of the tasks, we selected two product categories summarized in Table 2. The tasks were specified in such a way that there was a unique product that satisfied the query. Figure 7 displays the mock-app setup for the instruction following tasks.

Task	Product Category	Specified Task
Budget-Constrained	fitness watch	The budget constraint is $25
Budget-Constrained	toilet paper	The budget constraint is $10
Color-based	mousepad	Choose the pink color product
Color-based	stapler	Choose the pink color product
Brand-based	iphone 16 pro cover	Choose the Otterbox brand
Brand-based	toothpaste	Choose the Colgate brand

Price-Based Rationality Experiments

We have two types of price-based rationality experiments. In both, all products are identical except for price. In one setting (i) all price are equal, except for the price of one listing which is decreased by $\alpha$-fraction and in a second setting $(ii)$ we assign random prices to each of the listings drawn from a normal distribution $\mathcal{N}(\mu,\sigma^2)$ with mean $\mu$ and variance $\sigma^2$. For $(i)$, we consider $\alpha \in

$\{0.1, 0.05, 0.01\}$ which corresponds to a 10%, 5% and 1% discount in the prices with respect to the other prices respectively (see fig:8a). For $(ii)$, we set the mean $\mu$ as the average of the eight product prices in their respective category. For the variance, we consider $\sigma= \$0.3$ (low variance; see fig:8c) and $\sigma= 0.2\mu$ (high variance; see fig:8e).

Rating-Based Rationality Experiments

We have three rating-based experiments: $(i)$ the rating of one listing is increased by 0.1 (see fig:8b), $(ii)$ random ratings for the eight products with ratings ranging from 4.4 to 4.7 (low variance; see fig:8d) and $(iii)$ random ratings for the eight products with ratings ranging from 3.0 to 4.5 (high variance; see fig:8f). Note that except for the ratings, all the other product attributes are the same.

Choice Behavior and Biases

To estimate the sensitivity of the AI agent to different platform levers, we create a dataset with exogenous variation across a number of variables associated with the products (see Table 3). Given these variations are exogenous to the inherent attractiveness of the products, they allow us to obtain causal estimates to the impact of different platform levers like position, tags and product attributes like price, rating and number of reviews. We also provide a visualization of the exogenous variation of price, rating and number of reviews around their original value for a few product categories in Figure 9. Figure 10 provides an example of different mock-app screens for two different experiment runs. Notice that the assortment of the eight products is the same across the two experiments. However the position of the eight products change across the two experiments along with their price, rating and number of reviews. Also notice that the Sponsored, Overall Pick and Only X Remaining (scarcity) tags are assigned to different products across the two experiments.

Ind. Variables	Exogenous Variation
Position	Randomly permute the position of the eight products
Sponsored Tag	Randomly assign a Sponsored tag to $X$ listings, $X \sim\operatorname{Unif}(\{1,\ldots,4\})$
Overall Pick Tag	Randomly assign a Overall Pick tag to a listing without Sponsored tag
Scarcity Tag	Randomly assign a Only $X$ Remaining tag to a listing without a Sponsored or Overall Pick tag, $X \sim\operatorname{Unif}(\{1,\ldots,5\})$
Price	Randomly perturb the original price $p_j$ of each product $j$, $p'_j \leftarrow p_j \cdot f_j$, where $f_j \sim\operatorname{logNormal}(\mu=0,\sigma=0.3)$
Rating	Randomly perturb the original rating $r_j$ of each product $j$, $r'_j \leftarrow r_j+\alpha_j\cdot(5-r_j)$, where $\alpha_j \sim\operatorname{Unif}([-0.8,0.8])$
Number of reviews	Randomly perturb the original number of reviews $N_j$ of each product $j$, $N'_j \leftarrow N_j\cdot f_j$, where $f_j \sim\operatorname{logNormal}(\mu=0,\sigma=1)$

Seller Response to AI-assisted shopping

In this section, we aim to design an experiment to measure the impact of seller-side AI agents recommendations for the product description when buyers use a particular agent. To estimate the causal impact of the recommendation made by the AI seller agent, we have the following experimental setup.

1. Simulate the purchase behavior of the AI buying agent across different product categories. We run 200 experiment trials where we present the AI buying agent (Claude Sonnet 4, GPT-4.1 and Gemini 2.5 Flash) with eight products per category. The eight products are shuffled uniformly at random across the 200 experiments. Note that we only shuffle the position of the

products on the screen but do not change any of their other attributes like price, description, etc.

2. For each category, we select uniformly at random a single product from the eight-product assortment. The selected product becomes the focal product/seller that optimizes its description via the AI seller agent.

3. Prompt the AI seller agent (e.g., GPT-4.1) to generate recommendations for the product description. We provide the AI seller agent a screenshot of the platform with the eight products along with the features of the seller’s product, some context and the sales data of competing sellers induced by the buying AI agent. The exact prompt is reproduced below.

Prompt Template for AI seller agent

Context: There is an AI agent (a vision language model) which is given a screenshot of an e-commerce website selling a particular product and it decides on the products based on the given attributes. I have the sales data on all the products including mine. I want you to act as an agent on my behalf and suggest changes to the product title so that I can increase my sales by making my product more appealing to the AI agent.

My product is {product name}. The product features are {detailed features from Amazon}

The sales data on the competition is below.

{simulated purchase data from the 200 experiments with the original descriptions}.

It is important that the title changes you suggest align with the product features provided to you. Do not make up product features or add spurious keywords. Just use the product feature information provided.

4. Change the product description of the seller with the one recommended by the AI seller agent and simulate the purchase behavior of the AI agents across different product categories. Repeat the 200 experiment trials as before with the only change being the modified product description of the seller.

Note that for simplicity in this study of seller response, we assume that only one seller uses an AI agent to optimize their product description, and the seller tries only one AI generated recommendation (through one interaction with their AI agent).

Main Results Details

Market Share of different models across different categories

We run 1000 experiments for each product category in which we randomly permute the position of the products across experiments, and deploy an AI buying agent. For each product category, we compute the induced market shares, representing the selection rates of different products across the 1000 experiments. We present the market shares induced by different AI models in Figure 1 for the fitness watch and stapler categories and in Figure 11 for six additional product categories.

Instruction Following and Basic Rationality

Instruction Following Experiments

We summarize our experiments, and the associated fail rates, in Table 4.

Price-Based Rationality Experiments

We summarize our experiments and the associated fail rates in Table 5.

Rating-Based Rationality Experiments

We summarize our experiments, and the associated fail rates, in Table 6.

Analysis of Failures

On the experiments where the models failed, we probed the explanation provided by the models. Broadly speaking, we identified three types of explanations.

• Perceptual Limitations. The model treated all products as identical, failing to recognize differences in price or rating. This type of error was particularly prevalent when the difference was subtle, for e.g., in the case when the price of one listing was reduced by 1%.

• Unjustified Sub-optimality. The model recognized the optimal product (e.g., the cheapest or highest-rated) but still made a sub-optimal choice without providing any specific justification for its decision.

• Dismissal with justification. The model acknowledged the optimal product but explicitly provided a reason for not selecting it. These reasons often included:

  • Prioritizing “Standard” Options. The model would state it “chose the standard price/rating” or the “most common price point”, indicating a preference for typical values over slight deviations.

  • Attributing difference to errors. The model sometimes dismissed the optimal option by attributing the observed difference to a “display error”, a “temporary discount”, an “outdated listing”, or a “pricing error” on the platform.

  • Perceived Insignificance. The model might justify its sub-optimal choice by stating the price difference was minimal or that there was “no apparent/significant advantage” or “no functional difference” compared to other options.

Choice Behavior and Biases

Understanding the trade-offs between position, tags and listing attributes. We fit a conditional logit with the utility specification in (1). The estimates are provided in Table 1. We can use the estimates in Table 1, (1) and (2) to understand how the different models trade-off different attributes. For example, we can be interested in understanding how the choice probability changes if there is a Sponsored tag or an Overall Pick tag or rating changes by $\pm0.1$. Lets say that the baseline selection probability of a product $j$ is $p_j$ and assume that we change co-variate $z$ with coefficient $\beta_z$ by $\Delta z$. Let $o_j = p_j/(1-p_j)$ be the odd-ratio for product $j$, then the new selection probability $p'_j$ is given as

where ($a$) follows from (2), ($b$) follows by dividing numerator and denominator by $\sum_k \exp(U_k)$ and the fact that $p_j = \exp(U_j) / \sum_{k \ne i} \exp(U_k)$ and ($c$) follows from the definition of $o_j$. Using this, we can calculate how the probability will change if we change some covariate.

We can also obtain price-equivalent trade-offs: a convenient way to compare levers is to ask: by how much can a seller raise (or must cut) price to keep the utility constant when the seller adds (or loses) a feature worth $\Delta U = \beta_z \Delta z$. Let say, we change the price by a multiplicative factor of $\lambda$, i.e., $P' = \lambda P$ where $P$ is the original price of the product. Since we are keeping the utility constant, we have that $\beta_{\mathrm{price}} \ln\lambda+ \beta_z \Delta z = 0$, this implies that $\lambda= \exp(-\beta_z \Delta z / \beta_{\mathrm{price}})$. This allows to be get price-based exchange rate between different attributes. In Table 7, we present the price-equivalent results for different covariates.

Seller Response to AI-Assisted Shopping

Case Study of Seller’s Response to AI-assisted shopping

We consider the illustrative example of the mousepad product category. Within this category, our focal product (randomly selected) is the brand Aothia. We prompt an AI seller agent (GPT-4.1 in this case) with the simulated demand on the original product descriptions of the eight products in the catalog using GPT-4.1 as the AI buying agent. Below is the response of the AI seller agent on the recommended product description change.

AI Seller Agent’s (GPT-4.1) Response

Here’s how to make your Aothia Leather Desk Pad Protector title more AI- and customer-friendly, using only your provided features and aligning with what’s selling well on this page:

Competitive Patterns

• Top competitors (Amazon Basics, KTRIO, MROCO) lead with the product’s main use (“Ergonomic Mouse Pad,” “Gaming Mouse Pad”) and then surface key features: material, size, comfort, non-slip, waterproof, color.

• Titles are concise, not overloaded, but cover all core attributes and benefits.

Current Title

Aothia Leather Desk Pad Protector, Mouse Pad, Office Desk Mat, Non-Slip PU Leather Desk Blotter, Laptop Desk Pad, Waterproof Desk Writing Pad for Office and Home (Black, 23.6” × 13.7”)

Weaknesses

• Good breadth, but the most important features are spread out.

• “Leather Desk Pad Protector” is slightly generic—PU Leather is a value signal.

• “Non-Slip” and “Waterproof” should be earlier.

• “Large” or “XL” size callout (if true) is attractive for this market.

Optimized Title Suggestions

Option 1: Feature-Rich, Direct

Aothia Large PU Leather Desk Pad, Non-Slip Waterproof Mouse Pad & Desk Mat, 23.6” × 13.7”, Office and Home, Black

Why

• Leads with brand and size (“Large”).

• “PU Leather” specifies the material.

• “Desk Pad, Non-Slip, Waterproof” are high-signal attributes and not buried.

• “Mouse Pad & Desk Mat” covers multi-purpose use.

• Size and color at the end for clarity and direct comparison.

Option 2: Even More Focused

Aothia PU Leather Desk Pad Protector, Large Non-Slip Waterproof Mouse Pad, 23.6” × 13.7”, Office Desk Mat, Black

Why

• Highlights core value (protection, size, non-slip, waterproof) right after the brand.

• Short and very clear.

Option 3: Ultra-Compact

Aothia Large PU Leather Desk Pad, Non-Slip Waterproof, 23.6” $\times$ 13.7”, Black

Why

• Super concise for listings where title space is limited.

Why These Work

• All keywords (PU Leather, Large, Non-Slip, Waterproof, Mouse Pad, Desk Mat, Size, Black) are present and near the front.

• Follows competitor structure for quick parsing by AI agents.

• No added or exaggerated claims—uses your real features only.

Final Recommendation

To compete with both Amazon Basics and KTRIO, and help the AI agent recognize your desk pad’s versatility and core selling points, use:

Aothia Large PU Leather Desk Pad, Non-Slip Waterproof Mouse Pad & Desk Mat, 23.6” $\times$ 13.7”, Office and Home, Black

We run a new set of experiments where we just change the product description (see fig:12a) of the focal product (Aothia in this case), with different AI buying agents (Claude Sonnet 4, GPT-4.1 and Gemini 2.5 Flash). In this case, we find that this rather innocuous looking change in the product description can significantly shift the market share. In fig:12b, we plot the market share of different products for the mousepad category when using GPT-4.1 as the AI buying agent. We observe that there is a statistically significant and large increase of 21.8% in the market share of the focal product Aothia. This comes at the expense of a decrease in market share of other products like the Amazon Basics Mousepad, MROCO and SteelSeries. Note that the only change across the two sets of experiments is the change in the product description, hence this impact is causal. This example illustrates the potential of changing market shares by simply optimizing the product description to appear more attractive to an AI buying agent. This is very much akin to search engine optimization (SEO), which is the practice of optimizing keywords in a website in order to have it rank higher in search results. We note that the gains in market share are not uniform across different AI buying agents. For the case of Claude Sonnet 4, we see that the change in market share of Aothia due to the product description change is around 5.3% and statistically significant while for the case of Gemini 2.5 Flash, the change in market share is only 2.0% (and not statistically significant), cf. fig:12c and fig:12d. This demonstrates that there can be non-uniform impact of such interventions. In the next section, we provide the change in market shares across different product categories.

Category-wise change in market share

Figure 13 reports, for each category and AI agent model, the product-level selection shares before and after modifying the focal product’s description, with the focal item highlighted in red.

Model Update: The case of Gemini 2.5 Flash Preview to Gemini 2.5 Flash

Our initial experiments used Gemini 2.5 Flash Preview, the then-current frontier model by Google Deepmind. During writing, the provider replaced it with Gemini 2.5 Flash and deprecated the Preview version. We therefore re-ran a subset of experiments with the “updated” model to measure how an upstream change propagates to AI-mediated demand and position effects.

Market-share shifts under a model upgrade. Figure 14 compares product-level selection shares when the buyer agent “updates” from Gemini 2.5 Flash Preview to Gemini 2.5 Flash, holding the catalog and attributes fixed and only randomizing on-page positions. Shares move substantially, with frequent flips in the modal product: e.g., in office lamp category the mode shifts from TORCHSTAR (Gemini 2.5 Flash Preview) to SUNMORY (Gemini 2.5 Flash); in toilet paper from Cottonelle to Angel Soft. Several products experience large swings in share (e.g., Fitbit Versa:

−25.1%; MROCO: −16.5%; Kenmore: −25.6%).

Position effects. Table 8 shows that position biases also change: Gemini 2.5 Flash Preview exhibits a negative top-row bias, while Gemini 2.5 Flash’s is positive. Both versions penalize Columns 1–2; Gemini 2.5 Preview is roughly indifferent between Columns 3–4, whereas Gemini 2.5 Flash strictly prefers Column 3 over Column 4. Figure 15 shows the heatmap of selection probabilities across different positions on the page for Gemini 2.5 Flash Preview and Gemini 2.5 Flash.

	Gemini 2.5 Flash Preview	Gemini 2.5 Flash
Position effects
Row 1	$-0.071^{*}$<br>$(0.031)$	$0.586^{***}$<br>$(0.042)$
Column 1	$-0.461^{***}$<br>$(0.044)$	$-0.138^{*}$<br>$(0.057)$
Column 2	$-0.581^{***}$<br>$(0.044)$	$-0.700^{***}$<br>$(0.061)$
Column 3	$0.026$<br>$(0.043)$	$0.160^{**}$<br>$(0.055)$
Product Fixed Effects	Yes	Yes
Observations	31649	15723
Choice Sets (Groups)	8166	3933
Pseudo R-squared	0.40	0.28
Significance is indicated as: $^{*}p < 0.05$, $^{**}p < 0.01$, $^{***}p < 0.001$.

Implications. These results underscore that upstream model updates can function as exogenous demand shocks—relabeling “best sellers” and redistributing shares even when products are unchanged. For platforms, this volatility affects the stability of ranking and endorsement signals (e.g., badges, recommendations) and argues for model-aware evaluation pipelines. For sellers, the findings motivate continuous monitoring and agile listing management: content tuned to yesterday’s model may underperform after an upgrade. More broadly, because model changes alter both attribute sensitivities and position bias magnitudes, identical designs can yield different outcomes solely due to an upstream revision—reinforcing the need for ongoing auditing, disclosure of impactful updates, and diversification of strategies across agent ecosystems.