WorksComputer Science
Straight to the Product: What Online Retailers Miss When Shoppers Ask AI First
Abstract
When people ask an AI assistant what to buy, what happens next at the store? We link the same people's assistant conversations to their later browsing at twenty large online retailers. People go to the product the assistant named: among shoppers who had not visited the store in the past week, visits to the named product within a day rise from 0.02 to 2.07 per 100 answers (+2.0 points [1.7, 2.4]), and on days with a product-page visit at the store, 44% show a product from their own answer, against 1.5% for a matched answer given to someone else. The result replicates in a separate five-month period. Few arrive through the assistant's link: 89% of visits carry no sign of it, and 54% follow a web search, the same share as their usual visits. When a search comes just before the first visit, it names the answer's product 28% of the time (1% in earlier weeks), as if the search box were an address bar. The chat names the product; the store logs a search. Once there, shoppers keep browsing. We propose product follow-through, the share of answers followed by a visit to the product they named, as a way to evaluate shopping assistants and agents by what people do next: it needs no referral data or provider logs and can be computed for each assistant.
Introduction
A shopper asks an assistant for a light running watch with at least 30 hours of GPS. The answer names one, the Contrail Run 5, and a store that sells it. A few minutes later the shopper types “contrail run 5” into a search engine, clicks a result and lands on the watch’s page at the store. The store’s analytics record a search visit to a product page. The conversation that named the watch leaves no trace there (Figure 1).
Figure 1. One shopping journey, invented for illustration (the watch and the store are fictional), with the paper’s three findings. The assistant names a product; many shoppers reach the store through search and keep browsing. The store sees only the last step.
How common is this journey, and what does it mean for retailers? Earlier panel work found that brands an assistant recommends get more searches and site visits [6]. Here we go down to the product. We link the same people’s assistant answers to their browsing at twenty large retailers and follow the journey in order:
People go to the product the assistant named. On 44% of days with a product-page visit at the named store, they view a product from their own answer, against 1.5% for someone else’s matched answer.
The store rarely sees the assistant. 89% of visits after a recommending answer show no link from it. Most follow a web search, as usual visits do, often for the product’s name.
Then they keep browsing. Against all their other visits, they view two or more products more often, in both periods,
CCS Concepts
• Information systems → Online shopping; Recommender systems; Users and interactive retrieval; Web log analysis. and otherwise shop much as usual: how often a visit opens on a product page or uses the store’s recommendations changes by a few points at most (Figure 2).
Figure 2. (a) People view the product from their own answer, not someone else’s, in both periods; the more specific the step, the larger the ratio to the baseline; recommendations move store visits far more than mentions; a quoted price goes with a larger rise. (b) At the store, visits open on a product page only slightly more often than usual, shoppers browse a little more, and the named product often opens the visit. Rings and dots are the groups labeled in each block; the pale bar for products the answer did not name spans the lower and upper bounds.
The store misses these visits for a simple reason. A search for the product’s name follows 9.3% of specific-product answers, about 24 times the usual rate, and the store’s analytics log the visit as ordinary search traffic.
The study makes two contributions. It gives product-level evidence, across sites and without referral data, that an assistant’s answer shapes which product people go to see. It also turns the answer-to-product connection into a measure, product follow-through (§ 3.1), which shows retailers and assistant builders what referral reports cannot.
Related work. Most shopping-assistant evaluations stop at the answer. Conversational recommenders are studied for how they elicit preferences and present options [8, 13], LLM-based ones with simulated users [5], and shopping agents in simulated stores [12] or against expert results [4]. Work on generative engine optimization asks how products get named in answers [1, 2]. None of these follows the shopper out of the conversation.
Studies of what happens next see one side of the traffic. Kaiser and Schulze measure retailer sessions that carry a ChatGPT referral [9]; Pew finds that people click search results less often when an AI summary appears [3]; usage data from one travel platform show chat and search interleaved in the same purchase journey [11]; and a controlled experiment compares searching with chatting [10]. Panel studies link assistant use to later brand visits [6] and to the order of searching and asking [7]. We go to the product level and follow the same people across sites, from the answer to the product page, with or without a referral.
Data and Measurement
Panel and answers. An opt-in consumer panel in two English-speaking markets records the web pages people visit and, for the same people, their conversations with ChatGPT, Gemini, Claude, Perplexity and Grok.¹ The main period runs from July to October 2026, and February to June 2026 is a held-out replication: its analysis plan was committed before any of its data were pulled, and it reuses the same code. For each person and each of twenty large retailers, we take the first answer that names the store. A language-model judge sorts the answers: an answer names a specific product there, like the Contrail Run 5, suggests the store for the person’s need, or only mentions it. The first two are recommending answers. The main group keeps answers in which the assistant, not the person, first named the store, for people who had not been to that store in the past week.
Comparisons. Every shopper is measured against themselves. The 24 hours after an answer are set against the same person’s visits to that store 14, 21 and 28 days earlier (the earlier-week baseline). For the product match, each answer is also paired with someone else’s answer that names the same store, in the same month, for the same kind of request, country and product category; the judge deciding whether a visited page shows the answer’s product is not told whose answer it is. And because people who chat may simply be busy online, we also take matched moments: the same person and store, the same three-hour time of day, weekday or weekend, recent activity and past-week visit status, at least three days away and with no answer naming any of the twenty stores within a day. Around those moments store visits do not rise; they dip by 0.8 points. Paired answer by answer (the 53% of specific-product answers with a matched moment), store visits after a specific-product answer still rise 7.8 points [6.8, 9.0] more than around the moment (a more active subset than the +5.1 in Figure 2a).
Each comparison addresses a different alternative explanation: the earlier weeks account for a person’s habits at a store, the swapped answer for a store’s popular products and the category, and the matched moments hold fixed how busy the person was online and when.
Matching answers to products. An answer names a store when its text matches one of the twenty retailers’ names. For each such answer we keep the person’s last message before it and the parts of the answer around the store’s first two mentions. Both judges are classification tasks run on one underlying model, which picks among written options, each with a criterion, instead of writing free text. The answer-type judge reads the request and the answer excerpts. The product-match judge reads the same excerpts and the product pages the person then visits at that store, each described by its page title and web address. It counts a match when a page shows a product the answer named, allowing a different size, color or pack or another listing of the same item. A page of the same kind of product, or a page it cannot judge, is not a match. The steps need only answer text and later page views, so they carry over to other assistants and panels.
Label validation. Two other frontier models re-label samples from every judge; where they agree with each other, they confirm 96% of the product-match judge’s matches after an answer and 64% in the earlier-week baseline, and agree with the answer-type judge on 83% of answers. A reader blind to the judge’s labels and to the time window audited 160 product-match items, 7 of which it could not judge: the judge’s matches were confirmed in 93% [?, ?] of 90 after an answer and 54% [?, ?] of 39 in the earlier-week baseline, where true matches are rare, and called 4% of 24 pages the judge had rejected a match; adjusting for these precisions (missed matches are not corrected) leaves the named-product rate at 1.9 points against 2.0 unadjusted. On a class-balanced sample of 102 answers, the same reader agreed with the answer-type judge 75% [?, ?] of the time. Its errors run one way: the reader confirmed every mention, but called 29% of the judge’s recommending answers mentions, which would shrink the recommending rises, not inflate them.
Findings
People go to the product the assistant named
Take the days on which a main-group shopper opened a product page at the store a specific-product answer named: on 44% of them, a page shows a product from the shopper’s own answer. Swap in someone else’s matched answer and the share drops to 1.5% (Figure 2a). The result replicates in a separate five-month period: 42% against 1.2%.
Counted over all such answers, whether or not a product page followed, 2.07% lead to a visit to the named product within a day, against 0.02% in the earlier-week baseline: a rise of 2.0 points [1.7, 2.4], and 2.2 [1.9, 2.6] in the earlier period. Every step of the journey rises over the baseline (Figure 2a, middle), and both kinds of recommendation move store visits far more than a passing mention does. The more specific the step, the larger the ratio to the baseline: store visits rise 2.8 times, searches for the product’s name about 24 times, and visits to the named product’s page about a hundred times, from 0.02 to 2.07 per 100 answers. Recommending goes with far more store visits than mentioning: a store suggestion adds 2.7 points [2.0, 3.5] more than a bare mention, which adds under one visit per 100 answers. Only 8% of matched products had been named by the shopper earlier in the conversation (a separate judge reads their earlier messages), and without them the rise barely moves (1.9 points [1.5, 2.2]).
Four checks leave this pattern in place: someone else’s answer, the earlier period, a blind human reader, and moments with no answer. Around those moments, with pages scored against the same answer, visits to its product do not rise: 0.04 against 0.05 per 100 answers (earlier period 0.00 against 0.03). For the 53% of specific-product answers with a moment (58% earlier), the rise after the answer is 3.1 points [2.5, 3.6] larger than around it, paired answer by answer (earlier period 3.0 [2.5, 3.4]). That subset is more active than the overall +2.0 (+2.2 earlier). Nor is it driven by a few heavy users or one big store: the named-product rise is 2.3 points with each person counted once, 2.4 without the 5% of shoppers with the most answers, 2.2 without the largest retailer, and 2.1 and 2.0 in the two halves of the period.
We call this rate product follow-through, defined in the box. It can be reported separately for each assistant: follow-through rises 2.2 points [1.8, 2.7] after ChatGPT answers and 1.0 [0.5, 1.6] after Gemini answers. It also differs with what the answer contains (Figure 2a, bottom). Answers that quote a price near the store’s name are followed by a larger rise than those that do not (+0.93 points [0.19, 1.63]), while naming one product or several, or linking to the store, makes no detectable difference in named-product visits. These are comparisons across answers, not experiments, but they point builders at the content of the answer.
The store rarely sees the assistant
Our shopper did not click the assistant’s link, and most visits show no sign of one. A visit comes through the link when it carries an AI referrer or an AI marker in its address, and follows a web search when it has a search referral, a search ad or a search in the 30 Product follow-through. The share of answers naming a specific product that are followed, within 24 hours, by a visit to that product’s page at the named store.
Counted over: every such answer, whether or not the person visits the store.
A match: the visited page shows a product the answer named; size, color and pack variants count.
Compared with: the same person’s visits to that store 14, 21 and 28 days earlier.
Needs: the answer text and the same person’s later page views; no referral link, provider logs or integration with the assistant.
Worked example. The Contrail Run 5 answer counts once in the denominator. The shopper’s visit to the watch’s page a few minutes later counts as a match. Across main-group answers naming a specific product, 2.07% follow through, against 0.02% in earlier weeks. On the narrower set of days with a product-page visit at the store, 44% show the answer’s product, against 1.5% for someone else’s answer.
minutes before; the remaining visits are typed or direct, or come from another site or a reopened tab, checked in that order. Of 100 visits after a recommending answer, 11 come through the link (2 in their usual visits); the rest show no sign of it, and 54 follow a web search (Figure 3), as 55% of their usual visits do. Search visits double, from 56 to 110 per 1,000 answers, so the extra traffic looks like ordinary search traffic. Counting only the extra visits on each route (visits in the day after, minus the same person’s earlier-week average) changes little: 81% still show no sign of the link.

Figure 3. Most visits after a recommending answer arrive by web search; only a thin slice comes through the assistant’s link, and the link is used more by shoppers who had not visited the store in the past week. Raw shares of visits, February to October; the bottom block counts first arrivals and extra visits above each person’s earlier weeks.
The link is used most by shoppers who had not been to the store lately. Among first arrivals at a store, people who had not visited it in the past week came through the link 27% of the time, against 12% for recent visitors (+14.7 points [11.9, 17.6]). After a specific-product answer, the link carries 42.5% of the extra visits from shoppers with no visit in the past week and 16.8% from recent visitors (Figure 3).
Then they keep browsing
Once on the watch’s page, our shopper looked at another watch. The named product is often where the visit starts: after specific-product answers, a named product is the first page of the visit 34% of the time, other products 18%, and most of those landings follow a web search. Deeper in the visit, the store’s own search and category pages lead to the named product about as often as to other products (56% against 51% of later product views, at the three stores that record where a view came from). The answer shapes which product people look at; the store’s own pages guide the rest of the visit. Against the same shopper’s other visits, from every route and with link clicks left out, these visits open on a product page 39.3% of the time against 36.7% (Figure 2b; earlier period +1.6 [−0.4, 3.4]). We compare against all their visits because the route strongly affects where a visit opens: averaged within shoppers, typed visits rarely open on a product page (20.5%; 25% as a raw share in Figure 3), generic searches often do (55.2%).
Shoppers keep browsing. They view two or more products in 31.4% of visits against 26.2% on other visits (+5.2 points [3.0, 7.4]; earlier period +4.3 [2.8, 6.1]), the share of visits with at least two products the answer did not name shows no detectable change (0.1 points [−2.4, 2.5]), and at the three stores that record it they view the store’s own recommendations about as often as usual (14.5% against 13.4%; a little more in the earlier period, +2.0 [0.5, 3.4]). Counted over all specific-product answers, including those with no store visit, a view of a product the answer did not name follows 1.4 answers in 100 in the earlier weeks and between 2.6 and 3.5 after the answer (at least +1.1 points [0.8, 1.5]; earlier period at least +1.6 [1.2, 1.9]; Figure 2a), so the higher store-visit rate comes with more browsing of other products, not only the named one. The two figures are bounds because the match label says whether a day’s product pages include the named product, not how many do: the lower bound counts days with product pages and no named product, and the upper bound adds days with two or more products of which one is named.
Discussion
Referral reports miss these visits. An answer names a product, and within minutes the shopper goes to look at it. Most show no What a retailer can do. A referral report records the route into the store; the recommendation may have happened earlier.
89% show no sign of the assistant’s link. Report assistant referrals next to product-page arrivals from search and direct traffic, and track product-name queries in search-console data. Link-only reporting captures about one in nine visits after an answer, and about one in five of the extra visits.
Searches name the product. Make product pages easy to find by the product names assistants recommend, in titles, addresses and site search.
Recent visitors rarely click. Read the assistant’s referral line as a signal about shoppers absent in the past week; its influence on recent visitors arrives by other routes.
Shoppers keep browsing. Keep related products and recommendations within reach of the product page.
sign of the assistant’s link; many search for the product instead or go straight to the store, so the visit is logged as ordinary search or direct traffic. And because shoppers who had not visited the store in the past week use the link far more often than recent visitors, referral reports see the assistant best when it brings someone back, and least for recent visitors.
Back to the running watch. Trailmere Sports records a search visit to the Contrail Run 5; the journey began earlier, with the assistant’s answer. That gap between where a decision is made and where it is recorded is what referral reports cannot see.
Evaluating assistants and agents. Product follow-through gives a behavioral measure of a recommendation, without referral links or provider-side logs. It can be reported separately for each assistant, as above, and could extend to shopping agents that act on a person’s behalf, measured on purchases when the agent buys.
For builders of shopping agents. People go to look at what the answer named, so the product an agent names, and what it says about it, shape which product people go to see. And the answer is not the end of the journey: shoppers still browse other products at the store, so an agent that hands over a clear product name, one a search engine and a store both recognize, makes that step easy.
Open questions. Telling whether these visits lead to purchases requires order data from the named stores. The causal test is to randomize which of several suitable products an assistant names and measure the visits that follow. And when agents buy on a person’s behalf, the shopper may never open the store, so follow-through will have to be measured on the agent’s purchases rather than on page views.
Limitations. This is an observational study. We read the named-product and store-visit rises as effects of the answer under one assumption: without the answer, a person’s visits would have followed their earlier weeks and their equally active moments with no answer. Intent formed just before asking could break it; the at-store and route results are descriptive. We see visits, not purchases. We chose the main group after first results; its store-visit rise (+5.1 points) is smaller than over all answers in which the assistant named the store (+10.5). The study covers two markets, twenty retailers and February to October 2026.
Ethics Statement
This is secondary analysis of opt-in, de-identified browsing, search and assistant-conversation data collected under the panel provider’s terms; no participant was contacted and no new data were collected. Conversation text and visited pages were processed by third-party models as those terms allow. Results are reported only in aggregate, and the linkage of conversations to browsing is used only for aggregate measurement, never to profile individuals. AI coding assistants helped write the analysis and figure code and edit the text. The authors specified the design, verified every estimate and citation, and take full responsibility for all claims.
References
- [1]Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan, and Ameet Deshpande. 2024. GEO: Generative Engine Optimization. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, New York, NY, USA, 5–16. https://doi.org/10.1145/3637528.3671900
- [2]Puneet S. Bagga, Vivek F. Farias, Tamar Korkotashvili, Tianyi Peng, and Yuhang Wu. 2025. E-GEO: A Testbed for Generative Engine Optimization in E-Commerce. arXiv:2511.20867
- [3]Athena Chepakis and Anna Lieb. 2025. Google Users Are Less Likely to Click on Links When an AI Summary Appears in the Results. Pew Research Center. https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/ Short read, July 22, 2025.
- [4]Shuxian Fan, Seonwoo Min, Youna Hu, Botao Xia, Jayakrishnan Unnikrishnan, Rowan Musselmann, Yifan Gao, Qingyu Yin, Priyanka Nigam, and Bing Yin. 2026. Shopping Reasoning Bench: An Expert-Authored Benchmark for Multi-Turn Conversational Shopping Assistants. arXiv:2606.12608arxiv.org/abs/2606.12608
- [5]Luke Friedman, Sameer Ahuja, David Allen, Zhenning Tan, Hakim Sidahmed, Changbo Long, Jun Xie, Gabriel Schubiner, Ajay Patel, Harsh Lara, Brian Chu, Zexi Chen, and Manoj Tiwari. 2023. Leveraging Large Language Models in Conversational Recommender Systems. arXiv:2305.07961arxiv.org/abs/2305.07961
- [6]Michael Iannelli and Alan Ai. 2026. From Prompt to Purchase: How AI Brand Recommendations Move Consumers on the Open Web. arXiv:2606.10907 Pre-sented at the 5th Workshop on End-to-End Customer Journey Optimization (KDD 2026).arxiv.org/abs/2606.10907
- [7]Michael Iannelli and Alan Ai. 2026. The New Shape of Search: How Conversational AI Recomposes Information Seeking. arXiv:2607.04282arxiv.org/abs/2607.04282
- [8]Dietmar Jannach, Athasham Noor, Wenling Cai, and Li Chen. 2021. A Survey on Conversational Recommender Systems. Comput. Surveys 54, 5, Article 105 (2021), 36 pages. https://doi.org/10.1145/3453154
- [9]Maximilian Kaiser and Christian Schulze. 2026. Frontiers: ChatGPT Referrals to E-Commerce Websites: How Do LLMs Compare Against Traditional Referral Channels? Marketing Science 45, 4 (2026), 699–715. https://doi.org/10.1287/mksc.2025.0489
- [10]Ruiyun Xu, Yue Feng, and Hailiang Chen. 2023. ChatGPT Vs. Google: A Comparative Study of Search Performance and User Experience. arXiv:2307.01135arxiv.org/abs/2307.01135
- [11]Se Yan, Han Zhong, Zemin Zhong, and Wenyu Zhou. 2026. Shopping with a Platform AI Assistant: Who Adopts, When in the Journey, and What For. arXiv:2603.24947arxiv.org/abs/2603.24947
- [12]Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents. In Advances in Neural Information Processing Systems 35. Neural Information Processing Systems Foundation, Inc., San Diego, CA, USA, 20744–20757. https://doi.org/10.5220/068431-1508 arXiv:2207.01206
- [13]Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W. Bruce Croft. 2018. Towards Conversational Search and Recommendation: System Ask, User Respond. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM ’18). Association for Computing Machinery, New York, NY, USA, 177–186. https://doi.org/10.1145/3269206.3271776

