At a glance
- Source
- Fang, Yuan, Zhang, Donati and Sarvary (2025). Generative AI and Sales Productivity: Field Experiments in Online Retail. arXiv 2510.12049, version 6 (29 June 2026). arxiv.org · Not peer-reviewed: other researchers have not yet checked it for a journal.
- How it was done
- The platform built generative AI into seven of its workflows. In each, shoppers or products were assigned by chance to the AI version or the usual one, and the researchers compared sales. Randomised experiment. People were assigned by chance to work with or without the tool, and the two groups were compared.
- Who took part
- Millions of shoppers and products, in seven experiments run between September 2023 and June 2024.
- Where
- One of the world’s largest cross-border online retail platforms, not named in the paper, serving third-party sellers.
“these effects are not associated with worse post-purchase outcomes, as product return rates and customer ratings do not deteriorate.”
What it found
- Pre-sale chatbot: +16.3% sales. Shoppers in the comparison group got a message that live help was not available. The platform kept its human agents for questions after a sale. Many smaller sellers are in the same position: they cannot answer questions live, in several languages.
- AI with a person behind it. AI that could hand a shopper to a human agent raised sales by 25% against no service. Against a service run only by human agents, it raised sales by 11.5%. That last result is at a weaker level of statistical confidence.
- Search: +2.9% sales. AI refined what shoppers typed into the search box.
- Product descriptions: +2.1% sales overall. For products with fewer than 50 words of description, AI-written text raised sales by 6.5%. Products that were already well described gained nothing.
- Marketing messages: +1.6%, not clearly beyond chance.
- AI-written Google ad titles: −4.5%, also not clearly beyond chance.
- After the sale. Return rates and customer ratings did not get worse.
Across the four tools that raised sales, the authors put the extra value at about $5 per shopper per year. Most of the gain came from more shoppers buying, not from larger baskets. Newer and less active shoppers gained most.
When the result holds
AI added most where the platform had been doing little or nothing: unanswered questions, thin descriptions. Where the existing service was already good, the gain was small or zero.
The sellers did nothing. The platform switched the AI on for them. For small sellers, the AI that matters most may be the platform’s, which they do not choose or control.
Results for smaller and newer sellers were often larger than for others, but the authors say these differences are imprecisely estimated.
Limits
- Two authors were consultants to the platform during the project, and one was its employee. Results for two further experiments come from the platform’s own data team, without data the authors could check.
- The effects are short-term. One platform.
- A working paper, not peer-reviewed.
Practices that use this study
This study supports:
- Measure time saved per task against a baseline Record how long a task takes and how good the result is before you add AI. Without that starting figure you cannot tell a real gain from a hopeful one. Do first · Effort: low (days, one person)
- Start with one costly process, not a company-wide rollout Pick one process that visibly costs time or money. Apply AI there, with a person checking the output, and expand only when the numbers move. Do first · Effort: medium (weeks, a few people)
- Start with ready-made AI in software you already use, before building your own Your office, accounting, support and sales software probably has AI features now. Try those on one task, with a few licences and a measured result, before you pay anyone to build something custom. Do first · Effort: low (days, one person)
- For each kind of task, test whether a person with AI beats either one alone Do not assume that adding a person to AI, or AI to a person, gives the best result. Test it for each type of task before you settle the process. Do next · Effort: medium (weeks, a few people)