What it is
Start with the people who have been in the job for the shortest time. Most rollouts start with volunteers or senior staff. For routine, well-defined work, the evidence points the other way.
It holds where the task has a clear standard and someone checks the output against it. On open-ended work with no check, the weakest gained nothing or lost.
What the evidence shows
In a study of 5,172 customer-support agents at work, an AI assistant raised the number of issues resolved per hour by 15% on average. The gain was not even. Less experienced and lower-skilled agents got faster and better. The most experienced agents got slightly faster, and the quality of their work fell slightly.
In an experiment with 453 professionals, ChatGPT cut the time for writing tasks by 40% and raised quality by 18%. The gap between stronger and weaker performers narrowed.
The likely reason: the AI has absorbed how good work is usually done. A beginner does not know that yet. An expert already does.
The other side. Other studies point the other way, so the evidence is disputed.
- In software, experienced people gained most. A study of 160,097 developers’ code on GitHub estimated that AI raised their output by 3.6% on average. Experienced programmers captured nearly all of that gain.
- Weak business owners did worse with an AI adviser. In a randomised trial with 640 small-business owners in Kenya, half got a GPT-4 business mentor on WhatsApp. On average it made no difference to revenue or profit. Owners who were doing poorly at the start did nearly 10% worse with it. Those doing well may have gained over 15%. The gap came from which advice owners chose to act on.
- Students gained with the tool and lost without it. In a randomised trial with nearly 1,000 secondary-school students in Turkey, unrestricted GPT-4 raised practice scores by 48%. On the exam, taken without AI, the same students scored 17% below those who never had AI. A version limited to hints raised practice scores without that loss.
The studies may both hold. The supporting studies measured output with the tool, on tasks with a clear standard. The others measured open-ended work, output over a longer period, or ability once the tool was gone.
Who is not in these studies. Juniors can gain inside the job while fewer juniors are hired. Whether AI is cutting junior hiring is disputed. See keep the entry-level tasks.
How to do it
- Pick a task that is frequent and has a clear standard for good work.
- Give the assistant first to staff in their first months in the role.
- Keep a comparison group without it for a few weeks, so you can see the difference.
- Ask experienced staff to review what the assistant suggests. Experienced staff will spot where it is wrong.
- Extend it to experienced staff as an option, and watch their quality figures.
What to measure
- Output per hour and quality, split by time in the job.
- How long a new hire takes to reach the team’s usual level, before and after.
- Quality among your most experienced people. It may dip.
- Performance without the tool, on a sample of cases, for new staff after a few months.
Where it fails
- Tasks the AI handles badly. A beginner cannot tell when the AI is wrong. On such tasks, AI help can make results worse. See check by task type.
- Learning the job. If the assistant does the part of the work that teaches the job, new staff may stop developing. A study of surgical trainees shows how that happens when a technology takes over the practice. See keep the entry-level tasks.
- Gains that vanish without the tool. In the school trial, students who practised with an unrestricted chatbot did worse once it was removed. Test new staff without the tool now and then. See check what staff can still do without the tool.
- Experts. Do not expect the same gain from your best people, and do not force the tool on them. In a randomised trial with experienced developers working on their own code, tasks took 19% longer with AI. In a field experiment in Alibaba’s customer service, the quality of the best agents’ work fell with AI help, on customer ratings and on how often customers came back with the same problem. That study is a working paper.
By company size
- Micro and small: you may have one or two new people. Give them the tool, and have the owner or a senior colleague review their work for the first weeks.
- Mid-market and enterprise: roll out by time in the job, and keep a comparison group so the result can be measured.
Company cases
Companies that did this
Sources
- Brynjolfsson, Li and Raymond (2025). Generative AI at Work. The Quarterly Journal of Economics, 140(2), 889–942. What this study did and found
- Noy and Zhang (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. What this study did and found
- Beane (2018). Shadow Learning: Building Robotic Surgical Skill When Approved Means Fail. Administrative Science Quarterly, 64(1), 87–123. Limits or contradicts this practice. What this study did and found
- Daniotti, Wachs, Feng and Neffke (2026). Who is using AI to code? Global diffusion and impact of generative AI. Science, 391(6787), 831–835. Limits or contradicts this practice. What this study did and found
- Otis, Clarke, Delecourt, Holtz and Koning (2026). The Uneven Impact of Generative AI on Entrepreneurial Performance: Evidence from a Field Experiment in Kenya. Management Science (published 10 July 2026). Limits or contradicts this practice. What this study did and found
- Bastani, Bastani, Sungu, Ge, Kabakcı and Mariman (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. Limits or contradicts this practice. What this study did and found
- Ni, Wang, Feng, Lu, Wang and Zhou (2026). Generative AI in Action: Field Experimental Evidence from Alibaba's Customer Service Operations. arXiv 2603.29888. Working paper, not peer-reviewed. arxiv.org
Claims on this topic, checked against these studies:
- “AI made customer support agents 14% more productive, and novices 34% (Stanford and MIT)” Needs correcting
- “Consultants using AI did 12.2% more tasks, 25% faster, with 40% higher quality (BCG and Harvard)” Needs correcting
Related practices
- Keep the entry-level tasks that newcomers learn on Before you automate routine work, check whether it is how juniors learn the job. If it is, keep some of it for them, or replace the learning another way. Do later · Effort: medium (weeks, a few people)
- For each kind of task, test whether a person with AI beats either one alone Do not assume that adding a person to AI, or AI to a person, gives the best result. Test it for each type of task before you settle the process. Do next · Effort: medium (weeks, a few people)
- Measure time saved per task against a baseline Record how long a task takes and how good the result is before you add AI. Without that starting figure you cannot tell a real gain from a hopeful one. Do first · Effort: low (days, one person)