At a glance
- Source
- Noy and Zhang (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. doi:10.1126/science.adh2586
- How it was done
- An online experiment, registered in advance. Half of the participants were chosen by chance to use ChatGPT for a paid writing task typical of their job. Randomised experiment. People were assigned by chance to work with or without the tool, and the two groups were compared.
- Who took part
- 453 college-educated professionals.
- Where
- Online; participants worked on writing tasks matched to their occupation.
“Our results show that ChatGPT substantially raised productivity: The average time taken decreased by 40% and output quality rose by 18%.”
What it found
- Time taken fell by 40% on average.
- The quality of the output rose by 18%.
- Differences between workers shrank. Weaker performers moved closer to stronger ones.
- People who used ChatGPT in the experiment were twice as likely to report using it in their real job two weeks later, and 1.6 times as likely two months later.
When the result holds
The tasks were mid-level professional writing, matched to each participant’s occupation and paid by result. This is work that AI writing tools handle well.
Limits
- The tasks were short and done once, outside the participants’ real jobs.
- The experiment did not test accuracy on tasks that need facts the AI does not have.
Practices that use this study
This study supports:
- Measure time saved per task against a baseline Record how long a task takes and how good the result is before you add AI. Without that starting figure you cannot tell a real gain from a hopeful one. Do first · Effort: low (days, one person)
- Give AI assistance to newer staff first When you roll out an AI assistant for a well-defined task, start with your less experienced people. Studies find they gain the most. Do next · Effort: low (days, one person)