Experimental evidence on the productivity effects of generative artificial intelligence.
Level 2 - randomized trial
Randomized controlled trial (graded by non-clinical design analogy).
PubMed 37440646 · doi:10.1126/science.adh2586
What was done
In a preregistered online experiment, 453 college-educated professionals were assigned occupation-specific, incentivized midlevel writing tasks. Half of the participants were randomly assigned access to ChatGPT, while the other half completed the tasks without it.
What was found
ChatGPT exposure decreased the average time taken by 40% and increased output quality by 18%. Performance inequality between workers decreased, and participant concern and excitement about AI temporarily rose. Exposed workers were 2 times as likely to report using ChatGPT in their real jobs at 2 weeks and 1.6 times as likely at 2 months post-experiment.
Why it matters
It provides causal experimental evidence that generative AI assistance can improve speed and quality simultaneously on midlevel knowledge tasks while reducing disparities among workers.
Limits
The study tested short-duration, isolated online writing tasks rather than full workplace workflows. Quality scoring methods and metrics are not detailed in the abstract, and long-term effects on skill retention or task accuracy were not measured.
Cited by
- context An MIT study found that individuals with higher competency in a topic utilize generative AI tools in ways that maintain germane cognitive load while accelerating learning.