At a glance
- Company
- UK Department for Business and TradeNational government department
- Size
- Enterprise (1,000+ employees) The 1,000 licences went to part of the department’s UK-based staff.
- Where and when
- United Kingdom. October – December 2024, published August 2025.
- Who produced the figures
- Official recordThe figures come from the department’s own evaluation team. The main source was a one-week diary that 32% of licence holders filled in. A small number of timed tasks compared users with non-users.
What they did
The department had 1,000 licences for Microsoft 365 Copilot from October to December 2024. Copilot is an AI assistant that sits inside Outlook, Teams, Word, Excel and PowerPoint. About 70% of the licences went to volunteers and about 30% to staff picked at random.
The department’s evaluation team then measured what happened. Licence holders kept a diary for one week. The team interviewed users and non-users. It also sat a small number of users and non-users down with the same tasks, timed them, and had a second analyst score the results without knowing who had used Copilot.
The numbers
From the diaries (300 analysed, a 32% response rate):
- 72% of respondents were satisfied or very satisfied.
- Writing tasks saved the most time: summarising, reviewing and drafting.
- Scheduling and making images took longer with Copilot than without.
- 22% of respondents had spotted a hallucination, meaning content the AI had made up. 43% had not, and 11% were not sure.
From the timed tasks:
| Task | With Copilot | Without | Quality with Copilot |
|---|---|---|---|
| Summarising reports | 12 min 37 s | 41 min 34 s | Better |
| PowerPoint slides | 10 min 47 s | 18 min 30 s | Worse |
| Data analysis in Excel | 25 min 01 s | 20 min 33 s | Worse |
What the evaluation concluded
- Time saved did not show up as productivity. The report says it “did not find evidence that time savings have led to improved productivity”. Colleagues without a licence had not noticed any change in the output of those who had one. The report adds that measuring productivity was not a main aim.
- Some time was added. Tasks took longer when Copilot produced poor output, or when people did extra work only because they had the tool.
- Training mattered, and the kind mattered. Staff who taught themselves were more satisfied than those who went to the department’s formal sessions.
- Checking was uneven. Users did not check Copilot’s output in a consistent way.
- Value for money was left open. The report says a separate assessment was still under way.
The wider trial
This pilot was part of a UK government trial with about 20,000 staff across departments. There, users said they saved 26 minutes a day on average. That figure is self-reported. It came from one survey question with answers given in ranges, and the report says it was not possible to identify how the time saved was spent.
A separate evaluation at HMRC, the UK tax authority, gave 3,000 licences to staff picked at random. It found about 60 minutes a week, after HMRC cut the self-reported figures by about 20% to allow for bias. That figure is self-reported too. See the HMRC case.
What the case shows
- Trying the AI inside software you already use is a cheap way to find out. The department learned which tasks gained and which lost within three months.
- Satisfaction is not productivity. Most users liked the tool. The measured tasks told a more mixed story.
- Results differ by task. Summarising was three times faster and better. Spreadsheet analysis was slower and worse.
- What users report and what a timed test shows can differ. The diaries reported time saved on data analysis. The timed test showed time lost.
What this says about the practices
Practices the company applied
- Start with ready-made AI in software you already use, before building your own The department tried an assistant built into the email, document and meeting software its staff already used, with 1,000 licences for three months. Do first · Effort: low (days, one person)
- Measure time saved per task against a baseline The evaluators did not rely on what users said. They timed the same tasks for users and non-users and had the results scored blind. Do first · Effort: low (days, one person)
- Train staff on the tools you actually chose Training was a significant factor in satisfaction. Staff who taught themselves were more satisfied than those who attended the department’s formal sessions. Do first · Effort: medium (weeks, a few people)
Sources
- UK Department for Business and Trade (August 2025). The evaluation of the M365 Copilot pilot in the Department for Business and Trade. Report. assets.publishing.service.gov.uk
- UK Government Digital Service (2 June 2025). Microsoft 365 Copilot experiment: cross-government findings report. The wider trial this pilot was part of. gov.uk
- HM Revenue & Customs (9 July 2026). Evaluating the impact of Microsoft Copilot in HMRC. Phase III trial evaluation report. gov.uk