At a glance
- Source
- Pinski, Adam and Benlian (2023). AI Knowledge: Improving AI Delegation through Human Enablement. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3544548.3580794
- How it was done
- Participants were split into two groups. One group was first taught how AI works and where human judgement goes wrong. Both then chose which tasks to hand to an AI. Randomised experiment. People were assigned by chance to work with or without the tool, and the two groups were compared.
- Who took part
- 111 participants in two groups.
- Where
- A controlled experiment run by a university research team in Germany.
“We find that AI knowledge-enabled humans align their delegation decisions more closely with their assessment of how suitable a task is for humans or AI.”
What it found
- People often hand tasks to AI badly: they keep tasks the AI would do better and give away tasks they would do better.
- People who first learned how AI works matched their choices more closely to how well each task suited a human or an AI.
- Choices matched in this way led to better results on the task.
- The same knowledge lowered people’s stated intention to use AI later.
When the result holds
The participants could choose, task by task, whether to do the work themselves or hand it over. The finding applies to settings where staff have that choice.
Limits
- A single session, not real work over time.
- The drop in willingness to use AI was a stated intention. The study did not track what people later did.
Practices that use this study
This study supports:
- Train staff on the tools you actually chose Show every user what your chosen AI tool is, where it goes wrong, and what they must check. Use your own tasks and your own rules, give people time to practise, and include temporary staff. Do first · Effort: medium (weeks, a few people)
- For each kind of task, test whether a person with AI beats either one alone Do not assume that adding a person to AI, or AI to a person, gives the best result. Test it for each type of task before you settle the process. Do next · Effort: medium (weeks, a few people)
- Test AI knowledge instead of asking staff to rate their own People judge their own skills poorly. To find out what your staff know about AI, use a short knowledge test or a second opinion, not only a self-rating. Do next · Effort: medium (weeks, a few people)
Assessments linked to this study
- How far do staff trust one named AI system? Trust in Automation Scale, short form (S-TIAS) The people who use that system · 3 questions, or 12 in the long form Free to use and adapt
- What do staff know about AI? Scale of Artificial Intelligence Literacy for All (SAIL4ALL) Staff, each taking a test · 56 questions in 4 parts. Each part can be used on its own Free to use and adapt