At a glance
- Source
- Senoner, Schallmoser, Kratzwald, Feuerriegel and Netland (2024). Explainable AI improves task performance in human–AI collaboration. Scientific Reports, 14, 31150. doi:10.1038/s41598-024-82501-9
- How it was done
- Two experiments registered in advance. Participants were assigned by chance to see the AI’s score alone, or the score with a picture marking the areas the AI had reacted to. The AI was the same in both groups. Randomised experiment. People were assigned by chance to work with or without the tool, and the two groups were compared.
- Who took part
- 48 factory workers who made 9,600 assessments of electronic products, and 113 radiologists who made 5,650 assessments of chest X-rays.
- Where
- A Siemens electronics factory, and an online experiment with practising radiologists.
“Our results demonstrate that domain experts make subpar decisions when they are supported by a black-box AI algorithm with opaque predictions.”
What it found
Both experiments used a visual inspection task. An AI gave each image a score from 0 to 100. Half the participants saw only the score. The other half also saw the image with coloured areas marking what the AI had reacted to. Accuracy below gives equal weight to faulty and faultless items.
In the factory:
- Score only: 88.6% accurate. Score and picture: 96.3%. The AI alone reached 95.6%.
- More defects caught. Workers with the picture caught 93.0% of defective products. Workers with the score alone caught 82.0%.
- Fewer right answers thrown away. Workers with the score alone were 3.6 times more likely to overrule the AI when it was right.
- More wrong answers caught. Workers with the picture overruled 96.9% of the AI’s wrong predictions. Workers with the score alone overruled 86.4%.
- 73.1% of workers with the picture did better than the AI alone.
With radiologists:
- Score only: 79.1% accurate. Score and picture: 83.8%. The AI alone reached 82.2%.
- No difference in lesions found. Both groups found 90.4% of the images that showed a lung lesion. The gain came from fewer false alarms.
- Wrong answers were not caught more often. Radiologists with the picture overruled 50.8% of the AI’s wrong predictions. Those with the score alone overruled 57.7%.
When the result holds
The participants were experienced: 13.8 years in the job on average for the factory workers and 13.5 for the radiologists. They were told that the AI could be wrong. The explanation was a picture, which an expert can compare with the image at a glance.
Limits
- The authors name the main limit themselves: one kind of task, visual inspection, and one kind of explanation, a marked-up picture.
- The experiments used a fixed set of images under a time limit: 200 product images in the factory and 50 X-rays for the radiologists.
- One of the authors reports research funding from Siemens.
Practices that use this study
This study supports: