AI-first company

Study

A review of 16 questionnaires that measure AI knowledge

Of 16 questionnaires that measure AI literacy, 13 ask people to rate their own knowledge. Only 3 test it. None had been checked for whether it works the same way in different cultures.

By
Last reviewed
Type of study
Systematic review
Published
2024
Corrections
Report an error

At a glance

Source
Lintner (2024). A systematic review of AI literacy scales. npj Science of Learning, 9, 50. doi:10.1038/s41539-024-00264-4
How it was done
The author found every published study that tested an AI-literacy questionnaire and scored each questionnaire against a standard checklist for measurement quality (COSMIN). Systematic review. It searches for all published studies on a question, by fixed rules, and sums up what they show.
Who took part
22 studies testing 16 questionnaires, aimed at the general public, students and teachers.
Where
International.

“None of the scales have been tested for cross-cultural validity and measurement error.”

What it found

AI literacy means understanding what AI is, being able to use it, and being able to judge its output.

  • 13 of the 16 questionnaires rely on self-rating. Only 3 measure performance.
  • Overall, the questionnaires were well built: their questions fit together consistently.
  • Few were tested for whether results stay stable over time or change after training.
  • None was tested across cultures or for measurement error.
  • Most gave no guidance on how to read a score.

When the result holds

The review covers questionnaires published up to 2024. None of them was built for employees specifically; they target the general public, students and teachers.

Limits

  • New questionnaires, including knowledge tests, have appeared since.

Practices that use this study

This study supports:

Assessments linked to this study