Developing and Validating the Artificial Intelligence Literacy Concept Inventory: an Instrument to Assess Artificial Intelligence Literacy among Middle School Students
Name
40593_2024_398_ReferencePDF.pdf
Size
4.35 MB
Format
Adobe PDF
Checksum (MD5)
0852f935729dd828afca8b6f7b366d02
Author(s)
Zhang, Helen
Perry, Anthony
Lee, Irene
Date Issued
May 5, 2024
Journal
International Journal of Artificial Intelligence in Education
Publisher
Springer New York
Citation
Zhang, H., Perry, A. & Lee, I. Developing and Validating the Artificial Intelligence Literacy Concept Inventory: an Instrument to Assess Artificial Intelligence Literacy among Middle School Students. Int J Artif Intell Educ 35, 398–438 (2025).
Version
Author's final manuscript
Abstract
The rapid expansion of Artificial Intelligence (AI) in our society makes it urgent and necessary to develop young students’ AI literacy so that they can become informed citizens and critical consumers of AI technology. Over the past decade many efforts have focused on developing curricular materials that make AI concepts accessible and engaging to young learners; and yet, limited research investigated how to assess learners’ AI literacy, which is critically important to inform the teaching and learning of AI. This paper addresses this issue by reporting the development and validation findings of the AI Literacy Concept Inventory Assessment (AI-CI), a set of multiple-choice questions designed to assess understanding of AI literacy concepts among middle school students. The AI-CI consists of 20 multiple choice questions examining student understanding of four topics: AI general concepts, logic systems, machine learning general concepts, and supervised learning. The content validity of AI-CI was established through multiple rounds of expert panel reviews with AI educators and experts, observations of student learning of AI, and cognitive validation interviews. The validity of the AI-CI was established with a sample of 981 students and the pre-posttest reliability was established with a sample of 108 middle school students who learned AI through experiencing the Developing AI literacy (DAILy) curriculum. The findings show that the AI-CI is a valid and reliable tool to assess AI literacy at the middle school level.
MIT Department
Massachusetts Institute of Technology. Program in Comparative Media Studies/Writing
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s40593-024-00398-x
