Using transfer learning, spectrogram audio classification, and MIT app inventor to facilitate machine learning understanding

Abstract

Recent advancements in deep learning have brought machine learning and its many applications to the forefront of our everyday lives. As technology has become more and more integrated into our educational curriculum, researchers have focused on creating deep learning tools that allow students to interact with machine learning in a way that incites curiosity and teaches important concepts. My research contribution focuses on applying transfer learning and spectrogram audio classification methods to teach basic machine learning concepts to students. I introduce the Personal Audio Classifier (PAC), a web application that allows users to train and test custom audio classification models that can classify 1-2 second sound bites recorded by the user. Alongside this in-browser machine learning tool, I provide a set of best practices for spectrogram audio conversion in machine learning applications. I also contribute a custom App Inventor extension that allows users to use the output of the web interface to create App Inventor applications that rely on their trained custom audio classification model. Finally, I provide a high school workshop curriculum based on PAC and the App Inventor extension and detail the results of running workshops with three high school classes at the Boston Latin Academy. My results show that high school students with no prior machine learning knowledge are able to grasp important technical concepts related to machine learning and its applications, as well as build and explore custom machine learning models in the browser through the exploration of a hands-on curriculum based around PAC.