Tiny Trainable Instruments

Abstract

Can we build flexible and reusable multimedia instruments that are trained instead of programmed? How can we build and publish our own personal databases for artistic purposes? What are the new choreographies and techniques that machine learning running on microcontrollers offer for artists and activists?

Tiny Trainable Instruments is a collection of multimedia devices, running machine learning algorithms on microcontrollers, for artistic purposes. It includes techniques for capturing data, building databases, training machine learning models, and deploying on microcontrollers. The software library created for this project allows for the creation of instruments that react to different inputs, including color, gesture, and speech, to control different multimedia outputs, including sound, light, and movement, using machine learning and embedded sensors.

This thesis emphasizes open source software and artificial intelligence ethics, and includes all the steps for creating these bridges between machine learning and media arts, that are respectful of privacy and consent because of their offline and off-the-grid nature.