Scalable Embedded Tiny Machine Learning (SETML): A General Framework for Embedded Distributed Inference

Abstract

The growth of machine learning applications has increased the necessity of lightweight, energyefficient solutions for resource-constrained devices such as the STM32C011F6 microcontroller. However, such devices struggle with supporting larger models even after miniaturization techniques such as quantization and pruning. To facilitate machine learning inference on such devices, this work introduces Scalable Embedded Tiny Machine Learning (SETML), a general framework for distributed machine learning inference on microcontrollers. Furthermore, the framework is designed to be compatible with sensor-based applications that can take advantage of small hardware, such as gesture recognition, by testing binary size constraints with an accelerometer and its supporting library. This work evaluates the latency, power consumption, and cost trade-offs of using multiple small and efficient devices versus a larger device. The STM32C011F6 microcontroller is used as the primary hardware in the tested device network, while evaluation of the system is done in comparison with a device using a similar core processing element, the Seeeeduino XIAO SAMD21.