Applications of AI on Resource-ConstrainedHardware with a focus on Anomaly Detection

Abstract

This thesis addresses the challenges of improving the performance of AI models on resource-constrained microcontrollers (MCUs). As the complexity of modern models continues to grow and the demand for smaller mobile devices increases, optimizing model latency, memory usage, and accuracy on tiny devices remains a persistent problem. This thesis makes contributions to the field by (1) benchmarking common AI inference engines to identify trade-offs between them, (2) developing a framework that can assist neural-architecture searches to discover more efficient models, (3) proposing model conversion techniques that enable online-learning on MCUs, resulting in improved real-world accuracy, (4) creating a novel visual anomaly detector for MCUs, and (5) collecting a new dataset for anomaly detection benchmarks. The task of visual anomaly detection is to discern between known "Good" objects and objects that are slightly damaged or have imperfects. Being able to spot defective parts has huge applications in industrial and manufacturing settings. The proposed anomaly detector, MCU-PatchCore, is based on PatchCore – a state-of-the-art anomaly detector. MCU-PatchCore achieves a mean accuracy of 86% on the widely used MVTec AD dataset, which contains images of screws, cloth, glass bottles, etc., and their anomalous chipped, torn, cracked, etc., counterparts. While MCU-PatchCore’s accuracy is not as competitive as the GPU-based PatchCore detector, it only requires 200KB of RAM and less than 1MB of storage to run. Additionally, MCU-PatchCore outperforms a few other anomaly detectors in the literature, and shows promising potential for future improvement.