Design Optimizations for Action Recognition Applications

Abstract

There are many problems that exist within the relatively new field of action recognition that make it difficult for the immediate use of existing models for specific applications. My work at the MIT-IBM Watson lab revolved around utilizing existing assets and optimizing performance for achieving action detection in construction-centric videos. There were several pretrained general action recognition models at our disposal, each one with its own limitations. In addition to fine-tuning, there are other computer vision methods and processing techniques that were explored for performance optimization including background subtraction, optical flow, and frame selection algorithms. Though raw accuracy score gains through adopting these modalities were marginal, other improvements like faster training time and the potential for faster prediction time were observed. The process of building this experimental pipeline and the results obtained offered insight into what was feasible and effective with current technology in this unique problem space. This includes proof of concept with regards to a real-time action detection tool as well as potential modifications to optimize the tool's performance in this context.