Applying high performance computing to early fusion video action recognition
Author(s)Hutchinson, Matthew S.,M. Eng.Massachusetts Institute of Technology.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Charles E. Leiserson and Vijay Gadepally.
MetadataShow full item record
Over the past few years, there has been significant interest in video action recognition systems and models. However, direct comparison of accuracy and computational performance results remain clouded by differing training environments, hardware specifications, hyperparameters, pipelines, and inference methods. Additionally, the literature demonstrates a fixedness on late fusion approaches to audio-video multimodal problems. This project provides a side-by-side comparison of several 2-Dimensional Convolutional Neural Network (2D-CNN) video action recognition approaches and investigates the effectiveness and efficiency of new audio-video early fusion, slicing, and sampling methods. Model accuracy is evaluated using standard Top-1 and Top-5 metrics in addition to novel p-ROC metrics, and this project demonstrates the usefulness of the latter. Computational performance is measured via total training time and training time per epoch on a variety of high-performance computing (HPC) training configurations.
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, May, 2020Cataloged from the official PDF of thesis.Includes bibliographical references (pages 91-99).
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology
Electrical Engineering and Computer Science.