Action Recognition by Hierarchical Sequence Summarization
Author(s)Song, Yale; Morency, Louis-Philippe; Davis, Randall
MetadataShow full item record
Recent progress has shown that learning from hierarchical feature representations leads to improvements in various computer vision tasks. Motivated by the observation that human activity data contains information at various temporal resolutions, we present a hierarchical sequence summarization approach for action recognition that learns multiple layers of discriminative feature representations at different temporal granularities. We build up a hierarchy dynamically and recursively by alternating sequence learning and sequence summarization. For sequence learning we use CRFs with latent variables to learn hidden spatio-temporal dynamics, for sequence summarization we group observations that have similar semantic meaning in the latent space. For each layer we learn an abstract feature representation through non-linear gate functions. This procedure is repeated to obtain a hierarchical sequence summary representation. We develop an efficient learning method to train our model and show that its complexity grows sub linearly with the size of the hierarchy. Experimental results show the effectiveness of our approach, achieving the best published results on the Arm Gesture and Canal9 datasets.
DepartmentMassachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory; Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition
Song, Yale, Louis-Philippe Morency, and Randall Davis. “Action Recognition by Hierarchical Sequence Summarization.” 2013 IEEE Conference on Computer Vision and Pattern Recognition (n.d.).
Author's final manuscript