Video based system monitoring

Abstract

In this work we develop new algorithms for video comparison, for video alignment, and for determining the similarity between entire video clips or detecting similarities between sub-videos. The intent of this work is to develop video-based techniques for autonomous monitoring of systems in industrial, manufacturing, and research environments. We develop an algorithm, Dynamic Time and Space Warping, to determine a model-free similarity between an example and an unknown video. The algorithm optimally shifts space and warps time according to local measures of video similarity. The resulting similarity measure is an optimal path of similarity versus space and time used to optimally align or compare the two video. We demonstrate the applicability of such similarity measures to industrial wear monitoring, failure prediction, and assembly-line feedback control and to non-industrial settings with examples in sports and entertainment. We extend the similarity machinery and introduce a new technique for video event-detection. The local similarity is integrated along the optimal space-time path in order to determine a cumulative similarity.

(cont.) We demonstrate the applicability to content query and surveillance; we identify the temporal and spatial location inside of a large video stream which is similar to a query, or template, video. We explore applications in video classification. We investigate the performance degradation and robustness of the algorithms to noise via distortion of real examples and simulation. We develop techniques to aid engineers in the selection of a video template that is relevant to their monitoring application and locally robust to noise. We explore the structure and computational complexity of the algorithms. We demonstrate that the algorithms are highly-parallelizable and that the fast processing rates necessary for many industrial monitoring applications are achievable.