Multiple mice tracking using Microsoft Kinect

Abstract

Mouse tracking is integral to any attempt to automate mouse behavioral analysis in neuroscience. Systems that rely on vision have successfully tracked a single mouse in one cage[10], but when attempting to track multiple mice, video-based systems often struggle when the mice interact physically. In this thesis, I develop a novel vision-based tracking system that addresses the challenge of tracking multiple deformable mice with identical appearance, especially during complex occlusions. The system integrates both image and depth modalities to identify the boundary of two occluding mice, and then performs pose estimation to locate nose and tail locations of each mouse. Detailed performance evaluation shows that the system is robust and reliable, with low rate of identity swap after each occlusion event and accurate pose estimation during occlusion. To evaluate the tracking system, I introduce a dataset containing two 30-minute videos recorded with Microsoft's Kinect from the top view. Each video records the social reciprocal experiment of a pair of mice. I also explore applying the new tracking system to automated social behavior analysis, by detecting social interactions defined with position- and orientation-based features from tracking data. The preliminary results enable us to characterize lowered social activity of the Shank3 knockout mouse, and demonstrate the potential of this system for quantitaive study of mice social behavior.