Improved 3D Markerless Mouse Pose Estimation Using Temporal Semi-supervision
Name
11263_2023_1756_ReferencePDF.pdf
Size
65.16 MB
Format
Adobe PDF
Checksum (MD5)
c3418c350124b33949a7ce4867e36c88
Author(s)
Li, Tianqing
Severson, Kyle S.
Wang, Fan
Dunn, Timothy W.
Date Issued
February 22, 2023
Publisher
Springer US
Citation
Li, Tianqing, Severson, Kyle S., Wang, Fan and Dunn, Timothy W. 2023. "Improved 3D Markerless Mouse Pose Estimation Using Temporal Semi-supervision."
Version
Author's final manuscript
Abstract
Abstract
Three-dimensional markerless pose estimation from multi-view video is emerging as an exciting method for quantifying the behavior of freely moving animals. Nevertheless, scientifically precise 3D animal pose estimation remains challenging, primarily due to a lack of large training and benchmark datasets and the immaturity of algorithms tailored to the demands of animal experiments and body plans. Existing techniques employ fully supervised convolutional neural networks (CNNs) trained to predict body keypoints in individual video frames, but this demands a large collection of labeled training samples to achieve desirable 3D tracking performance. Here, we introduce a semi-supervised learning strategy that incorporates unlabeled video frames via a simple temporal constraint applied during training. In freely moving mice, our new approach improves the current state-of-the-art performance of multi-view volumetric 3D pose estimation and further enhances the temporal stability and skeletal consistency of 3D tracking.
MIT Department
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s11263-023-01756-3
