Geometric deep learning enables 3D kinematic profiling across species and environments
Name
nihms-1743737.pdf
Description
Accepted version
Size
2.63 MB
Format
Unknown
Checksum (MD5)
0d2b11b6f68ffef7af3a182f9f2e350d
Author(s)
Severson, Kyle S
Wang, Fan
Date Issued
May 2021
Journal
Nature Methods
Publisher
Springer Science and Business Media LLC
Citation
Dunn, Timothy W, Marshall, Jesse D, Severson, Kyle S, Aldarondo, Diego E, Hildebrand, David GC et al. 2021. "Geometric deep learning enables 3D kinematic profiling across species and environments." Nature Methods, 18 (5).
Version
Final published version
Abstract
Comprehensive descriptions of animal behavior require precise three-dimensional (3D) measurements of whole-body movements. Although two-dimensional approaches can track visible landmarks in restrictive environments, performance drops in freely moving animals, due to occlusions and appearance changes. Therefore, we designed DANNCE to robustly track anatomical landmarks in 3D across species and behaviors. DANNCE uses projective geometry to construct inputs to a convolutional neural network that leverages learned 3D geometric reasoning. We trained and benchmarked DANNCE using a dataset of nearly seven million frames that relates color videos and rodent 3D poses. In rats and mice, DANNCE robustly tracked dozens of landmarks on the head, trunk, and limbs of freely moving animals in naturalistic settings. We extended DANNCE to datasets from rat pups, marmosets, and chickadees, and demonstrate quantitative profiling of behavioral lineage during development.
MIT Department
McGovern Institute for Brain Research at MIT
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1038/S41592-021-01106-6
