Trajectory Prediction with Linguistic Representations
Name
2110.09741.pdf
Size
618.3 KB
Format
Adobe PDF
Checksum (MD5)
d984672b061cb4c0077752b4fb625099
Author(s)
Kuo, Yen-Ling
Huang, Xin
Barbu, Andrei
McGill, Stephen G.
Katz, Boris
Leonard, John J.
Rosman, Guy
Date Issued
May 23, 2022
Publisher
IEEE
Citation
Kuo, Yen-Ling, Huang, Xin, Barbu, Andrei, McGill, Stephen G., Katz, Boris et al. 2022. "Trajectory Prediction with Linguistic Representations."
Version
Author's final manuscript
Abstract
Language allows humans to build mental models that interpret what is happening around them resulting in more accurate long-term predictions. We present a novel trajectory prediction model that uses linguistic intermediate representations to forecast trajectories, and is trained using trajectory samples with partially-annotated captions. The model learns the meaning of each of the words without direct per-word supervision. At inference time, it generates a linguistic description of trajectories which captures maneuvers and interactions over an extended time interval. This generated description is used to refine predictions of the trajectories of multiple agents. We train and validate our model on the Argoverse dataset, and demonstrate improved accuracy results in trajectory prediction. In addition, our model is more interpretable: it presents part of its reasoning in plain language as captions, which can aid model development and can aid in building confidence in the model before deploying it.
Description
2022 IEEE International Conference on Robotics and Automation (ICRA) May 23-27, 2022. Philadelphia, PA, USA
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Center for Brains, Minds, and Machines
Terms of Use
Creative Commons Attribution-Noncommercial-ShareAlike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/icra46639.2022.9811928
