Principles and Guidelines for Evaluating Social Robot Navigation Algorithms
Name
3700599.pdf
Size
33.14 MB
Format
Adobe PDF
Checksum (MD5)
62f6d44a0b222fa299f7795f0b05feee
Author(s)
Francis, Anthony
P?rez-D'Arpino, Claudia
Li, Chengshu
Xia, Fei
Alahi, Alexandre
Alami, Rachid
Bera, Aniket
Biswas, Abhijat
Biswas, Joydeep
Chandra, Rohan
Date Issued
December 27, 2024
Journal
ACM Transactions on Human-Robot Interaction
Publisher
ACM
Citation
Francis, Anthony, P?rez-D'Arpino, Claudia, Li, Chengshu, Xia, Fei, Alahi, Alexandre et al. 2024. "Principles and Guidelines for Evaluating Social Robot Navigation Algorithms." ACM Transactions on Human-Robot Interaction.
Version
Final published version
Abstract
A major challenge to deploying robots widely is navigation in human-populated environments, commonly referred to as social robot navigation. While the field of social navigation has advanced tremendously in recent years, the fair evaluation of algorithms that tackle social navigation remains hard because it involves not just robotic agents moving in static environments but also dynamic human agents and their perceptions of the appropriateness of robot behavior. In contrast, clear, repeatable, and accessible benchmarks have accelerated progress in fields like computer vision, natural language processing and traditional robot navigation by enabling researchers to fairly compare algorithms, revealing limitations of existing solutions and illuminating promising new directions. We believe the same approach can benefit social navigation. In this paper, we pave the road towards common, widely accessible, and repeatable benchmarking criteria to evaluate social robot navigation. Our contributions include (a) a definition of a socially navigating robot as one that respects the principles of safety, comfort, legibility, politeness, social competency, agent understanding, proactivity, and responsiveness to context, (b) guidelines for the use of metrics, development of scenarios, benchmarks, datasets, and simulators to evaluate social navigation, and (c) a design of a social navigation metrics framework to make it easier to compare results from different simulators, robots and datasets.
MIT Department
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3700599
