PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception
Name
CBMM-Memo-123.pdf
Size
3.08 MB
Format
Adobe PDF
Checksum (MD5)
b2729b84aa44c5a4a63558db9e060bb0
Author(s)
Netanyahu, Aviv
Shu, Tianmin
Katz, Boris
Barbu, Andrei
Tenenbaum, Joshua B.
Date Issued
March 19, 2021
Publisher
Center for Brains, Minds and Machines (CBMM), The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI), 2021
Series/Report no.
CBMM Memo;123
Abstract
The ability to perceive and reason about social interactions in the context of physical environments
is core to human social intelligence and human-machine cooperation. However, no prior dataset or
benchmark has systematically evaluated physically grounded perception of complex social interactions
that go beyond short actions, such as high-fiving, or simple group activities, such as gathering. In this
work, we create a dataset of physically-grounded abstract social events, PHASE, that resemble a wide
range of real-life social interactions by including social concepts such as helping another agent. PHASE
consists of 2D animations of pairs of agents moving in a continuous space generated procedurally
using a physics engine and a hierarchical planner. Agents have a limited field of view, and can interact
with multiple objects, in an environment that has multiple landmarks and obstacles. Using PHASE,
we design a social recognition task and a social prediction task. PHASE is validated with human
experiments demonstrating that humans perceive rich interactions in the social events, and that the
simulated agents behave similarly to humans. As a baseline model, we introduce a Bayesian inverse
planning approach, SIMPLE (SIMulation, Planning and Local Estimation), which outperforms state-of-
the-art feedforward neural networks. We hope that PHASE can serve as a difficult new challenge for
developing new models that can recognize complex social interactions.
Persistent DSpace Link
