Bidirectional gaze guiding and indexing in human-robot interaction through a situated architecture

Abstract

In this body of work, I present a situated and interactive agent perception system that can index into its world and, through a bidirectional exchange of referential gesture, direct its internal indexing system toward both well-known objects as well as simple visuo-spatial indexing in the world. The architecture presented incorporates a novel method for synthetic human-robot joint attention, an internal and automatic crowdsourcing system that provides opportunistic and lifelong robotic socio-visual learning, supports the bidirectional process of following referential behavior; and generates referential behavior useful for directing the gaze of human peers. This document critically probes questions in human-robot interaction around our understanding of gaze manipulation and memory imprinting on human partners in similar architectures and makes recommendations that may improve human-robot peer-to-peer learning.