Non-verbal signals for grounding in embodied conversational agent

Abstract

In face-to-face conversation, speakers present non-verbal signals collateral with verbal information. Nodding and gazing at a speaker are known to provide positive feedback from listeners, which contributes to establishing common ground (a process called grounding). However, previous theories and computational models of grounding were mainly concerned with verbal grounding acts, and there have not been enough discussion about how nonverbal behaviors are used in the process of grounding. This thesis first compares face-to-face conversation to conversation without co-presence, revealing how nonverbal behaviors are used in the process of grounding in human communication. Results of the analysis show that, in face-to-face communication, non-verbal behaviors are changing during an utterance and a typical transition pattern of non-verbal behaviors is also different depending on the type of verbal act. Then, the implementation of grounding functionality onto an Embodied Conversational Agent is presented. The dialogue state updating mechanism in the Dialogue Manager accesses non-verbal information conveyed by a user and judges the groundedness of presented materials based on the results of empirical study.