Discourse Topic and Gestural Form
Name
Eisenstein_Discourse topic.pdf
Size
282.59 KB
Format
Adobe PDF
Checksum (MD5)
afde53606e9b5e45e97720ee2f6c30bd
Author(s)
Eisenstein, Jacob
Barzilay, Regina
Davis, Randall
Date Issued
2008
Journal
Proceedings of the 23rd national conference on Artificial intelligence - Volume 2 (AAAI '08)
Publisher
Association for Computing Machinery (ACM)
Citation
Jacob Eisenstein, Regina Barzilay, and Randall Davis. 2008. Discourse topic and gestural form. In Proceedings of the 23rd national conference on Artificial intelligence - Volume 2 (AAAI'08), Anthony Cohn (Ed.), Vol. 2. AAAI Press 836-841. © 2008, Association for the Advancement of Artificial Intelligence
Version
Final published version
Abstract
Coverbal gesture provides a channel for the visual expression of ideas. While some gestural emblems have culturally predefined forms (e.g., "thumbs up"), the relationship between gesture and meaning is, in general, not conventionalized. It is natural to ask whether such gestures can be interpreted in a speaker-independent way, or whether gestural form is determined by the speaker's idiosyncratic view of the discourse topic. We address this question using an audiovisual dataset across multiple speakers and topics. Our analysis employs a hierarchical Bayesian author-topic model, in which gestural patterns are stochastically generated by a mixture of speaker-specific and topic-specific priors. These gestural patterns are characterized using automatically-extracted visual features, based on spatio-temporal interest points. This framework detects significant cross-speaker patterns in gesture that are governed by the discourse topic, suggesting that even unstructured gesticulation can be interpreted across speakers. In addition, the success of this approach shows that the semantic characteristics of gesture can be detected via a low-level, interest point representation.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
http://dl.acm.org/citation.cfm?id=1620163.1620202
