Multidimensional Labeling of Gesture in Communication: the M3D Proposal
Name
41701_2025_Article_197.pdf
Size
997.45 KB
Format
Adobe PDF
Checksum (MD5)
bfc291b74fc7292fc37e00a2214f80d8
Author(s)
Rohrer, Patrick L.
Tütüncübasi, Ulya
Florit-Pons, Júlia
Vilà-Giménez, Ingrid
Esteve-Gibert, Núria
Ren-Mitchell, Ada
Shattuck-Hufnagel, Stefanie
Prieto, Pilar
Date Issued
June 26, 2025
Journal
Corpus Pragmatics
Publisher
Springer International Publishing
Citation
Rohrer, P.L., Tütüncübasi, U., Florit-Pons, J. et al. Multidimensional Labeling of Gesture in Communication: the M3D Proposal. Corpus Pragmatics (2025).
Version
Final published version
Abstract
Communication is multimodal in that speakers use not only their voices, but also co-speech gestures to communicate. Recent insights suggest that gestural behavior has a strong association with prosodic structure and that a single gesture can communicate various semantic and pragmatic meanings. This highlights the importance of developing a comprehensive, flexible, and transparent approach to gesture annotation that accounts for multiple dimensions of gesture, including a gesture’s form, prosodic properties, and semantic and pragmatic contributions. To address this need for an increasingly dimensionalized approach to multimodal data annotation, the main goal of this paper is to present and describe a novel labeling system for manual gestures. The MultiModal MultiDimensional (M3D) system consists of an open access package that has been developed in coordination with five different labs working on gesture and its interaction with speech. The package includes a set of reliable annotation guidelines, a validated training program, and two annotated audiovisual corpora that represent over 60 minutes of lecture-style speech.
MIT Department
Massachusetts Institute of Technology. Research Laboratory of Electronics
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s41701-025-00197-2
