Visually grounded virtual accelerometers : a longitudinal video investigation of dyadic bodily dynamics around the time of word acquisition
Author(s)Tsourides, Kleovoulos (Kleovoulos Leo)
Longitudinal video Investigation of dyadic bodily dynamics around the time of word acquisition
Massachusetts Institute of Technology. Dept. of Architecture. Program in Media Arts and Sciences.
MetadataShow full item record
Human movement encodes information about internal states and goals. When these goals involve dyadic interactions, such as in language acquisition, the nature of the movement and proximity become representative, allowing parts of our internal states to manifest. We propose an approach called Visually Grounded Virtual Accelerometers (VGVA), to aid with ecologically-valid video analysis investigations, involving humans during dyadic interactions. Utilizing the Human Speechome (HSP)  video corpus database, we examine a dyadic interaction paradigm taken from the caregiver-child ecology, during language acquisition. We proceed to characterize human interaction in a video cross-modally; by visually detecting and assessing the child's bodily dynamics in a video, grounded on the caregiver's bodily dynamics of the same video and the related HSP speech transcriptions . Potential applications include analyzing a child's language acquisition, establishing longitudinal diagnostic means for child developmental disorders and generally establishing a metric of effective human communication on dyadic interactions under a video surveillance system. In this thesis, we examine word-learning transcribed video episodes before and after the age of the word's acquisition (AOA). As auditory stimulus is uttered from the caregiver, points along the VGVA tracked sequences corresponding to the onset and post-onset of the child-caregiver bodily responses, are used to longitudinally mark and characterize episodes of word learning. We report a systematic shift in terms of caregiver-child synchrony in motion and turning behavior, tied to exposures of the target word around the time the child begins to understand and thus respond to instances of the spoken word. The systematic shift, diminishes gradually after the age of word acquisition (AOA).
Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2010.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student submitted PDF version of thesis.Includes bibliographical references (p. 105-110).
DepartmentMassachusetts Institute of Technology. Dept. of Architecture. Program in Media Arts and Sciences.
Massachusetts Institute of Technology
Architecture. Program in Media Arts and Sciences.