Semantic spaces : behavior, language and word learning in the Human Speechome corpus

Abstract

The Human Speechome Project is an unprecedented attempt to record, analyze and understand the process of language acquisition. It is composed of over 90,000 hours of video and 150,000 hours of audio, capturing roughly 80% of the waking hours of a single child from his birth until age 3. This thesis proposes and develops a method for representing and analyzing a video corpus of this scale that is both compact and efficient, while retaining much of the important information about large scale behaviors of the recorded subjects. This representation is shown to be useful for the unsupervised modeling, clustering and exploration of the data, particularly when it is combined with text transcripts of the speech. Novel methods are introduced to perform Spatial Latent Semantic Analysis - extending the popular framework for topic modeling to cover behavior as well. Finally, the representation is used to analyze the inherent "spatiality" of individual words. A surprising connection is demonstrated between the uniqueness of a word's spatial distribution and how early it is learned by the child.