FMRI studies of effects of hearing status on audio-visual speech perception

Abstract

The overall goal of this research is to acquire a more complete picture of the neurological processes of visual influences on speech perception and to investigate effects of hearing status on AV speech perception. More specifically, functional magnetic resonance imaging (fMRI) was used to investigate the brain activity underlying audio-visual speech perception in three groups of subjects: (1) normally hearing, (2) congenitally deafened signers (American Sign Language) who do not use hearing aids, and (3) congenitally hearing impaired individuals with hearing aids. FMRI data were collected while subjects experienced three different types of speech stimuli: video of a speaking face with audio input, audio speech without visual input, and video of a speaking face without audio input. The cortical areas found to be active for speechreading included: visual cortex, auditory cortex (but not primary auditory cortex), speech motor network areas, supramarginal gyrus, thalamus, superior parietal cortex and fusiform gyrus. For hearing impaired subjects, in addition to the areas listed above, Heschl's gyrus, right angular gyrus (AG), cerebellum and regions around right inferior frontal sulcus (IFS) in the frontal lobe were also found to be active.

(cont.) Results from our study added to existing evidence of the engagement of motor-articulatory strategies in visual speech perception. We also found that an individual's speechreading ability is related to the amount of activity in superior temporal cortical areas, including primary auditory cortex, pre-SMA, IFS and right AG during visual speech perception. Results from effective connectivity analyses suggest that posterior superior temporal sulcus may be a potential AV speech integration site; and that AG serves a critical role in visual speech perception when auditory information is absent for hearing subjects, and when auditory information is available for hearing impaired subjects. Also, strong excitatory projections from STS to inferior frontal gyrus (IFG) and premotor/motor areas, and a strong inhibitory projection from IFG to STS seem to play an important role in visual speech perception in all subject groups. Finally, correlation analyses revealed that in hearing aid users, the amount of acoustic and speech signal gained by using hearing aids were significantly correlated with activity in IFG.