Interactions between the auditory and vibrotactile senses : a study of perceptual effects

Abstract

This project is an experimental study of perceptual interactions between auditory and tactile stimuli. These experiments present vibrotactile stimuli to the fingertip and auditory tones diotically in broadband noise. Our hypothesis states that if the auditory and tactile systems integrate, the performance of the two sensory stimuli presented simultaneously will be different from the performance of the individual sensory stimuli. The research consists of work in two major areas: (1) Studies of the detection of auditory and tactile sinusoidal stimuli at levels near the threshold of perception (masked thresholds for auditory stimuli and absolute thresholds for tactile stimuli); and (2) Studies of loudness matching employing various combinations of auditory and tactile stimuli presented at supra-threshold levels. Results were compared to three models of auditory-tactile integration. The objective detection studies explore the effects of three major variables on perceptual integration: (a) the starting phase of the auditory relative to the tactile stimulus; (b) the temporal synchrony of stimulation within each of the two modalities; and (c) the frequency of stimulation within each modality. Detection performance for combined auditory-tactile (A+T) presentations was measured using stimulus levels that yielded 63%-77%-correct unimodal performance in a 2-Interval, 2-Alternative Forced- Choice procedure. Results for combined vibrotactile and auditory detection indicated: (1) For synchronous presentation of 500-msec, 250 Hz sinusoidal stimuli, percent-correct scores in the combined A+T conditions were significantly higher than scores within each single modality;

(cont.) (2) Scores in the A+T conditions were not affected by the relative phase of the 250 Hz auditory and tactile stimuli; (3) For asynchronous presentation of auditory and tactile 250 Hz stimuli, scores on the A+T conditions improved only when the tactile stimulus preceded the auditory stimulus (and not vice versa); and (4) The highest rates of detection in the combined-modality stimulus were obtained when stimulating frequencies in the two modalities were equal or closely spaced (and within the Pacinian range). The lack of phase effect suggests that integration operates on the envelopes rather than on temporal fine structure. The effects of asynchronous presentation imply a shorter time constant in the auditory compared to the tactile modality and are consistent with time constants deduced from single-modality masking experiments. The effects of frequency depend both on absolute frequency and on relative frequency of stimulation within each modality. In general, we found that an additive sensitivity model best explained detection performance when tones were presented synchronously and of the same frequency. In the second area of research, loudness matching was employed in a subjective study of the effects of frequency on auditory-tactile integration for stimuli presented at supra-threshold levels. These experiments, which were derived from previous auditory studies demonstrating the dependence of loudness on critical-band spacing of tonal signals, employed various combinations of auditory and tactile stimuli that were presented at equally loud levels in isolation.

(cont.) Loudness matches were obtained for auditory-only (A+A) and auditory-tactile (A+T) stimuli that were both close as well as farther apart in frequency. The results show that the matched loudness of an auditory pure tone is greater when the frequencies of combined stimuli (both A+A and A+T) are farther apart in frequency than when they are close in frequency. These results are consistent with the results found in the previous experiment exploring the frequency relationships at near-threshold levels, as well as with results in the psychoacoustic literature, and suggest that the auditory and tactile systems are interacting in a frequency-specific manner similar to the interactions of purely auditory stimuli. The research conducted here demonstrates objective and subjective perceptual effects that support the mounting anatomical and physiological evidence for interactions between the auditory and tactual sensory systems.