HapticHearing: A Haptic Feedback System for Complementing Auditory Speech Perception for Mild-to-Moderate Hearing Loss
Name
3663547.3759754.pdf
Size
5.32 MB
Format
Adobe PDF
Checksum (MD5)
e1c068e4e2df358cb416dc00224e772d
Author(s)
Chin, Sam
Fitz-Gibbon, Emmie
Huang, Bingjian
Paradiso, Joseph
Date Issued
October 22, 2025
Publisher
ACM|The 27th International ACM SIGACCESS Conference on Computers and Accessibility
Citation
Sam Chin, Emmie Fitz-Gibbon, Bingjian Huang, and Joseph A. Paradiso.
2025. HapticHearing: A Haptic Feedback System for Complementing Auditory Speech Perception for Mild-to-Moderate Hearing Loss. In The 27th
International ACM SIGACCESS Conference on Computers and Accessibility
(ASSETS ’25), October 26–29, 2025, Denver, CO, USA. ACM, New York, NY,
USA, 5 pages.
Version
Final published version
Abstract
Age-related hearing loss is often caused by cochlear hair cell degradation. This creates a challenge for hearing aids, which rely on sound amplification. Once hearing ability in a specific frequency is lost, amplification alone provides little benefit. Previous haptic systems have tried to solve this with complete sensory substitution, converting audio signals like phonemes to tactile patterns. However, these systems require significant amount of time to learn, and induce high cognitive load in haptic perception. Our system, HapticHearing, takes an alternative approach of leveraging a user’s residual hearing and complementing it with tactile feedback. We present a custom multi-actuator haptic device, designed to translate phonemic information from speech into tactile patterns that are customized to a user’s hearing loss and speech perception abilities. The system consists of a microphone for speech capture, four-band energy envelope extraction with vowel embedding, a custom USB-to-haptic driver PCB, and wearable devices containing eight vibrotactile actuators that deliver personalized tactile feedback based on the user’s audiogram. Psychophysical validation (n=9) showed neck-worn devices achieved better spatial localization (67% vs 53%) while while bracelet and necklace devices had lower detection thresholds than over-ear (thresholds 0.09 vs 0.18).
Description
ASSETS ’25, Denver, CO, USA
MIT Department
Massachusetts Institute of Technology. Media Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3663547.3759754
