Wearable-assisted social interaction as assistive technology for the blind

Abstract

This work presents an end-to-end wearable system designed to learn and assist its (potentially blind) wearers with daily social interactions. In particular, it visually identifies nearby acquaintances and provides timely, discreet notifications of their presence to the wearer. Offline, the system learns the people with whom the wearer interacts by automatically detecting social interactions through egocentric audio, video and accelerometer data and querying the wearer for the identities of persons unknown to the system.