Hover : A wearable object identification system for audio augmented reality interactions

Abstract

We design, implement, and test a wearable object identification system which allows users to "hover" their hands over objects of interest and get access to contextual information that may be tied to them, through an intelligent personal assistant. The system uses a fusion of sensors to be able to perform the identification of an object under a variety of conditions. Among these sensors there is a camera (operating in the visible and infrared spectrum), a small solid-state radar, and multi-spectral light spectroscopy sensors. Users can interact with contextual information tied to an object through conversations with an intelligent assistant to permit a hands-free, non-obtrusive, and personalized experience. The system explores audio interfacing with augmented reality content without the hassle of phones or head mounted devices.