Digital Synesthesia : using mobile technology and sensory substitution to interact with our world

Abstract

Digital Synesthesia is the concept of using sensors and mobile technology to create a user experience that mimics the way people use their senses and enables the perception of information that is outside of our sensory spectrum. Modern technology already offers the ability to detect this information, but what has not been achieved is a way for our brains and body to incorporate this new information as a part of our sensory tool-belt. Then we can truly begin to understand our surrounding world in a new and undiscovered way. The vision of Digital Synesthesia is to help the current idea of Human-Computer Interface evolve so that it gives way for a new Human-Environment Interface. Digital Synesthesia aims to keep our mobile devices in our pockets while allowing us to experience the world by sensing information outside of our sensory capabilities. The system will not only collect data through sensors, but also find the best way to pass the information to the body while bypassing visual and auditory channels. This way, the brain will be able to interpret the new information without shifting or dividing attention from the current task.