Deep video-to-video transformations for accessibility applications

Abstract

We develop a class of visual assistive technologies that can learn visual transforms to improve accessibility as an alternative to traditional methods that mostly rely on extracted symbolic information. In this thesis, we mainly focus on how we can apply this class of systems to address photosensitivity. People with photosensitivity may have seizures, migraines or other adverse reactions to certain visual stimuli such as flashing images and alternating patterns. We develop deep learning models that learn to identify and transform video sequences containing such stimuli whilst preserving video quality and content. Using descriptions of the adverse visual stimuli, we train models to learn transforms to remove such stimuli. We show that these deep learning models are able to generalize to real-world examples of images with these problematic stimuli. From our experimental trials, human subjects rated video sequences transformed by our models as having significantly less problematic stimuli than their input. We extend these ideas; we show how these deep transformation networks can be applied in other visual assistive domains through demonstration of an application addressing the problem of emotion recognition in those with the Autism Spectrum Disorder.