Totems: Verifying the Integrity of Visual Information using Neural Light Field

Abstract

In this work, we introduce a new approach to image forensics: physically placing a totem into the scene before taking a photo that needs to be protected from manipulations. A totem is any reflective or refractive object such that when placed in a scene, it displays a distorted version of the scene, which is called a totem view. When an image contains a totem, an adversary needs to modify both the totem view and the rest of the image (camera view) in a geometrically consistent manner in order to not have the manipulation detected. We assume that the adversary does not have access to totem shape and index of refraction (IoR), so achieving this consistency would be extremely difficult. Our work focuses on designing such algorithms that detect inconsistencies between the totem view and camera view given totem shape and IoR. In contrast to prior learning-based approaches that require large datasets of manipulated images, our methods are physics-based and work on a single image.