Inverse Inverse Graphics
Name
chandra-kach-sm-eecs-2023-thesis.pdf
Description
Thesis PDF
Size
23.79 MB
Format
Adobe PDF
Checksum (MD5)
c9bde51fb8bfd2e0dfc92035fa63246b
Author(s)
Chandra, Kartik
Advisor(s)
Ragan-Kelley, Jonathan
Tenenbaum, Joshua B
Date Issued
June 2023
Publisher
Massachusetts Institute of Technology
Abstract
To be human is to express—not only facts about the world, but also experiences, emotions, and ideas. We paint, explain, compose, persuade, dance, teach, sculpt, and sing, not merely to communicate bits of information from person to person, but rather to fan the windmills of one another's minds.
Well then, how do we do it? This thesis suggests that when engaging in artistic expression, we reflect on our shared perceptual and cognitive faculties—"common sense," so to speak—and then construct stimuli to evoke experiences in our audience's brains. Drawing on ideas from computer graphics, cognitive science, and literary theory, I offer ways to think about images (Chandra et al., 2022) and stories (Chandra et al., 2023a, 2023b) as objects to be designed with respect to a model of the audience. In particular, if we think of the audience's mind as solving inverse problems—perception as inverse rendering, action understanding as inverse planning—then we can think of expression as solving a kind of *inverse* inverse problem.
I then show how to implement such "inverse inverse" methods computationally. Starting with classic Bayesian models of vision and social cognition, I present algorithms for optimizing over inference to create "adversarial examples" that evoke various desired inferences in those models. For example, we optimize images that are visual illusions (Chapter 1), and animations that tell surprising stories (Chapter 2). Because the Bayesian models capture human intuitions well (better than, say, typical neural network models), the optimized stimuli "transfer" to evoke similar experiences in humans. I demonstrate this with a variety of human subject studies.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
