Reconstructing Observation Data in Diffusion Based Visuomotor Policies
Name
evren-evrenis-meng-eecs-2026-thesis.pdf
Size
32.18 MB
Format
Adobe PDF
Checksum (MD5)
2f501842cd53bdee82398350ab7ad432
Author(s)
Evren, Suat
Advisor(s)
Tedrake, Russ
Date Issued
May 2026
Publisher
Massachusetts Institute of Technology
Abstract
Humanoid and household robots are poised to enter private homes, and the diffusion policies that drive them are trained on raw camera observations of the spaces they operate in. As policy weights are increasingly distributed publicly in robotics, it is natural to ask whether those weights expose the training observations. Given white-box access to a trained diffusion policy and a single target action chunk can an adversary recover the observation that produced that action with minimal knowledge possible? We take a step toward answering that question by treating it as a model-inversion problem. We frame the attack as gradient descent on the diffusion denoising loss with respect to the conditioning vector, regularized by structural priors. We run the attack on four diffusion policies: state-only Push-T, image-and-state Push-T, bin-sorting, and a real-robot policy trained on the DROID dataset. We successfully reconstruct 11 of 100 Push-T training states essentially exactly, learn a decoder that turns recovered vision features into recognizable images, and show that a synthetic-image initialization built only from the public vision encoder matches in-distribution initialization while recovering visibly recognizable training scenes in 22 of 100 bin-sorting samples. On DROID, we don’t achieve immediate success, and we hypothesize that it is because the policy is evidently underfit rather than a fundamental privacy guarantee. We decode DROID images by training a latent diffusion model that maps 384-D DINOv2 CLS latents to 1024-D SD-VAE latents, an example of the Platonic representation hypothesis. Along the way we uncover a 16:1 sensitivity asymmetry between state and vision conditioning. We find that end-to-end-finetuned vision encoders leak measurably less than frozen readouts, suggesting that task-specialized encoders discard image content the action head does not need. We reveal insights for attacks and potential defense strategies.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
