2026-09-19T04:29:57Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1590812025-09-03T03:35:59Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Tedrake, Russ Yu, Christina Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2025-04-14T14:04:20Z 2025-04-14T14:04:20Z 2025-02 2025-04-03T14:06:36.742Z https://hdl.handle.net/1721.1/159081 Diffusion models have achieved remarkable progress in recent years across various domains and applications, but how diffusion models generalize is still not well understood. While prior work predominantly focuses on unconditional diffusion models, in this thesis we focus on understanding generalization for conditional diffusion models, which is especially relevant for modern text- or observation- conditioned applications. In particular, we are interested in the inductive biases of conditional diffusion models which predispose them to certain forms of interpolation in regions outside the support of the training data. We observe that neural networks are capable of learning qualitatively different forms of interpolation, which may be influenced by the architecture and capacity of the network and other aspects of neural network training. We develop a potential framework to model the interpolation behavior of neural networks via nonparametric estimation, which happens to have the property of being schedule consistent, or truly denoising at every time step. We find that, assuming a neural network with sufficient capacity, conditional diffusion models are biased towards smoothing, which can lead to non-schedule consistent behavior away from the training data and has a number of interesting consequences. M.Eng. Massachusetts Institute of Technology In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ On the Inductive Biases of Conditional Diffusion Models Thesis application/pdf Master Master of Engineering in Electrical Engineering and Computer Science Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="82b9d9d0-a849-4ac5-a891-81dfd0a300e1"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Title>On the Inductive Biases of Conditional Diffusion Models</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2025-02</PublicationDate> <Authors> <Author> <DisplayName>Yu, Christina</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>https://rightsstatements.org/page/InC-EDU/1.0/</License> <Abstract>Diffusion models have achieved remarkable progress in recent years across various domains and applications, but how diffusion models generalize is still not well understood. While prior work predominantly focuses on unconditional diffusion models, in this thesis we focus on understanding generalization for conditional diffusion models, which is especially relevant for modern text- or observation- conditioned applications. In particular, we are interested in the inductive biases of conditional diffusion models which predispose them to certain forms of interpolation in regions outside the support of the training data. We observe that neural networks are capable of learning qualitatively different forms of interpolation, which may be influenced by the architecture and capacity of the network and other aspects of neural network training. We develop a potential framework to model the interpolation behavior of neural networks via nonparametric estimation, which happens to have the property of being schedule consistent, or truly denoising at every time step. We find that, assuming a neural network with sufficient capacity, conditional diffusion models are biased towards smoothing, which can lead to non-schedule consistent behavior away from the training data and has a number of interesting consequences.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>