2026-09-19T04:48:51Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1514702023-08-01T04:02:20Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Tenenbaum, Joshua B.
Wei, Megan
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2023-07-31T19:42:19Z
2023-07-31T19:42:19Z
2023-06
2023-06-06T16:34:40.489Z
https://hdl.handle.net/1721.1/151470
Humans are able to build complex representations of our world – representing the world as compositional combinations of both objects and their interdependent relations. Recent work in text-guided diffusion models have produced impressive results in generating photorealistic images, but such models often fail to capture spatial relationships between objects, and will often generate scenes where individual specified relations are incorrectly captured. An underlying cause is that such models are not explicitly compositional – when given a relational text description such as fork on plate or plate on fork, models will regress to generating the previously seen images, and will only generate images with a fork on a plate. We propose an approach to more accurately capture relations by decomposing the image probability density as a hierarchical product between lifted density representing abstract relations between objects and individual densities representing each object. We illustrate how this approach is simple to implement in practice and enables us to scale to accurately capture relations between objects across simulated and realistic scenes.
M.Eng.
Massachusetts Institute of Technology
In Copyright - Educational Use Permitted
Copyright retained by author(s)
https://rightsstatements.org/page/InC-EDU/1.0/
Composing Visual Relations with Composable Diffusion Models
Thesis
application/pdf
Master
Master of Engineering in Electrical Engineering and Computer Science
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="4be67cc4-5eea-4c24-92bc-671cdafa3d08">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>Composing Visual Relations with Composable Diffusion Models</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2023-06</PublicationDate>
<Authors>
<Author>
<DisplayName>Wei, Megan</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>https://rightsstatements.org/page/InC-EDU/1.0/</License>
<Abstract>Humans are able to build complex representations of our world – representing the world as compositional combinations of both objects and their interdependent relations. Recent work in text-guided diffusion models have produced impressive results in generating photorealistic images, but such models often fail to capture spatial relationships between objects, and will often generate scenes where individual specified relations are incorrectly captured. An underlying cause is that such models are not explicitly compositional – when given a relational text description such as fork on plate or plate on fork, models will regress to generating the previously seen images, and will only generate images with a fork on a plate. We propose an approach to more accurately capture relations by decomposing the image probability density as a hierarchical product between lifted density representing abstract relations between objects and individual densities representing each object. We illustrate how this approach is simple to implement in practice and enables us to scale to accurately capture relations between objects across simulated and realistic scenes.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>