Toward Affordance-Based Generation for 3D Generative AI

Abstract

Recent advances in 3D content creation with generative AI have made it easier to generate 3D models using text and images as input. However, translating these digital designs into usable objects in the physical world is still an open challenge. Since these 3D models are generated to be aesthetically similar to their inputs, the resulting models tend to have the visual features the user desires but often lack the functionality required for their use cases. This thesis proposes a novel approach to generative AI in 3D modeling, shifting the focus from replicating specific objects to generating affordances. We trained models that allow users to create point clouds that satisfy physical properties called affordances, which are properties that describe how an object should behave in the real world. By ensuring that the generated objects have the expected affordances, we explore how existing tools can be augmented to generate 3D objects whose functionality is consistent with their appearances.