Enabling AI Copilots for Engineering Design With Parametric, Graph, And Component Inputs

Abstract

Engineering design demands the synthesis of multimodal and often incomplete data—ranging from detailed parametric specifications, assembly graphs, visual references, and textual descriptions. Despite growing interest in generative models for design ideation and exploration, state-of-the-art approaches struggle with incomplete inputs, lack of support for modalities other than text and image, and limited controllability. This thesis addresses these gaps by unifying two complementary advances:

First, we introduce a graph-guided diffusion approach for parametric data completion. By coupling Graph Attention Networks with a diffusion-based imputation mechanism, our method acts as a highly accurate and creative design auto-completion system for incomplete partial designs. On a dataset of 12,500 bicycles, this design imputation framework achieves a root mean square error (RMSE) of approximately 0.92 on numerical features and an error rate of around 0.18 for categorical attributes, outperforming both classical imputation methods such as MissForest, hotDeck, PPCA and advanced diffusion-based baselines such as TabCSDI. Moreover, it achives a Diversity Score of 3.10, surpassing all baselines, illustrating that the imputation process transforms incomplete data into multiple creative designs.

Second, we develop a multimodal control architecture that can extend foundation models to condition their generation processes with all or a subset of parametric inputs, assembly graphs, component images, and textual constraints. This model tremendously enhances both the controllability and precision of the generation process of foundational generative models, enabling controlling modalities that were not possible before. We first show that our model excels at tasks that state-of-the-art models struggle on. We further validate the performance of our model with surrogate models that investigate individual features. Our model achieves 95% or greater R^2 scores on different continuous parameters. Further, we show that our model is able to generate creative and novel designs while maintaining a high level of precision. This enables engineers to guide generative outputs along precise dimensional, aesthetic, and functional targets. Across numerous trials of different settings, we observe that our pipeline robustly fuses tabular parametric information, assembly graphs, and reference component images to produce results aligned with both specification precision and creativity.

Together, these contributions establish a coherent framework for AI-augmented design exploration. By viewing missing parameters as an opportunity for data-driven design autocompletion and by tightly integrating multimodal control over foundation models, this work elevates generative AI from a niche conceptual tool to a reliable design copilot. The implications of this thesis are profound: we show the possibilities and the pathways to AI copilot systems that can reduce data bottlenecks, broaden design spaces, and offer more thorough, constraint-adherent design candidates. As engineering problems grow in complexity and scale, the synergy of high-fidelity parametric imputation and multimodal control promises to accelerate innovation, cut development cycles, and guide human designers toward more inventive and manufacturable solutions.