Parametric-ControlNet: Multimodal Control in Foundation Models for Precise Engineering Design Synthesis
Name
md-24-1872.pdf
Description
Published version
Size
1.07 MB
Format
Adobe PDF
Checksum (MD5)
409b42386402decd8dd9b9c148e97a83
Author(s)
Zhou, Rui
Zhang, Yanxia
Yuan, Chenyang
Permenter, Frank
Arechiga, Nikos
Klenk, Matt
Ahmed, Faez
Date Issued
June 5, 2025
Journal
Journal of Mechanical Design
Publisher
ASME International
Citation
Zhou, R., Zhang, Y., Yuan, C., Permenter, F., Arechiga, N., Klenk, M., and Ahmed, F. (June 5, 2025). "Parametric-ControlNet: Multimodal Control in Foundation Models for Precise Engineering Design Synthesis." ASME. J. Mech. Des. December 2025; 147(12): 121704.
Version
Final published version
Abstract
This article introduces a generative model designed for multimodal control over text-to-image foundation generative artificial intelligence (AI) models such as Stable Diffusion, specifically tailored for engineering design synthesis. Our model proposes parametric, image, and text control modalities to enhance design precision and diversity. First, it handles both partial and complete parametric inputs using a diffusion model that acts as a design autocomplete copilot, coupled with a parametric encoder to process the information. Second, the model utilizes assembly graphs to systematically assemble input component images, which are then processed through a component encoder to capture essential visual data. Third, textual descriptions are integrated via CLIP encoding, ensuring a comprehensive interpretation of design intent. These diverse inputs are synthesized through a multimodal fusion technique, creating a joint embedding that acts as the input to a module inspired by ControlNet. This integration allows the model to apply robust multimodal control to foundation models, facilitating the generation of complex and precise engineering designs. This approach broadens the capabilities of AI-driven design tools and demonstrates significant advancements in precise control based on diverse data modalities for enhanced design generation.
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1115/1.4068661
