Multi-Modal Machine Learning in Engineering Design: A Review and Future Directions
Name
jcise_24_1_010801.pdf
Description
Published version
Size
1011.59 KB
Format
Adobe PDF
Checksum (MD5)
42a77b84e39f9c8fac6d7a635f09585f
Author(s)
Song, Binyang
Zhou, Rui
Ahmed, Faez
Date Issued
November 24, 2023
Journal
Journal of Computing and Information Science in Engineering
Publisher
ASME International
Citation
Song, B., Zhou, R., and Ahmed, F. (November 24, 2023). "Multi-Modal Machine Learning in Engineering Design: A Review and Future Directions." ASME. J. Comput. Inf. Sci. Eng. January 2024; 24(1): 010801.
Version
Final published version
Abstract
In the rapidly advancing field of multi-modal machine learning (MMML), the convergence of multiple data modalities has the potential to reshape various applications. This paper presents a comprehensive overview of the current state, advancements, and challenges of MMML within the sphere of engineering design. The review begins with a deep dive into five fundamental concepts of MMML: multi-modal information representation, fusion, alignment, translation, and co-learning. Following this, we explore the cutting-edge applications of MMML, placing a particular emphasis on tasks pertinent to engineering design, such as cross-modal synthesis, multi-modal prediction, and cross-modal information retrieval. Through this comprehensive overview, we highlight the inherent challenges in adopting MMML in engineering design, and proffer potential directions for future research. To spur on the continued evolution of MMML in engineering design, we advocate for concentrated efforts to construct extensive multi-modal design datasets, develop effective data-driven MMML techniques tailored to design applications, and enhance the scalability and interpretability of MMML models. MMML models, as the next generation of intelligent design tools, hold a promising future to impact how products are designed.
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1115/1.4063954
