Fast and accurate Bayesian optimization with pre-trained transformers for constrained engineering problems
Name
158_2025_Article_3987.pdf
Size
3.76 MB
Format
Adobe PDF
Checksum (MD5)
ee7a1646b13a5bc72a3f6adbc3ce831f
Author(s)
Yu, Rosen T.
Picard, Cyril
Ahmed, Faez
Date Issued
April 10, 2025
Journal
Structural and Multidisciplinary Optimization
Publisher
Springer Berlin Heidelberg
Citation
Yu, R.TY., Picard, C. & Ahmed, F. Fast and accurate Bayesian optimization with pre-trained transformers for constrained engineering problems. Struct Multidisc Optim 68, 66 (2025).
Version
Final published version
Abstract
Bayesian Optimization (BO) is a foundational strategy in engineering design optimization for efficiently handling black-box functions with many constraints and expensive evaluations. This paper introduces a novel constraint-handling framework for Bayesian Optimization (BO) using Prior-data Fitted Networks (PFNs), a foundation transformer model. Unlike traditional approaches requiring separate Gaussian Process (GP) models for each constraint, our framework leverages PFN’s transformer architecture to evaluate objectives and constraints simultaneously in a single forward pass using in-context learning. Through comprehensive benchmarking across 15 test problems spanning synthetic, structural, and engineering design challenges, we demonstrate an order of magnitude speedup while maintaining or improving solution quality compared to conventional GP-based methods with constrained expected improvement (CEI). Our approach particularly excels at engineering problems by rapidly finding feasible, optimal solutions. This benchmark framework for evaluating new BO algorithms in engineering design will be published at https://github.com/rosenyu304/BOEngineeringBenchmark .
MIT Department
Massachusetts Institute of Technology. Center for Computational Science and Engineering
Massachusetts Institute of Technology. Department of Mechanical Engineering
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s00158-025-03987-z
