Towards Zero-Shot Pretrained Models for Efficient Black-Box Optimization
Name
meindl-jmeindl-meng-eecs-2025-thesis.pdf
Description
Thesis PDF
Size
998.91 KB
Format
Adobe PDF
Checksum (MD5)
bbdb2476fe820ed1ff9aa53bbfcdce4d
Author(s)
Meindl, Jamison Chivvis
Advisor(s)
Matusik, Wojciech
Date Issued
September 2025
Publisher
Massachusetts Institute of Technology
Abstract
Global optimization of expensive, derivative-free black-box functions requires extreme sample efficiency. While Bayesian optimization (BO) is the current state-of-the-art, its performance hinges on surrogate and acquisition function hyperparameters that are often hand-tuned and fail to generalize across problem landscapes. We present ZeroShotOpt, the first general-purpose, pretrained model for continuous black-box optimization tasks ranging from 2 D to 20 D. Our approach leverages offline reinforcement learning on large-scale optimization trajectories collected from 12 BO variants. To scale pretraining, we generate millions of synthetic Gaussian process-based functions with diverse landscapes, enabling the model to learn transferable optimization policies. As a result, ZeroShotOpt achieves robust zero-shot generalization on a wide array of unseen synthetic and real-world benchmarks, matching or surpassing the sample efficiency of leading global optimizers, including BO, while also offering a reusable foundation for future extensions.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
