Scaling Bayesian optimization for engineering design : lookahead approaches and multifidelity dimension reduction

Abstract

The objective functions and constraints that arise in engineering design problems are often non-convex, multi-modal and do not have closed-form expressions. Evaluation of these functions can be expensive, requiring a time-consuming computation (e.g., solving a set of partial differential equations) or a costly experiment (e.g., conducting wind-tunnel measurements). Accordingly, whether the task is formal optimization or just design space exploration, there is often a finite budget specifying the maximum number of evaluations of the objectives and constraints allowed. Bayesian optimization (BO) has become a popular global optimization technique for solving problems governed by such expensive functions. BO iteratively updates a statistical model and uses it to quantify the expected benefits of evaluating a given design under consideration. The next design to evaluate can be selected in order to maximize such benefits. Most existing BO algorithms are greedy strategies, making decisions to maximize the immediate benefits, without planning over several steps. This is typically a suboptimal approach. In the first part of this thesis, we develop a novel BO algorithm with planning capabilities. This algorithm selects the next design to evaluate in order to maximize the long-term expected benefit obtained at the end of the optimization. This lookahead approach requires tools to quantify the effects a decision has over several steps in the future. To do so, we use Gaussian processes as generative models and combine them with dynamic programming to formulate the optimal planning strategy. We first illustrate the proposed algorithm on unconstrained optimization problems. In the second part, we demonstrate how the proposed lookahead BO algorithm can be extended to handle non-linear expensive inequality constraints, a ubiquitous situation in engineering design. We illustrate the proposed lookahead constrained BO algorithm on a reacting flow optimization problem. In the last part of this thesis, we develop techniques to scale BO to high dimension by exploiting a special structure arising when the objective function varies only in a low-dimensional subspace. Such a subspace can be detected using the (randomized) method of Active Subspaces. We propose a multifidelity active subspace algorithm that reduces the computational cost by leveraging a cheap-to-evaluate approximation of the objective function. We analyze the number of evaluations sufficient to control the error incurred, both in expectation and with high probability. We illustrate the proposed algorithm on an ONERA M6 wing shape-optimization problem.