Seminar 2024-29-03

Recent Advances in Bayesian Optimization for the Physical Sciences

Simon Mak

Assistant Professor

Department of Statistical Science

Duke University

Friday, March 29^th, 2024

11:00 A.M. – 12:00 P.M. Eastern Time

Nguyen Engineering Building, Room 1109

4511 Patriot Circle, Fairfax, VA

With fundamental advances in scientific computing, computer simulations are increasingly used for investigating complex physical phenomena, such as rocket propulsion and universe expansion. For many such applications, scientific decision-making involves optimizing the simulator output, which can be very costly given the expensive nature of each simulation. While Bayesian optimization (BO) offers a promising solution, there are key challenges that limit the use of existing BO methods in the physical sciences. The first challenge is the presence of noise parameters: parameters that are controllable in the computer simulation but uncontrollable or unknown in reality. For this, we propose a new Targeted Variance Reduction (TVR) method, for optimizing a black-box simulator given random uncertainty on noise parameters. Using a carefully specified Gaussian process surrogate model, the TVR admits a closed-form acquisition function for efficient sequential sampling via normalizing flows. It also reveals a novel exploration-exploitation-precision trade-off for robust black-box optimization. We explore the effectiveness of TVR over the state-of-the-art in numerical experiments and an application for automobile brake design under operational uncertainties. The second challenge is the need for diverse optimization solutions, which provides users with a basket of "good" solutions for downstream decision-making. For example, in drug optimization, a scientist may wish to find a diverse set of promising molecule structures to fully explore design options of a drug. For this, we propose a new Diverse Expected Improvement (DEI) method, which extends the popular Expected Improvement method to encourage diversity between near-optimal solutions. The DEI jointly targets two goals: the exploration of diverse near-optimal solutions over the parameter space, and the exploitation of promising solutions that improve upon the current-best solution. We explore the effectiveness of the DEI in two practical applications, the first on rover trajectory optimization and the second for real-time control of unmanned aerial vehicles. 

Simon Mak is an Assistant Professor in the Department of Statistical Science at Duke University. His research involves integrating domain knowledge (e.g. scientific theories, guiding principles) as prior information for cost-efficient statistical inference, prediction, and decision-making. This gives a holistic framework for interpretable statistical learning, providing a principled way for scientists to validate theories from data, and for statisticians to integrate scientific knowledge. His research tackles methodological, theoretical, and algorithmic challenges in this integration, via the building of probabilistic models on complex objects (e.g., functions, manifolds, networks), and the development of efficient learning algorithms and data collection methods. His current research is motivated from ongoing interdisciplinary collaborative projects in high-energy physics, aerospace engineering, public policy and computational advertising.

Nicholas Rios

David Kepplinger