Batch Bayesian Quadrature with Batch Updating Using Future Uncertainty Sampling

Abstract

We consider the approximation of unknown or intractable integrals using quadrature when the evaluation of the integrand is considered very costly. This is a central problem both within and without machine learning, including model averaging, (hyper-)parameter marginalization, and computing posterior predictive distributions. Recently, Batch Bayesian Quadrature has successfully combined the probabilistic integration techniques of Bayesian Quadrature with the parallelization techniques of Batch Bayesian Optimization, resulting in improved performance when compared to state-of-the-art Markov Chain Monte Carlo techniques, especially when parallelization is increased. While the selection of batches in Batch Bayesian Quadrature mitigates costs associated with individual point selection, every point within every batch is nevertheless chosen serially, which impedes the realization of the full potential of batch selection. We resolve this shortcoming. We have developed a novel Batch Bayesian Quadrature method that allows for the updating of points within a batch without incurring the costs traditionally associated with non-serial point selection. We show that our method efficiently reduces uncertainty, leads to lower error estimates of the integrand, and therefore results in more numerically robust estimates of the integral. We demonstrate our method and support our findings using a synthetic test function from the Batch Bayesian Quadrature literature.