research-article

A Variational Inference-Based Heteroscedastic Gaussian Process Approach for Simulation Metamodeling

Authors:

Linchang YangAuthors Info & Claims

ACM Transactions on Modeling and Computer Simulation (TOMACS), Volume 29, Issue 1

Article No.: 6, Pages 1 - 22

https://doi.org/10.1145/3299871

Published: 24 January 2019 Publication History

Abstract

In this article, we propose a variational Bayesian inference-based Gaussian process metamodeling approach (VBGP) that is suitable for the design and analysis of stochastic simulation experiments. This approach enables statistically and computationally efficient approximations to the mean and variance response surfaces implied by a stochastic simulation, while taking into full account the uncertainty in the heteroscedastic variance; furthermore, it can accommodate the situation where either one or multiple simulation replications are available at every design point. We demonstrate the superior performance of VBGP compared with existing simulation metamodeling methods through two numerical examples.

Supplementary Material

a6-wang-supp.pdf (wang.zip)

Supplemental movie, appendix, image and software files for, A Variational Inference-Based Heteroscedastic Gaussian Process Approach for Simulation Metamodeling

Download
149.31 KB

References

[1]

B. E. Ankenman, B. L. Nelson, and J. Staum. 2010. Stochastic kriging for simulation metamodeling. Operations Research 58 (2010), 371--382.

Digital Library

[2]

A. Banerjee, D. B. Dunson, and S. T. Tokdar. 2012. Efficient Gaussian process regression for large datasets. Biometrika 100 (2012), 75--89.

[3]

M. J. Beal. 2003. Variational Algorithms for Approximate Bayesian Inference. Ph.D. Dissertation. University of London.

[4]

M. Binois, R. B. Gramacy, and M. Ludkovski. 2018. Practical heteroskedastic Gaussian process modeling for large simulation experiments. Journal of Computational and Graphical Statistics 27 (2018), 808–821.

[5]

C. M. Bishop. 2006. Pattern Recognition and Machine Learning. Springer-Verlag.

Digital Library

[6]

D. M. Blei, A. Kucukelbir, and J. D. McAuliffe. 2017. Variational inference: A review for statisticians. Journal of the American Statistical Association 112 (2017), 859--877.

[7]

N. A. C. Cressie. 2015. Statistics for Spatial Data (revised ed.). John Wiley 8 Sons.

[8]

N. A. C. Cressie and G. Johannesson. 2008. Fixed rank kriging for very large spatial data sets. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 70 (2008), 209--226.

[9]

S. Efromovich. 2007. Sequential design and estimation in heteroscedastic nonparametric regression. Sequential Analysis 26 (2007), 3--25.

[10]

C. W. Fox and S. J. Roberts. 2012. A tutorial on variational Bayesian inference. Artificial Intelligence Review 38 (2012), 85--95.

[11]

M. C. Fu and K. J. Healy. 1992. Simulation optimization of (s, S ) inventory systems. In Proceedings of the 24th Conference on Winter Simulation. ACM, 506--514.

Digital Library

[12]

M. Fuentes, B. Reich, and G. Lee. 2008. Spatial-temporal mesoscale modeling of rainfall intensity using gage and radar data. Annals of Applied Statistics 2, 4 (2008), 1148--1169.

[13]

P. W. Goldberg, C. K. I. Williams, and C. M. Bishop. 1998. Regression with input-dependent noise: A Gaussian process treatment. In Advances in Neural Information Processing Systems. 493--499.

Digital Library

[14]

M. C. Kennedy and A. O’Hagan. 2000. Predicting the output from a complex computer code when fast approximations are available. Biometrika 87 (2000), 1--13.

[15]

K. Kersting, C. Plagemann, P. Pfaff, and W. Burgard. 2007. Most likely heteroscedastic Gaussian process regression. In Proceedings of the 24th International Conference on Machine Learning. ACM, 393--400.

Digital Library

[16]

J. P. C. Kleijnen. 2015. Design and Analysis of Simulation Experiments (2nd ed.). Springer.

Digital Library

[17]

J. P. C. Kleijnen and W. C. van Beers. 2005. Robustness of kriging when interpolating in random simulation with heterogeneous variances: Some experiments. European Journal of Operational Research 165 (2005), 826--834.

[18]

M. Lázaro-Gredilla and M. Titsias. 2011. Variational heteroscedastic Gaussian process regression. In Proceedings of the 28th International Conference on Machine Learning.

Digital Library

[19]

S. H. Ng and J. Yin. 2012. Bayesian kriging analysis and design for stochastic simulations. ACM Transactions on Modeling and Computer Simulation 22 (2012), 1--26.

Digital Library

[20]

H. Nguyen, N. A. C. Cressie, and A. Braverman. 2012. Spatial statistical data fusion for remote sensing applications. Journal of the American Statistical Association 107 (2012), 1004--1018.

[21]

C. J. Paciorek. 2012. Combining spatial information sources while accounting for systematic errors in proxies. Journal of the Royal Statistical Society: Series C (Applied Statistics) 61 (2012), 429--451.

[22]

C. E. Rasmussen and C. K. I. Williams. 2006. Gaussian Processes for Machine Learning. MIT Press.

Digital Library

[23]

J. Sacks, S. B. Schiller, and W. J. Welch. 1989. Designs for computer experiments. Technometrics 31 (1989), 41--47.

[24]

T. J. Santner, B. J. Williams, and W. I. Notz. 2013. The Design and Analysis of Computer Experiments. Springer Science 8 Business Media.

[25]

V. Tolvanen, P. Jylänki, and A. Vehtari. 2014. Expectation propagation for nonstationary heteroscedastic Gaussian process regression. In Proceedings of the 2014 IEEE International Workshop on Machine Learning for Signal Processing. 1--6.

[26]

O. A. Vanli, C. Zhang, L. Chen, K. K. Wang, and B. Wang. 2010. A Bayesian approach for integration of physical and computer experiments for quality improvement in nano-composite manufacturing. Quality and Reliability Engineering International 26 (2010), 749--764.

[27]

T. Wagner, C. Bröcker, N. Saba, D. Biermann, A. Matzenmiller, and K. Steinhoff. 2010. Modelling of a thermomechanically coupled forming process based on functional outputs from a finite element analysis and from experimental measurements. AStA Advances in Statistical Analysis 94 (2010), 389--404.

[28]

B. Wang and D. M. Titterington. 2004. Convergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence. AUAI Press, 577--584.

Digital Library

[29]

W. Wang and X. Chen. 2016. The effects of estimation of heteroscedasticity on stochastic kriging. In Proceedings of the 2016 Winter Simulation Conference. IEEE Press, 326--337.

Digital Library

[30]

W. Wang and X. Chen. 2018. An adaptive two-stage dual metamodeling approach for stochastic simulation experiments. IISE Transactions 50 (2018), 820--836.

[31]

Y. Wang and D. M. Blei. 2018. Frequentist consistency of variational Bayes. Journal of the American Statistical Association 0 (2018), 1--15.

[32]

W. Whitt. 1989. Planning queueing simulations. Management Science 35 (1989), 1341--1366.

Digital Library

[33]

H. Xia, Y. Ding, and J. Wang. 2008. Gaussian process method for form error assessment using coordinate measurements. IIE Transactions 40 (2008), 931--946.

[34]

Y. Xiong, W. Chen, D. Apley, and X. Ding. 2007. A non-stationary covariance-based kriging method for metamodelling in engineering design. International Journal for Numerical Methods in Engineering 71 (2007), 733--756.

[35]

J. Yin, S. H. Ng, and K. M. Ng. 2011. Kriging metamodel with modified nugget-effect: The heteroscedastic variance case. Computers 8 Industrial Engineering 61 (2011), 760--777.

Digital Library

Cited By

Liang TWang FWang SLi KMa XMo X(2024)Towards unified aleatory and epistemic uncertainty quantification for machinery health prognostic through sequential heteroscedastic Gaussian process regressionAdvanced Engineering Informatics10.1016/j.aei.2024.10271962(102719)Online publication date: Oct-2024
https://doi.org/10.1016/j.aei.2024.102719
Zhang YChen X(2024)Sequential metamodel‐based approaches to level‐set estimation under heteroscedasticityStatistical Analysis and Data Mining10.1002/sam.1169717:3Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1002/sam.11697
Zhang YChen X(2022)Empirical Uniform Bounds for Heteroscedastic MetamodelingProceedings of the Winter Simulation Conference10.5555/3586210.3586211(1-12)Online publication date: 11-Dec-2022
https://dl.acm.org/doi/10.5555/3586210.3586211
Show More Cited By

Index Terms

A Variational Inference-Based Heteroscedastic Gaussian Process Approach for Simulation Metamodeling
1. Mathematics of computing
  1. Probability and statistics
    1. Probabilistic algorithms

Recommendations

Stochastic variational inference

We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. We develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models, latent Dirichlet ...
Variational inference in nonconjugate models

Mean-field variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, mean-field methods approximately compute the posterior with a coordinate-ascent optimization algorithm. When the ...
Variational Tobit Gaussian Process Regression
Abstract
We propose a variational inference-based framework for training a Gaussian process regression model subject to censored observational data. Data censoring is a typical problem encountered during the data gathering procedure and requires ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Computer Simulation

ACM Transactions on Modeling and Computer Simulation Volume 29, Issue 1

January 2019

149 pages

ISSN:1049-3301

EISSN:1558-1195

DOI:10.1145/3309768

Editor:
Adelinde M. Uhrmacher
University of Rostock, Germany

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 January 2019

Accepted: 01 November 2018

Revised: 01 September 2018

Received: 01 April 2018

Published in TOMACS Volume 29, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
274
Total Downloads

Downloads (Last 12 months)41
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liang TWang FWang SLi KMa XMo X(2024)Towards unified aleatory and epistemic uncertainty quantification for machinery health prognostic through sequential heteroscedastic Gaussian process regressionAdvanced Engineering Informatics10.1016/j.aei.2024.10271962(102719)Online publication date: Oct-2024
https://doi.org/10.1016/j.aei.2024.102719
Zhang YChen X(2024)Sequential metamodel‐based approaches to level‐set estimation under heteroscedasticityStatistical Analysis and Data Mining10.1002/sam.1169717:3Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1002/sam.11697
Zhang YChen X(2022)Empirical Uniform Bounds for Heteroscedastic MetamodelingProceedings of the Winter Simulation Conference10.5555/3586210.3586211(1-12)Online publication date: 11-Dec-2022
https://dl.acm.org/doi/10.5555/3586210.3586211
Zhang YChen X(2022)Empirical Uniform Bounds For Heteroscedastic Metamodeling2022 Winter Simulation Conference (WSC)10.1109/WSC57314.2022.10015525(1-12)Online publication date: 11-Dec-2022
https://doi.org/10.1109/WSC57314.2022.10015525
Wang QZhang CMa ZNi Y(2022)Modelling and forecasting of SHM strain measurement for a large-scale suspension bridge during typhoon events using variational heteroscedastic Gaussian processEngineering Structures10.1016/j.engstruct.2021.113554251(113554)Online publication date: Jan-2022
https://doi.org/10.1016/j.engstruct.2021.113554
Noack MDoerk GLi RStreit JVaia RYager KFukuto M(2020)Autonomous materials discovery driven by Gaussian process regression with inhomogeneous measurement noise and anisotropic kernelsScientific Reports10.1038/s41598-020-74394-110:1Online publication date: 19-Oct-2020
https://doi.org/10.1038/s41598-020-74394-1
Wang WChen XSon YHaas P(2019)Distributed variational inference-based heteroscedastic gaussian process metamodelingProceedings of the Winter Simulation Conference10.5555/3400397.3400428(380-391)Online publication date: 8-Dec-2019
https://dl.acm.org/doi/10.5555/3400397.3400428
Wang WChen X(2019)Distributed Variational Inference-Based Heteroscedastic Gaussian Process Metamodeling2019 Winter Simulation Conference (WSC)10.1109/WSC40007.2019.9004911(380-391)Online publication date: Dec-2019
https://doi.org/10.1109/WSC40007.2019.9004911

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents