Abstract
We consider the regression problem based on Gaussian processes. We assume that the prior distribution on the vector of parameters in the corresponding model of the covariance function is non-informative. Under this assumption, we prove the Bernstein-von Mises theorem that states that the posterior distribution on the parameters vector is close to the corresponding normal distribution. We show results of numerical experiments that indicate that our results apply in practically important cases.
Similar content being viewed by others
References
Rasmussen, C.E. and Williams, C.K.I., Gaussian Processes for Machine Learning, Cambridge: MIT Press, 2006.
Chervonenkis, A.Ya., Chernova, S.S., and Zykova, T.V., Applications of Kernel Ridge Estimation to the Problem of Computing the Aerodynamical Characteristics of a Passenger Plane (in Comparison with Results Obtained with Artificial Neural Networks), Autom. Remote Control, 2011, vol. 72, no. 5, pp. 1061–1067.
Forrester, A., Sobester, A., and Keane, A., Engineering Design via Surrogate Modelling: A Practical Guide, Chichester: Wiley, 2008.
Panov, M.E., Burnaev, E.V., and Zaitsev, A.A., On Methods of Introducing Regularization in Regression Based on Gaussian processes, Tr. konf. “Matematicheskie metody raspoznavaniya obrazov-15” (Proc. Conf. “Mathematical Methods of Image Recognition-15”), 2011, pp. 142–145.
Kennedy, M.C. and Hagan, A.O., Bayesian Calibration of Computer Models, J. R. Statist. Soc., Ser. B (Stat. Met.), 2001, vol. 63, no. 3, pp. 425–464.
Qian, P.Z.G. and Wu, C.F.G., Bayesian Hierarchical Modeling for Integrating Low-accuracy and High-accuracy Experiments, Technometrics, 2008, vol. 50, no. 2, pp. 192–204.
Kaufman, C.G., Schervish, M.J., and Nychka, D.W., Covariance Tapering for Likelihood-based Estimation in Large Spatial Data Sets, J. Am. Statist. Ass., 2008, vol. 103, no. 484, pp. 1545–1555.
Eidsvik, J., Finley, A.O., Banerjee, S., et al., Approximate Bayesian Inference for Large Spatial Datasets Using Predictive Process Models, Comput. Statist. Data Anal., 2011, vol. 56, no. 6, pp. 1362–1380.
Shaby, B. and Ruppert, D., Tapered Covariance: Bayesian Estimation and Asymptotics, J. Comput. Graphical Statist., 2012, vol. 21, no. 2, pp. 433–452.
Spokoiny, V., Parametric Estimation. Finite Sample Theory, Ann. Statist., 2012, vol. 40, no. 6, pp. 2877–2909.
Spokoiny, V., Bernstein-von Mises Theorem for Growing Parameter Dimension, preprint, 2013, arXiv: 1302.3430v2.
Kok, S., The Asymptotic Behaviour of the Maximum Likelihood Function of Kriging Approximations Using the Gaussian Correlation Function, Int. Conf. on Engin. Optimization (EngOpt 2012), Rio de Janeiro, Brazil, 2012.
Nagy, B., Loeppky, J.L., and Welch, W.J., Correlation Parameterization in Random Function Models to Improve Normal Approximation of the Likelihood or Posterior, Technical Report, Vancouver: Univ. of British Columbia, 2007.
Author information
Authors and Affiliations
Additional information
Original Russian Text © A.A. Zaitsev, E.V. Burnaev, V.G. Spokoiny, 2013, published in Avtomatika i Telemekhanika, 2013, No. 10, pp. 55–67.
Rights and permissions
About this article
Cite this article
Zaitsev, A.A., Burnaev, E.V. & Spokoiny, V.G. Properties of the posterior distribution of a regression model based on Gaussian random fields. Autom Remote Control 74, 1645–1655 (2013). https://doi.org/10.1134/S0005117913100056
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S0005117913100056