Abstract
Expected information gain (EIG) is an important criterion in Ba yesian optimal experimental design. Nested Monte Carlo and M ulti-level Monte Carlo (MLMC) methods have been used to compute EIG. However, in cases where the forward output function is not analytically tractable, even MLMC can not achieve its best rate. In this paper, we use Multi-index Monte Carlo to compute the EIG, which can give \( O(\varepsilon ^{-2}) \) computation work. Both theoretical analysis and numerical results are presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
No datasets were generated or analysed during the current study.
References
Bartuska, A., Carlon, A.G., Espath, L., Krumscheid, S., Tempone, R.: Double-loop quasi-Monte Carlo estimator for nested integration. arXiv:2302.14119 (2023)
Beck, J., Dia, B.M., Espath, L.F.R., Long, Q., Tempone, R.: Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain. Comput. Methods Appl. Mech. Eng. 334, 523–553 (2018). https://doi.org/10.1016/j.cma.2018.01.053
Beck, J., Mansour Dia, B., Espath, L., Tempone, R.: Multilevel double loop Monte Carlo and stochastic collocation methods with importance sampling for Bayesian optimal experimental design. Int. J. Numer. Methods Eng. 121(15), 3482–3503 (2020). https://doi.org/10.1002/nme.6367
Bierig, C., Chernov, A.: Convergence analysis of multilevel Monte Carlo variance estimators and application for random obstacle problems. Numer. Math. 130, 579–613 (2015)
Box, G.E.P., Hill, W.J.: Discrimination among mechanistic models. Technometrics 9(1), 57–71 (1967)
Chaloner, K., Verdinelli, I.: Bayesian experimental design: a review. Stat. Sci. 10(3), 273–304 (1995). https://doi.org/10.1214/ss/1177009939
Cliffe, K.A., Giles, M.B., Scheichl, R., Teckentrup, A.L.: Multilevel Monte Carlo methods and applications to elliptic pdes with random coefficients. Comput. Vis. Sci. 14, 3–15 (2011)
Fisher, R.A.: Design of experiments. BMJ 1(3923), 554 (1936)
Giles, M.B.: Multilevel Monte Carlo path simulation. Oper. Res. 56(3), 607–617 (2008). https://doi.org/10.1287/opre.1070.0496
Giles, M.B.: Multilevel Monte Carlo methods. Acta Numer. 24, 259–328 (2015). https://doi.org/10.1017/S096249291500001X
Giles, M.B., Goda, T.: Decision-making under uncertainty: using mlmc for efficient estimation of evppi. Stat. Comput. 29(4), 739–751 (2019). https://doi.org/10.1007/s11222-018-9835-1
Goda, T., Hironaka, T., Iwamoto, T.: Multilevel Monte Carlo estimation of expected information gains. Stoch. Anal. Appl. 38(4), 581–600 (2020). https://doi.org/10.1080/07362994.2019.1705168
Goda, T., Hironaka, T., Kitade, W., Foster, A.: Unbiased mlmc stochastic gradient-based optimization of Bayesian experimental designs. SIAM J. Sci. Comput. 44(1), 286–311 (2022). https://doi.org/10.1137/20M1338848
Haji-Ali, A.-L., Tempone, R.: Multilevel and multi-index Monte Carlo methods for the Mckean–Vlasov equation. Stat. Comput. 28, 923–935 (2018)
Haji-Ali, A.-L., Nobile, F., Tempone, R.: Multi-index Monte Carlo: when sparsity meets sampling. Numer. Math. 132(4), 767–806 (2016). https://doi.org/10.1007/s00211-015-0734-5
Hoel, H., Von Schwerin, E., Szepessy, A., Tempone, R.: Implementation and analysis of an adaptive multilevel Monte Carlo algorithm. Monte Carlo Methods Appl. 20(1), 1–41 (2014)
Huan, X., Marzouk, Y.M.: Simulation-based optimal Bayesian experimental design for nonlinear systems. J. Comput. Phys. 232(1), 288–317 (2013)
Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951). https://doi.org/10.1214/aoms/1177729694
Lindley, D.V.: On a measure of the information provided by an experiment. Ann. Math. Stat. 27(4), 986–1005 (1956)
Lindley, D.V.: On a measure of the information provided by an experiment. Ann. Math. Stat. 27(4), 986–1005 (1956). https://doi.org/10.1214/aoms/1177728069
O’Hagan, A.: Bayesian analysis of computer code outputs: a tutorial. Reliab. Eng. Syst. Saf. 91(10), 1290–1300 (2006)
Ryan, K.J.: Estimating expected information gains for experimental designs with application to the random fatigue-limit model. J. Comput. Graph. Stat. 12(3), 585–603 (2003)
Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(3), 379–423 (1948). https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
Vauhkonen, P.J., Vauhkonen, M., Savolainen, T., Kaipio, J.P.: Three-dimensional electrical impedance tomography based on the complete electrode model. IEEE Trans. Biomed. Eng. 46(9), 1150–1160 (1999). https://doi.org/10.1109/10.784147
Wilcoxon, F.: Individual Comparisons by Ranking Methods, pp. 196–202. Springer, Berlin (1997)
Zhijian He, X.W. Hejin, W.: Quasi-Monte Carlo and importance sampling methods for Bayesian inverse problems. arXiv:2403.11374 (2024)
Acknowledgements
The authors would like to thank Professor Zhijian He, Xiaoqun Wang for helpful comments and suggestions.
Author information
Authors and Affiliations
Contributions
Xinting Du wrote the main manuscript and the finish all the experiments. Hejin Wang Revised and organized into the format for final submission.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Du, X., Wang, H. Efficient estimation of expected information gain in Bayesian experimental design with multi-index Monte Carlo. Stat Comput 34, 200 (2024). https://doi.org/10.1007/s11222-024-10522-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11222-024-10522-5