research-article

Batching Adaptive Variance Reduction

Authors:

Reiichiro KawaiAuthors Info & Claims

ACM Transactions on Modeling and Computer Simulation, Volume 33, Issue 1-2

Article No.: 3, Pages 1 - 24

https://doi.org/10.1145/3573386

Published: 28 February 2023 Publication History

Abstract

Adaptive Monte Carlo variance reduction is an effective framework for running a Monte Carlo simulation along with a parameter search algorithm for variance reduction, whereas an initialization step is required for preparing problem parameters in some instances. In spite of the effectiveness of adaptive variance reduction in various fields of application, the length of the preliminary phase has often been left unspecified for the user to determine on a case-by-case basis, much like in typical sequential frameworks. This uncertain element may possibly be even fatal in realistic finite-budget situations, since the pilot run may take most of the budget, or possibly use up all of it. To unnecessitate such an ad hoc initialization step, we develop a batching procedure in adaptive variance reduction, and provide an implementable formula of the learning rate in the parameter search which minimizes an upper bound of the theoretical variance of the empirical batch mean. We analyze decay rates of the minimized upper bound towards the minimal estimator variance with respect to the predetermined computing budget, and provide convergence results as the computing budget increases progressively when the batch size is fixed. Numerical examples are provided to support theoretical findings and illustrate the effectiveness of the proposed batching procedure.

References

[1]

Christos Alexopoulos and David Goldsman. 2004. To batch or not to batch? ACM Transactions on Modeling and Computer Simulation 14, 1 (2004), 76–114.

Digital Library

[2]

Bouhari Arouna. 2004. Adaptative Monte Carlo method, a variance reduction technique. Monte Carlo Methods and Applications 10, 1 (2004), 1–24.

[3]

S. K. Au and J. L. Beck. 1999. A new adaptive importance sampling scheme for reliability calculations. Structural Safety 21, 2 (1999), 135–158.

[4]

Vivek S. Borkar. 1997. Stochastic approximation with two time scales. Systems and Control Letters 29, 5 (1997), 291–294.

Digital Library

[5]

Chiahon Chien, David Goldsman, and Benjamin Melamed. 1997. Large-sample results for batch means. Management Science 43, 9 (Sep.1997), 1288–1295.

[6]

Fang Chu and Marvin K. Nakayama. 2012. Confidence intervals for quantiles when applying variance-reduction techniques. ACM Transactions on Modeling and Computer Simulation 22, 2 (2012), 1–25.

Digital Library

[7]

Halim Damerdji. 1994. Strong consistency of the variance estimator in steady-state simulation output analysis. Mathematics of Operations Research 19, 2 (1994), 494–512.

Digital Library

[8]

John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12 (2011), 2121–2159.

Digital Library

[9]

Frank Grooteman. 2008. Adaptive radial-based importance sampling method for structural reliability. Structural Safety 30, 6 (2008), 533–542.

[10]

Frank Grooteman. 2011. An adaptive directional importance sampling method for structural reliability. Probabilistic Engineering Mechanics 26, 2 (2011), 134–41.

[11]

Shane G. Henderson and Burt Simon. 2004. Adaptive simulation using perfect control variates. Journal of Applied Probability 41, 3 (2004), 859–876. http://www.jstor.org/stable/4141358.

[12]

Guangxin Jiang, Chenglong Xu, and Michael C. Fu. 2016. On sample average approximation algorithms for determining the optimal importance sampling parameters in pricing financial derivatives on Lévy processes. Operations Research Letters 44, 1 (2016), 44–49.

Digital Library

[13]

Reiichiro Kawai. 2007. Adaptive Monte Carlo variance reduction with two-time-scale stochastic approximation. Monte Carlo Methods and Applications 13, 3 (2007).

[14]

Reiichiro Kawai. 2008. Adaptive Monte Carlo variance reduction for Lévy processes with two-time-scale stochastic approximation. Methodology and Computing in Applied Probability 10, 2 (2008), 199–223.

Digital Library

[15]

Reiichiro Kawai. 2009. Optimal importance sampling parameter search for Lévy processes via stochastic approximation. SIAM J. Numer. Anal. 47, 1 (Jan.2009), 293–307.

Digital Library

[16]

Reiichiro Kawai. 2010. Asymptotically optimal allocation of stratified sampling with adaptive variance reduction by strata. ACM Transactions on Modeling and Computer Simulation 20, 2 (2010), 1–17.

Digital Library

[17]

Reiichiro Kawai. 2017. Acceleration on adaptive importance sampling with sample average approximation. SIAM Journal on Scientific Computing 39, 4 (2017), A1586–A1615. arXiv:

Digital Library

[18]

Reiichiro Kawai. 2017. Adaptive importance sampling Monte Carlo simulation for general multivariate probability laws. J. Comput. Appl. Math. 319 (2017), 440–459.

Digital Library

[19]

Reiichiro Kawai. 2018. Optimizing adaptive importance sampling by stochastic approximation. SIAM Journal on Scientific Computing 40, 4 (2018), A2774–A2800. arXiv:

Digital Library

[20]

Reiichiro Kawai. 2020. Adaptive importance sampling and control variates. J. Math. Anal. Appl. 483, 1 (2020), 123608.

[21]

Sujin Kim and Shane G. Henderson. 2007. Adaptive control variates for finite-horizon simulation. Mathematics of Operations Research 32, 3 (Aug.2007), 508–527.

Digital Library

[22]

Harold J. Kushner and G. George Yin. 2003. Stochastic Approximation and Recursive Algorithms and Applications. Springer New York.

[23]

Bernard Lapeyre and Jérôme Lelong. 2011. A framework for adaptive Monte Carlo procedures. Monte Carlo Methods and Applications 17, 1 (2011), 1569–3961.

[24]

Vincent Lemaire and Gilles Pagès. 2010. Unconstrained recursive importance sampling. Annals of Applied Probability 20, 3 (2010), 1029–1067.

[25]

Theodoros Mamalis, Dusan Stipanovic, and Petros Voulgaris. 2021. Accelerated almost-sure convergence rates for nonconvex stochastic gradient descent using stochastic learning rates. arXiv:2110.12634 (2021).

[26]

Theodoros Mamalis, Dusan Stipanovic, and Petros Voulgaris. 2021. Stochastic learning rate optimization in the stochastic approximation and online learning settings. arXiv:2110.10710 (2021).

[27]

Marvin K. Nakayama. 2014. Confidence intervals for quantiles using sectioning when applying variance-reduction techniques. ACM Transactions on Modeling and Computer Simulation 24, 4 (2014), 1–21.

Digital Library

[28]

Yurii Nesterov. 1983. A method for unconstrained convex minimization problem with the rate of convergence \(O(1/k^2)\). Doklady ANSSSR (Translated as Soviet.Math.Docl.) 269 (1983), 543–547.

[29]

Atsushi Nitanda. 2016. Accelerated stochastic gradient descent for minimizing finite sums. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research), Arthur Gretton and Christian C. Robert (Eds.), Vol. 51. PMLR, Cadiz, Spain, 195–203. http://proceedings.mlr.press/v51/nitanda16.html.

[30]

Man-Suk Oh and James O. Berger. 1992. Adaptive importance sampling in Monte Carlo integration. Journal of Statistical Computation and Simulation 41, 3-4 (1992), 143–168.

[31]

Qiyun Pan, Eunshin Byon, Young Myoung Ko, and Henry Lam. 2020. Adaptive importance sampling for extreme quantile estimation with stochastic black box computer models. Naval Research Logistics 67, 7 (2020), 524–547.

[32]

B. T. Polyak and A. B. Juditsky. 1992. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization 30, 4 (1992), 838–855. arXiv:https://doi.org/10.1137/0330046

Digital Library

[33]

Ning Qian. 1999. On the momentum term in gradient descent learning algorithms. Neural Networks 12, 1 (1999), 145–151.

Digital Library

[34]

Sheldon M. Ross and Kyle Y. Lin. 2001. Applying variance reduction ideas in queuing simulations. Probability in the Engineering and Informational Sciences 15 (2001), 481–494.

Digital Library

[35]

Bruce Schmeiser. 1982. Batch size effects in the analysis of simulation output. Operations Research 30, 3 (1982), 556–568.

Digital Library

[36]

A. Shapiro, D. Dentcheva, and A. Ruszczyński. 2009. Lectures on Stochastic Programming. Society for Industrial and Applied Mathematics. arXiv:

[37]

Shing Chih Tsai, Jun Luo, Guangxin Jiang, and Wei Cheng Yeh. 2022. Adaptive fully sequential selection procedures with linear and nonlinear control variates. IISE Transactions. DOI:.

[38]

Xiaoxia Wu, Rachel Ward, and Leon Bottou. 2018. WNGrad: Learn the learning rate in gradient descent. arXiv:1803.02865 (2018).

Cited By

Song CKawai R(2024)Sampling and Change of Measure for Monte Carlo Integration on SimplicesJournal of Scientific Computing10.1007/s10915-024-02461-098:3Online publication date: 13-Feb-2024
https://dl.acm.org/doi/10.1007/s10915-024-02461-0
SONG C(2023)MONTE CARLO VARIANCE REDUCTION METHODS WITH APPLICATIONS IN STRUCTURAL RELIABILITY ANALYSISBulletin of the Australian Mathematical Society10.1017/S0004972723000874108:3(518-521)Online publication date: 31-Aug-2023
https://doi.org/10.1017/S0004972723000874

Index Terms

Batching Adaptive Variance Reduction
1. Mathematics of computing
  1. Probability and statistics
    1. Probabilistic algorithms

Recommendations

Asymptotically optimal allocation of stratified sampling with adaptive variance reduction by strata

To enhance efficiency in Monte Carlo simulations, we develop an adaptive stratified sampling algorithm for allocation of sampling effort within each stratum, in which an adaptive variance reduction technique is applied. Given the number of replications ...
Variance reduction for Markov chains with application to MCMC
Abstract
In this paper, we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive ...
Variance reduction of estimators arising from Metropolis---Hastings algorithms

The Metropolis---Hastings algorithm is one of the most basic and well-studied Markov chain Monte Carlo methods. It generates a Markov chain which has as limit distribution the target distribution by simulating observations from a different proposal ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Computer Simulation

ACM Transactions on Modeling and Computer Simulation Volume 33, Issue 1-2

April 2023

159 pages

ISSN:1049-3301

EISSN:1558-1195

DOI:10.1145/3572857

Editor:
Francesco Quaglia
University of Rome Tor Vergata, Italy

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 February 2023

Online AM: 01 December 2022

Accepted: 22 November 2022

Revised: 03 October 2022

Received: 09 December 2021

Published in TOMACS Volume 33, Issue 1-2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

JSPS Grants-in-Aid for Scientific Research

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
178
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)6

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Song CKawai R(2024)Sampling and Change of Measure for Monte Carlo Integration on SimplicesJournal of Scientific Computing10.1007/s10915-024-02461-098:3Online publication date: 13-Feb-2024
https://dl.acm.org/doi/10.1007/s10915-024-02461-0
SONG C(2023)MONTE CARLO VARIANCE REDUCTION METHODS WITH APPLICATIONS IN STRUCTURAL RELIABILITY ANALYSISBulletin of the Australian Mathematical Society10.1017/S0004972723000874108:3(518-521)Online publication date: 31-Aug-2023
https://doi.org/10.1017/S0004972723000874

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents