research-article

Adaptive Newton-based multivariate smoothed functional algorithms for simulation optimization

Author:

Shalabh BhatnagarAuthors Info & Claims

ACM Transactions on Modeling and Computer Simulation (TOMACS), Volume 18, Issue 1

Article No.: 2, Pages 1 - 35

https://doi.org/10.1145/1315575.1315577

Published: 12 December 2007 Publication History

Abstract

In this article, we present three smoothed functional (SF) algorithms for simulation optimization. While one of these estimates only the gradient by using a finite difference approximation with two parallel simulations, the other two are adaptive Newton-based stochastic approximation algorithms that estimate both the gradient and Hessian. One of the Newton-based algorithms uses only one simulation and has a one-sided estimate in both the gradient and Hessian, while the other uses two-sided estimates in both quantities and requires two simulations. For obtaining gradient and Hessian estimates, we perturb each parameter component randomly using independent and identically distributed (i.i.d) Gaussian random variates.

The earlier SF algorithms in the literature only estimate the gradient of the objective function. Using similar techniques, we derive two unbiased SF-based estimators for the Hessian and develop suitable three-timescale stochastic approximation procedures for simulation optimization. We present a detailed convergence analysis of our algorithms and show numerical experiments with parameters of dimension 50 on a setting involving a network of M/G/1 queues with feedback. We compare the performance of our algorithms with related algorithms in the literature. While our two-simulation Newton-based algorithm shows the best results overall, our one-simulation algorithm shows better performance compared to other one-simulation algorithms.

References

[1]

Bertsekas, D. P. 1999. Nonlinear Programming. Athena Scientific, Belmont.

[2]

Bertsekas, D. P. and Tsitsiklis, J. N. 1989. Parallel and Distributed Computation. Prentice Hall, Englewood Cliffs, NJ.

Digital Library

[3]

Bhatnagar, S. 2005. Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization. ACM Trans. Model. Comput. Sim. 15, 1, 74--107.

Digital Library

[4]

Bhatnagar, S. and Borkar, V. S. 1998. A two time scale stochastic approximation scheme for simulation based parametric optimization. Prob. Eng. Info. Sci. 12, 519--531.

[5]

Bhatnagar, S. and Borkar, V. S. 2003. Multiscale chaotic SPSA and smoothed functional algorithms for simulation optimization. Simulation 79, 10, 568--580.

[6]

Bhatnagar, S., Fu, M. C., Marcus, S. I., and Bhatnagar, S. 2001. Two timescale algorithms for simulation optimization of hidden Markov models. IIE Trans. 33, 3, 245--258.

[7]

Bhatnagar, S., Fu, M. C., Marcus, S. I., and Wang, I.-J. 2003. Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences. ACM Trans Model. Comput. Sim. 13, 2, 180--209.

Digital Library

[8]

Borkar, V. S. 1995. Probability Theory: An Advanced Course. Springer Verlag, New York, NY.

[9]

Brandiere, O. 1998. Some pathological traps for stochastic approximation. SIAM J. Contr. Optim. 36, 1293--1314.

Digital Library

[10]

Cassandras, C. G. 1993. Discrete Event Systems: Modeling and Performance Analysis. Aksen Associates, Boston, MA.

[11]

Chin, D. C. 1997. Comparative study of stochastic algorithms for system optimization based on gradient approximation. IEEE Trans. Sys. Man. Cyber. Part B 27, 2, 244--249.

Digital Library

[12]

Fabian, V. 1971. Stochastic approximation. In Optimizing Methods in Statistics, J.J.Rustagi, Ed. Academic Press, New York, NY. 439--470.

[13]

Hirsch, M. W. 1989. Convergent activation dynamics in continuous time networks. Neur. Netw. 2, 331--349.

Digital Library

[14]

Ho, Y. C. and Cao, X. R. 1991. Perturbation Analysis of Discrete Event Dynamical Systems. Kluwer, Boston, MA.

[15]

Katkovnik, V. Y. and Kulchitsky, Y. 1972. Convergence of a class of random search algorithms. Automat. Remote Cont. 8, 1321--1326.

[16]

Kiefer, E. and Wolfowitz, J. 1952. Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23, 462--466.

[17]

Kushner, H. J. and Clark, D. S. 1978. Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer Verlag, New York, NY.

[18]

Kushner, H. J. and Yin, G. G. 1997. Stochastic Approximation Algorithms and Applications. Springer Verlag, New York, NY.

[19]

Lasalle, J. P. and Lefschetz, S. 1961. Stability by Liapunov's Direct Method with Applications. Academic Press, New York, NY.

[20]

L'Ecuyer, P. and Glynn, P. W. 1994. Stochastic optimization by simulation: Convergence proofs for the GI/G/1 queue in steady-state. Manage. Sci. 40, 11, 1562--1578.

Digital Library

[21]

Pemantle, R. 1990. Nonconvergence to unstable points in urn models and stochastic approximations. Ann. Prob. 18, 698--712.

[22]

Robbins, H. and Monro, S. 1951. A stochastic approximation method. Ann. Math. Statist. 22, 400--407.

[23]

Rubinstein, R. Y. 1981. Simulation and the Monte Carlo Method. Wiley, New York, NY.

Digital Library

[24]

Ruppert, D. 1985. A Newton-Raphson version of the multivariate Robbins-Monro procedure. Ann. Statist. 13, 236--245.

[25]

Spall, J. C. 1992. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Auto. Contr. 37, 3, 332--341.

[26]

Spall, J. C. 1997. A one-measurement form of simultaneous perturbation stochastic approximation. Automatica 33, 109--112.

Digital Library

[27]

Spall, J. C. 2000. Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Trans. Auto. Contr. 45, 1839--1853.

[28]

Spall, J. C. 2006. Convergence analysis for feedback- and weighting-based Jacobian estimates in the adaptive simultaneous perturbation algorithm. In Proceedings of the IEEE Conference on Decision and Control (San Diego, CA, Dec. 13--15). 5669--5674.

[29]

Styblinski, M. A. and Opalski, L. J. 1986. Algorithms and software tools for IC yield optimization based on fundamental fabrication parameters. IEEE Trans. Comput. Aid. Des. CAD-5, 1, 79--89.

Digital Library

[30]

Styblinski, M. A. and Tang, T.-S. 1990. Experiments in nonconvex optimization: Stochastic approximation with function smoothing and simulated annealing. Neur. Netw. 3, 467--483.

Digital Library

[31]

Zhu, X. and Spall, J. C. 2002. A modified second-order SPSA optimization algorithm for finite samples. Int. J. Adapt. Contr. Sign. Process. 16, 397--409.

Digital Library

Cited By

Mandal LDiddigi RBhatnagar S(2024)Variance-Reduced Deep Actor–Critic With an Optimally Subsampled Actor RecursionIEEE Transactions on Artificial Intelligence10.1109/TAI.2024.33791095:7(3607-3623)Online publication date: Jul-2024
https://doi.org/10.1109/TAI.2024.3379109
Mondal AL.A. PBhatnagar S(2024)Truncated Cauchy random perturbations for smoothed functional-based stochastic optimizationAutomatica10.1016/j.automatica.2024.111528162(111528)Online publication date: Apr-2024
https://doi.org/10.1016/j.automatica.2024.111528
Bhatnagar SL.A. P(2023)Generalized Simultaneous Perturbation Stochastic Approximation with Reduced Estimator Bias2023 57th Annual Conference on Information Sciences and Systems (CISS)10.1109/CISS56502.2023.10089720(1-6)Online publication date: 22-Mar-2023
https://doi.org/10.1109/CISS56502.2023.10089720
Show More Cited By

Index Terms

Adaptive Newton-based multivariate smoothed functional algorithms for simulation optimization
1. Computing methodologies
  1. Modeling and simulation
    1. Simulation theory
2. Mathematics of computing
  1. Probability and statistics

Recommendations

Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization

We develop in this article, four adaptive three-timescale stochastic approximation algorithms for simulation optimization that estimate both the gradient and Hessian of average cost at each update epoch. These algorithms use four, three, two, and one ...
Quasi-Newton smoothed functional algorithms for unconstrained and constrained simulation optimization

We propose a multi-time scale quasi-Newton based smoothed functional (QN-SF) algorithm for stochastic optimization both with and without inequality constraints. The algorithm combines the smoothed functional (SF) scheme for estimating the gradient with ...
Newton-based stochastic optimization using q -Gaussian smoothed functional algorithms

We present the first q -Gaussian smoothed functional (SF) estimator of the Hessian and the first Newton-based stochastic optimization algorithm that estimates both the Hessian and the gradient of the objective function using q -Gaussian perturbations. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Computer Simulation

ACM Transactions on Modeling and Computer Simulation Volume 18, Issue 1

December 2007

108 pages

ISSN:1049-3301

EISSN:1558-1195

DOI:10.1145/1315575

Issue’s Table of Contents

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 December 2007

Accepted: 01 May 2007

Revised: 01 January 2007

Received: 01 March 2006

Published in TOMACS Volume 18, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Department of Science and Technology, Ministry of Science and Technology

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

55
Total Citations
View Citations
560
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mandal LDiddigi RBhatnagar S(2024)Variance-Reduced Deep Actor–Critic With an Optimally Subsampled Actor RecursionIEEE Transactions on Artificial Intelligence10.1109/TAI.2024.33791095:7(3607-3623)Online publication date: Jul-2024
https://doi.org/10.1109/TAI.2024.3379109
Mondal AL.A. PBhatnagar S(2024)Truncated Cauchy random perturbations for smoothed functional-based stochastic optimizationAutomatica10.1016/j.automatica.2024.111528162(111528)Online publication date: Apr-2024
https://doi.org/10.1016/j.automatica.2024.111528
Bhatnagar SL.A. P(2023)Generalized Simultaneous Perturbation Stochastic Approximation with Reduced Estimator Bias2023 57th Annual Conference on Information Sciences and Systems (CISS)10.1109/CISS56502.2023.10089720(1-6)Online publication date: 22-Mar-2023
https://doi.org/10.1109/CISS56502.2023.10089720
Mok RAhmad M(2022)Fast and optimal tuning of fractional order PID controller for AVR system based on memorizable-smoothed functional algorithmEngineering Science and Technology, an International Journal10.1016/j.jestch.2022.10126435(101264)Online publication date: Nov-2022
https://doi.org/10.1016/j.jestch.2022.101264
Sharma ALakshmanan KGupta RGupta A(2021)Multi-Time Scale Smoothed Functional With Nesterov’s AccelerationIEEE Access10.1109/ACCESS.2021.31037679(113489-113499)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3103767
L. A. PBhatnagar SBhavsar NFu MMarcus S(2020)Random Directions Stochastic Approximation With Deterministic PerturbationsIEEE Transactions on Automatic Control10.1109/TAC.2019.293082165:6(2450-2465)Online publication date: Jun-2020
https://doi.org/10.1109/TAC.2019.2930821
Bhatnagar SPatel SKarmeshu (2018)A stochastic approximation approach to active queue managementTelecommunications Systems10.1007/s11235-017-0377-168:1(89-104)Online publication date: 1-May-2018
https://dl.acm.org/doi/10.1007/s11235-017-0377-1
L. A. PBhatnagar SFu MMarcus S(2017)Adaptive System Optimization Using Random Directions Stochastic ApproximationIEEE Transactions on Automatic Control10.1109/TAC.2016.260064362:5(2223-2238)Online publication date: May-2017
https://doi.org/10.1109/TAC.2016.2600643
Lakshmanan KBhatnagar S(2017)Quasi-Newton smoothed functional algorithms for unconstrained and constrained simulation optimizationComputational Optimization and Applications10.1007/s10589-016-9875-466:3(533-556)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1007/s10589-016-9875-4
Li LPeng HChen XCheng JGao D(2016)Visualization of boundaries in CT volumetric data sets using dynamic M - | ∇ f | histogramComputers in Biology and Medicine10.1016/j.compbiomed.2015.10.01868:C(109-120)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1016/j.compbiomed.2015.10.018
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents