skip to main content
research-article

Gradient Extrapolated Stochastic Kriging

Published: 18 November 2014 Publication History

Abstract

We introduce an approach for enhancing stochastic kriging in the setting where additional direct gradient information is available (e.g., provided by techniques such as perturbation analysis or the likelihood ratio method). The new approach, called gradient extrapolated stochastic kriging (GESK), incorporates direct gradient estimates by extrapolating additional responses. For two simplified settings, we show that GESK reduces mean squared error (MSE) compared to stochastic kriging under certain conditions on step sizes. Since extrapolation step sizes are crucial to the performance of the GESK model, we propose two different approaches to determine the step sizes: maximizing penalized likelihood and minimizing integrated mean squared error. Numerical experiments are conducted to illustrate the performance of the GESK model and to compare it with alternative approaches.

References

[1]
J. J. Alonso and H. S. Chung. 2002. Using gradients to construct cokriging approximation models for high-dimensional design optimization problems. In 40th AIAA Aerospace Sciences Meeting and Exhibit, AIAA. 2002--0317.
[2]
B. E. Ankenman, B. L. Nelson, and J. Staum. 2010. Stochastic kriging for simulation metamodeling. Operations research 58, 2 (March 2010), 371--382.
[3]
R. R. Barton. 2009. Simulation optimization using metamodels. In Proceedings of the 2009 Winter Simulation Conference, M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin, and R. G. Ingalls (Eds.). Institute of Electrical and Electronics Engineers, Inc, Piscataway, New Jersey, 230--238.
[4]
R. R. Barton and M. Meckesheimer. 2006. Metamodel-based simulation optimization. In Handbooks in Operations Research and Management Science: Simulation, S. G. Henderson and B. L. Nelson (Eds.). Elsevier, 535--574.
[5]
X. Chen, B. E. Ankenman, and B. L. Nelson. 2012. The effects of common random numbers on stochastic kriging metamodels. ACM Transactions on Modeling and Computer Simulation 22, 2 (March 2012), Article 7, 20 pages.
[6]
X. Chen, B. E. Ankenman, and B. L. Nelson. 2013. Enhancing stochastic kriging metamodels with gradient estimators. Operations Research 61, 2 (2013), 512--528.
[7]
T. Chu, J. Zhu, and H. Wang. 2011. Penalized maximum likelihood estimation and variable selection in geostatistics. Annals of Statistics 39, 5 (2011), 2607--2625.
[8]
M. C. Fu. 2008. What you should know about simulation and derivatives. Naval Research Logistics 55, 8 (2008), 723--736.
[9]
M. C. Fu and H. Qu. 2014. Regression models augumented with direct stochastic gradient estimators. INFORMS Journal on Computing 26, 3, 484--499.
[10]
P. Glasserman. 1991. Gradient Estimation via Perturbation Analysis. Kluwer Academic, Boston, MA.
[11]
Y. C. Ho, L. Shi, L. Dai, and W. Gong. 1992. Optimizing discrete event dynamic systems via the gradient surface method. Discrete Event Dynamic Systems: Theory and Applications 2 (Jan. 1992), 99--120.
[12]
J. P. C. Kleijnen and W. C. M. van Beers. 2005. Robustness of kriging when interpolating in random simulation with heterogeneous variances: Some experiments. European Journal of Operational Research 165, 3 (2005), 826--834.
[13]
P. L’Ecuyer. 1990. A unified view of the IPA, SF, and LR gradient estimation techniques. Management Science 36 (1990), 1364--1383.
[14]
R. Li and A. Sudjianto. 2005. Analysis of computer experiments using penalized likelihood in Gaussian kriging models. Technometrics 47, 2 (May 2005), 111--121.
[15]
W. Liu. 2003. Development of Gradient-enhanced Kriging Approximations for Multidisciplinary Design Optimization. Ph.D. Dissertation. University of Notre Dame.
[16]
H. Qu and M. C. Fu. 2012. On direct gradient enhanced simulation metamodels. In Proceedings of the 2012 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, Article 43, 12 pages.
[17]
R. Y. Rubinstein and A. Shapiro. 1993. Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method. John Wiley & Sons.
[18]
T. J. Santner, B. Williams, and W. Notz. 2003. The Design and Analysis of Computer Experiments. Springer-Verlag.
[19]
J. Staum. 2009. Better simulation metamodeling: The why, what, and how of stochastic kriging. In Proceedings of the 2009 Winter Simulation Conference, M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin, and R. G. Ingalls (Eds.). Institute of Electrical and Electronics Engineers, Piscataway, NJ, 119--133.
[20]
W. C. M. van Beers and J. P. C. Kleijnen. 2008. Customized sequential designs for random simulation experiments: Kriging metamodeling and bootstrapping. European Journal of Operational Research 186, 3 (2008), 1099--1113.
[21]
W. Xie, B. L. Nelson, and J. Staum. 2010. The influence of correlation functions on stochastic kriging metamodels. In Proceedings of the 2010 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, Piscataway, NJ, 1067--1078.
[22]
F. Zhang and Q. Zhang. 2006. Eigenvalue inequalities for matrix product. IEEE Transactions on Automatic Control 51, 9 (Sept. 2006), 1506--1509.

Cited By

View all

Index Terms

  1. Gradient Extrapolated Stochastic Kriging

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Modeling and Computer Simulation
    ACM Transactions on Modeling and Computer Simulation  Volume 24, Issue 4
    Special Issue on Emerging Methodologies and Applications
    August 2014
    132 pages
    ISSN:1049-3301
    EISSN:1558-1195
    DOI:10.1145/2617568
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 November 2014
    Accepted: 01 July 2014
    Revised: 01 July 2014
    Received: 01 February 2013
    Published in TOMACS Volume 24, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Stochastic kriging
    2. response surface
    3. simulation
    4. stochastic gradients

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)15
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Real-Time Derivative Pricing and Hedging with Consistent MetamodelsINFORMS Journal on Computing10.1287/ijoc.2023.029236:5(1168-1189)Online publication date: Sep-2024
    • (2023)Gradient-Based Algorithms for Convex Discrete Optimization via SimulationOperations Research10.1287/opre.2022.229571:5(1815-1834)Online publication date: 1-Sep-2023
    • (2021)Flat chance!Proceedings of the Winter Simulation Conference10.5555/3522802.3523013(1-12)Online publication date: 13-Dec-2021
    • (2021)Neural network-assisted simulation optimization with covariatesProceedings of the Winter Simulation Conference10.5555/3522802.3522817(1-12)Online publication date: 13-Dec-2021
    • (2021)Surrogate-Based Simulation OptimizationTutorials in Operations Research: Emerging Optimization Methods and Modeling Techniques with Applications10.1287/educ.2021.0225(287-311)Online publication date: 18-Oct-2021
    • (2021)Neural Network-Assisted Simulation Optimization with Covariates2021 Winter Simulation Conference (WSC)10.1109/WSC52266.2021.9715516(1-12)Online publication date: 12-Dec-2021
    • (2021)Flat Chance! Using Stochastic Gradient Estimators to Assess Plausible Optimality for Convex Functions2021 Winter Simulation Conference (WSC)10.1109/WSC52266.2021.9715288(1-12)Online publication date: 12-Dec-2021
    • (2021)Optimal Computing Budget Allocation for regression with gradient informationAutomatica10.1016/j.automatica.2021.109927134(109927)Online publication date: Dec-2021
    • (2020)Faster KrigingOperations Research10.1287/opre.2019.186068:1(233-249)Online publication date: 1-Jan-2020
    • (2020)Stochastic Oblique Impact on Composite Laminates: A Concise Review and Characterization of the Essence of Hybrid Machine Learning AlgorithmsArchives of Computational Methods in Engineering10.1007/s11831-020-09438-wOnline publication date: 27-Aug-2020
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media