Skip to main content
Springer Nature Link
Log in

A study into the potential of GPUs for the efficient construction and evaluation of Kriging models

  • Original Article
  • Published:
Engineering with Computers Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

The surrogate modelling technique known as Kriging, and its various derivatives, requires an optimization process to effectively determine the model’s defining parameters. This optimization typically involves the maximisation of a likelihood function which requires the construction and inversion of a correlation matrix dependent on the selected modelling parameters. The construction of such models in high dimensions and with a large numbers of sample points can, therefore, be considerably expensive. Similarly, once such a model has been constructed the evaluation of the predictor, error and other related design and model improvement criteria can also be costly. The following paper investigates the potential for graphical processing units to be used to accelerate the evaluation of the Kriging likelihood, predictor and error functions. Five different Kriging formulations are considered including, ordinary, universal, non-stationary, gradient-enhanced and multi-fidelity Kriging. Other key contributions include the derivation of the adjoint of the likelihood function for a fully and partially gradient-enhanced Kriging model as well as the presentation of novel schemes to accelerate the likelihood optimization via a mixture of single and double precision calculations and by automatically selecting the best hardware to perform the evaluations on.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29

Similar content being viewed by others

Notes

  1. https://developer.nvidia.com/cuBLAS.

  2. http://icl.cs.utk.edu/magma/.

  3. http://www.culatools.com/.

References

  1. Akbariyeh A, Carrigan T, Dennis B, Chan W, Wang B, Lawrence K (2012) Application of gpu-based computing to large scale finite element analysis of three-dimensional structures. In: Proceedings of the 8th international conference on engineering computational technology

  2. Angelikopoulos P, Papadimitriou C (2012) Bayesian uncertainty quantification and propagation in molecular dynamics simulations. In: ECCOMAS 2012—European congress on computational methods in applied sciences and engineering

  3. Appleyard J, Drikakis D (2011) Higher-order cfd and interface tracking methods on highly-parallel mpi and gpu systems. Comput Fluids 46(1):101–105. doi:10.1016/j.compfluid.2010.10.019

    Article  MathSciNet  MATH  Google Scholar 

  4. Brandstetter A, Artusi A (2008) Radial basis function networks gpu-based implementation. IEEE Trans Neural Netw 19(12):2150–2154. doi:10.1109/TNN.2008.2003284

    Article  Google Scholar 

  5. Brooks C, Forrester A, Keane A, Shahpar S (2011) Multi-fidelity design optimisation of a transonic compressor rotor. In: 9th European turbomachinery conference. Istanbul

  6. Broyden C (1970) The convergence of a class of double-rank minimization algorithms. J Inst Math Appl 6(1):76–90

    Article  MathSciNet  MATH  Google Scholar 

  7. Challis V, Roberts A, Grotowski J (2014) High resolution topology optimization using graphics processing units (gpus). Struct Multidiscip Optim 49:315–325. doi:10.1007/s00158-013-0980-z

    Article  Google Scholar 

  8. Cheng T (2013) Accelerating universal Kriging interpolation algorithm using cuda-enable gpu. Comput Geosci 54:178–183. doi:10.1016/j.cageo.2012.11.013

    Article  Google Scholar 

  9. Cressie N (1993) Statistics for spatial data, probability and mathematical statistics. Wiley, New York

    Google Scholar 

  10. Czapinski M, Barnes S (2011) Tabu search with two approaches to parallel flowshop evaluation on cuda platform. J Parallel Distrib Comput 71(6):802–811. doi:10.1016/j.jpdc.2011.02.006

    Article  Google Scholar 

  11. Demir I, Westermann R (2013) Progressive high-quality response surfaces for visually guided sensitivity analysis. Comput Graph Forum 32(3):2130. doi:10.1111/cgf.12089

    Google Scholar 

  12. Dwight R, Han Z (2009) Efficient uncertainty quantification using gradient-enhanced kriging. In: 50th AIAA/ASME/ASCE/AHS/ASC Structures, structural dynamics, and materials conference. doi: Y10.2514/6.2009-2276

  13. Ferreiro AM, Garcfa JA, Lpez-Salas JG, Vzquez C (2013) An efficient implementation of parallel simulated annealing algorithm in gpus. J Glob Optim 57(3):863–890. doi:10.1007/s10898-012-9979-z

    Article  MathSciNet  Google Scholar 

  14. Forrester A, Bressloff N, Keane A (2006) Optimization using surrogate models and partially converged computational fluid dynamics simulations. Proc R Soc A 462(2071):2177–2204. doi:10.1098/rspa.2006.1679

    Article  MATH  Google Scholar 

  15. Forrester A, Keane A (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79. doi:10.1016/j.paerosci.2008.11.001

    Article  Google Scholar 

  16. Forrester A, Keane A, Bressloff N (2006) Design and analysis of “noisy” computer experiments. AIAA Journal 44(10):2331–2339

    Article  Google Scholar 

  17. Forrester A, Sóbester A, Keane A (2008) Engineering design via surrogate modelling. Wiley, London

    Book  MATH  Google Scholar 

  18. Fuentes M (2001) A high frequency Kriging approach for non-stationary environmental processes. Environmetrics 12:469–483

    Article  Google Scholar 

  19. Gibbs M (1997) Bayesian Gaussian processes for regression and classficiation. Ph.D. Dissertation, University of Cambridge

  20. Giles M (2008) Collected matrix derivative results for forward and reverse mode algorithmic differentiation. Lect Notes Comput Sci Eng 64:35–44. doi:10.1007/978-3-540-68942-3-4

    Article  MathSciNet  MATH  Google Scholar 

  21. Giles M, Pierce N (2000) An introduction to the adjoint approach to design. Flow Turbul Combust 65(3–4):393–415

    Article  MATH  Google Scholar 

  22. Griewank A (2000) Evaluating derivatives: principles and techniques of algorithmic differentiation. SIAM, Philadelphia

  23. Haas T (1990) Lognormal and moving window methods of estimating acid deposition. J Am Stat Assoc 85(412):950–963

    Article  Google Scholar 

  24. Hofmann J, Limmer S, Fey D (2013) Performance investigations of genetic algorithms on graphics cards. Swarm Evol Comput 12:33–47. doi:10.1016/j.swevo.2013.04.003

    Article  Google Scholar 

  25. Jameson A (1988) Aerodynamic design via control theory. J Sci Comput 3(3):233–260. doi:10.1007/BF01061285

    Article  MATH  Google Scholar 

  26. Jia X, Gu X, Graves YJ, Folkerts M, Jiang S (2011) Gpu-based fast Monte Carlo simulation for radiotherapy dose calculation. Phys Med Biol 56(22):7017–7031. doi:10.1088/0031-9155/56/22/002

    Article  Google Scholar 

  27. Jones D (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21(4):345–383. doi:10.1023/A:1012771025575

    Article  MathSciNet  MATH  Google Scholar 

  28. Jones D, Schonlau M, Welch W (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492. doi:10.1023/A:1008306431147

    Article  MathSciNet  MATH  Google Scholar 

  29. Kampolis I, Trompoukis X, Asouti V, Giannakoglou K (2010) Cfd-based analysis and two-level aerodynamic optimization on graphics processing units. Comput Methods Appl Mech Eng 199(9–12):712–722. doi:10.1016/j.cma.2009.11.001

    Article  MathSciNet  MATH  Google Scholar 

  30. Keane A (2006) Statistical improvement criteria for use in mulitobjective design optimization. AIAA J 44(4):879–891

    Article  Google Scholar 

  31. Kennedy M, O’Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87(1):1–13. doi:10.1093/biomet/87.1.1

    Article  MathSciNet  MATH  Google Scholar 

  32. Krige D (1951) A statistical approach to some basic mine valuation problems on the witwatersrand. J Chem Metallurigical Min Eng Soc S Afr 52(6):119–139. doi:10.2307/3006914

    Google Scholar 

  33. Laurenceau J, Sagaut P (2008) Building efficient response surfaces of aerodynamic functions with Kriging and cokriging. AIAA J 46(2):498–507. doi:10.2514/1.32308

    Article  Google Scholar 

  34. Leary S, Bhaskar A, Keane A (2004) Global approximation and optimisation using adjoint computational fluid dynamics codes. AIAA J 42(3):631–641

    Article  Google Scholar 

  35. Leithead W, Zhang Y (2007) \(O(N^2)\)-operation approximation of covariance matrix inverse in gaussian process regression based on quasi-newton BFGS method. Commun Stat Simul Comput 36(2):367–380

    Article  MathSciNet  MATH  Google Scholar 

  36. Li Q, Salman R, Kecman V (2010) An intelligent system for accelerating parallel SVM classification problems on large datasets using GPU. In: Proceedings of the 2010 10th international conference on intelligent systems design and applications, pp 1131–1135. doi:10.1109/ISDA.2010.5687033

  37. Liao Q, Wang J, Webster Y, Watson I (2009) Gpu accelerated support vector machines for mining high-throughput screening data. J Chem Inf Model 49(12):2718–2725

    Article  Google Scholar 

  38. Lophaven S, Nielsen H, Søndergaard J (2002) Dace: A matlab Kriging toolbox, imm-tr-2002-12. Tech. rep., Informatics and Mathematical Modelling, Technical University of Denmark

  39. Luo Z, Liu H (2005) Artificial neural network computation on graphic process unit. In: Proceedings of the international joint conference on neural networks. doi:10.1109/IJCNN.2005.1555903

  40. Mishra V, Suresh K (2011) GPU-friendly preconditioners for efficient 3-d finite element analysis of thin structures. In: Proceedings of the ASME 2011 international design engineering technical conferences and computers and information in engineering conference

  41. Paciorek C, Schervish M (2004) Nonstationary covariance functions for Gaussian process regression. Adv Neural Inf Process Syst 16:273–280

    Google Scholar 

  42. Pintore A, Holmes C (2004) Spatially adaptive non-stationary covariance functions via spatially adaptive spectra. Technical Report, University of Oxford, U.K

  43. Gutiérrez de Ravé E, Jiménez-Hornero F, Ariza-Villaverde A, Gómez-Lpez J (2014) Using general-purpose computing on graphics processing units (GPGPU) to accelerate the ordinary Kriging algorithm. Comput Geosci 64:1–6. doi:10.1016/j.cageo.2012.11.013

  44. Rumpfkeil M (2013) Optimization under uncertainty using gradients, hessians and surrogate models. AIAA J 51(2):444–451. doi:10.2514/1.J051847

    Article  Google Scholar 

  45. Rupesh S, Deb K (2013) An evolutionary based bayesian design optimization approach under incomplete information. Eng Optim 45(2). doi:10.1080/0305215X.2012.661730

  46. Sacks J, Welch W, Mitchell T, Wynn H (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–435. doi:10.2307/2245858

    Article  MathSciNet  MATH  Google Scholar 

  47. Sampson P, Guttorp P (1992) Nonparametric estimation of nonstationary spatial covariance structure. J Am Stat Assoc 87(417):108–119

    Article  Google Scholar 

  48. Sankaran R, Grout R (2012) Accelerating the computation of detailed chemical reaction kinetics for simulating combustion of complex fuels. In: 50th AIAA aerospace sciences meeting including the new horizons forum and aerospace exposition. doi:Y10.2514/6.2012-720

  49. Simpson T, Peplinski J, Kock P, Allen J (2001) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17(2):129–150. doi:10.1007/PL00007198

    Article  MATH  Google Scholar 

  50. Toal D, Bressloff N, Keane A (2008) Kriging hyperparameter tuning strategies. AIAA J 46(5):1240–1252. doi:10.2514/1.34822

    Article  Google Scholar 

  51. Toal D, Bressloff N, Keane A, Holden C (2011) The development of a hybridized particle swarm for Kriging hyperparameter tuning. Eng Optim. doi:10.1080/0305215X.2010.508524. (Accepted for Publication)

  52. Toal D, Forrester A, Bressloff N, Keane A, Holden C (2009) An adjoint for likelihood maximization. Proc R Soc A 465(2111):3267–3287. doi:10.1098/rspa.2009.0096

    Article  MathSciNet  MATH  Google Scholar 

  53. Toal D, Keane A (2011) Efficient multi-point aerodynamic design optimization via co-Kriging. J Aircr 48(5):1685–1695. doi:10.2514/1.C031342

    Article  Google Scholar 

  54. Toal D, Keane A (2011) Non-stationary Kriging for design optimization. Eng Optim. doi:10.1080/0305215X.2011.607816

  55. Uchida A, Ito Y, Nakano K (2013) Accelerating ant colony optimisation for the travelling salesman problem on the GPU. Int J Parallel Emergent Distrib Syst. doi:10.1080/17445760.2013.842568

  56. Viana F, Simpson T, Balabanov V, Toropov V (2014) Metamodeling in multidisciplinary design optimization: how far have we come. AIAA J. doi:10.2514/1.J052375

  57. Voutchkov I, Keane A, Fox R (2006) Robust structural design of a simplified jet engine model using multiobjective optimization. In: 11th AIAA/ISSMO Multidisciplinary analysis and optimization conference. Portsmouth

  58. Wachowiak M, Lambe Foster AE (2012) GPU-based asynchronous global optimization with particle swarm. In: High performance computing symposium 2012

  59. Wang G, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. ASME J Mech Des 129:370–380. doi:10.1115/1.2429697

    Article  Google Scholar 

  60. Xiong Y, Chen W, Apley D, Ding X (2007) A non-stationary covariance-based Kriging method for metamodelling in engineering design. Int J Numer Methods Eng 71(6):733–756

    Article  MATH  Google Scholar 

  61. Zbierski M (2011) A simulated annealing algorithm for GPU clusters. In: 9th International conference on parallel processing and applied mathematics

  62. Zegard T, Paulino G (2013) Toward GPU accelerated topology optimization on unstructured meshes. Struct Multidiscip Optim 48:473–485. doi:10.1007/s00158-013-0920-y

    Article  Google Scholar 

  63. Zhang Y, Leithead W (2005) Exploiting hessian matrix and trust-region algorithm in hyperparameters estimation of Gaussian process. Appl Math Comput 171(2):1264–1281

    MathSciNet  MATH  Google Scholar 

  64. Zhongwen L, Hongzhi L, Zhengping Y, Xincai W (2005) Self-organizing maps computing on graphic process unit. In: Proceedings—13th European symposium on artificial neural networks

Download references

Acknowledgments

The support of Rolls-Royce in carrying out this work is greatly appreciated.

Author information

Authors and Affiliations

  1. Faculty of Engineering and the Environment, University of Southampton, Southampton, SO16 7QF, UK

    David J. J. Toal

Authors
  1. David J. J. Toal
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to David J. J. Toal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Toal, D.J.J. A study into the potential of GPUs for the efficient construction and evaluation of Kriging models. Engineering with Computers 32, 377–404 (2016). https://doi.org/10.1007/s00366-015-0421-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00366-015-0421-2

Keywords