Hostname: page-component-8448b6f56d-dnltx Total loading time: 0 Render date: 2024-04-23T12:59:44.297Z Has data issue: false hasContentIssue false
  • English
  • Français

Ensemble of surrogates and cross-validation for rapid and accurate predictions using small data sets

Published online by Cambridge University Press:  18 October 2019

Reza Alizadeh ,
Liangyue Jia ,
Anand Balu Nellippallil ,
Guoxin Wang Open the ORCID record for Guoxin Wang [Opens in a new window] ,
Jia Hao ,
Janet K. Allen  and
Farrokh Mistree
Reza Alizadeh
Affiliation:
Systems Realization Laboratory, University of Oklahoma, Norman, OK, USA
Liangyue Jia
Affiliation:
School of Mechanical Engineering, Institute for Industrial Engineering, Beijing Institute of Technology, Beijing, China
Anand Balu Nellippallil
Affiliation:
Center for Advanced Vehicular Systems, School of Mechanical Engineering, Mississippi State University, Starkville, MS, USA
Guoxin Wang*
Affiliation:
School of Mechanical Engineering, Institute for Industrial Engineering, Beijing Institute of Technology, Beijing, China
Jia Hao
Affiliation:
School of Mechanical Engineering, Institute for Industrial Engineering, Beijing Institute of Technology, Beijing, China
Janet K. Allen
Affiliation:
Systems Realization Laboratory, University of Oklahoma, Norman, OK, USA
Farrokh Mistree
Affiliation:
Systems Realization Laboratory, University of Oklahoma, Norman, OK, USA
*
Author for correspondence: Guoxin Wang, E-mail: wangguoxin@bit.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

In engineering design, surrogate models are often used instead of costly computer simulations. Typically, a single surrogate model is selected based on the previous experience. We observe, based on an analysis of the published literature, that fitting an ensemble of surrogates (EoS) based on cross-validation errors is more accurate but requires more computational time. In this paper, we propose a method to build an EoS that is both accurate and less computationally expensive. In the proposed method, the EoS is a weighted average surrogate of response surface models, kriging, and radial basis functions based on overall cross-validation error. We demonstrate that created EoS is accurate than individual surrogates even when fewer data points are used, so computationally efficient with relatively insensitive predictions. We demonstrate the use of an EoS using hot rod rolling as an example. Finally, we include a rule-based template which can be used for other problems with similar requirements, for example, the computational time, required accuracy, and the size of the data.

Keywords

Ensemble of surrogateskrigingresponse surface modelingsmall data setssurrogate models
Type
Research Article
Information
AI EDAM , Volume 33 , Special Issue 4: Intelligent Interaction Design , November 2019 , pp. 484 - 501
Copyright
Copyright © Cambridge University Press 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Acar, E (2015) Effect of error metrics on optimum weight factor selection for ensemble of metamodels. Expert Systems with Applications 42, 2703–2709. doi:10.1016/j.eswa.2014.11.020CrossRefGoogle Scholar
Acar, E and Rais-Rohani, M (2009) Ensemble of metamodels with optimized weight factors. Structural and Multidisciplinary Optimization 37, 279–294.CrossRefGoogle Scholar
Adhav, R, Samad, A and Kenyery, F (2015) Design optimization of electric centrifugal pump by multiple surrogate models. Paper Presented at the SPE Middle East Oil & Gas Show and Conference, Manama, Bahrain.CrossRefGoogle Scholar
Alizadeh, R, Lund, PD, Beynaghi, A, Abolghasemi, M and Maknoon, R (2016) An integrated scenario-based robust planning approach for foresight and strategic management with application to energy industry. Technological Forecasting and Social Change 104, 162–171. doi:10.1016/j.techfore.2015.11.030CrossRefGoogle Scholar
Alizadeh, R, Allen, JK and Mistree, F (2019) Managing computational complexity using surrogate models: a critical review. Research in Engineering Design. Under Review.Google Scholar
Arias-Montano, A, Coello, CAC and Mezura-Montes, E (2012) Multi-objective airfoil shape optimization using a multiple-surrogate approach. Paper Presented at the IEEE Congress on Evolutionary Computation (CEC), Brisbane, Australia.CrossRefGoogle Scholar
Babaei, M and Pan, I (2016) Performance comparison of several response surface surrogate models and ensemble methods for water injection optimization under uncertainty. Computers & Geosciences 91, 19–32. doi:10.1016/j.cageo.2016.02.022CrossRefGoogle Scholar
Badhurshah, R and Samad, A (2015) Multiple surrogate based optimization of a bidirectional impulse turbine for wave energy conversion. Renewable Energy 74, 749–760. doi:10.1016/j.renene.2014.09.001CrossRefGoogle Scholar
Basudhar, A (2012) Selection of anisotropic kernel parameters using multiple surrogate information. Paper Presented at the 12th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference and 14th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Indianapolis, Indiana.Google Scholar
Bellary, SAI and Samad, A (2017) An alternative approach to surrogate averaging for a centrifugal impeller shape optimisation. International Journal of Computer Aided Engineering and Technology 9, 62–83. doi:10.1504/ijcaet.2017.080769CrossRefGoogle Scholar
Bellary, SAI, Adhav, R, Siddique, MH, Chon, B-H, Kenyery, F and Samad, A (2016) Application of computational fluid dynamics and surrogate-coupled evolutionary computing to enhance centrifugal-pump performance. Engineering Applications of Computational Fluid Mechanics 10, 171–181. doi:10.1080/19942060.2015.1128359CrossRefGoogle Scholar
Bellucci, JP and Bauer, KW Jr (2017) A Taylor series approach to the robust parameter design of computer simulations using kriging and radial basis function neural networks. International Journal of Quality Engineering and Technology 6, 137–160.CrossRefGoogle Scholar
Beynaghi, A, Moztarzadeh, F, Shahmardan, A, Alizadeh, R, Salimi, J and Mozafari, M (2016) Makespan minimization for batching work and rework process on a single facility with an aging effect: a hybrid meta-heuristic algorithm for sustainable production management. Journal of Intelligent Manufacturing 30, 33–45. doi:10.1007/s10845-016-1223-0CrossRefGoogle Scholar
Bhat, S, Viana, FAC, Lind, R and Haftka, R (2010) Control-oriented design using H-infinity synthesis and multiple surrogates. Paper Presented at the 51st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Orlando, Florida.CrossRefGoogle Scholar
Bhattacharjee, KS, Singh, HK, Ray, T and Branke, J (2016) Multiple surrogate assisted multiobjective optimization using improved pre-selection. Paper Presented at the 2016 IEEE Congress on Evolutionary Computation (CEC).CrossRefGoogle Scholar
Bhattacharjee, KS, Singh, HK and Tapabrata, R (2018) Multiple surrogate assisted many-objective optimization for computationally expensive engineering design. ASME Journal of Mechanical Design 140, 051403.CrossRefGoogle Scholar
Bishop, CM (1995) Neural Networks for Pattern Recognition. Birmingham, UK: Department of Computer Science and Applied Mathematics Aston University, Oxford University Press.Google Scholar
Bodnar, R and Hansen, S (1994) Effects of austenite grain size and cooling rate on Widmanstätten ferrite formation in low-alloy steels. Metallurgical and Materials Transactions A 25, 665–675.CrossRefGoogle Scholar
Chaudhuri, A and Haftka, RT (2014) Efficient global optimization with adaptive target setting. AIAA Journal 52, 1573–1578. doi:10.2514/1.J052930Google Scholar
Chaudhuri, A, Haftka, RT, Ifju, P, Chang, K, Tyler, C and Schmitz, T (2015) Experimental flapping wing optimization and uncertainty quantification using limited samples. Structural and Multidisciplinary Optimization 51, 957–970. doi:10.1007/s00158-014-1184-xCrossRefGoogle Scholar
Ezhilsabareesh, K, Rhee, SH and Samad, A (2018) Shape optimization of a bidirectional impulse turbine via surrogate models. Engineering Applications of Computational Fluid Mechanics 12, 1–12. doi:10.1080/19942060.2017.1330709CrossRefGoogle Scholar
Goel, T, Haftka, RT, Shyy, W and Queipo, NV (2006) Ensemble of surrogates. Structural and Multidisciplinary Optimization 33, 199–216. doi:10.1007/s00158-006-0051-9CrossRefGoogle Scholar
Goel, T, Haftka, RT, Shyy, W and Queipo, NV (2007) Ensemble of surrogates. Structural and Multidisciplinary Optimization 33, 199–216. doi:10.1007/s00158-006-0051-9CrossRefGoogle Scholar
Habib, A, Kumar Singh, H and Ray, T (2017) A multiple surrogate assisted evolutionary algorithm for optimization involving iterative solvers. Engineering Optimization 49, 1–20. doi:10.1080/0305215X.2017.1401068Google Scholar
Jägle, E (2007) Modelling of Microstructural Banding During Transformations in Steel (Master of Philosophy Dissertation). University of Cambridge, Cambridge, UK.Google Scholar
Jones, S and Bhadeshia, H (1997) Kinetics of the simultaneous decomposition of austenite into several transformation products. Acta Materialia 45, 2911–2920.CrossRefGoogle Scholar
Jones, SJ and Bhadeshia, HKDH (2017) Program STRUCTURE on the Materials Algorithm Project Web Site. Available at http://www.msm.cam.ac.uk/map/steel/programs/structure.html.Google Scholar
Kaleibari, SS, Beiragh, RG, Alizadeh, R and Solimanpur, M (2016) A framework for performance evaluation of energy supply chain by a compatible network data envelopment analysis model. Scientia Iranica. Transaction E, Industrial Engineering 23, 1904–1917.Google Scholar
Khademi, A, Ghorbani Renani, N, Mofarrahi, M, Rangraz Jeddi, A and Mohd Yusof, N (2013) The best location for speed bump installation using experimental design methodology. Promet - Traffic and Transportation 25, 565–574. doi:10.7307/ptt.v25i6.1188CrossRefGoogle Scholar
Kleijnen, JP (2017) Regression and Kriging metamodels with their experimental designs in simulation: a review. European Journal of Operational Research 256, 1–16.CrossRefGoogle Scholar
Korda, AA, Mutoh, Y, Miyashita, Y, Sadasue, T and Mannan, S (2006) In situ observation of fatigue crack retardation in banded ferrite–pearlite microstructure due to crack branching. Scripta Materialia 54, 1835–1840.CrossRefGoogle Scholar
Krauss, GB (2003) Solidification, segregation, and banding in carbon and alloy steels. Metallurgical and Materials Transactions A 34, 781–792.CrossRefGoogle Scholar
Lim, D, Ong, Y-S, Jin, Y and Sendhoff, B (2007) A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation. Paper Presented at the Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, London, England.CrossRefGoogle Scholar
Liu, K, Tovar, A, Nutwell, E and Detwiler, D (2015) Thin-walled compliant mechanism component design assisted by machine learning and multiple surrogates. Paper Presented at the SAE 2015 World Congress & Exhibition, Detroit, Michigan.CrossRefGoogle Scholar
Lv, Z, Zhao, J, Wang, W and Liu, Q (2018) A multiple surrogates based PSO algorithm. Artificial Intelligence Review, 1–22. doi:10.1007/s10462-017-9601-3Google Scholar
Mack, Y, Goel, T, Shyy, W, Haftka, R and Queipo, N (2005) Multiple surrogates for the shape optimization of bluff body-facilitated mixing. Paper Presented at the 43rd AIAA Aerospace Sciences Meeting and Exhibit, Reno, Nevada. doi:10.2514/6.2005-333CrossRefGoogle Scholar
Mirjalili, S (2019) Evolutionary Radial Basis Function Networks. In Mirjalili, S (ed.), Evolutionary Algorithms and Neural Networks: Theory and Applications, Vol. 2. Cham: Springer International Publishing, pp. 105–139.CrossRefGoogle Scholar
Montgomery, DC (2017) Design and Analysis of Experiments. Hoboken, NJ: John Wiley & Sons.Google Scholar
Nellippallil, AB, Rangaraj, V, Gautham, BP, Singh, AK, Allen, JK and Mistree, F (2017) A goal-oriented, inverse decision-based design method to achieve the vertical and horizontal integration of models in a hot rod rolling process chain. Paper Presented at the ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Cleveland, Ohio, USA.CrossRefGoogle Scholar
Nellippallil, AB, Rangaraj, V, Gautham, BP, Singh, AK, Allen, JK and Mistree, F (2018) An inverse, decision-based design method for integrated design exploration of materials, products and manufacturing processes. ASME Journal of Mechanical Design 140, 111403–111417.CrossRefGoogle Scholar
Qiu, Q, Li, B and Feng, P (2016) Optimal design of hydraulic excavator working device based on multiple surrogate models. Advances in Mechanical Engineering 8. doi:10.1177/1687814016647947CrossRefGoogle Scholar
Razavi, S, Tolson, BA and Burn, DH (2012) Review of surrogate modeling in water resources. Water Resources Research 48, 1–32. doi:10.1029/2011WR011527CrossRefGoogle Scholar
Samad, A, Kim, K-Y, Goel, T, Haftka, RT and Shyy, W (2006) Shape optimization of turbomachinery blade using multiple surrogate models. Paper Presented at the ASME 2006 2nd Joint U.S.-European Fluids Engineering Summer Meeting Collocated With the 14th International Conference on Nuclear Engineering, Miami, Florida, USA.CrossRefGoogle Scholar
Samad, A, Lee, K-D, Kim, K-Y and Haftka, R (2007) Application of multiple-surrogate model to optimization of a dimpled channel. Paper Presented at the 7th World Congresses of Structural and Multidisciplinary Optimization, Seoul, Korea.Google Scholar
Shankar Bhattacharjee, K, Kumar Singh, H and Ray, T (2016) Multi-objective optimization with multiple spatially distributed surrogates. ASME Journal of Mechanical Design 138, 091401–091410. doi:10.1115/1.4034035CrossRefGoogle Scholar
Shi, R, Liu, L, Long, T and Liu, J (2016) An efficient ensemble of radial basis functions method based on quadratic programming. Engineering Optimization 48, 1202–1225. doi:10.1080/0305215X.2015.1100470CrossRefGoogle Scholar
Shyy, W, Tucker, PK and Vaidyanathan, R (2001) Response surface and neural network techniques for rocket engine injector optimization. Journal of Propulsion and Power 17, 391–401.CrossRefGoogle Scholar
Song, X, Lv, L, Li, J, Sun, W and Zhang, J (2018) An advanced and robust ensemble surrogate model: extended adaptive hybrid functions. Journal of Mechanical Design 140, 041402–041409. doi:10.1115/1.4039128CrossRefGoogle Scholar
Spitzig, W (1983) Effect of sulfide inclusion morphology and pearlite banding on anisotropy of mechanical properties in normalized C-Mn steels. Metallurgical Transactions A 14, 271–283.CrossRefGoogle Scholar
Tomita, Y (1995) Effect of modified austemper on tensile properties of 0·52%C steel. Materials Science and Technology 11, 994–997.CrossRefGoogle Scholar
Viana, FAC and Haftka, RT (2008) Using multiple surrogates for minimization of the RMS Error in meta-modeling. Paper Presented at the ASME 2008 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Brooklyn, New York, USA.CrossRefGoogle Scholar
Viana, F, Haftka, R, Steffen, JV, Butkewitsch, S and Leal, MF (2008) Optimal use of multiple surrogate for reduced RMS error in meta-model. Paper Presented at the NSF Engineering Research and Innovation Conference, Knoxville, Tennessee.Google Scholar
Viana, FA, Picheny, V and Haftka, RT (2009) Conservative prediction via safety margin: design through cross-validation and benefits of multiple surrogates. Paper Presented at the ASME 2009 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, San Diego, California, USA.Google Scholar
Viana, F, Haftka, R and Watson, L (2010) Why not run the efficient global optimization algorithm with multiple surrogates? Paper Presented at the 51st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Orlando, Florida.CrossRefGoogle Scholar
Viana, FA, Haftka, RT and Watson, LT (2013) Efficient global optimization algorithm assisted by multiple surrogate techniques. Journal of Global Optimization 56, 669–689.CrossRefGoogle Scholar
Villanueva, D, Haftka, RT, Le Riche, R and Picard, G (2013) Locating multiple candidate designs with surrogate-based optimization. Paper Presented at the 10th World Congress on structural and multidisciplinary optimization, Orlando, Florida, USA.Google Scholar
Wang, H, Ye, F, Li, E and Li, G (2016) A comparative study of expected improvement-assisted global optimization with different surrogates. Engineering Optimization 48, 1432–1458. doi:10.1080/0305215X.2015.1115645CrossRefGoogle Scholar
Xu, J and Zeger, SL (2001) The evaluation of multiple surrogate endpoints. Biometrics 57, 81–87.CrossRefGoogle ScholarPubMed
Yin, H, Fang, H, Wen, G, Gutowski, M and Xiao, Y (2018) On the ensemble of metamodels with multiple regional optimized weight factors. Structural and Multidisciplinary Optimization 58, 245–263. doi:10.1007/s00158-017-1891-1CrossRefGoogle Scholar
Zamani Sabzi, H, Abudu, S, Alizadeh, R, Soltanisehat, L, Dilekli, N and King, JP (2018) Integration of time series forecasting in a dynamic decision support system for multiple reservoir management to conserve water sources. Energy Sources, Part A: Recovery, Utilization, and Environmental Effects 40, 1398–1416. doi:10.1080/15567036.2018.1476934.CrossRefGoogle Scholar
Zerpa, LE, Queipo, NV, Pintos, S and Salager, J-L (2005) An optimization methodology of alkaline–surfactant–polymer flooding processes using field scale numerical simulation and multiple surrogates. Journal of Petroleum Science and Engineering 47, 197–208.CrossRefGoogle Scholar
Zhou, Q, Wang, Y, Choi, S, Jiang, P, Shao, X, Hu, J and Shu, L (2018) A robust optimization approach based on multi-fidelity metamodel. Structural and Multidisciplinary Optimization 57, 775–797.CrossRefGoogle Scholar