Skip to main content
SpringerLink
Log in

Design optimization by integrating limited simulation data and shape engineering knowledge with Bayesian optimization (BO-DK4DO)

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Surrogate models have been widely studied for optimization tasks in the domain of engineering design. However, the expensive and time-consuming simulation cycles needed for complex products always result in limited simulation data, which brings a challenge for building high accuracy surrogate models because of the incomplete information contained in the limited simulation data. Therefore, a method that builds a surrogate model and conducts design optimization by integrating limited simulation data and engineering knowledge through Bayesian optimization (BO-DK4DO) is presented. In this method, the shape engineering knowledge is considered and used as derivative information which is integrated with the limited simulation data with a Gaussian process (GP). Then the GP is updated sequentially by sampling new simulation data and the optimal design solutions are found by maximizing the GP. The aim of BO-DK4DO is to significantly reduce the required number of computer simulations for finding optimal design solutions. The BO-DK4DO is verified by using benchmark functions and an engineering design problem: hot rod rolling. In all scenarios, the BO-DK4DO shows faster convergence rate than the general Bayesian optimization without integrating engineering knowledge, which means the required amount of data is decreased.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://en.wikipedia.org/wiki/Test_functions_for_optimization.

References

  • Abu-Mostafa, Y. S. (1990). Learning from hints in neural networks. Journal of Complexity, 6(2), 192–198.

    Article  Google Scholar 

  • Aguirre, L. A., & Furtado, E. C. (2007). Building dynamical models from data and prior knowledge: The case of the first period-doubling bifurcation. Physical Review E, 76(4), 046219.

    Article  Google Scholar 

  • Amsallem, D., Cortial, J., Carlberg, K., & Farhat, C. (2009). A method for interpolating on manifolds structural dynamics reduced-order models. International Journal for Numerical Methods in Engineering, 80(9), 1241–1258.

    Article  Google Scholar 

  • Andersen, M. R., Siivola, E., & Vehtari, A. (2017). Bayesian Optimization of Unimodal Functions. In 31st Conference on Neural Information Processing Systems (NIPS 2017).

  • Aughenbaugh, J., & Herrmann, J. (2007). Updating uncertainty assessments: A comparison of statistical approaches. In ASME 2007 international design engineering technical conferences and computers and information in engineering conference (pp. 1195–1209). https://doi.org/10.1115/detc2007-35158.

  • Bhattacharyya, A., Conlan-Smith, C., & James, K. A. (2019). Design of a bi-stable airfoil with tailored snap-through response using topology optimization. Computer-Aided Design, 108, 42–55. https://doi.org/10.1016/j.cad.2018.11.001.

    Article  Google Scholar 

  • Brochu, E., Cora, V. M., & de Freitas, N. (2010). A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. ArXiv, 49. doi:10.1007/9783642532580?COVERIMAGEURL=HTTPS://STATICCONTENT.SPRINGER.COM/COVER/BOOK/9783642532580.JPG.

  • Calandra, R., Seyfarth, A., Peters, J., & Deisenroth, M. P. (2016). Bayesian optimization for learning gaits under uncertainty. Annals of Mathematics and Artificial Intelligence, 76(1), 5–23. https://doi.org/10.1007/s10472-015-9463-9.

    Article  Google Scholar 

  • Calvin, J. M., Chen, Y., & Žilinskas, A. (2012). An adaptive univariate global optimization algorithm and its convergence rate for twice continuously differentiable functions. Journal of Optimization Theory and Applications, 155(2), 628–636.

    Article  Google Scholar 

  • Daniels, H., & Velikova, M. (2010). Monotone and partially monotone neural networks. IEEE Transactions on Neural Networks, 21(6), 906–917.

    Article  Google Scholar 

  • Daróczy, L., Janiga, G., & Thévenin, D. (2018). Computational fluid dynamics based shape optimization of airfoil geometry for an H-rotor using a genetic algorithm. Engineering Optimization, 50(9), 1483–1499. https://doi.org/10.1080/0305215X.2017.1409350.

    Article  Google Scholar 

  • Di Bella, A., Fortuna, L., Grazianil, S., Napoli, G., Xibilia, M. G., & Doria, V. A. (2007). Development of a soft sensor for a thermal cracking unit using a small experimental data set. In 2007 IEEE international symposium on intelligent signal processing.

  • Dougherty, E. R., Dalton, L. A., & Alexander, F. J. (2015). Small data is the problem. In 2015 49th Asilomar conference on signals, systems and computers (pp. 418–422). IEEE.

  • Du, T.-S., Ke, X.-T., Liao, J.-G., & Shen, Y.-J. (2018). DSLC-FOA : Improved fruit fly optimization algorithm for application to structural engineering design optimization problems. Applied Mathematical Modelling, 55, 314–339. https://doi.org/10.1016/j.apm.2017.08.013.

    Article  Google Scholar 

  • Du, G., Xia, Y., Jiao, R. J., & Liu, X. (2019). Leader-follower joint optimization problems in product family design. Journal of Intelligent Manufacturing, 30(3), 1387–1405. https://doi.org/10.1007/s10845-017-1332-4.

    Article  Google Scholar 

  • Fatemeh, D. B., Loo, C. K., & Kanagaraj, G. (2019). Shuffled Complex Evolution based Quantum Particle Swarm Optimization algorithm for mechanical design optimization problems. Journal of Modern Manufacturing Systems and Technology, 02, 23–32.

    Google Scholar 

  • Fengjie, T., & Lahmer, T. (2018). Shape optimization based design of arch-type dams under uncertainties. Engineering Optimization, 50(9), 1470–1482. https://doi.org/10.1080/0305215X.2017.1409348.

    Article  Google Scholar 

  • Forrester, A. I. J., Sbester, A., & Keane, A. J. (2008). Engineering design via surrogate modelling. Chichester: Wiley.

    Book  Google Scholar 

  • Fortuna, L., Graziani, S., & Xibilia, M. G. (2009). Comparison of soft-sensor design methods for industrial plants using small data sets. IEEE Transactions on Instrumentation and Measurement, 58(8), 2444–2451.

    Article  Google Scholar 

  • Gorissen, D., & Dhaene, T. (2010). A surrogate modeling and adaptive sampling toolbox for computer based design. Journal of Machine Learning Research, 11(1), 2051–2055.

    Google Scholar 

  • Gupta, M., Bahri, D., Cotter, A., & Canini, K. (2018). Diminishing returns shape constraints for interpretability and regularization. In Advances in neural information processing systems (Vol. 31, pp. 6834–6844). Curran Associates, Inc. http://papers.nips.cc/paper/7916-diminishing-returns-shape-constraints-for-interpretability-and-regularization.pdf.

  • Hao, J., Ye, W., Wang, G., Jia, L., & Wang, Y. (2018). Evolutionary neural network-based method for constructing surrogate model with small scattered dataset and monotonicity experience. In Proceedings of the 2018 soft computing & machine intelligence, Nairobi, Kenya.

  • Hennig, P., & Schuler, C. J. (2012). Entropy search for information-efficient global optimization. The Journal of Machine Learning Research, 13(1), 1809–1837.

    Google Scholar 

  • Jauch, M., & Peña, V. (2016). Bayesian optimization with shape constraints. In 29th conference on neural information processing systems, Barcelona, Spain.

  • Jones, D. R., Schonlau, M., & William, J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4), 455–492.

    Article  Google Scholar 

  • Kang, G., Wu, L., Guan, Y., & Peng, Z. (2019). A virtual sample generation method based on differential evolution algorithm for overall trend of small sample data: Used for lithium-ion battery capacity degradation data. IEEE Access, 7, 123255–123267. https://doi.org/10.1109/ACCESS.2019.2937550.

    Article  Google Scholar 

  • Kotlowski, W., & Slowinski, R. (2013). On nonparametric ordinal classification with monotonicity constraints. IEEE Transactions on Knowledge and Data Engineering, 25(11), 2576–2589.

    Article  Google Scholar 

  • Kushner, H. J. (1964). A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. Journal of Basic Engineering, 86(1), 97–106.

    Article  Google Scholar 

  • Lenk, P. J., & Choi, T. (2017). Bayesian analysis of shape-restricted functions using Gaussian process priors. Statistica Sinica, 27, 43–70.

    Google Scholar 

  • Li, D.-C., Chang, C.-C., Liu, C.-W., & Chen, W.-C. (2013). A new approach for manufacturing forecast problems with insufficient data: The case of TFT–LCDs. Journal of Intelligent Manufacturing, 24(2), 225–233.

    Article  Google Scholar 

  • Li, D.-C., Chen, L.-S., & Lin, Y.-S. (2003). Using functional virtual population as assistance to learn scheduling knowledge in dynamic manufacturing environments. International Journal of Production Research, 41(17), 4011–4024.

    Article  Google Scholar 

  • Li, D.-C., Chen, H.-Y., & Shi, Q.-S. (2018a). Learning from small datasets containing nominal attributes. Neurocomputing, 291, 226–236. https://doi.org/10.1016/j.neucom.2018.02.069.

    Article  Google Scholar 

  • Li, D.-C., Fang, Y.-H., Liu, C.-W., & Juang, C. (2012). Using past manufacturing experience to assist building the yield forecast model for new manufacturing processes. Journal of Intelligent Manufacturing, 21(4), 1–12.

    Google Scholar 

  • Li, C., Santu, R., Gupta, S., Nguyen, V., Venkatesh, S., Sutti, A., et al. (2018b). Accelerating experimental design by incorporating experimenter hunches. In 2018 IEEE international conference on data mining (ICDM) (pp. 257–266). https://doi.org/10.1109/icdm.2018.00041.

  • Liu, B., Koziel, S., & Zhang, Q. (2016). A multi-fidelity surrogate-model-assisted evolutionary algorithm for computationally expensive optimization problems. Journal of Computational Science, 12, 28–37. https://doi.org/10.1016/j.jocs.2015.11.004.

    Article  Google Scholar 

  • Liu, H., Ong, Y.-S., & Cai, J. (2018). A survey of adaptive sampling for global metamodeling in support of simulation-based complex engineering design. Structural and Multidisciplinary Optimization, 57(1), 393–416. https://doi.org/10.1007/s00158-017-1739-8.

    Article  Google Scholar 

  • Min, A. T. W., Sagarna, R., Gupta, A., Ong, Y., & Goh, C. K. (2017). Knowledge transfer through machine learning in aircraft design. IEEE Computational Intelligence Magazine, 12(4), 48–60.

    Article  Google Scholar 

  • Monisha, R., & Peter, D. J. (2017). Triple band PIFA antenna using knowledge based neural networks. Asian Journal of Applied Science and Technology (AJAST), 1(3), 271–274.

    Google Scholar 

  • Nagarajan, H. P. N., Mokhtarian, H., Jafarian, H., Dimassi, S., Bakrani-Balani, S., Hamedi, A., et al. (2018). Knowledge-based design of artificial neural network topology for additive manufacturing process modeling: A new approach and case study for fused deposition modeling. Journal of Mechanical Design, 141(2), 1. https://doi.org/10.1115/1.4042084.

    Article  Google Scholar 

  • Ning, J., Nguyen, V., Huang, Y., Hartwig, K. T., & Liang, S. Y. (2018). Inverse determination of Johnson–Cook model constants of ultra-fine-grained titanium based on chip formation model and iterative gradient search. The International Journal of Advanced Manufacturing Technology, 99(5), 1131–1140. https://doi.org/10.1007/s00170-018-2508-6.

    Article  Google Scholar 

  • Parrado-Hernández, E., Ambroladze, A., Shawe-Taylor, J., & Sun, S. (2012). PAC-bayes bounds with data dependent priors. Journal of Machine Learning Research, 13(1), 3507–3531.

    Google Scholar 

  • Pratap, S., Daultani, Y., Tiwari, M. K., & Mahanty, B. (2018). Rule based optimization for a bulk handling port operations. Journal of Intelligent Manufacturing, 29(2), 287–311. https://doi.org/10.1007/s10845-015-1108-7.

    Article  Google Scholar 

  • Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. Cambridge, Massachusetts, London: The MIT Press.

    Google Scholar 

  • Riihimäki, J., & Vehtari, A. (2010). Gaussian processes with monotonicity information. Journal of Machine Learning Research, 9, 645–652.

    Google Scholar 

  • Russo, D., & Van Roy, B. (2014). Learning to optimize via posterior sampling. Mathematics of Operations Research, 39(4), 949–1348.

    Article  Google Scholar 

  • Sahnoun, M., Bettayeb, B., Bassetto, S.-J., & Tollenaere, M. (2016). Simulation-based optimization of sampling plans to reduce inspections while mastering the risk exposure in semiconductor manufacturing. Journal of Intelligent Manufacturing, 27(6), 1335–1349. https://doi.org/10.1007/s10845-014-0956-x.

    Article  Google Scholar 

  • Schneider, J. (2015). High dimensional bayesian optimisation and bandits via additive models. In ICML’15 Proceedings of the 32nd international conference on international conference on machine learning (Vol. 37, pp. 295–304).

  • Sefat, M., Salahshoor, K., Jamialahmadi, M., & Moradi, B. (2012). A new approach for the development of fast-analysis proxies for petroleum reservoir simulation. Petroleum Science and Technology, 30(18), 1920–1930. https://doi.org/10.1080/10916466.2010.512885.

    Article  Google Scholar 

  • Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & de Freitas, N. (2016). Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1), 148–175.

    Article  Google Scholar 

  • Shavlik, J. W. (1994). Combining symbolic and neural learning. Machine Learning, 14(3), 321–331. https://doi.org/10.1007/BF00993982.

    Article  Google Scholar 

  • Sill, J. (1998). Monotonic networks. In Proceedings of the 1997 conference on advances in neural information processing systems (Vol, 10, pp. 661–667). Cambridge: MIT Press.

  • Sill, J., & Abu-Mostafa, Y. S. (1997). Monotonicity hints. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems (Vol. 6, pp. 634–640). Cambridge: MIT Press.

    Google Scholar 

  • Smola, A. J. (2012). Exponential regret bounds for gaussian process bandits with deterministic observations. In ICML’12 Proceedings of the 29th international coference on international conference on machine learning (pp. 955–962).

  • Song, X., Sun, G., & Li, Q. (2016). Sensitivity analysis and reliability based design optimization for high-strength steel tailor welded thin-walled structures under crashworthiness. Thin-Walled Structures, 109, 132–142. https://doi.org/10.1016/j.tws.2016.09.003.

    Article  Google Scholar 

  • Srinivas, N., Krause, A., Kakade, S., & Seeger, M. (2010). Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th international conference on international conference on machine learning (pp. 1015–1022). USA: Omnipress.

  • Towell, G. G., & Shavlik, J. W. (1994). Knowledge-based artificial neural networks. Artificial Intelligence, 70(1), 119–165. https://doi.org/10.1016/0004-3702(94)90105-8.

    Article  Google Scholar 

  • Trucano, T. G., Swiler, L. P., Igusa, T., Oberkampf, W. L., & Pilch, M. (2006). Calibration, validation, and sensitivity analysis: What’s what. Reliability Engineering & System Safety, 91(10), 1331–1357. https://doi.org/10.1016/j.ress.2005.11.031.

    Article  Google Scholar 

  • Tsai, T., & Li, D. (2008). Utilize bootstrap in small data set learning for pilot run modeling of manufacturing systems. Expert Systems with Applications, 35(3), 1293–1300.

    Article  Google Scholar 

  • Vidal, A., & Archer, R. (2016). Calibration of a geothermal reservoir model using a global method based on surrogate modeling. In 41st workshop on geothermal reservoir engineering (pp. 1–8). Stanford University.

  • Wang, X., & Berger, J. O. (2016). Estimating shape constrained functions using Gaussian processes. SIAM/ASA Journal on Uncertainty Quantification, 4(1), 1–25.

    Article  Google Scholar 

  • Wang, L., Beeson, D., Akkaram, S., & Wiggs, G. (2005). Gaussian process metamodels for efficient probabilistic design in complex engineering design spaces. In ASME 2005 international design engineering technical conferences and computers and information in engineering conference (pp. 785–798). https://doi.org/10.1115/DETC2005-85406.

  • Wang, H., Jin, Y., & Doherty, J. (2017). Committee-based active learning for surrogate-assisted particle swarm optimization of expensive problems. IEEE Transactions on Cybernetics, 47(9), 2664–2677. https://doi.org/10.1109/TCYB.2017.2710978.

    Article  Google Scholar 

  • Wang, W., & Welch, W. J. (2018). Bayesian optimization using monotonicity information and its application in machine learning hyperparameter tuning. In Proceedings of AutoML 2018 @ ICML/IJCAI-ECAI (pp. 1–13).

  • Wu, J., & Feb, M. L. (2017). Bayesian optimization with gradients. In 31st conference on neural information processing systems (NIPS 2017) (pp. 1–17).

  • Yoshimura, M., Shimoyama, K., Misaka, T., & Obayashi, S. (2017). Topology optimization of fluid problems using genetic algorithm assisted by the Kriging model. International Journal for Numerical Methods in Engineering, 109(4), 514–532. https://doi.org/10.1002/nme.

    Article  Google Scholar 

  • Yu, L., Wang, L., & Yu, J. (2008). Identification of product definition patterns in mass customization using a learning-based hybrid approach. The International Journal of Advanced Manufacturing Technology, 38(11), 1061–1074. https://doi.org/10.1007/s00170-007-1152-3.

    Article  Google Scholar 

  • Zhang, Z., Chai, N., Liu, Y., & Xia, B. (2019). Base types selection of PSS based on a priori algorithm and knowledge-based ANN. IET Collaborative Intelligent Manufacturing, 1(2), 29–38. https://doi.org/10.1049/iet-cim.2018.0003.

    Article  Google Scholar 

  • Zhang, X., Wang, S., Yi, L., Xue, H., & Yang, S. (2018). An integrated ant colony optimization algorithm to solve job allocating and tool scheduling problem. Journal of Engineering Manufacture, 232(1), 172–182. https://doi.org/10.1177/0954405416636038.

    Article  Google Scholar 

  • Zhao, T., Montoya-Noguera, S., Phoon, K.-K., & Wang, Y. (2017). Interpolating spatially varying soil property values from sparse data for facilitating characteristic value selection. Canadian Geotechnical Journal, 55(2), 171–181. https://doi.org/10.1139/cgj-2017-0219.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Professor Janet K. Allen and Professor Farrokh Mistree from the University of Oklahoma for their valuable comments. The author also appreciates the strong support provided by the National Natural Science Foundation of China (NSFC 51505032), the Beijing Natural Science Foundation (BJNSF 3172028).

Author information

Authors and Affiliations

  1. Beijing Institute of Technology, No. 5 South Street, Zhongguancun, Haidian District, Beijing, China

    Jia Hao, Mengying Zhou, Guoxin Wang, Liangyue Jia & Yan Yan

Authors
  1. Jia Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mengying Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guoxin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liangyue Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yan Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jia Hao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hao, J., Zhou, M., Wang, G. et al. Design optimization by integrating limited simulation data and shape engineering knowledge with Bayesian optimization (BO-DK4DO). J Intell Manuf 31, 2049–2067 (2020). https://doi.org/10.1007/s10845-020-01551-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-020-01551-8

Keywords

Search

Navigation