Skip to main content
SpringerLink
Log in

New loss functions to improve deep learning estimation of heat transfer

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Deep neural networks (DNNs) are promising alternatives to simulate physical problems. These networks are capable of eliminating the requirement of numerical iterations. The DNNs could learn the governing physics of engineering problems through a learning process. The structure of deep networks and parameters of the training process are two basic factors that influence the simulation accuracy of DNNs. The loss function is the main part of the training process that determines the goal of training. During the training process, lost function regularly is used to adapt parameters of the deep network. The subject of using DNNs to learn the physical images is a novel topic and demands novel loss functions to capture the physical meanings. Thus, for the first time, the present study aims to develop new loss functions to enhance the training process of DNNs. Here, three novel loss functions were introduced and examined to estimate the temperature distributions in thermal conduction problems. The images of temperature distribution obtained in the present research were systematically compared with the literature data. The results showed that one of the introduced loss functions could significantly outperformance the literature loss functions available in the literature. Using a new loss function improved the mean error by 67.1%. Moreover, using new loss functions eliminated the pixels predictions (with large errors) by 96%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Wang J, Jiang J (2020) SA-Net: A deep spectral analysis network for image clustering. Neurocomputing 383:10–23

    Article  Google Scholar 

  2. Chan T-H, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2015) PCANet: A simple deep learning baseline for image classification? IEEE Trans Image Process 24(12):5017–5032

    Article  MathSciNet  Google Scholar 

  3. Qing Y, Zeng Y, Li Y, Huang G-B (2020) Deep and wide feature based extreme learning machine for image classification. Neurocomputing 412:426–436

    Article  Google Scholar 

  4. Graves A, Mohamed A-r, Hinton G (eds) (2013) Speech recognition with deep recurrent neural networks. In: 2013 IEEE international conference on acoustics, speech and signal processing. IEEE

  5. Pascual S, Bonafonte A, Serrà J (2017) SEGAN: speech enhancement generative adversarial network. arXiv:170309452.2017

  6. Yıldırım Ö, Baloglu UB, Acharya UR (2018) A deep convolutional neural network model for automated identification of abnormal EEG signals. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3889-z

    Article  Google Scholar 

  7. Hemanth DJ, Deperlioglu O, Kose U (2020) An enhanced diabetic retinopathy detection and classification approach using deep convolutional neural network. Neural Comput Appl 32(3):707–721

    Article  Google Scholar 

  8. Woźniak M, Siłka J, Wieczorek M (2021) Deep neural network correlation learning mechanism for CT brain tumor detection. Neural Comput Appl. https://doi.org/10.1007/s00521-021-05841-x

    Article  Google Scholar 

  9. He K, Zhang X, Ren S, Sun J (eds) (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision

  10. Han K, Mun YY, Gweon G, Lee J-G (eds) (2013) Understanding the difficulty factors for learning materials: a qualitative study. In: International conference on artificial intelligence in education. Springer

  11. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:150203167.2015

  12. Agarap AF (2018) Deep learning using rectified linear units (relu). arXiv:180308375.2018

  13. Raissi M, Yazdani A, Karniadakis GE (2020) Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science 367(6481):1026–1030

    Article  MathSciNet  Google Scholar 

  14. Berg J, Nyström K (2018) A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing 317:28–41

    Article  Google Scholar 

  15. Lin Q, Hong J, Liu Z, Li B, Wang J (2018) Investigation into the topology optimization for conductive heat transfer based on deep learning approach. Int Commun Heat Mass Transf 97:103–109

    Article  Google Scholar 

  16. Liu X, Zhang H, Kong X, Lee KY (2020) Wind speed forecasting using deep neural network with feature selection. Neurocomputing 397:393–403

    Article  Google Scholar 

  17. Sharma R, Farimani AB, Gomes J, Eastman P, Pande V (2018) Weakly-supervised deep learning of heat transport via physics informed loss. arXiv:180711374.2018

  18. Ciresan D, Giusti A, Gambardella LM, Schmidhuber J (eds) (2012) Deep neural networks segment neuronal membranes in electron microscopy images. Advances in neural information processing systems

  19. Bergman TL, Incropera FP, Lavine AS, DeWitt DP (2011) Introduction to heat transfer. John Wiley & Sons, USA

    Google Scholar 

  20. Farimani AB, Gomes J, Pande VS (2017) Deep learning the physics of transport phenomena. arXiv:170902432.2017

  21. Edalatifar M, Tavakoli MB, Ghalambaz M, Setoudeh F (2020) A dataset for conduction heat transer and deep learning. Mendeley Data 1:10–7632

    Article  Google Scholar 

  22. Edalatifar M, Tavakoli MB, Ghalambaz M, Setoudeh F (2020) Using deep learning to learn physics of conduction heat transfer. J Therm Anal Calorim 146:1435–1452

    Article  Google Scholar 

  23. Nadipally M (2019) Optimization of methods for image-texture segmentation using ant colony optimization. In: Hemanth DJ, Gupta D, Emilia Balas V (eds) Intelligent data analysis for biomedical applications. Academic Press, Cambridge, pp 21–47

    Chapter  Google Scholar 

  24. Botchkarev A (2018) Performance metrics (error measures) in machine learning regression, forecasting and prognostics: properties and typology. arXiv:180903006.2018

  25. Masci J, Meier U, Cireşan D, Schmidhuber J (eds) (2011) Stacked convolutional auto-encoders for hierarchical feature extraction. In: international conference on artificial neural networks. Springer

  26. Wu C, Khishe M, Mohammadi M, Taher Karim SH, Rashid TA (2021) Evolving deep convolutional neutral network by hybrid sine–cosine and extreme learning machine for real-time COVID19 diagnosis from X-ray images. Soft Comput. https://doi.org/10.1007/s00500-021-05839-6

    Article  Google Scholar 

  27. Hu T, Khishe M, Mohammadi M, Parvizi G-R, Taher Karim SH, Rashid TA (2018) Real-time COVID-19 diagnosis from X-Ray images using deep CNN and extreme learning machines stabilized by chimp optimization algorithm. Biomed Signal Process Control 68:102764

    Article  Google Scholar 

  28. Rashid TA, Abbas DK, Turel YK (2019) A multi hidden recurrent neural network with a modified grey wolf optimizer. PLoS ONE 14(3):e0213237

    Article  Google Scholar 

  29. Rashid TA, Fattah P, Awla DK (2018) Using accuracy measure for improving the training of LSTM with metaheuristic algorithms. Procedia Comput Sci 140:324–333

    Article  Google Scholar 

  30. Jabar AL, Rashid TA (2018) A modified particle swarm optimization with neural network via euclidean distance. Int J Recent Contrib Eng Sci IT (iJES) 6(1):4–18

    Article  Google Scholar 

  31. Lemaréchal C (2012) Cauchy and the gradient method. Doc Math Extra 251(254):10

    MATH  Google Scholar 

  32. Cheridito P, Jentzen A, Riekert A, Rossmannek F (2022) A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions. arXiv preprint arXiv:210209924.2021.

  33. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536

    Article  Google Scholar 

  34. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge

    MATH  Google Scholar 

  35. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C et al (2015) TensorFlow: large-scale machine learning on heterogeneous systems

  36. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:14126980.2014

  37. Glorot X, Bengio Y (eds) (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Arak Branch, Islamic Azad University, Arak, Iran

    Mohammad Edalatifar & Mohammad Bagher Tavakoli

  2. Metamaterials for Mechanical, Biomechanical and Multiphysical Applications Research Group, Ton Duc Thang University, Ho Chi Minh City, Vietnam

    Mohammad Ghalambaz

  3. Faculty of Applied Sciences, Ton Duc Thang University, Ho Chi Minh City, Vietnam

    Mohammad Ghalambaz

  4. Faculty of Electrical Engineering, Arak University of Technology, Arak, Iran

    Farbod Setoudeh

Authors
  1. Mohammad Edalatifar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Ghalambaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Bagher Tavakoli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Farbod Setoudeh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Ghalambaz.

Ethics declarations

Conflict of interest

The authors clarify that there is no conflict of interest for report.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Edalatifar, M., Ghalambaz, M., Tavakoli, M.B. et al. New loss functions to improve deep learning estimation of heat transfer. Neural Comput & Applic 34, 15889–15906 (2022). https://doi.org/10.1007/s00521-022-07233-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07233-1

Keywords

Search

Navigation