Skip to main content
SpringerLink
Log in

BC-PINN: an adaptive physics informed neural network based on biased multiobjective coevolutionary algorithm

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Physics informed neural network (PINN) has become a promising method for solving partial differential equations (PDEs). The loss function of PINN is a weighted sum of multiple items. This makes it easy to fall into local optima, especially the gradient pathologies when solving high frequency problems. The value of penalty coefficients has a crucial impact on the prediction results. Therefore, a new PINN with adaptive penalty coefficients iteratively optimized by biased multiobjective coevolutionary algorithm (BC-PINN) is presented. In BC-PINN, a two-stage optimization mechanism is used to search for parameters of neural network and penalty coefficients respectively. This method involves constructing the fitness function of penalty coefficients based on the biased dominance ranking by data item and regularization item. Compared with the previous works of others, the accuracy of fitting the initial conditions and boundary conditions is considered to be given priority, which is more conducive to PINN converging to the particular solution of PDE. In addition, the set of penalty coefficients is divided into multiple populations to improve the optimization efficiency through coevolutionary algorithm. The empirical results show that: (1) Our method can improve the gradient pathologies and effectively capture the high-frequency features. (2) Compared to the original PINN, it reduces the MSE by 1–6 orders of magnitude in our benchmark functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Data Availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Strang G, Fix GJ, Griffin DS (1974) An analysis of the finite-element method. J Appl Mech 41:115–126

    Google Scholar 

  2. Ramakrishnan S, Collis SS (2004) Turbulence control simulation using the variational multiscale method. AIAA J 42:745–753

    Google Scholar 

  3. Glatzel T, Litterst C, Cupelli C, Lindemann T, Moosmann C, Niekrawietz R, Streule W, Zengerle R, Koltay P (2008) Computational fluid dynamics (CFD) software tools for microfluidic applications—a case study. Comput Fluids 37(3):218–235

    MATH  Google Scholar 

  4. Farmer R, Pike R, Cheng G (2005) CFD analyses of complex flows. Comput Chem Eng 29:2386–2403

    Google Scholar 

  5. Jeong W, Seong J (2014) Comparison of effects on technical variances of computational fluid dynamics (CFD) software based on finite element and finite volume methods. Int J Mech Sci 78:19–26

    Google Scholar 

  6. Dissanayake MWMG, Phan-Thien N (1994) Neural-network-based approximations for solving partial differential equations. Commun Numer Methods Eng 10:195–201

    MATH  Google Scholar 

  7. Van Milligen BP, Tribaldos V, Jiménez JA (1995) Neural network differential equation and plasma equilibrium solver. Phys Rev Lett 75:3594

    Google Scholar 

  8. Lagaris IE, Likas A, Fotiadis DI (1998) Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans Neural Networks 9:987–1000

    Google Scholar 

  9. Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707

    MathSciNet  MATH  Google Scholar 

  10. Raissi M, Yazdani A, Karniadakis GE (2020) Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science (New York), p 367, 1026–1030.

  11. Ellington EH, Bastille-Rousseau G, Austin C, Landolt KN, Pond BA, Rees EE, Robar N, Murray DL (1981) Automatic differentiation: techniques and applications. J Appl Water Eng Res22:1548–1555

    Google Scholar 

  12. Li R, Wang JX, Lee E, Luo T (2022) Physics-informed deep learning for solving phonon Boltzmann transport equation with large temperature non-equilibrium. Npj Comput Mater, 8.

  13. P.J. Baddoo, B. Herrmann, B.J. McKeon, J. Nathan Kutz, S.L. Brunton, Physics-informed dynamic mode decomposition, Proc. R. Soc. A Math. Phys. Eng. Sci. 479 (2023).

  14. Kunisch K, Walter D (2021) Semiglobal optimal feedback stabilization of autonomous systems via deep neural network approximation. ESAIM Control Optim Calc Var 27:16

    MathSciNet  MATH  Google Scholar 

  15. Jo H, Son H, Hwang HJ, Kim EH (2020) Deep neural network approach to forward-inverse problems. Networks Heterog Media 15:247–259

    MathSciNet  MATH  Google Scholar 

  16. Kashefi M, Mukerji T (2022) Physics-informed PointNet: a deep learning solver for steady-state incompressible flows and thermal fields on multiple sets of irregular geometries. J Comput Phys 402:115041

    MathSciNet  MATH  Google Scholar 

  17. Yuan L, Ni YQ, Deng XY, Hao S (2022) A-PINN: auxiliary physics informed neural networks for forward and inverse problems of nonlinear integro-differential equations. J Comput Phys 462:111260

    MathSciNet  MATH  Google Scholar 

  18. Xu C, Cao BT, Yuan Y, Meschke G (2022) Transfer learning based physics-informed neural networks for solving inverse problems in tunneling. Comput Methods Appl Mech Eng, 405.

  19. Fan T, Xu K, Pathak J, Darve E. Solving inverse problems in steady state Navier-Stokes equations using deep neural networks, arXiv preprint arXiv:2008.13074v1.

  20. Meng X, Karniadakis GE (2020) A composite neural network that learns from multi-fidelity data: application to function approximation and inverse PDE problems. J Comput Phys 401:109020

    MathSciNet  MATH  Google Scholar 

  21. Raissi M, Wang Z, Triantafyllou MS, Karniadakis GE (2019) Deep learning of vortex-induced vibrations. J Fluid Mech 861:119–137

    MathSciNet  MATH  Google Scholar 

  22. Cai S, Wang Z, Fuest F, Jeon YJ, Gray C, Karniadakis GE (2021) Flow over an espresso cup: Inferring 3-D velocity and pressure fields from tomographic background oriented Schlieren via physics-informed neural networks. J Fluid Mech 915:A102

    MathSciNet  MATH  Google Scholar 

  23. Luo S, Vellakal M, Koric S, Kindratenko V, Cui J (2020) Parameter identification of RANS turbulence model using physics-embedded neural network. Lect Notes Comput Sci 12321:137–149

    Google Scholar 

  24. Eivazi H, Tahani M, Schlatter P, Vinuesa R (2022) Physics-informed neural networks for solving Reynolds-averaged Navier-Stokes equations. Phys Fluids, 34.

  25. Almajid MM, Abu-Al-Saud MO (2022) Prediction of porous media fluid flow using physics informed neural networks. J Pet Sci Eng 208:109205

    Google Scholar 

  26. Haghighat E, Raissi M, Moure A, Gomez H. A deep learning framework for solution and discovery in solid mechanics: linear elasticity, arXiv preprint arXiv:2003.02751.

  27. Rao C, Sun H, Liu Y (2021) Physics-informed deep learning for computational Elastodynamics without labeled data. J Eng Mech 147:04021043

    Google Scholar 

  28. Li W, Bazant MZ, Zhu J (2021) A physics-guided neural network framework for elastic plates: comparison of governing equations-based and energy-based approaches. Comput Methods Appl Mech Eng 383:113933

    MathSciNet  MATH  Google Scholar 

  29. D.W Abueidda, Q.Lu, S.Koric, Meshless physics-informed deep learning method for three-dimensional solid mechanics, arXiv preprint arXiv:2012.01547v2.

  30. Pu J, Li J, Chen Y (2021) Solving localized wave solutions of the derivative nonlinear Schrödinger equation using an improved PINN method. Nonlinear Dyn 105:1723–1739

    Google Scholar 

  31. Chen Y, Lu L, Karniadakis GE, Dal Negro L (2020) Physics-informed neural networks for inverse problems in nano-optics and metamaterials. Opt Express 28:11618–11633

    Google Scholar 

  32. Pun GPP, Batra R, Ramprasad R, Mishin Y (2019) Physically informed artificial neural networks for atomistic modeling of materials. Nat Commun 10:2339

    Google Scholar 

  33. bin Waheed U, Haghighat E, Alkhalifah T, Song C, Hao Q (2021) PINNeik: Eikonal solution using physics-informed neural networks. Comput Geosci 155:104833.

  34. Wang L, Yan Z (2021) Data-driven rogue waves and parameter discovery in the defocusing nonlinear Schrödinger equation with a potential using the PINN deep learning. Phys Lett Sect A Gen At Solid State Phys 404:127408.

  35. Mishra S, Molinaro R (2021) Physics informed neural networks for simulating radiative transfer. J Quant Spectrosc Radiat Transf 270:107705

    Google Scholar 

  36. Martin CH, Oved A, Chowdhury RA, Ullmann E, Peters NS, Bharath AA. Ep-pinns: cardiac electrophysiology characterisation using physics-informed neural networks. arXiv preprint arXiv:2112.07703.

  37. Patel RG, Manickam I, Trask NA, Wood MA, Lee M, Tomas I, Cyr EC (2022) Thermodynamically consistent physics-informed neural networks for hyperbolic systems. J Comput Phys 449:110754

    MathSciNet  MATH  Google Scholar 

  38. Lu L, Meng X, Mao Z, Karniadakis GE (2021) DeepXDE: a deep learning library for solving differential equations. SIAM Rev 63:208–228

    MathSciNet  MATH  Google Scholar 

  39. Peng W, Zhang J, Zhou W, Zhao X, Yao W, Chen X. IDRLnet: a Physics-Informed Neural Network Library. arXiv preprint arXiv:2107.04320.

  40. Chen F, Sondak D, Protopapas P, Mattheakis M, Liu S, Agarwal D, Di Giovanni M (2020) NeuroDiffEq: a python package for solving differential equations with neural networks. J Open Source Softw 5:1931

    Google Scholar 

  41. Yang L, Meng X, Karniadakis GE (2021) B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data. J Comput Phys 425:109913

    MathSciNet  MATH  Google Scholar 

  42. Cheng C, Meng H, Li YZ, Zhang GT (2021) Deep learning based on PINN for solving 2 DOF vortex induced vibration of cylinder. Ocean Eng 240:109932

    Google Scholar 

  43. Fang Z (2021) A high-efficient hybrid physics-informed neural networks based on convolutional neural network. IEEE Trans Neural Netw Learn Syst, pp 1–13.

  44. Gao H, Zahr MJ, Wang JX. Physics-informed graph neural Galerkin networks: aunified framework for solving PDE-governed forward and inverse problems. arXiv preprint arXiv:2107.12146v1.

  45. Arora R. Machine learning-accelerated computational solid mechanics: application to linear elasticity. arXiv preprint arXiv:2112.08676.

  46. Chen X, Duan J, Karniadakis GEM (2021) Learning and meta-learning of stochastic advection-diffusion-reaction systems from sparse measurements. Eur J Appl Math 32:397–420

    MathSciNet  MATH  Google Scholar 

  47. Wight CL, Zhao J (2021) Solving allen-cahn and cahn-hilliard equations using the adaptive physics informed neural networks, Commun. Comput Phys 29:930–954

    MathSciNet  MATH  Google Scholar 

  48. Jagtap AD, Kawaguchi K, Karniadakis GE (2020) Adaptive activation functions accelerate convergence in deep and physics-informed neural networks. J Comput Phys 404:109136

    MathSciNet  MATH  Google Scholar 

  49. Dwivedi V, Parashar N, Srinivasan B (2021) Distributed learning machines for solving forward and inverse problems in partial differential equations. Neurocomputing 420:299–316

    Google Scholar 

  50. Meng X, Li Z, Zhang D, Karniadakis GE (2020) PPINN: Parareal physics-informed neural network for time-dependent PDEs. Comput Methods Appl Mech Eng 370:113250

    MathSciNet  MATH  Google Scholar 

  51. Jagtap AD, Kharazmi E, Karniadakis GE (2020) Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems. Comput Methods Appl Mech Eng 365:113028

    MathSciNet  MATH  Google Scholar 

  52. Moseley B, Markham A, Nissen-Meyer T. Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations. arXiv preprint arXiv:2107.07871.

  53. Kim Y, Choi Y, Widemann D, Zohdi T (2022) A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder. J Comput Phys 451:110841

    MathSciNet  MATH  Google Scholar 

  54. Mishra S, Molinaro R (2022) Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems for PDEs. IMA J Numer Anal 42:981–1022

    MathSciNet  MATH  Google Scholar 

  55. Yu J, Lu L, Meng X, Karniadakis GE (2022) Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems. Comput Methods Appl Mech Eng 393:1–22

    MathSciNet  MATH  Google Scholar 

  56. Cuomo S, Di Cola VS, Giampaolo F, Rozza G, Raissi M, Piccialli F (2022) Scientific machine learning through physics–informed neural networks: where we are and What’s Next. J Sci Comput, 92.

  57. Wang S, Teng Y, Perdikaris P (2021) Understanding and mitigating gradient pathologies in physics-informed neural networks. SIAM J Sci Comput 43:A3055–A3081

    MathSciNet  MATH  Google Scholar 

  58. Krishnapriyan S, Gholami A, Zhe S, Kirby Mahoney RM. Characterizing possible failure modes in physics-informed neural networks. arXiv preprint arXiv:2109.01050.

  59. Sun L, Gao H, Pan S, Wang JX (2020) Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data. Comput Methods Appl Mech Eng 361:112732

    MathSciNet  MATH  Google Scholar 

  60. Wang S, Yu X, Perdikaris P (2022) When and why PINNs fail to train: a neural tangent kernel perspective. J Comput Phys 449:110768

    MathSciNet  MATH  Google Scholar 

  61. Xiang Z, Peng W, Zheng X, Zhao X, Yao W. Self-adaptive loss balanced physics-informed neural networks for the incompressible navier-stokes equations. arXiv preprint arXiv:2104.06217.

  62. Cipolla R, Gal Y, Kendall A (2018) Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. In: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2018, pp. 7482–749.

  63. Rall LB (1981) Automatic computation of gradients, Jacobians, Hessians, and applications to optimization. In: Rall LB (ed) Automatic differentiation: techniques and applications. LNCS, vol 120. Springer, Heidelberg, pp 91–111. https://doi.org/10.1007/3-540-10861-0_6

  64. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6:182–197

    Google Scholar 

  65. Li X, Zhang G (2015) Biased multiobjective optimization for constrained single-objective evolutionary optimization. In: Proc. World Congr. Intell. Control Autom, 2015, pp 891–896.

  66. Potter MA, De Jong KA (2000) Cooperative coevolution: an architecture for evolving coadapted subitem. Evol Comput 8:1–29

    Google Scholar 

  67. García-Pedrajas N, Hervás-Martínez C, Ortiz-Boyer D (2005) Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Trans Evol Comput 9:271–302

    Google Scholar 

  68. Panait L, Luke S, Wiegand RP (2006) Biasing coevolutionary search for optimal multiagent behaviors. IEEE Trans Evol Comput 10:629–645

    Google Scholar 

  69. Droste S, Jansen T, Wegener I (2002) On the analysis of the (1+1) evolutionary algorithm. Theor Comput Sci 276:51–81

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors appreciate the strong support provided by the National Key R&D Program (2021YFB1714500), the National Ministry Project of China (2019-JCJQ-ZD-049-02, JCKY2019204B021). The authors would like to thank Dr. Liangyue Jia and Dr. Wenbin Ye for their helpful discussions during this work. The authors also thank the anonymous reviewers for their comments and suggestions about improving the manuscript.

Author information

Authors and Affiliations

  1. Beijing Institute of Technology, No. 5 South Street, Zhongguancun, Haidian District, Beijing, 100081, China

    Zhicheng Zhu, Jia Hao, Jin Huang & Biao Huang

Authors
  1. Zhicheng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Biao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jia Hao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, Z., Hao, J., Huang, J. et al. BC-PINN: an adaptive physics informed neural network based on biased multiobjective coevolutionary algorithm. Neural Comput & Applic 35, 21093–21113 (2023). https://doi.org/10.1007/s00521-023-08876-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08876-4

Keywords

Search

Navigation