Skip to main content

Advertisement

An enhanced hybrid adaptive physics-informed neural network for forward and inverse PDE problems

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Physics-informed neural networks (PINNs) have emerged as a powerful tool for solving partial differential equations (PDEs) in various scientific and engineering applications. PINNs integrate the PDEs into the loss functions of neural networks through automatic differentiation, which comprises a weighted combination of the PDE residuals, boundary and initial constraints, and observed data. However, the accuracy and efficiency of PINNs need substantial improvement to meet the requirements across a wider range of challenging problems. In this paper, we propose a hybrid adaptive (HA) sampling method and a feature embedding layer for PINN. The spatiotemporal points for PINN training (called residual points) are resampled during iteration procedure via the proposed HA method. The HA method ensures randomness in the selection of points while also prioritizing those with large PDE residuals, thereby ensuring efficient training by focusing the network’s learning on areas where the model’s predictions are less accurate. Moreover, the vanilla PINN architecture is further enhanced by a feature embedding layer, which transforms the raw input into a higher-dimensional space, enabling the network to better capture complex relationships underlying in PDEs and improve its fitting ability. The numerical experiments on a variety of forward and inverse PDE problems have shown that the proposed methods significantly improve the accuracy and efficiency of PINN with reduced reliance on the number of sampling points. The L2 relative error of vanilla PINN can be reduced by approximately 1\(\sim \)2 orders of magnitude and the proposed methods outperform state-of-the-art baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The data in this study is available from the corresponding author upon reasonable request.

References

  1. Chernyshenko AY, Olshanskii MA (2015) An adaptive octree finite element method for PDEs posed on surfaces. Comput Methods Appl Mech Eng 291:146–172. https://doi.org/10.1016/j.cma.2015.03.025

    Article  MathSciNet  MATH  Google Scholar 

  2. Strikwerda JC (1984) Finite difference methods for the stokes and navier-stokes equations. SIAM J Sci Stat Comput 5(1):56–68. https://doi.org/10.1137/0905004

    Article  MathSciNet  MATH  Google Scholar 

  3. Javidi M (2006) A numerical solution of the generalized burger’s-huxley equation by spectral collocation method. Appl Math Comput 178(2):338–344. https://doi.org/10.1016/j.amc.2005.11.051

    Article  MathSciNet  MATH  Google Scholar 

  4. Darbon J, Osher S (2016) Algorithms for overcoming the curse of dimensionality for certain hamilton-jacobi equations arising in control theory and elsewhere. Res Math Sci 3(1). https://doi.org/10.1186/s40687-016-0068-7

  5. Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707. https://doi.org/10.1016/j.jcp.2018.10.045

    Article  MathSciNet  MATH  Google Scholar 

  6. Zhang D, Guo L, Karniadakis GE (2020) Learning in modal space: Solving time-dependent stochastic pdes using physics-informed neural networks. SIAM J Sci Comput 42(2):639–665. https://doi.org/10.1137/19m1260141

    Article  MathSciNet  MATH  Google Scholar 

  7. Pang G, Lu L, Karniadakis GE (2019) fPinns: Fractional physics-informed neural networks. SIAM J Sci Comput 41(4):2603–2626. https://doi.org/10.1137/18m1229845

    Article  MathSciNet  MATH  Google Scholar 

  8. Jin X, Cai S, Li H, Karniadakis GE (2021) Nsfnets (navier-stokes flow nets): Physics-informed neural networks for the incompressible navier-stokes equations. J Comput Phys 426:109951. https://doi.org/10.1016/j.jcp.2020.109951

    Article  MathSciNet  MATH  Google Scholar 

  9. Raissi M, Yazdani A, Karniadakis GE (2020) Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science 367(6481):1026–1030. https://doi.org/10.1126/science.aaw4741

    Article  MathSciNet  MATH  Google Scholar 

  10. Yin M, Zheng X, Humphrey JD, Karniadakis GE (2021) Non-invasive inference of thrombus material properties with physics-informed neural networks. Comput Methods Appl Mech Eng 375:113603. https://doi.org/10.1016/j.cma.2020.113603

    Article  MathSciNet  MATH  Google Scholar 

  11. Chen Y, Lu L, Karniadakis GE, Dal Negro L (2020) Physics-informed neural networks for inverse problems in nano-optics and metamaterials. Opt Express 28(8):11618. https://doi.org/10.1364/oe.384875

    Article  MATH  Google Scholar 

  12. Sahli Costabal F, Yang Y, Perdikaris P, Hurtado DE, Kuhl E (2020) Physics-informed neural networks for cardiac activation mapping. Front Phys 8. https://doi.org/10.3389/fphy.2020.00042

  13. Olivares E, Ye H, Herrero A, Nia BA, Ren Y, Donovan RP, Flaviis F (2021) Applications of information channels to physics-informed neural networks for wifi signal propagation simulation at the edge of the industrial internet of things. Neurocomputing 454:405–416. https://doi.org/10.1016/j.neucom.2021.04.021

    Article  Google Scholar 

  14. Wu C, Zhu M, Tan Q, Kartha Y, Lu L (2023) A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks. Comput Methods Appl Mech Eng 403:115671. https://doi.org/10.1016/j.cma.2022.115671

    Article  MathSciNet  MATH  Google Scholar 

  15. Zeng S, Zhang Z, Zou Q (2022) Adaptive deep neural networks methods for high-dimensional partial differential equations. J Comput Phys 463:111232. https://doi.org/10.1016/j.jcp.2022.111232

    Article  MathSciNet  MATH  Google Scholar 

  16. Yu J, Lu L, Meng X, Karniadakis GE (2022) Gradient-enhanced physics-informed neural networks for forward and inverse pde problems. Comput Methods Appl Mech Eng 393:114823. https://doi.org/10.1016/j.cma.2022.114823

    Article  MathSciNet  MATH  Google Scholar 

  17. Buzaev F, Gao J, Chuprov I, Kazakov E (2023) Hybrid acceleration techniques for the physics-informed neural networks: a comparative analysis. Mach Learn 113(6):3675–3692. https://doi.org/10.1007/s10994-023-06442-6

    Article  MathSciNet  MATH  Google Scholar 

  18. Lu L, Meng X, Mao Z, Karniadakis GE (2021) DeepXDE: A deep learning library for solving differential equations. SIAM Rev 63(1):208–228. https://doi.org/10.1137/19m1274067

    Article  MathSciNet  MATH  Google Scholar 

  19. Nabian MA, Gladstone RJ, Meidani H (2021) Efficient training of physics-informed neural networks via importance sampling. Comput-Aid Civil Infr Eng 36(8):962–977. https://doi.org/10.1111/mice.12685

    Article  MATH  Google Scholar 

  20. Hao Z, Yao J, Su C, Su H, Wang Z, Lu F, Xia Z, Zhang Y, Liu S, Lu L, Zhu J (2023) PINNacle: A comprehensive benchmark of physics-informed neural networks for solving PDEs

  21. Hanna JM, Aguado JV, Comas-Cardona S, Askri R, Borzacchiello D (2022) Residual-based adaptivity for two-phase flow simulation in porous media using physics-informed neural networks. Comput Methods Appl Mech Eng 396:115100. https://doi.org/10.1016/j.cma.2022.115100

    Article  MathSciNet  MATH  Google Scholar 

  22. Tang K, Wan X, Yang C (2023) DAS-PINNs: a deep adaptive sampling method for solving high-dimensional partial differential equations. J Comput Phys 476:111868. https://doi.org/10.1016/j.jcp.2022.111868

    Article  MathSciNet  MATH  Google Scholar 

  23. Daw A, Bu J, Wang S, Perdikaris P, Karpatne A (2023) Mitigating propagation failures in physics-informed neural networks using retain-resample-release (R3) sampling. In: Proceedings of the 40th international conference on machine learning, vol. 202. pp 7264–7302. https://proceedings.mlr.press/v202/daw23a.html

  24. Jiao Y, Li D, Lu X, Yang JZ, Yuan C (2024) A Gaussian mixture distribution-based adaptive sampling method for physics-informed neural networks. Eng Appl Artif Intell 135:108770. https://doi.org/10.1016/j.engappai.2024.108770

    Article  MATH  Google Scholar 

  25. Lau GKR, Hemachandra A, Ng S-K, Low BKH (2024) PINNACLE: PINN adaptive collocation and experimental points selection. In: The twelfth international conference on learning representations

  26. Tang K, Zhai J, Wan X, Yang C (2024) Adversarial adaptive sampling: Unify PINN and optimal transport for the approximation of PDEs. In: The twelfth international conference on learning representations. https://openreview.net/forum?id=7QI7tVrh2c

  27. Zhang Z, Li J, Liu B (2024) Annealed adaptive importance sampling method in PINNs for solving high dimensional partial differential equations. arXiv:2405.03433. [math.NA]

  28. Yang Z, Qiu Z, Fu D (2023) DMIS: dynamic mesh-based importance sampling for training physics-informed neural networks. Proc AAAI Conf Artif Intell 37(4):5375–5383. https://doi.org/10.1609/aaai.v37i4.25669

    Article  MATH  Google Scholar 

  29. Wang S, Teng Y, Perdikaris P (2021) Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM J Sci Comput 43(5):3055–3081. https://doi.org/10.1137/20m1318043

    Article  MathSciNet  MATH  Google Scholar 

  30. Wang S, Yu X, Perdikaris P (2022) When and why pinns fail to train: a neural tangent kernel perspective. J Comput Phys 449:110768. https://doi.org/10.1016/j.jcp.2021.110768

    Article  MathSciNet  MATH  Google Scholar 

  31. Xiang Z, Peng W, Liu X, Yao W (2022) Self-adaptive loss balanced physics-informed neural networks. Neurocomputing 496:11–34. https://doi.org/10.1016/j.neucom.2022.05.015

    Article  MATH  Google Scholar 

  32. Lu L, Pestourie R, Yao W, Wang Z, Verdugo F, Johnson SG (2021) Physics-informed neural networks with hard constraints for inverse design. SIAM J Sci Comput 43(6):1105–1132. https://doi.org/10.1137/21m1397908

    Article  MathSciNet  MATH  Google Scholar 

  33. Gu Y, Yang H, Zhou C (2021) SelectNet: self-paced learning for high-dimensional partial differential equations. J Comput Phys 441:110444. https://doi.org/10.1016/j.jcp.2021.110444

    Article  MathSciNet  MATH  Google Scholar 

  34. Li W, Zhang C, Wang C, Guan H, Tao D (2022) Revisiting PINNs: generative adversarial physics-informed neural networks and point-weighting method. arXiv:2205.08754. [cs.LG]

  35. McClenny LD, Braga-Neto UM (2023) Self-adaptive physics-informed neural networks. J Comput Phys 474:111722. https://doi.org/10.1016/j.jcp.2022.111722

    Article  MathSciNet  MATH  Google Scholar 

  36. Song Y, Wang H, Yang H, Taccari ML, Chen X (2024) Loss-attentional physics-informed neural networks. J Comput Phys 501:112781. https://doi.org/10.1016/j.jcp.2024.112781

    Article  MathSciNet  MATH  Google Scholar 

  37. Psaros AF, Kawaguchi K, Karniadakis GE (2022) Meta-learning pinn loss functions. J Comput Phys 458:111121. https://doi.org/10.1016/j.jcp.2022.111121

    Article  MathSciNet  MATH  Google Scholar 

  38. Gao H, Sun L, Wang J-X (2021) Phygeonet: physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state pdes on irregular domain. J Comput Phys 428:110079. https://doi.org/10.1016/j.jcp.2020.110079

    Article  MathSciNet  MATH  Google Scholar 

  39. Ren P, Rao C, Liu Y, Wang J-X, Sun H (2022) Phycrnet: Physics-informed convolutional-recurrent network for solving spatiotemporal pdes. Comput Methods Appl Mech Eng 389:114399. https://doi.org/10.1016/j.cma.2021.114399

    Article  MathSciNet  MATH  Google Scholar 

  40. Kumar R (2022) A Lyapunov-stability-based context-layered recurrent pi-sigma neural network for the identification of nonlinear systems. Appl Soft Comput 122:108836. https://doi.org/10.1016/j.asoc.2022.108836

    Article  MATH  Google Scholar 

  41. Kumar R (2023) Memory recurrent Elman neural network-based identification of time-delayed nonlinear dynamical system. IEEE Trans Syst Man Cybern Syst 53(2):753–762. https://doi.org/10.1109/tsmc.2022.3186610

    Article  MATH  Google Scholar 

  42. Kumar R (2023) Double internal loop higher-order recurrent neural network-based adaptive control of the nonlinear dynamical system. Soft Comput 27(22):17313–17331. https://doi.org/10.1007/s00500-023-08061-8

    Article  MATH  Google Scholar 

  43. Kumar R (2024) Recurrent context layered radial basis function neural network for the identification of nonlinear dynamical systems. Neurocomputing 580:127524. https://doi.org/10.1016/j.neucom.2024.127524

    Article  MATH  Google Scholar 

  44. Guan W, Yang K, Chen Y, Liao S, Guan Z (2023) A dimension-augmented physics-informed neural network (dapinn) with high level accuracy and efficiency. J Comput Phys 491:112360. https://doi.org/10.1016/j.jcp.2023.112360

    Article  MathSciNet  MATH  Google Scholar 

  45. Wang S, Wang H, Perdikaris P (2021) On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks. Comput Methods Appl Mech Eng 384:113938. https://doi.org/10.1016/j.cma.2021.113938

    Article  MathSciNet  MATH  Google Scholar 

  46. Li X, Wu J, Tai X, Xu J, Wang Y-G (2024) Solving a class of multi-scale elliptic PDEs by Fourier-based mixed physics informed neural networks. J Comput Phys 508:113012. https://doi.org/10.1016/j.jcp.2024.113012

    Article  MathSciNet  MATH  Google Scholar 

  47. Jagtap AD, Kharazmi E, Karniadakis GE (2020) Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Comput Methods Appl Mech Eng 365:113028. https://doi.org/10.1016/j.cma.2020.113028

    Article  MathSciNet  MATH  Google Scholar 

  48. Shukla K, Jagtap AD, Karniadakis GE (2021) Parallel physics-informed neural networks via domain decomposition. J Comput Phys 447:110683. https://doi.org/10.1016/j.jcp.2021.110683

    Article  MathSciNet  MATH  Google Scholar 

  49. Wu W, Feng X, Xu H (2022) Improved deep neural networks with domain decomposition in solving partial differential equations. J Sci Comput 93(1). https://doi.org/10.1007/s10915-022-01980-y

  50. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  51. Zhu C, Byrd RH, Lu P, Nocedal J (1997) Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization. ACM Trans Math Softw 23(4):550–560. https://doi.org/10.1145/279232.279236

    Article  MathSciNet  MATH  Google Scholar 

  52. Mckay MD, Beckman RJ, Conover WJ (2000) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 42(1):55–61. https://doi.org/10.1080/00401706.2000.10485979

    Article  MATH  Google Scholar 

  53. Stein M (1987) Large sample properties of simulations using latin hypercube sampling. Technometrics 29(2):143–151. https://doi.org/10.1080/00401706.1987.10488205

    Article  MathSciNet  MATH  Google Scholar 

  54. Möller M, Kuzmin D (2006) Adaptive mesh refinement for high-resolution finite element schemes. Int J Numer Meth Fluids 52(5):545–569. https://doi.org/10.1002/fld.1183

  55. Mildenhall B, Srinivasan P, Tancik M, Barron J, Ramamoorthi R, Ng R (2020) Nerf: Representing scenes as neural radiance fields for view synthesis. In: European conference on computer vision. pp 405–421

  56. Tancik M, Srinivasan P, Mildenhall B, Fridovich-Keil S, Raghavan N, Singhal U, Ramamoorthi R, Barron J, Ng R (2020) Fourier features let networks learn high frequency functions in low dimensional domains. Adv Neural Inf Process Syst 33:7537–7547

    Google Scholar 

Download references

Acknowledgements

The work in this paper is partially supported by Guangdong Innovation and Entrepreneurship Team Project under Grant 2021ZT09X070, Guangdong Zhujiang Leadership Project under Grant 2021CX02X011, and Chinese Huoju Talents Project under Grant 2023HJJH01.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shaolin Liao.

Ethics declarations

Conflicts of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luo, K., Liao, S., Guan, Z. et al. An enhanced hybrid adaptive physics-informed neural network for forward and inverse PDE problems. Appl Intell 55, 255 (2025). https://doi.org/10.1007/s10489-024-06195-2

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06195-2

Keywords