Skip to main content
SpringerLink
Log in

Deep learning-based method coupled with small sample learning for solving partial differential equations

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Partial differential equations (PDEs) are existing widely in the field of mathematics, physics and engineering. They are often used to describe natural phenomena and model dynamical systems, but how to solve the equations efficiently is still a hard task. In this paper, we develop a deep learning-based general numerical method coupled with small sample learning (SSL) for solving PDEs. To be more specific, we approximate the solution via a deep feedforward neural network, which is trained to satisfy the PDEs with the initial and boundary conditions. Then the proposed method is modeled to solve an optimization problem by minimizing a designed cost function, which involves the residual of the differential equations, the initial/boundary conditions and the residual of a handful of observations. With a few of sample data, the model can be rectified effectively and the predictive accuracy can be improved. The effectiveness of the proposed method is demonstrated by a wide range of benchmark problems in mathematical physics, including the classical Burgers equations, Schrödinger equations, Buckley-Leverett equation, Navier-Stokes equation, and Carburizing diffusion equations, which are applied in carburizing diffusion problems in material science. And the results validate that the proposed algorithm is effective, flexible and robust without relying on trial solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Berg J, Nyström K (2018) A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing 317:28–41

    Article  Google Scholar 

  2. Berkani MS, Giurgea S, Espanet C, Coulomb JL, Kiefferet C (2013) Study on optimal design based on direct coupling between a FEM simulation model and L-BFGS-B algorithm. IEEE Trans Magn 49(5):2149–2152

    Article  Google Scholar 

  3. Weinan E, Han J, Jentzen A (2017) Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun Math Stat 5:349–380

    Article  MathSciNet  Google Scholar 

  4. Weinan E, Yu B (2018) The deep ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun Math Stat 6(1):1–12

    MathSciNet  MATH  Google Scholar 

  5. Eymard R, Gallouët T, Herbin R (2000) Finite volume methods. Handb Numer Anal 7:713–1018

    MathSciNet  MATH  Google Scholar 

  6. Fang Z, Li W, Zou J, Du Q (2016) Using CNN-based high-level features for remote sensing scene classification. In: 2016 IEEE international geoscience and remote sensing symposium, pp 2610– 2613

  7. Han J, Jentzen A, E W (2018) Solving high-dimensional partial differential equations using deep learning. Proc Natl Acad Sci 115(34):8505–8510

    Article  MathSciNet  Google Scholar 

  8. Kitchin R, Lauriault TP (2015) Small data in the era of big data. GeoJournal 80(4):463–475

    Article  Google Scholar 

  9. Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. In: Proceedings of the 25th international conference on neural information processing systems, vol 1, pp 1097–1105

  10. Lagaris IE, Likas A, Fotiadis DI (1998) Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans Neural Netw 9 (5):987–1000

    Article  Google Scholar 

  11. Lagaris IE, Likas AC, Papageorgiou DG (2000) Neural-network methods for boundary value problems with irregular boundaries. IEEE Trans Neural Netw 11(5):1041–1049

    Article  Google Scholar 

  12. Lake BM, Salakhutdinov R, Tenenbaum JB (2015) Human-level concept learning through probabilistic program induction. Science 350(6266):1332–1338

    Article  MathSciNet  Google Scholar 

  13. Lecun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  14. Meng X, Karniadakis GE (2020) A composite neural network that learns from multi-fidelity data: application to function approximation and inverse PDE problems. J Comput Phys 401:109020

    Article  MathSciNet  Google Scholar 

  15. Nabian MA, Meidani H (2019) A deep learning solution approach for high-dimensional random differential equations. Probabilistic Engineering Mechanics 57:14–25

    Article  Google Scholar 

  16. Ouyang W, Wang X, Zhang C, Yang X (2016) Factors in finetuning deep model for object detection with long-tail distribution. In: IEEE conference on computer vision and pattern recognition , pp 864–873

  17. Owhadi H, Scovel C, Sullivan T (2015) Brittleness of bayesian inference under finite information in a continuous world. Electron J Stat 9(1):1–79

    Article  MathSciNet  Google Scholar 

  18. Platte RB, Trefethen LN (2010) Chebfun: a new kind of numerical computing. Progress in Industrial Mathematics at ECMI, 69–87

  19. Raissi M, Karniadakis GE (2018) Hidden physics models: machine learning of nonlinear partial differential equations. J Comput Phys 357:125–141

    Article  MathSciNet  Google Scholar 

  20. Raissi M, Perdikaris P, Karniadakis GE (2017) Inferring solutions of differential equations using noisy multi-fidelity data. J Comput Phys 335:736–746

    Article  MathSciNet  Google Scholar 

  21. Raissi M, Perdikaris P, Karniadakis GE (2017) Machine learning of linear differential equations using gaussian processes. J Comput Phys 348:683–693

    Article  MathSciNet  Google Scholar 

  22. Raissi M, Perdikaris P, Karniadakis GE (2018) Numerical gaussian processes for time-dependent and nonlinear partial differential equations. SIAM J Sci Comput 40(1):A172–A198

    Article  MathSciNet  Google Scholar 

  23. Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707

    Article  MathSciNet  Google Scholar 

  24. Rudy SH, Brunton SL, Proctor Kutz JN (2017) Data-driven discovery of partial differential equations. Science Advances 3(4):e1602614

    Article  Google Scholar 

  25. Seeger M (2004) Gaussian processes for machine learning. Int J Neural Syst 14(02):69–106

    Article  Google Scholar 

  26. Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298

    Article  Google Scholar 

  27. Shirvany Y, Hayati M, Moradian R (2008) Numerical solution of the nonlinear Schrödinger equation by feedforward neural networks. Commun Nonlinear Sci Numer Simul 13(10):2132–2145

    Article  MathSciNet  Google Scholar 

  28. Shirvany Y, Hayati M, Moradian R (2009) Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations. Applied Soft Computing Journal 9(1):20–29

    Article  Google Scholar 

  29. Sirignano J, Spiliopoulos K (2018) DGM: a deep learning algorithm for solving partial differential equations. J Comput Phys 375:1339–1364

    Article  MathSciNet  Google Scholar 

  30. Squires TM, Mason TG (2010) Fluid mechanics of microrheology. Ann Rev Fluid Mech 42(1):413–438

    Article  Google Scholar 

  31. Stein M (1987) Large sample properties of simulations using latin hypercube sampling. Technometrics 29(2):143–151

    Article  MathSciNet  Google Scholar 

  32. Sun Z (2019) A meshless symplectic method for two-dimensional nonlinear Schrd̈inger equations based on radial basis function approximation. Eng Anal Bound Elem 104:1–7

    Article  MathSciNet  Google Scholar 

  33. Tatari M, Dehghan M (2010) A method for solving partial differential equations via radial basis functions: application to the heat equation. Eng Anal Bound Elem 34(3):206–212

    Article  MathSciNet  Google Scholar 

  34. Taylor CA, Hughes TJR, Zarins CK (1998) Finite element modeling of blood flow in arteries. Comput Methods Appl Mech Eng 158(1-2):155–196

    Article  MathSciNet  Google Scholar 

  35. Valan M, Makonyi K, Maki A, et al. (2019) Automated taxonomic identification of insects with expert-level accuracy using effective feature transfer from convolutional networks. Syst Biol 68(6):876–895

    Article  Google Scholar 

  36. Wang R, Utiyama M, Finch A, Liu L, Chen K, Sumita E (2018) Sentence selection and weighting for neural machine translation domain adaptation. 26(10):1727–1741

  37. Williams CKI, Barber D (1998) Bayesian classification with Gaussian processes. IEEE Trans Pattern Anal Mach Intell 20(12):1342–1351

    Article  Google Scholar 

  38. Yang X, Ge Y, Zhang L (2019) A class of high-order compact difference schemes for solving the Burgers’ equations. Appl Math Comput 358:394–417

    Article  MathSciNet  Google Scholar 

  39. Zhang Y (2009) A finite difference method for fractional partial differential equation. Appl Math Comput 215(2):524–529

    MathSciNet  MATH  Google Scholar 

  40. Zhu F, Ma Z, Li X, Chen G, Chien JT, Xue JH, Guo J (2019) Image-text dual neural network with decision strategy for small-sample image classification. Neurocomputing 328:182–188

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, China

    Ying Li & Fangjun Mei

Authors
  1. Ying Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fangjun Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Mei, F. Deep learning-based method coupled with small sample learning for solving partial differential equations. Multimed Tools Appl 80, 17391–17413 (2021). https://doi.org/10.1007/s11042-020-09142-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09142-8

Keywords

Search

Navigation