Abstract
Several neural network approaches for solving differential equations employ trial solutions with a feedforward neural network. There are different means to incorporate the trial solution in the construction, for instance one may include them directly in the cost function. Used within the corresponding neural network, the trial solutions define the so-called neural form. Such neural forms represent general, flexible tools by which one may solve various differential equations. In this article we consider time-dependent initial value problems, which requires to set up the trial solution framework adequately.
The neural forms presented up to now in the literature for such a setting can be considered as first order polynomials. In this work we propose to extend the polynomial order of the neural forms. The novel construction includes several feedforward neural networks, one for each order. The feedforward neural networks are optimised using a stochastic gradient descent method (ADAM). As a baseline model problem we consider a simple yet stiff ordinary differential equation. In experiments we illuminate some interesting properties of the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yadav, N., Yadav, A., Kumar, M.: An Introduction to Neural Network Methods for Differential Equations. SpringerBriefs in Applied Sciences and Technology, Netherlands (2015). https://doi.org/10.1007/978-94-017-9816-7
Maede, A.J., Jr., Fernandez, A.A.: The numerical solution of linear ordinary differential equations by feedforward neural networks. Math. Comput. Model. 19(12), 1–25 (1994). https://doi.org/10.1016/0895-7177(94)90095-7
Dissanayake, M.W.M.G., Phan-Thien, N.: Neural-network-based approximations for solving partial differential equations. Commun. Numer. Methods Eng. 10(3), 195–201 (1994). https://doi.org/10.1002/cnm.1640100303
Mall, S., Chakraverty, S.: Application of Legendre Neural Network for solving ordinary differential equations. Appl. Soft Comput. 43, 347–356 (2016). https://doi.org/10.1016/j.asoc.2015.10.069
Zjavka, L., Pedrycz, W.: Constructing general partial differential equations using polynomial and neural networks. Neural Netw. 73, 58–69 (2016). https://doi.org/10.1016/j.neunet.2015.10.001
Famelis, I.T., Kaloutsa, V.: Parameterized neural network training for the solution of a class of stiff initial value systems. Neural Comput. Appl. 33(8), 3363–3370 (2020). https://doi.org/10.1007/s00521-020-05201-1
Schneidereit, T., Breuß, M.: Solving ordinary differential equations using artificial neural networks - a study on the solution variance. In: Proceedings of the Conference Algoritmy, pp. 21–30 (2020)
Lagaris, I.E., Likas, A., Fotiadis, D.I.: Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Networks 9(5), 987–1000 (1998). https://doi.org/10.1109/72.712178
Lagari, P.L., Tsoukalas, L.H., Safarkhani, S., Lagaris, I.E.: Systematic construction of neural forms for solving partial differential equations inside rectangular domains, subject to initial, boundary and interface conditions. Int. J. Artif. Intell. Tools 29(5), 2050009 (2020). https://doi.org/10.1142/S0218213020500098
Lagaris, I.E., Likas, A., Papageorgiou, D.G.: Neural-network methods for boundary value problems with irregular boundaries. IEEE Trans. Neural Networks 11(5), 1041–1049 (2000). https://doi.org/10.1109/72.870037
Tsoulos, I.G., Gavrilis, D., Glavas, E.: Solving differential equations with constructed neural networks. Neurocomputing 72(10), 2385–2391 (2009). https://doi.org/10.1016/j.neucom.2008.12.004
Piscopo, M.L., Spannowsky, M., Waite, P.: Solving differential equations with neural networks: applications to the calculation of cosmological phase transitions. Phys. Rev. D 100(1), 016002 (2019). https://doi.org/10.1103/PhysRevD.100.016002
Rudd, K., Ferrari, S.: A constrained integration (CINT) approach to solving partial differential equations using artificial neural networks. Neurocomputing 155, 277–285 (2015). https://doi.org/10.1016/j.neucom.2014.11.058
Kingma, D.P., Ba, J.: ADAM: A Method for Stochastic Optimization. arXiv preprint:1412.6980 (2017)
Antia, H.M.: Numerical Methods for Scientists and Engineers, 1st edn. Hindustan Book Agency, New Delhi (2012)
Dahlquist, G.G.: G-stability is equivalent to A-stability. BIT Numer. Math. 18(4), 384–401 (1978). https://doi.org/10.1007/BF01932018
Fernández-Redondo, M., Hernández-Espinosa, C.: Weight initialization methods for multilayer feedforward. ESANN, pp. 119–124 (2001)
Acknowledgement
This publication was funded by the Graduate Research School (GRS) of the Brandenburg University of Technology Cottbus-Senftenberg. This work is part of the Research Cluster Cognitive Dependable.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Schneidereit, T., Breuß, M. (2021). Polynomial Neural Forms Using Feedforward Neural Networks for Solving Differential Equations. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2021. Lecture Notes in Computer Science(), vol 12854. Springer, Cham. https://doi.org/10.1007/978-3-030-87986-0_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-87986-0_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87985-3
Online ISBN: 978-3-030-87986-0
eBook Packages: Computer ScienceComputer Science (R0)