Application of Legendre Neural Network for solving ordinary differential equations

Highlights

• Numerical solution of ordinary differential equations using Legendre polynomial based Functional Link Artificial Neural Network (FLANN).

• It is a single layer neural network, so number of parameters is less than MLP and the hidden layer is eliminated by expanding the input pattern by Legendre polynomials.

• Unsupervised back propagation algorithm is used here.

• Obtained results are compared with the existing methods, plots and tables to show the powerfulness of the methodology.

Abstract

In this paper, a new method based on single layer Legendre Neural Network (LeNN) model has been developed to solve initial and boundary value problems. In the proposed approach a Legendre polynomial based Functional Link Artificial Neural Network (FLANN) is developed. Nonlinear singular initial value problem (IVP), boundary value problem (BVP) and system of coupled ordinary differential equations are solved by the proposed approach to show the reliability of the method. The hidden layer is eliminated by expanding the input pattern using Legendre polynomials. Error back propagation algorithm is used for updating the network parameters (weights). Results obtained are compared with the existing methods and are found to be in good agreement.

Introduction

It is well known that the differential equations are back bone of physical systems. Many problems in engineering, mathematics, physics, economics etc. may be modeled by ordinary or partial differential equations [1], [2], [3], [4]. In most cases analytical solutions of differential equations may not be obtained easily. So various numerical methods such as Runge–Kutta [5], predictor–corrector [6], finite difference, finite element [7], etc. have been developed to solve these equations. These numerical methods require the discretization of domain into the number of finite domains/points where the functions are approximated locally.

Recently, various machine intelligence methods in particular Artificial Neural Networks (ANN) are being used to solve initial and boundary value problems. The approximate solutions by ANN have many advantages. The trial solutions of ANN involve a single independent variable regardless of the dimension of the problem. The approximate solutions are continuous over all the domain of integration. Moreover, other numerical methods are usually iterative in nature, where we fix the step size before initiating the computation. After the solution is obtained, if we want to know the solution in between steps then again the procedure is to be repeated from initial stage. ANN may be one of the reliefs where we may overcome this repetition of iterations. Also we may use it as a black box to get numerical results at any arbitrary point in the domain.

In 1990, Lee and Kang [8] introduced a method to solve first order ordinary differential equation using Hopfield neural network models. Solution of linear and nonlinear ordinary differential equations using linear B₁ splines as basis function in feed forward neural network model has been approached by Meade and Fernandez [9], [10]. Liu and Jammes [11] proposed a hybrid numerical method based on both neural network and optimization techniques to solve higher order ordinary differential equations. Lagaris et al. [12] used multi layer perceptron in their network architecture to solve both ordinary and partial differential equations. Malek and Beidokhti [13] solved higher order ordinary differential equations using artificial neural networks and optimization technique. An unsupervised version of kernel least mean square algorithm for solving first and second order ordinary differential equations has been developed by Yazdi et al. [14]. Selvaraju and Samant [15] proposed new algorithms based on neural network for solving matrix Riccati differential equations. Evolutionary algorithm with neural network training has been proposed by Aarts and Van der Veer [16] for solving partial differential equation and initial value problems. Another method for solving mixed boundary value problems on irregular domains have been implemented by Hoda and Nagla [17]. Shirvany et al. [18] used multilayer perceptron and radial basis function (RBF) neural networks with a new unsupervised training method to obtain numerical solution of non linear Schrodinger equation. Mcfall and Mahan [19] introduced an artificial neural network method for solution of mixed boundary value problems with irregular domain. A multi-quadric radial basis function neural network has been used to solve linear differential equations (ordinary and elliptic partial differential equations) by Mai-Duy and Tran-Cong [20]. Recently, Mall and Chakraverty [21], [22] proposed regression based neural network model for solving lower as well as higher order ordinary differential equations. Also neural network method has been used by Ibraheem and Khalaf [23] to get the solution of boundary value problems. In another approach, Parisi et al. [24] steady-state heat transfer problem has been solved by using artificial neural network. Raja and Ahmad [25] implemented the solution of boundary value problems of one dimensional Bratu type equations using neural network.

A single layer Functional Link Artificial Neural Network (FLANN) model is introduced by Pao and Philips [26]. In FLANN, number of network parameters and number of iterations for training are less than that of multi layer perceptron (MLP) structure. In FLANN the hidden layer is replaced by a functional expansion block for enhancement of the input patterns using orthogonal polynomials such as Chebyshev, Legendre, etc. So single layer FLNN model is computationally efficient and having higher convergence speed. Chebyshev polynomial based Functional Link Artificial Neural Network has extensively applied to nonlinear dynamic system identification [27], [28], digital communication [29], channel equalization [30], function approximation [31], etc.

Here our target is to propose a single layer Legendre polynomial based Functional Link Artificial Neural Network called Legendre Neural Network (LeNN) to solve initial and boundary value problems. The Legendre Neural Network (LeNN) has been introduced by Yang and Tseng [32] for function approximation. Subsequently, LeNN has been used in channel equalization problems [33], [34], system identification [35], for prediction of machinery noise [36], etc. To the best of our knowledge, present paper may be the first of its kind where we use Legendre Neural Network (LeNN) model in solving differential equations. We propose a single layer neural network with increasing the dimension of the input pattern using Legendre polynomials. A feed forward model with principle of error back propagation algorithm has been used here. Initial weights of the single layered network model are considered as random. Some of the advantages of the new single layer Legendre Neural Network (LeNN) based model for solving differential equations are as follows:

• It is a single layer neural network, so number of parameters is less than MLP.

• Simple implementation and easy computation.

• The hidden layers are removed.

• The back propagation algorithm is unsupervised.

• No optimization technique is used.

To validate the present method we have considered various numerical examples and a problem of astrophysics viz. Lane–Emden equation.

In astrophysics, nonlinear singular initial value problems which describe the thermal behavior of a spherical cloud of gas acting under the mutual attraction of its molecules and subject to the classical laws of thermodynamics had been proposed by Lane [37] and Emden [38]. The governing differential equation then was known as Lane–Emden type equations. Further Fowler [39] studied Lane–Emden type equations in greater detail. The Lane–Emden equation is generally expressed as

$$y^{″}+\frac{\alpha }{x}{y}^{\prime }+f(x)h(y)=g(x)x\ge 0,\alpha \ge 0$$

subject to y(0) = a, y′(0) = bwhere a, b are constants and f(x), h(y) and g(x) are functions of x and y.

Lane–Emden type equations for α = 2, f(x) = 1 and with some special form of h(y) describes various phenomena in astrophysics such as theory of stellar structure, thermal behavior of a spherical cloud of gas and theory of thermionic currents [40], [41], [42].

The Lane–Emden type equations are singular at x = 0. The solution of Lane–Emden equation and other nonlinear IVPs (as mentioned above) in astrophysics are challenging because of the singular point at the origin. So various techniques based on series solutions such as Adomian decomposition [43], [44], [45], variational iteration [46], [47], [48] and homotopy perturbation [49], [50], [51], [52] methods have been used to handle the Lane–Emden equations.

Rest of the paper is organized as follows. In Section 2, we explain briefly the basic architecture of the Legendre Neural Network (LeNN) and its learning algorithm. Section 3 gives the details of problem formulation, construction of the appropriate form of LeNN trial solution and computation of gradient. Numerical examples and its results are presented in Section 4. In this section we also compare analytical and LeNN results and those are shown graphically. Conclusions are incorporated in the last section.

We now describe below the LeNN model and its learning method.