LS-SVR-based solving Volterra integral equations

doi:10.1016/j.amc.2012.05.028

Applied Mathematics and Computation

Volume 218, Issue 23, 1 August 2012, Pages 11404-11409

https://doi.org/10.1016/j.amc.2012.05.028 Get rights and content

Abstract

In this paper, a novel hybrid method is presented for solving the second kind linear Volterra integral equations. Due to the powerful regression ability of least squares support vector regression (LS-SVR), we approximate the unknown function of integral equations by using LS-SVR in intervals with known numerical solutions. The trapezoid quadrature is used to approximate subsequent integrations in intervals with unknown numerical solutions. The feasibility of the proposed method is examined on some integral equations. Experimental results of comparison with analytic and repeated modified trapezoid quadrature method’s solutions show that the proposed algorithm could reach a very high accuracy. The proposed algorithm could be a good tool for solving the second kind linear Volterra integral equations.

Introduction

Let’s consider the second kind linear integral equations with the form $f (x) + λ \int_{a}^{b} k (x, t) f (t) dt = g (x), a ⩽ x ⩽ b .$

In Eq. (1) the parameter $λ$ , functions $k (\cdot, \cdot)$ and $g (\cdot)$ are given and $f (x)$ is the unknown function to be determined. Eq. (1) is called Volterra integral equation, if the upper limit of integration satisfies $b = x$ , and it is called Fredholm integral equation, if $b$ is a constant [1], [2]. In this paper, we take into account only the Volterra case of the second kind linear integral equations.

Many approaches on numerical solutions of the second kind linear Volterra integral equations have been presented. Among others, Maleknejad et al. [3], [4] proposed the Runge–Kutta method and the Taylor-series expansion method. Rashed [5] proposed a Lagrange interpolation based approach. Jafar and Mahdi [6], [7] proposed a variable step quadrature method and a modified trapezoid quadrature method.

The support vector machine (SVM) based on statistical learning theory is a powerful tool for function regression and pattern recognition [8]. SVM maps input data into a high dimensional feature space where it may become linearly separable. Recently SVM has been applied to a wide variety of fields, such as nonlinear time series forecasting in economics field, DNA sequence prediction and protein structure prediction in bioinformatics field [9], [10], [11]. One reason that SVM often performs better than earlier conventional methods is that SVM is designed to minimize structural risk whereas previous techniques are usually based on minimization of empirical risk, which is the minimization of the number of misclassified points in the training set. So SVM is usually less vulnerable to over-fitting problem. Suykens and Vandewalle [12] proposed a modified version of SVM called least squares SVM (LS-SVM), which results in a set of linear equations instead of a quadratic programming problem and has been applied to financial time series prediction [13].

Motivated by the powerful regression ability of SVMs, we propose a hybrid approach based on LS-SVR and trapezoid quadrature to solve the second kind linear Volterra integral equations. We approximate the unknown function $f (x)$ in Eq. (1) by using LS-SVR and use the approximation of $f (x)$ step by step in the subsequent numerical solution.

The remainder of this paper is organized as follows. In Section 2, LS-SVM regression model is reviewed. The presented algorithm is described in Section 3. In Section 4, the proposed method is examined through numerical simulations. The conclusion is given in Section 5.

Section snippets

Least squares support vector regression (LS-SVR)

Consider a given training set ${(x_{i}, y_{i}) | x_{i} \in R^{n}, y_{i} \in R}_{i = 1}^{N}$ (in this paper $n = 1$ ), where $x_{i}$ is the input and $y_{i}$ the corresponding target value. The LS-SVMs model has the following form: $y = f (x) = w^{T} ϕ (x) + b,$ where the nonlinear function $ϕ (\cdot)$ maps the input into a higher dimensional feature space. For the function estimation, the optimization problems of LS-SVMs have the following form: $\min_{w, e_{i}} J (w, e) = \frac{1}{2} w^{T} w + \frac{γ}{2} \sum_{i = 1}^{N} e_{i}^{2}$ subjected to the equality constraints: $y_{i} = w^{T} ϕ (x_{i}) + b + e_{i} (i = 1, 2, \dots, N) .$

The Lagrangian corresponding

Proposed method

In this section we present a method to find numerical solutions of Eq. (1). To find an approximate solution of Eq. (1), the collocation method is employed, which assumes that a domain $I$ is discretized into a set of collocation points $\bar{I} = {x_{i}}_{i = 1}^{m}$ . Let $x_{1} = a$ and $ξ_{m} = b$ . Therefore, by substituting Eq. (7) into Eq. (1) and ensuring that Eq. (1) holds at $\bar{I}$ , the following equations can be obtained: $\{\begin{matrix} \hat{f} (x_{1}) + λ \int_{x_{1}}^{x_{1}} k (x_{1}, t) \hat{f} (t) dt = g (x_{1}), \\ \hat{f} (x_{2}) + λ \int_{x_{1}}^{x_{2}} k (x_{2}, t) \hat{f} (t) dt = g (x_{2}), \\ \hat{f} (x_{i}) + λ \int_{x_{1}}^{x_{i - 1}} k (x_{i}, t) \bar{f} (t) dt + λ \int_{x_{i - 1}}^{x_{i}} k \end{matrix}$

Numerical examples

In this section, in order to examine the proposed algorithm, we consider three elaborated Volterra integral equations that have analytic solutions. The Volterra integral equations are solved by using the proposed algorithm. All computations are implemented on a PC with 2.4 GHz CPU and 256 M bytes memory.

The parameters in regularization item and kernel function play an important role to the algorithm performance. In this paper the RBF kernel function was adopted. The parameters $σ$ and $γ$ are

Conclusion

A novel method based on least squares support vector machines (LS-SVMs) and trapezoid quadrature is proposed for solving the second kind linear Volterra integral equations. Due to the powerful regression ability of LS-SVR, we approximate the unknown function of integral equations by using LS-SVR in intervals with known numerical solutions. The trapezoid quadrature is used to approximate subsequent integrations in intervals with unknown numerical solutions. Results of comparison with analytic

Acknowledgement

The authors are grateful to the support of the NSFC (60673023, 10872077), the National High-technology Development Project of China (2007AA04Z114), the European Commission under grant No. TH/Asia Link/010 (111084), the Collaboration Project from Guangdong Province and MOE of China (2007B090400031), the Science-Technology Development Project from Jilin Province (20080708), and “985” project of Jilin University, the Natural Science Fund Project from Province Jilin (201215165).

References (14)

A. Golbabai et al.
Numerical solution of the second kind integral quations using radial basis function networks
Appl. Math. Comput.
(2006)
K. Maleknejad et al.
Using Runge–Kutta method for numerical solution of the system of Volterra integral equation
Appl. Math. Comput.
(2004)
K. Maleknejad et al.
Numerical solution of Volterra integral equations of the second kind with convolution kernel by using Taylor-series expansion method
Appl. Math. Comput.
(2005)
M.T. Rashed
Numerical solutions of functional integral equations
Appl. Math. Comput.
(2004)
K.J. Kim
Financial time series forecasting using support vector machines
Neurocomputing
(2003)
C. Chen et al.
Prediction of protein secondary structure content using support vector machine
Talanta
(2007)
A. Golbabai et al.
An iterative solution for the second kind integral equations using radial basis functions
Appl. Math. Comput.
(2006)

There are more references available in the full text version of this article.

Cited by (6)

A novel approach for solving linear Fredholm integro-differential equations via LS-SVM algorithm
2024, Applied Mathematics and Computation
In this paper, an innovative numerical methodology is introduced for solving multiple types of linear one-dimensional Fredholm integro-differential equations using least squares support vector machine (LS-SVM) algorithm, even in singular cases. The proposed method combines the original LS-SVM model with the composite Simpson method, which transforms the original solution problem into an optimization problem with equality constraints. Then the original problem is reduced to seeking the LS-SVM model parameters, which further reduces the calculation cost. An explicit expression for the numerical solution of the Fredholm integro-differential equations is given. Finally, the constructed LS-SVM model is evaluated through numerical experiments, and the obtained solutions demonstrate the robustness and validity of the constructed model.
A new approach to the numerical solution of Fredholm integral equations using least squares-support vector regression
2021, Mathematics and Computers in Simulation
Citation Excerpt :
Its regression model has been successfully applied for instance, in forecasting electricity consumption [20], predicting thermal conductivity [2] and traffic flow forecasting [12]. The method has been also extended for solving ODEs [27,30], PDEs [31], IEs [18]. In these works, the authors have used radial basis functions as the kernel.
In this paper, we develop a machine learning method with the Least Squares Support Vector Regression (LS-SVR) for the numerical solution of Fredholm integral equations. Two different approaches are proposed for training the network by using the shifted Legendre kernel, the collocation and Galerkin LS-SVR approaches. As with the standard LS-SVR for known dataset regression, the formulation of the method gives rise to an optimization problem. An equivalent system of algebraic equations is then derived and in linear cases discussed in terms of the sparsity of the matrices and computational efficiency. Finally, the method is carried out on some numerical examples, including nonlinear and multidimensional cases to show the accuracy and efficiency of the method.
Numerical solution of Volterra–Fredholm integral equations using the collocation method based on a special form of the Müntz–Legendre polynomials
2018, Journal of Computational and Applied Mathematics
Citation Excerpt :
Many approximate methods have been presented for solving of Volterra–Fredholm integral equations, such as the shifted Legendre polynomials [13], Legendre wavelets method [14], collocation method [15–17], spectral method [18], optimization technology [19] and parameterized pseudospectral integration matrices [20]. In particular, Guo et al. [21] proposed a least squares support vector regression method with trapezoid quadrature, and Wang et al. [22] proposed a numerical method by least squares approximation. In this study, we apply a hybrid approach of the Müntz–Legendre collocation and least squares approximation methods to investigate the Volterra–Fredholm integral equations of the form (1).
This paper presents a computational technique based on a special family of the Müntz–Legendre polynomials to solve a class of Volterra–Fredholm integral equations. The relationship between the Jacobi polynomials and Müntz–Legendre polynomials, in a particular state, are expressed. The proposed method reduces the integral equation into algebraic equations via the Chebyshev–Gauss–Lobatto points, so that the system matrix coefficients are obtained by the least squares approximation method. The useful properties of the Jacobi polynomials are exploited to analyze the approximation error. The performance and accuracy of our method are examined with some illustrative examples.
Numerical solution of Volterra-Fredholm integral equations based on ε-SVR method
2016, Journal of Computational and Applied Mathematics
In this paper, we try to study the numerical methods for solving integral equations from a new perspective—machine learning method. By means of the idea of kernel $ε$ -support vector regression machine ( $ε$ -SVR), we construct an optimization modeling for a class of Volterra–Fredholm integral equations and propose a novel numerical method for solving them. The proposed method has a certain versatility and can be used to solve some other kinds of integral equations. In order to verify the effectiveness of the proposed method, we perform a series of comparative experiments with six specific Volterra–Fredholm integral equations and a method proposed in Wang et al. (2014). Experimental results show that the proposed method has a good approximation property.
Approximate solutions to several classes of Volterra and Fredholm integral equations using the neural network algorithm based on the sine-cosine basis function and extreme learning machine
2023, Frontiers in Computational Neuroscience
Fuzzy C-means clustering SVR algorithm based on wavelet kernel
2014, Journal of Computational Information Systems

View full text

LS-SVR-based solving Volterra integral equations

Abstract

Introduction

Section snippets

Least squares support vector regression (LS-SVR)

Proposed method

Numerical examples

Conclusion

Acknowledgement

Appl. Math. Comput.

Appl. Math. Comput.

Appl. Math. Comput.

Appl. Math. Comput.

Neurocomputing

Talanta

An iterative solution for the second kind integral equations using radial basis functions

Appl. Math. Comput.