Neural network-based discrete-time Z-type model of high accuracy in noisy environments for solving dynamic system of linear equations

Abstract

To solve dynamic system of linear equations with square or rectangular system matrices in real time, a discrete-time Z-type model based on neural network is proposed and investigated. It is developed from and studied with the aid of a unified continuous-time Z-type model. Note that the framework of such a unified continuous-time Z-type model is generic and has a wide range of applications, such as robotic redundancy resolution with quadratic programming formulations. To do so, a one-step-ahead numerical differentiation formula and its optimal sampling-gap rule in noisy environments are presented. We compare the Z-type model extensively with E-type and N-type models. Theoretical results on stability and convergence are provided which show that the maximal steady-state residual errors of the Z-type, E-type and N-type models have orders $O(\tau ^3)$, $O(\tau ^2)$ and $O(\tau )$, respectively, where $\tau$ is the sampling gap. We also prove that the residual error of any static method that does not exploit the time-derivative information of a time-dependent system of linear equations has order $O(\tau )$ when applied to solve discrete real-time dynamic system of linear equations. Finally, several illustrative numerical experiments in noisy environments as well as two application examples to the inverse-kinematics control of redundant manipulators are provided and illustrated. Our analysis substantiates the efficacy of the Z-type model for solving the dynamic system of linear equations in real time.

Appendices

Appendix 1 (Z-type design formula)

By employing the Z-type design formula (Zhang formula) $\dot{\mathbf {e}}(t)=-\gamma \mathbf {e}(t)$, the continuous-time Z-type model for normal and underdetermined system of linear equations (i.e., for $A(t)\in \mathbb {R}^{m\times n}$ with $m\le n$) has the form (3) [32, 34], i.e.,

$$\begin{aligned} \dot{\mathbf {x}}(t)=A^+(t)(\dot{\mathbf {b}}(t)-\dot{A}(t)\mathbf {x}(t)-\gamma (A(t)\mathbf {x}(t)-\mathbf {b}(t))), \end{aligned}$$

where $\mathbf {x}(t)$ (with known initial value $\mathbf {x}(0)$) is the neural state corresponding to the exact dynamic solution $\mathbf {x}^{*}(t)$ of (1). For the case of overdetermined system of linear equations with $m>n$ [33], we have

$$\begin{aligned} \begin{aligned} A^{\mathrm{T}}(t)A(t)\dot{\mathbf {x}}(t)=&-A^{\mathrm{T}}(t)\dot{A}(t)\mathbf {x}(t)+A^{\mathrm{T}}(t)\dot{\mathbf {b}}(t)-\gamma A^{\mathrm{T}}(t)(A(t)\mathbf {x}(t)-\mathbf {b}(t)), \end{aligned} \end{aligned}$$

(18)

where $\mathbf {x}(t)$ is the neural state corresponding to an approximate dynamic solution (e.g., a pseudoinverse-type solution) $\mathbf {x}^{*}(t)$ of (1). If A(t) is of full column rank, then $A^{\mathrm{T}}(t)A(t)$ is invertible and $A^+(t)=(A^{\mathrm{T}}(t)A(t))^{-1}A^{\mathrm{T}}(t)\in \mathbb {R}^{n\times m}$ holds [33]. Thus (18) can be rewritten as (3), i.e.,

$$\begin{aligned} \dot{\mathbf {x}}(t)=A^+(t)(\dot{\mathbf {b}}(t)-\dot{A}(t)\mathbf {x}(t)-\gamma (A(t)\mathbf {x}(t)-\mathbf {b}(t))). \end{aligned}$$

Evidently, the formulations of these two Z-type models are the same. Thus, the continuous-time Z-type model for solving the dynamic system of linear equations with normal, overdetermined and underdetermined situations all handled can be uniformly expressed as (3). This discovery is also a contribution of this paper.

Appendix 2 (multistep methods)

All of the models in Table 2 are expressed there as multistep formulas for solving dynamic system of linear equations problem numerically. Multistep methods and general difference equations have been studied for centuries. We will explain some of their general concepts and theoretical results here, see e.g., [38, Chap. 17.6] for a better understanding of the theoretical results regarding stability and convergence in Sects. 3.3 and 4.1.

An N step method $\sum ^{N}_{j=0}\alpha _j\eta _{k+j}=\tau \sum ^{N}_{j=0}\beta _j\psi _{k+j}$ can be checked for 0-stability by determining the roots of the characteristic polynomial $P_{N}(q)=\sum ^{N}_{j=0}\alpha _j q^j$.

Result 1

[38, pp. 474–475] If all roots of $P_N(q)=0$ satisfy $|q|\le 1$ with $|q|=1$ being simple, then the N step method is 0-stable.

An N step method is said to be consistent of order p if the truncation error for the exact solution is of order $O(\tau ^p)$ where $p>0$.

Result 2

[38, Chap. 17.6] An N step method is convergent, i.e., $\eta _{[(t-t_0)/\tau ]}\rightarrow \eta ^{*}(t)$ for all $t\in [t_0,t_{\text {f}}]$ as $\tau \rightarrow 0$, if and only if the method is 0-stable and consistent. Thus 0-stability plus consistency results in convergence.

Moreover we have

Result 3

[38, pp. 452 (formulated for one-step methods)] A 0-stable consistent method converges with the order of its truncation error.

These three classical and well-known results from numerical analysis are essential for understanding the proofs of Theorems 3–6 in Sects. 3.3 and 4.1.

Appendix 3 (L-type models)

In general, stable and convergent discrete-time models cannot be generated from every numerical differentiation formula. To illustrate, three Lagrange-type one-step-ahead numerical differentiation formulas using multiple data points with high computational precision [46] are displayed in Table 3. Evidently these formulas are possible candidates to discretize the continuous-time Z-type model (3). In Table 4, we have constructed the respective L-type models based on these three Lagrange-type one-step-ahead numerical differentiation formulas. Recall Appendix 2 on the stability of multistep methods: Unfortunately one of the roots of the characteristic polynomials for the L-type models lies outside the unit disk, making each of these three L-type models not 0-stable. Besides, the performance of the L-type models through computer simulations of Example 1 is completely unacceptable, see Fig. 9. The documented divergence occurs during the solution process when using the four-point and six-point L-type models with step size $h=0.3$, sampling gap $\tau =0.01$ s, and randomly generated initial state $\mathbf {x}_0\in [-0.5, 0.5]^9$ as done in Example 1. Thus the ‘solutions’ computed via L-type models do not converge to the exact solutions at all. Other L-type models relying on multiple data points such as seven or eight produce similar bad results.

Table 3 One-step-ahead differentiation formulas using multiple data points from [46]

Table 4 Multiple-point L-type models using the differentiation formulas shown in Table 3

Fig. 9

Divergence phenomena occurring during the solution process when employing two multiple-point L-type models with the data of Example 1. a L-type model using four data points and b L-type model using six data points