Elsevier

Neurocomputing

Volume 394, 21 June 2020, Pages 84-94
Neurocomputing

Two neural dynamics approaches for computing system of time-varying nonlinear equations

https://doi.org/10.1016/j.neucom.2020.02.011Get rights and content

Abstract

The problem of solving time-varying nonlinear equations has received much attention in many fields of science and engineering. In this paper, firstly, considering that the classical gradient-based neural dynamics (GND) might result in nonnegligible residual error in handling time-varying nonlinear equations, an adaptive coefficient GND (AGND) model is constructed as an improvement. Besides, the secondly new designed model is the projected zeroing neural dynamics (PZND) to relieve the limitation on the available activation function, which can be of saturation and non-convexity different from that should be unbounded and convex described in the traditional zeroing neural dynamics (ZND) approach. Moreover, theoretical analyses on the AGND model and PZND model are provided to guarantee their effectiveness. Furthermore, computer simulations are conducted and analyzed to illustrate the efficacy and superiority of the two new neural dynamics models designed for online solving time-varying nonlinear equations. Finally, applications to robot manipulator motion generation and Lorenz system are provided to show the feasibility and practicability of the proposed approaches.

Introduction

It can be observed in various areas that many systems in scientific and engineering fields or even in social science are inherently of nonlinearity [1], [2], [3]. Due to the wide applications of nonlinear systems in many fields such as control theory, optimization, signal processing, robotics, and multi-agent systems and so on [4], [5], [6], [7], [8], [9], the problem of solving nonlinear equations has received much concern in recent years. For example, the inequality-constrained quadratic programming for robot control can be formulated as a system of nonlinear equations [10], [11], [12]. Generally, these nonlinear equations can not be solved via direct methods but aided with the recurrent approaches (e.g., numerical algorithms, neural dynamics) to approximate the theoretical solution until a predefined accuracy is achieved. For example, Taylor series, Newton method, Levenberg-Marquardt method and their various modifications [1], [5], [13], [14], [15], [16], [17], [18], as classical numerical algorithms, are exploited to solve nonlinear equations within different applications.

As an extension to the problem of static nonlinear equations, the time-varying ones are frequently involved in abundant practical applications [19], [20], [21], [22], [23], [24], e.g., robot manipulator motion generation [8], [17], [20], [25], switched system design [24]. It is worth noting that the classical numerical algorithms for solving time-varying nonlinear equations usually suffer from the separation between the current solution and the previous solution. Thus, the real-time solving requirements cannot be well satisfied in view of the time information being not available in the process of solving time-varying nonlinear equations [26], [27].

Benefited from the natures of high-speed parallel-processing, distributed-storage, and potential implementation by hardware, artificial neural dynamics method has been widely applied in various fields [28], [29], [30], [31], [32]. As one of the most commonly used artificial neural dynamics, recurrent neural dynamics (RND) is often developed for achieving the approximated solution to time-varying nonlinear equations owing to its capability of conveying time information and exhibiting the dynamic temporal behavior for a time sequence. This characteristic makes it applicable to time-varying tasks, such as, unsegmented, connected handwriting recognition or speech recognition [33]. As a result, a great deal of efforts are devoted to efficiently, stably, accurately and timely solving time-varying nonlinear equations by putting forward different types of recurrent neural dynamics models [13], [17], [34], [35].

Generally speaking, RND models for solving time-varying nonlinear equations can be classified into two categories, i.e., the gradient-based neural dynamics (GND) and zeroing neural dynamics (ZND). Specifically, for the GND model, defining a certain kind norm of the error as the objective function and starting with a randomly-generated initial state, it iteratively updates the candidate solution along the gradient-descent direction with a proportional manner [15], [34]. Then, the equilibrium solutions are recurrently generated until a predefined precision is achieved. Note that, parameters in the problem of time-varying nonlinear equations are changing over time, and the time-derivative information of the time-varying parameters may exert an important influence on the evolution direction. Thus, the time-derivative information should be considered as a compensation factor for more accurately predicting the solving process for the time-varying problems. Jin et al. point out that the GND method cannot effectively make use of the time-derivative information thus inevitably suffering from the separation between the current solution and the previous solution in the solving process of time-varying nonlinear equations with nonnegligible lagging errors [36]. Further, it is substantiated that the GND model can only approximately approach to the theoretical solution, instead of converging exactly in practical applications [37].

Obviously, for an iteratively computational method, it is a fundamental and important requirement to determine the correct evolution direction impelling the corresponding residual error to decrease as time evolves. To better determine the evolution direction in a predictive manner, Zhang et al. develop a type of ZND model by taking advantage of the time-derivative information of the time-varying parameters, which becomes more efficient for solving time-varying problems [26], [27]. As a development, several ZND models are presented for solving the time-varying matrix inversion, dynamics optimization and so on [38], [39], [40]. Furthermore, considering the fact that the ZND is difficult to be implemented directly on digital computers, the ZND model is further extended to the discrete version termed discrete-time zeroing neural dynamics (DTZND). It is worth pointing out that both the continuous-time and discrete-time ZND can achieve high accuracy. Thereinafter, we would only discuss the continuous-time version of ZND model.

The remainder of this paper is organized as follows. In Section 2, formulation of the problem of time-varying nonlinear equations is introduced and some existing results are provided in detail. In Section 3, the requirement for the fixed coefficient on time-varying nonlinear equations of traditional GND model is released by proposing an adaptive coefficient. Then, a new nonconvex activation function is constructed to remedy the convex restriction in the existing ZND model in Section 4. Section 5 provides illustrative simulation examples to substantiate the efficacy and superiority of the proposed models. Section 6 concludes the paper with final remarks. Before ending the introduction section, the main highlights of this paper are summarized as follows.

  • It is the first time to develop the adaptive coefficient gradient-based neural dynamics (AGND) model exploiting the adaptive coefficient for solving time-varying nonlinear equations, which is demonstrated to be of no theoretical errors and no matrix inversion operation.

  • The proposed projected zeroing neural dynamics (PZND) model with nonconvex activation functions relaxes the convex restriction in the existing ZND models to solve time-varying nonlinear equations.

  • Theoretical analyses are provided to show the robustness and global convergence of the proposed neural dynamics models. Besides, numerical experiments as well as applications to robot manipulator motion generation and the Lorenz system are presented, which verifies the validity and superiority of the proposed neural dynamics models.

Section snippets

Problem formulation and existing solution techniques

In this section, the problem formulation is discussed in detail. Firstly, the mathematical formulation of the time-varying nonlinear equations is presented. Then, the design procedures of the traditional GND and ZND models are revisited from the zero-finding or optimization perspective.

Adaptive gradient-based neural dynamics

In this section, firstly, the convergence upper bound of traditional GND model (4) is analyzed. Then, to remedy the shortcoming of the GND model (4), the proposed AGND model is designed by introducing an adaptive coefficient.

Projected zeroing neural dynamics

In this section, a PZND model is presented to relax the bounded and convex restriction of the activation function of ZND model (7). In addition, theoretical analyses are provided to validate the superiority of the proposed PZND model.

Simulation experiments

In Section 3 and Section 4, the proposed AGND model (18) and PZND model (20) are presented with theoretical analyses on their convergence performance. In this section, in order to demonstrate the effectiveness and superiority of AGND model (18) and PZND model (20), an illustrative example of nonlinear equations with time-varying parameters investigated in [42], [43] is studied as a benchmark. Further, the application to robot manipulator motion generation is conducted to illustrate the

Conclusions

In this paper, two novel neural dynamics approaches, i.e., the AGND model and PZND model, have been proposed for solving time-varying nonlinear equations. Firstly, with theoretical analyses presented, AGND model equipped with the global convergence performance and zero residual error has been constructed, which is superior than the traditional GND model. Secondly, progresses have been made in relaxing the bound and convex restriction activation function for the existing ZND model and then the

CRediT authorship contribution statement

Xiuchun Xiao: Writing - original draft, Conceptualization, Methodology, Software, Investigation. Dongyang Fu: Validation, Formal analysis, Visualization, Software. Guancheng Wang: Resources, Writing - review & editing, Supervision, Data curation. Shan Liao: Writing - review & editing, Supervision, Data curation. Yimeng Qi: Writing - review & editing. Haoen Huang: Writing - review & editing. Long Jin: Conceptualization, Methodology, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 61703189, in part by the International Science and Technology Cooperation Program of China under Grant 2017YFE0118900, in part by the Doctoral Initiating Project of Guangdong Ocean University under Grant E13428, in part by the Innovation and Strength Project of Guangdong Ocean University under Grant Q15090 and Grant 230419065, in part by the Research on Advanced Signal Processing Technology and

Xiuchun Xiao received his Ph.D. degree in communication and information system in 2013 from Sun Yat-sen University, Guangzhou, China. He is currently an Associate Professor with the School of Electronics and Information Engineering, Guangdong Ocean University, Zhanjiang, China. His current research interests include artificial neural network, image processing, and computer vision.

References (47)

  • L. Xiao et al.

    Nonlinear recurrent neural networks for finite-time solution of general time-varying linear matrix equations

    Neural Netw.

    (2018)
  • M. Mao et al.

    Enhanced discrete-time Zhang neural network for time-variant matrix inversion in the presence of bias noises

    Neurocomputing

    (2016)
  • L. Jin et al.

    Tracking control of modified lorenz nonlinear system using ZG neural dynamics with additive input or mixed inputs

    Neurocomputing

    (2016)
  • C. Yang et al.

    Adaptive parameter estimation and control design for robot manipulators with finite-time convergence

    IEEE Trans. Ind. Electron.

    (2018)
  • D. Wang et al.

    On mixed data and event driven design for adaptive-critic-based nonlinear H control

    IEEE Trans. Neural Netw. Learn. Syst.

    (2017)
  • J. Nocedal et al.

    Nonlinear Equations

    (2006)
  • J. Santiaguillo-Salinas et al.

    Time-varying containment problem for multi-agent systems

    Proceedings of the 10th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE 2013), Me´xico City

    (2013)
  • M.A.Z. Raja et al.

    Design of stochastic solvers based on genetic algorithms for solving nonlinear equations

    Neural Comput. Appl.

    (2015)
  • C. Yang et al.

    Robot learning system based on adaptive neural control and dynamic movement primitives

    IEEE Trans. Neural Netw. Learn. Syst.

    (2018)
  • H. Zhang et al.

    Adaptive fuzzy fault-tolerant tracking control for partially unknown systems with actuator faults via integral reinforcement learning method

    IEEE Trans. Fuzzy Syst.

    (2019)
  • I. Fister Jr et al.

    A review of chaos-based firefly algorithms: perspectives and research challenges

    Appl. Math. Comput.

    (2015)
  • L. Jin et al.

    Noise-suppressing neural algorithm for solving time-varying system of linear equations: a control-based approach

    IEEE Trans. Ind. Inform.

    (2018)
  • X. Xiao et al.

    A neural network for parameter estimation of the exponentially damped sinusoids

    Proceedings of the International Conference on Intelligent Science and Big Data Engineering

    (2013)
  • Cited by (24)

    • New discrete-time zeroing neural network for solving time-variant underdetermined nonlinear systems under bound constraint

      2022, Neurocomputing
      Citation Excerpt :

      Recently, a typical recurrent neural network, which is originally proposed by Zhang et al. and is termed the zeroing neural network (ZNN), has achieved a remarkable success in solving time-variant mathematical problems [20–22]. In particular, to solve the unconstrained TVUNS, different continuous-time ZNN (CTZNN) models with prominent properties (e.g., finite-time convergence and/or noise rejection) were developed and studied in [23–26]. Moreover, for possible hardware implementation, the CTZNN models were discretized by using difference formulas, and the discrete-time ZNN (DTZNN) models were established in [26–29].

    • Convergence and robustness of bounded recurrent neural networks for solving dynamic Lyapunov equations

      2022, Information Sciences
      Citation Excerpt :

      In recent decades, due to the great demand of the effective and powerful computing methods, many intelligent computing technologies have been developed in many application fields, including pattern recognition [9], motion planning of robots [10], and computational problems [11,12]. As one of the typical intelligent computing technologies with the property of multitude parallel processing and the potential for circuit implementation, the RNN model has also been widely investigated and applied [13–16]. Based on the gradient descent method, gradient-based neural network (GNN) models were designed to search the point where the energy function of the norm error decreases to zero along the gradient descent direction, and one of the GNN models was demonstrated to solve online dynamic Lyapunov equations effectively [13].

    View all citing articles on Scopus

    Xiuchun Xiao received his Ph.D. degree in communication and information system in 2013 from Sun Yat-sen University, Guangzhou, China. He is currently an Associate Professor with the School of Electronics and Information Engineering, Guangdong Ocean University, Zhanjiang, China. His current research interests include artificial neural network, image processing, and computer vision.

    Dongyang Fu received his Ph.D degree in South China Sea Institute of Oceanology Chinese Academy of Sciences and postdoctoral of State Key Laboratory of Satellite Ocean Environment Dynamics, Second Institute of Oceanography, State Oceanic Administration, Guangzhou, China. He is currently an Professor with the School of Electronics and Information Engineering, Guangdong Ocean University, Zhanjiang, China. His current research interests include ocean color remote sensing and its application, remote sensing in offshore water quality, response of upper ocean to typhoon and neural networks.

    Guancheng Wang received his B.E. degree in automation from Sun Yat-sen University, Guangzhou, in 2014, and the M.S. degree in electrical and computer engineering from the University of Macau, Macau, China, in 2017. He works at the School of Electronics and Information Engineering, Guangdong Ocean University, Zhanjiang, China. His current research interests include neural networks, signal processing and analog and mixed-signal circuit designs.

    Shan Liao received her master degree in software engineering in 2012 from Guangdong University of Technology, Guangzhou, China. She is currently studying in graduate school of Sichuan University for Doctor’s degree in cybersecurity. Her current research interests include remote sensing signal processing and algorithm, artificial neural network and communication technology.

    Yimeng Qi received the B.E. degree in electronic information science and technology from Lanzhou University, Lanzhou, China, in 2019. She is currently pursuing her M.E. degree in information and communication engineering at School of Information Science and Engineering, Lanzhou University, Lanzhou, China. Her main research interests include neural networks and robotics.

    Haoen Huang received the B.E. degree in electrical engineering and automation from Guangdong Ocean University, Zhanjiang, China, in 2019. He is currently pursuing the M.Agr. degree in agricultural engineering and information technology with the School of Electronics and Information Engineering, Guangdong Ocean University, Zhanjiang, China. His current research interests include Newton algorithm, neural networks and robotics.

    Long Jin received the B.E. degree in automation and the Ph.D. degree in information and communication engineering from Sun Yat-sen University, Guangzhou, China, in 2011 and 2016, respectively. He is currently a full professor with the School of Information Science and Engineering, Lanzhou University, Lanzhou, China. Before joining Lanzhou University in 2017, he was a Postdoctoral Fellow with the Department of Computing, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong. His main research interests include neural networks, robotics, and intelligent information processing. He is a member of the IEEE.

    View full text