Elsevier

Neurocomputing

Volume 383, 28 March 2020, Pages 314-323
Neurocomputing

Online singular value decomposition of time-varying matrix via zeroing neural dynamics

https://doi.org/10.1016/j.neucom.2019.11.036Get rights and content

Abstract

In this paper, the problem of online singular value decomposition (SVD) for time-varying matrix is proposed, analyzed and investigated. In order to solve this complex and difficult problem in real time, we consider to transform it into an equation system firstly. Then, by applying zeroing neural dynamics (ZND) method and a dimensional reduction technique, a continuous-time SVD (CTSVD) model is proposed. Besides, a high-precision eight-instant Zhang et al discretization (ZeaD) formula with theoretical analysis is proposed and studied. Furthermore, by using this new ZeaD formula to discretize the CTSVD model, an eight-instant discrete-time SVD (EIDTSVD) model is thus proposed. Moreover, three other discrete-time SVD (DTSVD) models termed Euler-type DTSVD (ETDTSVD) model, four-instant DTSVD (FIDTSVD) model and six-instant DTSVD (SIDTSVD) model are derived and proposed, respectively, for the purpose of comparison. Finally, numerical experiments and results further substantiate the great effectiveness, accuracy and superiority of the proposed CTSVD and EIDTSVD models.

Introduction

As we know, singular value decomposition (SVD) is a powerful tool, which widely exists in the fields of scientific research, medical treatment, industrial and agricultural applications, and daily life [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], such as no-reference stereopair quality assessment [1], plant disease recognition [3], automatic classification of electromyogram [4], damage localization of plates [5] and face recognition [10], [11]. Note that the previous research on SVD is almost time-invariant, because research on time-varying SVD is highly complex and difficult. Generally speaking, for the time-varying SVD problem, it is usually considered as a time-invariant problem under the assumption of short-time invariance [12], [13]. In [14], a method for solving time-varying SVD is introduced, and it is used to decompose a time-varying square matrix.

In recent years, a new method termed zeroing neural dynamics (ZND) [15], [16], [17], [18] has been proposed and widely used to solve time-varying problems, for which the conventional methods cannot perform well [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32]. Simply put, according to the general design procedures for the ZND method, when solving a time-varying problem, an error function is defined. Then, the value of the error function is made to converge to zero by using a design formula. Thus, the continuous-time model corresponding to the original problem is obtained. In addition, a new class of one-step forward finite difference formula termed Zhang et al discretization (ZeaD) formula, which is used to discretize continuous-time models, has been proposed, studied and applied [33], [34], [35], [36], [37], [38], [39] for potential hardware implementation.

Different from [14], this paper is mainly concerned with applying the ZND method to compute SVD of arbitrary time-varying rectangular/square matrix in real time. The remainder of this paper is organized into five sections. Firstly, the problem formulation and an equivalent equation system are presented in Section 2. In Section 3, a dimensional reduction technique for time-varying diagonal matrix vectorization is proposed and investigated. Besides, by employing the ZND method, the dimensional reduction technique, Kronecker product, vectorization and permutation matrix, a continuous-time SVD (CTSVD) model is proposed. In Section 4, a new eight-instant ZeaD formula is derived and obtained, and meanwhile, an eight-instant discrete-time SVD (EIDTSVD) model is proposed and investigated. Section 5 shows the numerical experiment results and comparison verifications. Section 6 concludes this paper with final remarks. Besides, the main contributions of this work are listed as below.

  • The SVD problem for time-varying rectangular/square matrix is first studied, and its equivalent equation system is formulated and investigated.

  • A CTSVD model for time-varying SVD solving is proposed and analyzed, by using the ZND method and the special dimensional reduction technique.

  • A new eight-instant ZeaD formula with O(τ4) precision is derived and studied. By exploiting this formula, a discrete-time model termed EIDTSVD model is proposed.

  • For the purpose of comparison, three other DTSVD models are also investigated in the numerical experiments, which show the superiority of the EIDTSVD model.

Section snippets

Preliminary and problem formulation

In general, the SVD problem of time-varying matrix can be expressed asA(t)=U(t)Σ(t)V*(t).Thereinto, A(t) is an m × n real or complex time-varying matirx; U(t) and V(t) are m × m and n × n real or complex time-varying unitary matrices, respectively; Σ(t) is an m × n time-varying diagonal matrix with non-negative real numbers on the diagonal; superscript * denotes the conjugate transpose operation of a matrix. Simply speaking, the SVD of A(t) is a factorization of the form U(t)Σ(t)V*(t).

It is

Continuous-time solution model for SVD

To obtain the SVD of time-varying matrix A(t), the ZND method is adopted, and the CTSVD model is proposed and investigated in this section.

We define three matrix-valued error functions, which are corresponding to the three equations in system (1) respectively, as follows:{Z1(t)=Σ(t)UT(t)A(t)V(t),(2a)Z2(t)=U(t)UT(t)Im,(2b)Z3(t)=V(t)VT(t)In.(2c)By substituting (2a), (2b) and (2c) into linear design formula Z˙j(t)=αjZj(t), with Z˙j(t) denoting the derivative of Zj(t) and j=1,2,3, the following

Discrete-time solution models for SVD

In this section, for the purpose of potential digital hardware realization, the discrete-time solution model for SVD is proposed, discussed, investigated, and also compared with other discrete-time solution models for SVD.

Numerical experiments and verifications

In order to verify the performance and validity of the proposed models, the numerical experiments are conducted. Moreover, the comparison and analysis are also given in this section.

Conclusion

In this paper, the problem of the online SVD of time-varying rectangular/square matrix has been analyzed and investigated for the first time. Firstly, the original problem has been transformed into a system of equations. Then a dimensional reduction technique for time-varying diagonal matrix vectorization has been proposed, and by using this technique and ZND method, a solution model termed CTSVD model (5) has been further derived and proposed. In addition, the new eight-instant ZeaD formula (6)

Declaration of Competing Interest

None.

Jianrong Chen received the B.S. degree in engineering management from Guangzhou University, Guangzhou, China, in 2005, the M.S. degree in computer application technology from Guangxi University for Nationalities, Nanning, China, in 2009. He is currently pursuing the Ph.D. degree in computer science and technology at the School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China. In addition, he is an assistant researcher at the Information and Education Technology Center,

References (45)

  • M. Baumann et al.

    Singular value decomposition of time-varying matrices

    Future Gen. Comput. Syst.

    (2003)
  • L. Xiao

    A nonlinearly activated neural dynamics and its finite-time solution to time-varying nonlinear equation

    Neurocomputing

    (2016)
  • Y. Zhang et al.

    Different Zhang functions leading to different Zhang-dynamics models illustrated via time-varying reciprocal solving

    Appl. Math. Model.

    (2012)
  • C. Yi et al.

    Comparison on neural solvers for the Lyapunov matrix equation with stationary & nonstationary coefficients

    Appl. Math. Model.

    (2013)
  • M. Mao et al.

    Enhanced discrete-time Zhang neural network for time-variant matrix inversion in the presence of bias noises

    Neurocomputing

    (2016)
  • L. Jin et al.

    Robot manipulator control using neural networks: a survey

    Neurocomputing

    (2018)
  • L. Jin et al.

    Nonconvex projection activated zeroing neurodynamic models for time-varying matrix pseudoinversion with accelerated finite-time convergence

    Appl. Soft Comput.

    (2018)
  • D. Chen et al.

    Zeroing neural-dynamics approach and its robust and rapid solution for parallel robot manipulators against superposition of multiple disturbances

    Neurocomputing

    (2018)
  • J. Li et al.

    Z-type neural-dynamics for time-varying nonlinear optimization under a linear equality constraint with robot application

    J. Comput. Appl. Math.

    (2018)
  • S. Qiao et al.

    Two finite-time convergent Zhang neural network models for time-varying complex matrix Drazin inverse

    Linear Algebra Appl.

    (2018)
  • L. Jin et al.

    Discrete-time Zhang neural network of O(τ3) pattern for time-varying matrix pseudoinversion with application to manipulator motion generation

    Neurocomputing

    (2014)
  • D. Guo et al.

    Theoretical analysis, numerical verification and geometrical representation of new three-step DTZD algorithm for time-varying nonlinear equations solving

    Neurocomputing

    (2016)
  • Cited by (0)

    Jianrong Chen received the B.S. degree in engineering management from Guangzhou University, Guangzhou, China, in 2005, the M.S. degree in computer application technology from Guangxi University for Nationalities, Nanning, China, in 2009. He is currently pursuing the Ph.D. degree in computer science and technology at the School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China. In addition, he is an assistant researcher at the Information and Education Technology Center, Youjiang Medical University for Nationalities, Baise, China. His main research interests include neural networks, robotics, numerical computation and intelligent optimization.

    Yunong Zhang received the B.S. degree in industrial electrical automation from Huazhong University of Science and Technology, Wuhan, China, in 1996, the M.S. degree in control theory and control engineering from South China University of Technology, Guangzhou, China, in 1999, and the Ph.D. degree in mechanical and automation engineering from Chinese University of Hong Kong, Shatin, Hong Kong, China, in 2003. He is currently a professor in the School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China. Before joining Sun Yat-sen University in 2006, he had been with National University of Singapore, University of Strathclyde, and National University of Ireland at Maynooth, since 2003. His main research interests include robotics, neural networks, computation and optimization. His web-page is now available at http://sdcs.sysu.edu.cn/content/2477.

    This work is supported by the National Natural Science Foundation of China (with number 61473323) and also by Shenzhen Science and Technology Plan Project (with number JCYJ20170818154936083). Besides, kindly note that both authors of the paper are jointly of the first authorship and jointly of the corresponding authorship.

    View full text