Elsevier

Pattern Recognition

Volume 41, Issue 6, June 2008, Pages 2133-2135
Pattern Recognition

Comments on “An analytical algorithm for generalized low-rank approximations of matrices”

https://doi.org/10.1016/j.patcog.2007.10.008Get rights and content

Abstract

Low rank approximations of matrices have been widely used in pattern recognition and machine learning. Based on a sequence of matrices, a generalized low rank approximation problem was presented and an iterative scheme was given by Liang and Shi recently proposed an analytical scheme for this approximation problem. In this paper, we identify the weakness in their scheme and prove that their algorithm is incorrect.

Introduction

The low rank approximation is a useful technique for dimension reduction, which is widely used in image compression, information retrieval, pattern recognition and so on. However, when applied to high-dimensional data, this technique meets with practical computational limits. Ye [1] proposed a novel approach to reduce the computational cost, which is based on a new data representation model. An iterative algorithm for these generalized low rank approximations of matrices (GLRAM) was also given, and it is proven to have superior performance over the traditional singular value decomposition (SVD) based methods.

Liang and Shi recently proposed a new algorithm for GLRAM in Ref. [2]. They concluded that their scheme is an analytical algorithm and has no iterative steps compared with the Ye's scheme. Unfortunately, as we will show shortly, this scheme is not analytical. Although Liang and Shi claimed that their scheme is a closed form solution for GLRAM, our analysis indicates that it cannot get the right solutions, because the theorem, which the algorithm is derived from, is incorrect. Furthermore, the upper and lower bounds of the objective function of GLRAM are also given in this paper. For simplicity, we call their scheme the LS scheme hereafter. Incorrectness of the LS scheme is also pointed out in Refs. [3], [4], [5], where a non-iterative algorithm is proposed to find an approximate solution of GLRAM in Ref. [3].

The rest of this paper is organized as follows. We give a brief review of the LS scheme in Section 2. Our analysis is presented in Section 3. The conclusion is in Section 4.

Section snippets

Review of the SL scheme

Denote the M original image matrices as AiRm×n, for i=1,,M. Let two matrices L and R be with orthonormal columns. The aim is to find proper L, R, and M matrices Di, such that LDiRT is a good approximation of Ai, for i=1,,M. This can be formulated as follows [1]:minLRm×l1:LTL=Il1RRn×l2:RTR=Il2DiRl1×l21Mi=1MAi-LDiRTF2.

To solve the minimization problem, some theorems and corollaries are needed, which can be found in Refs. [1], [2].

Theorem 1

Let L,R and {Di}i=1M be the optimal solution to the

Comments on the SL scheme

In Ref. [2], Liang and Shi claimed that their scheme is an analytical algorithm for GLRMM. However, we will show in Theorem 4 that it is not true.

Theorem 4

The LS scheme is not an analytical algorithm for GLRMM.

Proof

To prove the scheme is not an analytical algorithm, we just need to find a counterexample for which the result is not the optimal solution of Eq. (2). Define two 3×3 matrices asA1=0.30.90.50.900.70.90.50.1,A2=0.20.80.50.30.700.10.40.9.

Let l1=l2=2, and then according to the LS scheme, we can getL=

Conclusion

In this paper, we present an analysis of the analytical scheme for GLRAM proposed recently in Ref. [2]. Our results show that this scheme is incorrect, which is contrary to the original authors’ conclusion. Furthermore, the upper and lower bounds of the objective function of GLRAM are also given. We remark that it is still an open problem to construct effective and fast algorithms for low rank approximation and its applications.

Acknowledgments

The authors would like to thank the anonymous reviewers and the associate editor for their insightful comments, which led to a significantly improved presentation of the manuscript. This work was supported by the National Natural Science Foundation of China under Grant 60675002, and funded by the Basic Research Foundation of Tsinghua National Laboratory for Information Science and Technology (TNList).

References (6)

There are more references available in the full text version of this article.

Cited by (3)

View full text