Abstract
Low rank tensor approximation is an important subject with a wide range of applications. Most prevailing techniques for computing the low rank approximation in the Tucker format often first assemble relevant factors into matrices and then update by turns one factor matrix at a time. In order to improve two factor matrices simultaneously, a special system of nonlinear matrix equations over a certain product Stiefel manifold must be resolved at every update. The solution to the system consists of orbit varieties invariant under the orthogonal group action, which thus imposes challenges on its analysis. This paper proposes a scheme similar to the power method for subspace iterations except that the polar decomposition is used as the normalization process and that the iteration can be applied to both the orbits and the cross-sections. The notion of quotient manifold is employed to factor out the effect of orbital solutions. The dynamics of the iteration is completely characterized. An isometric isomorphism between the tangent spaces of two properly identified Riemannian manifolds is established to lend a hand to the proof of convergence.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
These two characteristics mentioned are innate to our operators \({\mathcal {A}}\) and \({\mathcal {B}}\) specifically derived for tensor applications. For general (2), if we assume that the bilinear operators \({\mathcal {A}}\) and \({\mathcal {B}}\) satisfy the symmetry (19), (20), and the invariance (21), then the remaining discussion can be equally applied.
Though it is not directly relevant to our discussion, in topology it can be proved that since \({\mathscr {O}}(q)\) is compact, the quotient space \({\mathscr {S}}(p,q)/{\mathscr {O}}(q)\) is Hausdorff. Furthermore, since the right group action is free, the quotient space is indeed a manifold.
To demonstrate its analogy to the conventional power method and the subspace iteration method, we give two examples in Sect. 4.
References
Bader, B.W., Kolda, T.G.: Efficient MATLAB computations with sparse and factored tensors. SIAM J. Sci. Comput. 30(1), 205–231 (2008). https://doi.org/10.1137/060676489
Chu, M.T., Trendafilov, N.T.: The orthogonally constrained regression revisited. J. Comput. Graph. Stat. 10(4), 746–771 (2001). https://doi.org/10.1198/106186001317243430
Chu, M.T., Trendafilov, J.L.: On a multivariate eigenvalue problem. I. algebraic theory and a power method. SIAM J. Sci. Comput. 14(5), 1089–1106 (1993). https://doi.org/10.1137/0914066
Derksen, H., Kemper, G.: Computational Invariant Theory. Encyclopaedia of Mathematical Sciences Invariant Theory and Algebraic Transformation Groups VIII. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-48422-7
Hein, M., Bühler, T.: An inverse power method for nonlinear eigenproblems with applications in 1-spectral clustering and sparse PCA. CoRR arXiv:1012.0774 (2010)
Higham, N.J.: Computing the polar decomposition—with applications. SIAM J. Sci. Stat. Comput. 7(4), 1160–1174 (1986). https://doi.org/10.1137/0907079
Langville, A.N., Stewart, W.J.: The Kronecker product and stochastic automata networks. J. Comput. Appl. Math. 167(2), 429–447 (2004). https://doi.org/10.1016/j.cam.2003.10.010
Ruhe, A.: Algorithms for the nonlinear eigenvalue problem. SIAM J. Numer. Anal. 10, 674–689 (1973). https://doi.org/10.1137/0710059
Sommese, A.J., Wampler, I.C.W.: The Numerical Solution of Systems of Polynomials Arising in Engineering and Science. World Scientific Publishing Co. Pte. Ltd., Hackensack (2005). https://doi.org/10.1142/9789812567727
Van Loan, C.F.: The ubiquitous Kronecker product. J. Comput. Appl. Math. 123(1–2), 85–100 (2000). https://doi.org/10.1016/S0377-0427(00)00393-9
Van Loan, C.F.: Structured matrix problems from tensors. In: Exploiting Hidden Structure in Matrix Computations: Algorithms and Applications, Lecture Notes in Math., vol. 2173, pp. 1–63. Springer, Cham (2016)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Bo Dong: This work was supported in part by the National Natural Science Foundation of China under Grant 11871136, the Chinese Scholarship Council and the Fundamental Research Funds for the Central Universities.
Nan Jiang: This work was supported in part by the Chinese Scholarship Council.
Moody T. Chu: This work was supported in part by the National Science Foundation under Grants DMS-1316779 and DMS-1912816.
Rights and permissions
About this article
Cite this article
Dong, B., Jiang, N. & Chu, M.T. Nonlinear power-like iteration by polar decomposition and its application to tensor approximation. Numer. Math. 144, 729–749 (2020). https://doi.org/10.1007/s00211-020-01100-8
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00211-020-01100-8