Skip to main content
SpringerLink
Log in

Properties and methods for finding the best rank-one approximation to higher-order tensors

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

The problem of finding the best rank-one approximation to higher-order tensors has extensive engineering and statistical applications. It is well-known that this problem is equivalent to a homogeneous polynomial optimization problem. In this paper, we study theoretical results and numerical methods of this problem, particularly focusing on the 4-th order symmetric tensor case. First, we reformulate the polynomial optimization problem to a matrix programming, and show the equivalence between these two problems. Then, we prove that there is no duality gap between the reformulation and its Lagrangian dual problem. Concerning the approaches to deal with the problem, we propose two relaxed models. The first one is a convex quadratic matrix optimization problem regularized by the nuclear norm, while the second one is a quadratic matrix programming regularized by a truncated nuclear norm, which is a D.C. function and therefore is nonconvex. To overcome the difficulty of solving this nonconvex problem, we approximate the nonconvex penalty by a convex term. We propose to use the proximal augmented Lagrangian method to solve these two relaxed models. In order to obtain a global solution, we propose an alternating least eigenvalue method after solving the relaxed models and prove its convergence.

Numerical results presented in the last demonstrate, especially for nonpositive tensors, the effectiveness and efficiency of our proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Fig. 1

Similar content being viewed by others

References

  1. Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear Programming: Theory and Algorithms, 2nd edn. Wiley, New York (1993)

    MATH  Google Scholar 

  2. Bomze, I.M., Palagi, L.: Quartic formulation of standard quadratic optimization problems. J. Glob. Optim. 32, 181–205 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2011)

    Article  Google Scholar 

  4. Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9, 717–772 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  5. Cardoso, J.F.: High-order contrasts for independent component analysis. Neural Comput. 11, 157–192 (1999)

    Article  Google Scholar 

  6. Chang, K.C., Pearson, K., Zhang, T.: Perron-Frobenius theorem for nonnegative tensors. Commun. Math. Sci. 6, 507–520 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  7. Chang, K.C., Pearson, K., Zhang, T.: Some variational principles for Z-eigenvalues of nonnegative tensors. Linear Algebra Appl. 438, 4166–4182 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  8. Chen, B., He, S., Li, Z., Zhang, S.: Maximum block improvement and polynomial optimization. SIAM J. Optim. 22, 87–107 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  9. De Lathauwer, L., De Moor, B., Vandewalle, J.: On the best rank-1 and rank-(R 1,R 2,…,R n ) approximation of higer-order tensors. SIAM J. Matrix Anal. Appl. 21, 1324–1342 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  10. Grippo, L., Sciandrone, M.: On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper. Res. Lett. 26, 127–136 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  11. Han, L.: An unconstrained optimization approach for finding real eigenvalues of even order symmetric tensors (2012). arXiv preprint. arXiv:1203.5150

  12. Hastad, J.: Clique is hard to approximate within n 1−ϵ. In: Proceedings of 37th Annual Symposium on Foundations of Computer Science, 1996, pp. 627–636. IEEE, New York (1996)

    Google Scholar 

  13. He, S., Jiang, B., Li, Z., Zhang, S.: Probability bounds for polynomial functions in random variables. Technical Report, Department of Industrial and Systems Engineering, University of Minnesota, Minneapolis (2012)

  14. He, S., Li, Z., Zhang, S.: Approximation algorithms for homogeneous polynomial optimization with quadratic constraints. Math. Program. 125, 353–383 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  15. Henrion, D., Lasserre, J.B., Löfberg, J.: GloptiPoly 3: moments, optimization and semidefinite programming. Optim. Methods Softw. 24, 761–779 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  16. Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MATH  MathSciNet  Google Scholar 

  17. Hu, S., Qi, L.: Algebraic connectivity of an even uniform hypergraph. J. Comb. Optim. 24, 564–579 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  18. Hu, Y., Zhang, D., Ye, J., Li, X., He, X.: Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Trans. Pattern Anal. 35, 2117–2130 (2013)

    Article  Google Scholar 

  19. Kofidis, E., Regalia, P.: On the best rank-1 approximation of higher-order supersymmetric tensors. SIAM J. Matrix Anal. Appl. 23, 863–884 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  20. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51, 455–500 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  21. Kolda, T.G., Mayo, J.R.: Shifted power method for computing tensor eigenpairs. SIAM J. Matrix Anal. Appl. 32, 1095–1124 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  22. Lim, L.H.: Singular values and eigenvalues of tensors: a variational approach. In: 1st IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, 2005, pp. 129–132 (2005)

    Google Scholar 

  23. Lin, Z., Chen, M., Ma, Y.: The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices (2010). arXiv preprint arXiv:1009.5055

  24. Luo, Z.Q., Zhang, S.: A semidefinite relaxation scheme for multivariate quartic polynomial optimization with quadratic constraints. SIAM J. Optim. 20, 1716–1736 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  25. Ma, S., Jiang, B., Zhang, S.: Tensor principal component analysis via convex optimization (2012). arXiv preprint arXiv:1212.2702

  26. Motzkin, T.S., Straus, E.G.: Maxima for graphs and a new proof of a theorem of Turán. Can. J. Math. 17, 533–540 (1965)

    Article  MATH  MathSciNet  Google Scholar 

  27. Ng, M., Qi, L., Zhou, G.: Finding the largest eigenvalue of a nonnegative tensor. SIAM J. Matrix Anal. Appl. 31, 1090–1099 (2009)

    Article  MathSciNet  Google Scholar 

  28. Ni, Q., Qi, L., Wang, F.: An eigenvalue method for the positive definiteness identification problem. IEEE Trans. Autom. Control 53, 1096–1107 (2008)

    Article  MathSciNet  Google Scholar 

  29. Powell, M.J.: A method for non-linear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization. Academic Press, New York (1967)

    Google Scholar 

  30. Powell, M.J.: On search directions for minimization algorithms. Math. Program. 4, 193–201 (1973)

    Article  MATH  Google Scholar 

  31. Qi, L.: Eigenvalues of a real supersymmetric tensor. J. Symb. Comput. 40, 1302–1324 (2005)

    Article  MATH  Google Scholar 

  32. Qi, L., Dai, H.H., Han, D.: Conditions for strong ellipticity and M-eigenvalues. Front. Math. China 4, 349–364 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  33. Qi, L., Wang, F., Wang, Y.: Z-eigenvalue methods for a global polynomial optimization problem. Math. Program. 118, 301–316 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  34. Ragnarsson, S., Van Loan, C.F.: Block tensors and symmetric embeddings. Linear Algebra Appl. 2, 853–874 (2013)

    Article  Google Scholar 

  35. Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52, 471–501 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  36. So, A.M.C.: Deterministic approximation algorithms for sphere constrained homogeneous polynomial optimization problems. Math. Program. 129, 357–382 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  37. Wang, H., Banerjee, A., Boley, D.: Modeling time varying covariance matrices in low dimensions. Tech. rep, Department of Computer Science and Engineering, University of Minnesota (2010)

  38. Wu, C., Tai, X.C., et al.: Augmented Lagrangian method, dual methods, and split Bregman iteration for rof, vectorial tv, and high order models. SIAM J. Imaging Sci. 3, 300–339 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  39. Yang, J., Yuan, X.: Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Math. Comput. 82, 301–329 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  40. Yang, Y., Yang, Q.: Further results for Perron-Frobenius theorem for nonnegative tensors. SIAM J. Matrix Anal. Appl. 31, 2517–2530 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  41. Zhang, X., Ling, C., Qi, L.: The best rank-1 approximation of a symmetric tensor and related spherical optimization problems. SIAM J. Matrix Anal. Appl. 33, 806–821 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  42. Zhou, G., Caccetta, L., Teo, K.L., Wu, S.Y.: Nonnegative polynomial optimization over unit spheres and convex programming relaxations. SIAM J. Optim. 22, 987–1008 (2012)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous referees for their suggestions, which help us to improve the paper. The second author was supported by the National Natural Science Foundation of China (Grant No. 11271206). The third author was supported by the Hong Kong Research Grant Council (Grant No. PolyU 501909, 502510, 502111 and 501212).

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, Katholieke Universiteit Leuven, Kasteelpark Arenberg 10, 3001, Leuven, Belgium

    Yuning Yang

  2. School of Mathematical Sciences and LPMC, Nankai University, Tianjin, 300071, P.R. China

    Qingzhi Yang

  3. Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, P.R. China

    Liqun Qi

Authors
  1. Yuning Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingzhi Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liqun Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuning Yang.

Additional information

Y. Yang is on leave from Nankai University, Tianjin, China.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yang, Y., Yang, Q. & Qi, L. Properties and methods for finding the best rank-one approximation to higher-order tensors. Comput Optim Appl 58, 105–132 (2014). https://doi.org/10.1007/s10589-013-9617-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-013-9617-9

Keywords

Search

Navigation