Skip to main content
SpringerLink
Log in

A Scaled Gradient Projection Method for Minimization over the Stiefel Manifold

  • Conference paper
  • First Online:
Advances in Soft Computing (MICAI 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11835))

Included in the following conference series:

Abstract

In this paper we consider a class of iterative gradient projection methods for solving optimization problems with orthogonality constraints. The proposed method can be seen as a forward-backward gradient projection method which is an extension of a gradient method based on the Cayley transform. The proposal incorporates a self-adaptive scaling matrix and the Barzilai-Borwein step-sizes that accelerate the convergence of the method. In order to preserve feasibility, we adopt a projection operator based on the QR factorization. We demonstrate the efficiency of our procedure in several test problems including eigenvalue computations and sparse principal component analysis. Numerical comparisons show that our proposal is effective for solving these kind of problems and presents competitive results compared with some state-of-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abrudan, T.E., Eriksson, J., Koivunen, V.: Steepest descent algorithms for optimization under unitary matrix constraint. IEEE Trans. Sig. Process. 56(3), 1134–1147 (2008)

    Article  MathSciNet  Google Scholar 

  2. Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8(1), 141–148 (1988)

    Article  MathSciNet  Google Scholar 

  3. Boufounos, P.T., Baraniuk, R.G.: 1-bit compressive sensing. In: 42nd Annual Conference on Information Sciences and Systems, 2008. CISS 2008, pp. 16–21. IEEE (2008)

    Google Scholar 

  4. Cedeño, O.S.D., Leon, H.F.O.: Projected nonmonotone search methods for optimization with orthogonality constraints. Comput. Appl. Math. 37(3), 1–27 (2017). https://doi.org/10.1007/s40314-017-0501-6

    Article  MathSciNet  Google Scholar 

  5. Chen, S., Ma, S., So, A.M.C., Zhang, T.: Proximal gradient method for manifold optimization. arXiv preprint arXiv:1811.00980 (2018)

  6. Dai, Y.H., Fletcher, R.: Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming. Numer. Math. 100(1), 21–47 (2005)

    Article  MathSciNet  Google Scholar 

  7. Dalmau-Cedeño, O., Oviedo, H.: A projection method for optimization problems on the stiefel manifold. In: Carrasco-Ochoa, J.A., Martínez-Trinidad, J.F., Olvera-López, J.A. (eds.) MCPR 2017. LNCS, vol. 10267, pp. 84–93. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59226-8_9

    Chapter  Google Scholar 

  8. Davis, T.A., Hu, Y.: The university of Florida sparse matrix collection. ACM Trans. Mathem. Softw. (TOMS) 38(1), 1 (2011)

    MathSciNet  MATH  Google Scholar 

  9. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)

    Article  MathSciNet  Google Scholar 

  10. Eldén, L., Park, H.: A procrustes problem on the stiefel manifold. Numer. Math. 82(4), 599–619 (1999)

    Article  MathSciNet  Google Scholar 

  11. Hu, J., Jiang, B., Lin, L., Wen, Z., Yuan, Y.: Structured quasi-newton methods for optimization with orthogonality constraints. arXiv preprint arXiv:1809.00452 (2018)

  12. Hu, J., Milzarek, A., Wen, Z., Yuan, Y.: Adaptive quadratically regularized newton method for riemannian optimization. SIAM J. Matrix Anal. Appl. 39(3), 1181–1207 (2018)

    Article  MathSciNet  Google Scholar 

  13. Joho, M., Mathis, H.: Joint diagonalization of correlation matrices by using gradient methods with application to blind signal separation. In: Sensor Array and Multichannel Signal Processing Workshop Proceedings, 2002, pp. 273–277. IEEE (2002)

    Google Scholar 

  14. Kokiopoulou, E., Chen, J., Saad, Y.: Trace optimization and eigenproblems in dimension reduction methods. Numer. Linear Algebra Appl. 18(3), 565–602 (2011)

    Article  MathSciNet  Google Scholar 

  15. Lai, R., Osher, S.: A splitting method for orthogonality constrained problems. J. Sci. Comput. 58(2), 431–449 (2014)

    Article  MathSciNet  Google Scholar 

  16. Laska, J.N., Wen, Z., Yin, W., Baraniuk, R.G.: Trust, but verify: fast and accurate signal recovery from 1-bit compressive measurements. IEEE Trans. Sig. Process. 59(11), 5289–5301 (2011)

    Article  Google Scholar 

  17. Liu, Y.F., Dai, Y.H., Luo, Z.Q.: On the complexity of leakage interference minimization for interference alignment. In: 2011 IEEE 12th international workshop on Signal Processing Advances in Wireless Communications, pp. 471–475. IEEE (2011)

    Google Scholar 

  18. Lu, Z., Zhang, Y.: An augmented lagrangian approach for sparse principal component analysis. Math. Program. 135(1–2), 149–193 (2012)

    Article  MathSciNet  Google Scholar 

  19. Manton, J.H.: Optimization algorithms exploiting unitary constraints. IEEE Trans. Sig. Process. 50(3), 635–650 (2002)

    Article  MathSciNet  Google Scholar 

  20. Nie, F., Zhang, R., Li, X.: A generalized power iteration method for solving quadratic problem on the stiefel manifold. Sci. China Inf. Sci. 60(11), 112101 (2017)

    Article  MathSciNet  Google Scholar 

  21. Oviedo, H., Lara, H., Dalmau, O.: A non-monotone linear search algorithm with mixed direction on stiefel manifold. Optim. Methods Softw. 34(2), 1–21 (2018)

    MathSciNet  MATH  Google Scholar 

  22. Oviedo, H.F.: A spectral gradient projection method for the positive semi-definite procrustes problem. arXiv /abs/1908.06497v1 (2019)

    Google Scholar 

  23. Oviedo, H.F., Lara, H.J.: A riemannian conjugate gradient algorithm with implicit vector transport for optimization in the stiefel manifold. Technical report, Technical report. UFSC-Blumenau, CIMAT (2018)

    Google Scholar 

  24. Saad, Y.: Numerical Methods for Large Eigenvalue Problems. Manchester University Press, Manchester (1992)

    MATH  Google Scholar 

  25. Sato, H.: Riemannian newton’s method for joint diagonalization on the stiefel manifold with application to ica. arXiv preprint arXiv:1403.8064 (2014)

  26. Seibert, M., Kleinsteuber, M., Hüper, K.: Properties of the bfgs method on riemannian manifolds. Mathematical System Theory C Festschrift in Honor of Uwe Helmke on the Occasion of his Sixtieth Birthday, pp. 395–412 (2013)

    Google Scholar 

  27. Theis, F.J., Cason, T.P., Absil, P.-A.: Soft dimension reduction for ICA by joint diagonalization on the stiefel manifold. In: Adali, T., Jutten, C., Romano, J.M.T., Barros, A.K. (eds.) ICA 2009. LNCS, vol. 5441, pp. 354–361. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00599-2_45

    Chapter  Google Scholar 

  28. Urdaneta, H.L., Leon, H.F.O.: Solving joint diagonalization problems via a riemannian conjugate gradient method in stiefel manifold. In: Proceeding Series of the Brazilian Society of Computational and Applied Mathematics, vol. 6, no. 2 (2018)

    Google Scholar 

  29. Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Math. Program. 142(1–2), 397–434 (2013)

    Article  MathSciNet  Google Scholar 

  30. Yang, B.: Projection approximation subspace tracking. IEEE Trans. Sig. Process. 43(1), 95–107 (1995)

    Article  Google Scholar 

  31. Yang, C., Meza, J.C., Lee, B., Wang, L.W.: KSSOLV – a matlab toolbox for solving the kohn-sham equations. ACM Trans. Math. Softw. (TOMS) 36(2), 10 (2009)

    Article  MathSciNet  Google Scholar 

  32. Zhang, H., Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14(4), 1043–1056 (2004)

    Article  MathSciNet  Google Scholar 

  33. Zhu, X.: A riemannian conjugate gradient method for optimization on the stiefel manifold. Comput. Optim. Appl. 67(1), 73–110 (2017)

    Article  MathSciNet  Google Scholar 

Download references

Aknowledgement

This work was supported in part by Consejo Nacional de Ciencia y Tecnología (CONACYT) (Mexico), grant 258033.

Author information

Authors and Affiliations

  1. Centro de Investigación en Matemáticas, CIMAT A.C., Guanajuato, Mexico

    Harry Oviedo & Oscar Dalmau

Authors
  1. Harry Oviedo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oscar Dalmau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Harry Oviedo .

Editor information

Editors and Affiliations

  1. Universidad Panamericana, Mexico City, Mexico

    Lourdes Martínez-Villaseñor

  2. Instituto Politecnico Nacional, Mexico, Mexico

    Ildar Batyrshin

  3. Universidad Veracruzana, Xalapa, Mexico

    Antonio Marín-Hernández

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Oviedo, H., Dalmau, O. (2019). A Scaled Gradient Projection Method for Minimization over the Stiefel Manifold. In: Martínez-Villaseñor, L., Batyrshin, I., Marín-Hernández, A. (eds) Advances in Soft Computing. MICAI 2019. Lecture Notes in Computer Science(), vol 11835. Springer, Cham. https://doi.org/10.1007/978-3-030-33749-0_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-33749-0_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-33748-3

  • Online ISBN: 978-3-030-33749-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation