Abstract
We extend and analyze the trust region method for solving smooth and unconstrained multicriteria optimization problems on Riemannian manifolds. At each iteration of this method, a quadratic model is assigned to each component of the vectorial objective function by considering the notion of retractions. Then, a subproblem is constructed and solved to find a new descent direction. Furthermore, we investigate the convergence behavior of the algorithm by considering radially Lipschitz continuously differentiable functions. In the end, the algorithm is implemented on three examples, and the corresponding numerical results showing the efficiency of the proposed method are reported as well.
Similar content being viewed by others
References
Absil, P.A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007). https://doi.org/10.1007/s10208-005-0179-9
Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms On Matrix Manifolds. Princeton University Press, Princeton (2009). https://doi.org/10.1515/9781400830244
Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. SIAM J. Optim. 22(1), 135–158 (2012). https://doi.org/10.1137/100802529
Ashry, G.A.: On globally convergent multi-objective optimization. Appl. Math. Comput. 183(1), 209–216 (2006). https://doi.org/10.1016/j.amc.2006.05.126
Bento, G.C., Cruz Neto, J.X.: A subgradient method for multiobjective optimization on Riemannian manifolds. J. Optim. Theory Appl. 159(1), 125–137 (2013). https://doi.org/10.1007/s10957-013-0307-7
Bento, G.C., Ferreira, O.P., Oliveira, P.R.: Unconstrained steepest descent method for multicriteria optimization on Riemannian manifolds. J. Optim. Theory Appl. 154, 88–107 (2012). https://doi.org/10.1007/s10957-011-9984-2
Bento, G.C., Cruz Neto, J.X., Santos, P.S.M.: An inexact steepest descent method for multicriteria optimization on Riemannian manifolds. J. Optim. Theory Appl. 159(1), 108–124 (2013). https://doi.org/10.1007/s10957-013-0305-9
Bento, G.C., Cruz Neto, J.X., Meireles, L.V.: Proximal point method for locally Lipschitz functions in multiobjective optimization of Hadamard manifolds. J. Optim. Theory Appl. 179(1), 37–52 (2018). https://doi.org/10.1007/s10957-018-1330-5
Berkemeier, M., Peitz, S.: Derivative-free multiobjective trust region descent method using radial basis function surrogate models. Math. Comput. Appl. 26(2), 31 (2021). https://doi.org/10.3390/mca26020031
Boumal, N.: An Introduction to Optimization on Smooth Manifolds. Available online, (2020) http://www.nicolasboumal.net/book
Cai, T., Song, L., Li G., Liao, M.: Multi-task learning with Riemannian optimization. In: Huang, D.S., Jo, K.H., Li, J., Gribova, V., Hussain, A. (eds.): International Conference on Intelligent Computing, 499–509, Springer, Cham (2021) https://doi.org/10.1007/978-3-030-84529-2_42
Cambier, L., Absil, P.A.: Robust low-rank matrix completion by Riemannian optimization. SIAM J. Sci. Comput. 38(5), 440–460 (2016). https://doi.org/10.1137/15M1025153
Carrizo, G.A., Lotito, P.A., Maciel, M.C.: Trust region globalization strategy for the nonconvex unconstrained multiobjective optimization problem. Math. Program. 159, 339–369 (2016). https://doi.org/10.1007/s10107-015-0962-6
Chen, S., Deng, Z., Ma, S., So, A.M.C.: Manifold proximal point algorithms for dual principal component pursuit and orthogonal dictionary learning. IEEE Trans. Signal Proc. 69, 4759–4773 (2021). https://doi.org/10.1109/TSP.2021.3099643
Ehrgott, M.: Multicriteria Optimization, vol. 491. Springer, New York (2005)
Ferreira, O.P., Iusem, A.N., Nmeth, S.Z.: Concepts and techniques of optimization on the sphere. TOP 22(3), 1148–1170 (2014). https://doi.org/10.1007/s11750-014-0322-3
Ferreira, O.P., Louzeiro, M.S., Prudente, L.F.: Gradient method for optimization on Riemannian manifolds with lower bounded curvature. SIAM J. Optim. 29(4), 2517–2541 (2019). https://doi.org/10.1137/18M1180633
Ferreira, O.P., Louzeiro, M.S., Prudente, L.F.: Iteration-complexity and asymptotic analysis of steepest descent method for multiobjective optimization on Riemannian manifolds. J. Optim. Theory Appl. 184(2), 507–533 (2020). https://doi.org/10.1007/s10957-019-01615-7
Fliege, J., Drummond, L.M.G., Svaiter, B.F.: Newton’s method for multiobjective optimization. SIAM J. Optim. 20(2), 602–626 (2009). https://doi.org/10.1137/08071692X
Golub, G.H., Van Loan, C.F.: Matrix Computations. John Hopkins University Press, Baltimore (1996)
Heidel, G., Schulz, V.: A Riemannian trustregion method for lowrank tensor completion. Numer. Linear Algebra Appl. 25(6), e2175 (2018). https://doi.org/10.1002/nla.2175
Hosseini, R., Sra, S.: Matrix manifold optimization for Gaussian mixtures. In: Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R., (eds.): Advances in Neural Information Processing Systems 28, 910–918 (2015)
Hosseini, R., Sra, S.: An alternative to EM for Gaussian mixture models: batch and stochastic Riemannian optimization. Math. Program. 181(1), 187–223 (2020). https://doi.org/10.1007/s10107-019-01381-4
Hosseini, R., Sra, S.: Recent advances in stochastic Riemannian optimization. In: Grohs, P., Holler, M., Weinmann, A., (eds.): Handbook of Variational Methods for Nonlinear Geometric Data, Springer, Cham 527–554 (2020) https://doi.org/10.1007/978-3-030-31351-7_19
Hu, J., Milzarek, A., Wen, Z., Yuan, Y.: Adaptive quadratically regularized Newton method for Riemannian optimization. SIAM J. Matrix Anal. Appl. 39(3), 1181–1207 (2018). https://doi.org/10.1137/17M1142478
Huang, W., Gallivan, K.A., Absil, P.A.: A Broyden class of quasi-Newton methods for Riemannian optimization. SIAM J. Optim. 25(3), 1660–1685 (2015). https://doi.org/10.1137/140955483
Jeuris, B., Vandebril, R., Vandereycken, B.: A survey and comparison of contemporary algorithms for computing the matrix geometric mean. Electron. Trans. Numer. Anal. 39, 379–402 (2012)
Lee, J.M.: Riemannian Manifolds: An Introduction to Curvature. Springer, New York (1997)
Lee, J.M.: Smooth Manifolds: Introduction to Smooth Manifolds. Springer, New York (2013). https://doi.org/10.1007/978-1-4419-9982-5-1
Lucambio Pérez, L.R., Prudente, L.F.: Nonlinear conjugate gradient methods for vector optimization. SIAM J. Optim. 28(3), 2690–2720 (2018). https://doi.org/10.1137/17M1126588
Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (2006)
Qu, S., Goh, M., Liang, B.: Trust region methods for solving multiobjective optimisation. Optim. Methods Softw. 28(4), 796–811 (2013). https://doi.org/10.1080/10556788.2012.660483
Sun, W., Yuan, Y.X.: Optimization Theory and Methods: Nonlinear Programming. Springer, New York (2006)
Thomann, J., Eichfelder, G.: A trust-region algorithm for heterogeneous multiobjective optimization. SIAM J. Optim. 29(2), 1017–1047 (2019). https://doi.org/10.1137/18M1173277
Udriste, C.: Convex Functions and Optimization Methods on Riemannian Manifolds. Mathematics and its Applications, vol. 297. Springer Science & Business Media, Doedecht (1994)
Vandereycken, B.: Low-rank matrix completion by Riemannian optimization. SIAM J. Optim. 23(2), 1214–1236 (2013). https://doi.org/10.1137/110845768
Villacorta, K.D., Oliveira, P.R., Soubeyran, A.: A trust-region method for unconstrained multiobjective problems with applications in satisficing processes. J. Optim. Theory Appl. 160(3), 865–889 (2014). https://doi.org/10.1007/s10957-013-0392-7
Wang, J., Wang, X., Li, C., Yao, J.C.: Convergence analysis of gradient algorithms on Riemannian manifolds without curvature constraints and application to Riemannian mass. SIAM J. Optim. 31(1), 172–199 (2021). https://doi.org/10.1137/19M1289285
Wei, H., Wei, H.Y.: A Riemannian subspace limited-memory SR1 trust region method. Optim. Lett. 10(8), 1705–1723 (2016). https://doi.org/10.1007/s11590-015-0977-1
Yuan, X., Huang, W., Absil, P.A., Gallivan, K.A.: A Riemannian quasi-Newton method for computing the Karcher mean of symmetric positive definite matrices, Technical Report FSU17-02, Florida State University (2017)
Zhang, T., Yang, Y.: Robust PCA by manifold optimization. J. Mach. Learn. Res. 19(80), 1–39 (2018)http://jmlr.org/papers/v19/17-473.html
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Ana Luisa Custodio.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Eslami, N., Najafi, B. & Vaezpour, S.M. A Trust Region Method for Solving Multicriteria Optimization Problems on Riemannian Manifolds. J Optim Theory Appl 196, 212–239 (2023). https://doi.org/10.1007/s10957-022-02142-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-022-02142-8