Skip to main content
Springer Nature Link
Log in

Alternative extension of the Hager–Zhang conjugate gradient method for vector optimization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Recently, Gonçalves and Prudente proposed an extension of the Hager–Zhang nonlinear conjugate gradient method for vector optimization (Comput Optim Appl 76:889–916, 2020). They initially demonstrated that directly extending the Hager–Zhang method for vector optimization may not result in descent in the vector sense, even when employing an exact line search. By utilizing a sufficiently accurate line search, they subsequently introduced a self-adjusting Hager–Zhang conjugate gradient method in the vector sense. The global convergence of this new scheme was proven without requiring regular restarts or any convex assumptions. In this paper, we propose an alternative extension of the Hager–Zhang nonlinear conjugate gradient method for vector optimization that preserves its desirable scalar property, i.e., ensuring sufficiently descent without relying on any line search or convexity assumption. Furthermore, we investigate its global convergence with the Wolfe line search under mild assumptions. Finally, numerical experiments are presented to illustrate the practical behavior of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

The codes are freely available at https://github.com/zlpjulie/vector-optimization1.git.

References

  1. Ansary, M.A.T., Panda, G.: A modified quasi-Newton method for vector optimization problem. Optimization 64(11), 2289–2306 (2015)

    MathSciNet  Google Scholar 

  2. Ansary, M.A.T.: A Newton-type proximal gradient method for nonlinear multi-objective optimization problems. Optim. Methods Softw. 38(3), 570–590 (2023)

    MathSciNet  Google Scholar 

  3. Beck, A.: First-Order Methods in Optimization. SIAM, Philadelphia (2017). https://doi.org/10.1137/1.9781611974997

    Book  Google Scholar 

  4. Bello Cruz, J.Y.: A subgradient method for vector optimization problems. SIAM J. Optim. 23(4), 2169–2182 (2013)

    MathSciNet  Google Scholar 

  5. Bonnel, H., Iusem, A.N., Svaiter, B.F.: Proximal methods in vector optimization. SIAM J. Optim. 15(4), 953–970 (2005)

    MathSciNet  Google Scholar 

  6. Burachik, R.S., Kaya, C.Y., Rizvi, M.M.: A new scalarization technique and new algorithms to generate Pareto fronts. SIAM J. Optim. 27(2), 1010–1034 (2017)

    MathSciNet  Google Scholar 

  7. Ceng, L.C., Mordukhovich, B.S., Yao, J.C.: Hybrid approximate proximal method with auxiliary variational inequality for vector optimization. J. Optim. Theory Appl. 146(2), 267–303 (2010)

    MathSciNet  Google Scholar 

  8. Chen, W., Yang, X.M., Zhao, Y.: Memory gradient method for multiobjective optimization. Appl. Comput. Math. 443, 127791 (2023)

    MathSciNet  Google Scholar 

  9. Chuong, T.D.: Newton-like methods for efficient solutions in vector optimization. Comput. Optim. Appl. 54(3), 495–516 (2013)

    MathSciNet  Google Scholar 

  10. Custódio, A.L., Madeira, J.F.A., Vaz, A.I.F., Vicente, L.N.: Direct multisearch for multiobjective optimization. SIAM J. Optim. 21(3), 1109–1140 (2011)

    MathSciNet  Google Scholar 

  11. Dai, Y.H., Yuan, Y.X.: A nonlinear conjugate gradients method with a strong global convergence property. SIAM J. Optim. 10, 177–182 (1999)

    MathSciNet  Google Scholar 

  12. Dai, Y.H., Kou, C.X.: A nonlinear conjugate gradients method with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23, 296–320 (2013)

    MathSciNet  Google Scholar 

  13. Das, I., Dennis, J.: Normal-boundary intersection: a new method for generating the Pareto surface in nonlinear multicriteria optimization problems. SIAM J. Optim. 8(3), 631–657 (1998)

    MathSciNet  Google Scholar 

  14. De, P., Ghosh, J.B., Wells, C.E.: On the minimization of completion time variance with a bicriteria extension. Oper. Res. 40(6), 1148–1155 (1992)

    Google Scholar 

  15. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)

    MathSciNet  Google Scholar 

  16. Eichfelder, G.: An adaptive scalarization method in multiobjective optimization. SIAM J. Optim. 19(4), 1694–1718 (2009)

    MathSciNet  Google Scholar 

  17. Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7, 149–154 (1964)

    MathSciNet  Google Scholar 

  18. Fliege, J., Vicente, L.N.: Multicriteria approach to bilevel optimization. J. Optim. Theory Appl. 131(2), 209–225 (2006)

    MathSciNet  Google Scholar 

  19. Fliege, J., Svaiter, B.F.: Steepest descent methods for multicriteria optimization. Math. Methods Oper. Res. 51(3), 479–494 (2000)

    MathSciNet  Google Scholar 

  20. Fliege, J., Drummond, L.M.G., Svaiter, B.F.: Newton’s method for multiobjective optimization. SIAM J. Optim. 20(2), 602–626 (2009)

    MathSciNet  Google Scholar 

  21. Fukuda, E.H., Graña Drummond, L.M.: Inexact projected gradient method for vector optimization. Comput. Optim. Appl. 54(3), 473–493 (2013)

    MathSciNet  Google Scholar 

  22. Fletcher, R.: Practical Methods of Optimization, Unconstrained Optimization, vol. 1. Wiley, New York (1987)

    Google Scholar 

  23. Geoffrion, A.M.: Proper efficiency and the theory of vector maximization. J. Math. Anal. Appl. 22(3), 618–630 (1968)

    MathSciNet  Google Scholar 

  24. Graña Drummond, L.M., Raupp, F.M.P., Svaiter, B.F.: A quadratically convergent Newton method for vector optimization. Optimization 63(5), 661–677 (2014)

    MathSciNet  Google Scholar 

  25. Graña Drummond, L.M., Svaiter, B.F.: A steepest descent method for vector optimization. J. Comput. Appl. Math. 175(2), 395–414 (2005)

    MathSciNet  Google Scholar 

  26. Graña Drummond, L.M., Iusem, A.N.: A projected gradient method for vector optimization problems. Comput. Optim. Appl. 28(1), 5–29 (2004)

    MathSciNet  Google Scholar 

  27. Gonçalves, M.L.N., Lima, F.S., Prudente, L.F.: Globally convergent Newton-type methods for multiobjective optimization. Comput. Optim. Appl. 83(2), 403–434 (2022)

    MathSciNet  Google Scholar 

  28. Gonçalves, M.L.N., Lima, F.S., Prudente, L.F.: A study of Liu–Storey conjugate gradient methods for vector optimization. Appl. Comput. Math. 425, 127099 (2022)

    MathSciNet  Google Scholar 

  29. Gonçalves, M.L.N., Prudente, L.F.: On the extension of the Hager–Zhang conjugate gradient method for vector optimization. Comput. Optim. Appl. 76(3), 889–916 (2020)

    MathSciNet  Google Scholar 

  30. Hager, W.W., Zhang, H.C.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16(1), 170–192 (2005)

    MathSciNet  Google Scholar 

  31. Hager, W.W., Zhang, H.C.: A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2(1), 35–58 (2006)

    MathSciNet  Google Scholar 

  32. Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49, 409–436 (1952)

    MathSciNet  Google Scholar 

  33. He, Q.R., Chen, C.R., Li, S.J.: Spectral conjugate gradient method for vector optimization. Comput. Optim. Appl. (2023). https://doi.org/10.1007/s10589-023-00508-w

    Article  MathSciNet  Google Scholar 

  34. Hillermeier, C.: Generalized homotopy approach to multiobjective optimization. J. Optim. Theory Appl. 110(3), 557–583 (2001)

    MathSciNet  Google Scholar 

  35. Hong, T.S., Craft, D.L., Carlsson, F.: Multicriteria optimization in intensity-modulated radiation therapy treatment planning for locally advanced cancer of the pancreatic head. Int. J. Radiat. Oncol. Biol. Phys. 72(4), 1208–1214 (2008)

    Google Scholar 

  36. Huband, S., Hingston, P., Barone, L.: A review of multiobjective test problems and a scalable test problem toolkit. IEEE Trans. Evol. Comput. 10(5), 477–506 (2006)

    Google Scholar 

  37. Jahn, J., Kirsch, A., Wagner, C.: Optimization of rod antennas of mobile phones. Math. Methods Oper. Res. 59(1), 37–51 (2004)

    MathSciNet  Google Scholar 

  38. Jin, Y., Olhofer, M., Sendhoff, B.: Dynamic weighted aggregation for evolutionary multi-objective optimization: why does it work and how? In: Proceedings of the genetic and evolutionary computation conference, pp. 1042–1049 (2001)

  39. Leschine, T.M., Wallenius, H., Verdini, W.A.: Interactive multiobjective analysis and assimilative capacity-based ocean disposal decisions. Eur. J. Oper. Res. 56(2), 278–289 (1992)

    Google Scholar 

  40. Lovison, A.: Singular continuation: generating piecewise linear approximations to pareto sets via global analysis. SIAM J. Optim. 21(2), 463–490 (2011)

    MathSciNet  Google Scholar 

  41. Lu, F., Chen, C.R.: Newton-like methods for solving vector optimization problems. Appl. Anal. 93(8), 1567–1586 (2014)

    MathSciNet  Google Scholar 

  42. Lucambio Pérez, L.R., Prudente, L.F.: A Wolfe line search algorithm for vector optimization. ACM Trans. Math. Softw. 45(4), 1–23 (2019)

    MathSciNet  Google Scholar 

  43. Lucambio Pérez, L.R., Prudente, L.F.: Nonlinear conjugate gradient methods for vector optimization. SIAM J. Optim. 28(3), 2690–2720 (2018)

    MathSciNet  Google Scholar 

  44. Luc, D.T.: Theory of Vector Optimization. Lectures Notes in Economics and Mathematical Systems, vol. 319. Springer, Berlin (1989)

    Google Scholar 

  45. Miglierina, E., Molho, E., Recchioni, M.C.: Box-constrained multiobjective optimization: a gradient-like method without a priori scalarization. Eur. J. Oper. Res. 188(3), 662–682 (2008)

    Google Scholar 

  46. Mita, K., Fukuda, E.H., Yamashita, N.: Nonmonotone line searches for unconstrained multiobjective optimization problems. J. Global Optim. 75, 63–90 (2019)

    MathSciNet  Google Scholar 

  47. Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7(1), 17–41 (1981)

    MathSciNet  Google Scholar 

  48. Preuss, M., Naujoks, B., Rudolph, G.: Pareto set and EMOA behavior for simple multimodal multiobjective functions. In: Parallel Problem Solving from Nature-PPSN IX. Springer, Berlin, Heidelberg, pp. 513–522 (2006)

  49. Polyak, B.T.: The conjugate gradients method in extreme problems. Comput. Math. Math. Phys. 9, 94–112 (1969)

    Google Scholar 

  50. Schütze, O., Laumanns, M., Coello, C.A., Dellnitz, M., Talbi, E.G.: Convergence of stochastic search algorithms to finite size Pareto set approximations. J. Global Optim. 41(4), 559–577 (2008)

    MathSciNet  Google Scholar 

  51. Tavana, M.: A subjective assessment of alternative mission architectures for the human exploration of Mars at NASA using multicriteria decision making. Comput. Oper. Res. 31(7), 1147–1164 (2004)

    Google Scholar 

  52. Thomann, J., Eichfelder, G.: Numerical results for the multiobjective trust region algorithm MHT. Data Brief 25, 104103 (2019)

    Google Scholar 

  53. Toint, P.L.: Test problems for partially separable optimization and results for the routine PSPMIN. Technical Report, The University of Namur, Department of Mathematics, Belgium, 1983

  54. White, D.J.: Epsilon-dominating solutions in mean-variance portfolio analysis. Eur. J. Oper. Res. 105(3), 457–466 (1998)

    Google Scholar 

  55. Zhao, X.P., Jolaoso, L.O., Shehu, Y., Yao, J.C.: Convergence of a nonmonotone projected gradient method for nonconvex multiobjective optimization. J. Nonlinear Var. Anal. 5, 441–457 (2021)

    Google Scholar 

Download references

Acknowledgements

The author wish to thank the anonymous referees for their constructive comments and suggestions on the paper, which improve this paper greatly.

Author information

Authors and Affiliations

  1. School of Mathematics and Computing Science, Guilin University of Electronic Technology, Guilin, 541002, People’s Republic of China

    Qingjie Hu & Liping Zhu

  2. Guangxi Colleges and Universities Key Laboratory of Data Analysis and Computation, Guilin, 541002, People’s Republic of China

    Qingjie Hu

  3. Center for Applied Mathematics of Guangxi (GUET), Guilin, 541002, People’s Republic of China

    Qingjie Hu

  4. School of Mathematics and Statistics, Guangxi Normal University, Guilin, 541004, People’s Republic of China

    Yu Chen

Authors
  1. Qingjie Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Liping Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yu Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Qingjie Hu.

Ethics declarations

Conflict of interest

This study does not have any conflicts to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by NNSF(No.11961011,11761014) of China and Guangxi Science and Technology Base and Talents Special Project (No.2021AC06001) and Innovation Project of Guangxi Graduate Education (No.YCSW2022282).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, Q., Zhu, L. & Chen, Y. Alternative extension of the Hager–Zhang conjugate gradient method for vector optimization. Comput Optim Appl 88, 217–250 (2024). https://doi.org/10.1007/s10589-023-00548-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-023-00548-2

Keywords