Skip to main content
SpringerLink
Log in

On the Properties of the Method of Minimization for Convex Functions with Relaxation on the Distance to Extremum

  • Optimization, System Analysis, and Operations Research
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

We present a subgradient method of minimization, similar to the method of minimal iterations for solving systems of equations, which inherits from the latter convergence properties on quadratic functions. The proposed algorithm, for a certain set of parameters, coincides with the previously known method of minimizing piecewise linear functions and is an element of the family of minimization methods with relaxation of the distance to extremum, developed by B.T. Polyak, where the step length is calculated based on the predefined minimum value of the function. We link parameters of this method to the constraint on the degree of homogeneity of the function and obtain estimates on its convergence rate on convex functions. We prove that on some classes of functions it converges at the rate of a geometric progression. We also discuss the computational capabilities of this approach for solving problems with high dimension.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Shor, N.Z., An Application of Gradient Descent to Solve a Network Transportation Problem, Proc. Seminar on Theory and Applied Prob. Cyber. Oper. Research, Kiev: Nauch. Sovet po Kibernetike AN USSR, 1962, no. 1, pp. 9–17.

    Google Scholar 

  2. Polyak, B.T., One General Approach to Solving Extreme Problems, Dokl. AN USSR, 1967, vol. 174, no. 1, pp. 33–36.

    Google Scholar 

  3. Polyak, B.T., Vvedenie v optimizatsiyu (Introduction to Optimization), Moscow: Nauka, 1983.

    MATH  Google Scholar 

  4. Wolfe, P., Note on a Method of Conjugate Subgradients for Minimizing Nondifferentiable Functions, Math. Program., 1974, vol. 7, no. 1, pp. 380–383.

    Article  MathSciNet  MATH  Google Scholar 

  5. Lemarechal, C., An Extension of Davidon Methods to Non-Differentiable Problems, Math. Program. Study, 1975, vol. 3, pp. 95–109.

    Article  MathSciNet  MATH  Google Scholar 

  6. Nemirovskii, A.S. and Yudin, D.B., Slozhnost’ zadach i effektivnost’ metodov optimizatsii (Complexity of Problems and Efficiency of Optimization Methods), Moscow: Nauka, 1979.

    Google Scholar 

  7. Shor, N.Z., Metody minimizatsii nedifferentsiruemykh funktsii i ikh prilozheniya (Minimization Methods for Non-Differentiable Functions and Their Applications), Kiev: Naukova Dumka, 1979.

    MATH  Google Scholar 

  8. Krutikov, V.N. and Petrova, T.V., A Relaxation Method of Minimization with Space Dilation in the Direction of Subgradient, Ekonom. Mat. Metody, 2003, vol. 39, no. 1, pp. 106–119.

    MATH  Google Scholar 

  9. Krutikov, V.N. and Gorskaya, T.A., A Family of Relaxation Subgradient Methods with Two-Rank Correction of the Metric Matrices, Ekonom. Mat. Metody, 2009, vol. 45, no. 4, pp. 37–80.

    Google Scholar 

  10. Nurminskii, E.A. and Tien, D., Method of Conjugate Subgradients with Constrained Memory, Autom. Remote Control, 2014, vol. 75, no. 4, pp. 646–656.

    Article  MathSciNet  MATH  Google Scholar 

  11. Krutikov, V.N. and Vershinin, Ya.N., A Multistep Subgradient Method for Solving Nonsmooth Minimization Problems of High Dimension, Vestn. Tomsk. Gos. Univ., Mat. Mekh., 2014, no. 3, pp. 5–19.

    Google Scholar 

  12. Krutikov, V.N. and Vershinin, Ya.N., A Subgradient Minimization Method with Correction of Descent Vectors Based on Pairs of Training Relations, Vestn. Kemer. Gos. Univ., 2014, no. 1–1 (57), pp. 46–54.

    Google Scholar 

  13. Gol’shtein, E.G., Nemirovskii, A.S., and Nesterov, Yu.E., Method of Levels, its Generalization and Applications, Ekonom. Mat. Metody, 1995, vol. 31, no. 3, pp. 164–180.

    MATH  Google Scholar 

  14. Nesterov, Yu.E., Smooth Minimization of Non-Smooth Functions, Math. Program., 2005, vol. 103, no. 1, pp. 127–152.

    Article  MathSciNet  MATH  Google Scholar 

  15. Nesterov, Yu., Universal Gradient Methods for Convex Optimization Problems, Math. Program., Ser. A, 2015, vol. 152, pp. 381–404.

    Article  MathSciNet  MATH  Google Scholar 

  16. Gasnikov, A.V. and Nesterov, Yu.E., A Universal Method for Stochastic Composite Optimization Problems, e-print, 2016. https://arxiv.org/ftp/arxiv/papers/1604/1604.05275.pdf

    Google Scholar 

  17. Nesterov, Yu., Subgradient Methods for Huge-Scale Optimization Problems, Math. Program., Ser. A, 2013, vol. 146, no. 1–2, pp. 275–297.

    Google Scholar 

  18. Polyak, B.T., Minimzation of Non-Smooth Functionals, Zh. Vychisl. Mat. Mat. Fiz., 1969, vol. 9, no. 3, pp. 507–521.

    Google Scholar 

  19. Samoilenko, N.S., Krutikov, V.N., and Meshechkin, V.V., A Study of One Variation of the Subgradient Method, Vestn. Kemer. Gos. Univ., 2015, no. 5 (2), pp. 55–58.

    Google Scholar 

  20. Fadeev, D.K. and Fadeeva, D.K., Vychislitel’nye metody lineinoi algebry (Computational Methods of Linear Algebra), Moscow: Fizmatgiz, 1963.

    Google Scholar 

  21. Voevodin, V.V. and Kuznetsov, Yu.A., Matritsy i vychisleniya (Matrices and Computations), Moscow: Nauka, 1984.

    MATH  Google Scholar 

  22. Camerini, P., Fratta, L., and Maffioli, F., On Improving Relaxation Methods by Modified Gradient Techniques, Math. Program., 1975, no. 3, pp. 26–34.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kemerovo State University, Kemerovo, Russia

    V. N. Krutikov, N. S. Samoilenko & V. V. Meshechkin

Authors
  1. V. N. Krutikov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. N. S. Samoilenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. V. V. Meshechkin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. N. Krutikov.

Additional information

Russian Text © V.N. Krutikov, N.S. Samoilenko, V.V. Meshechkin, 2019, published in Avtomatika i Telemekhanika, 2019, No. 1, pp. 126–137.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krutikov, V.N., Samoilenko, N.S. & Meshechkin, V.V. On the Properties of the Method of Minimization for Convex Functions with Relaxation on the Distance to Extremum. Autom Remote Control 80, 102–111 (2019). https://doi.org/10.1134/S0005117919010090

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0005117919010090

Keywords

Search

Navigation