Skip to main content
SpringerLink
Log in

Spectral bundle methods for non-convex maximum eigenvalue functions: first-order methods

  • Published:
Mathematical Programming Submit manuscript

Abstract

Many challenging problems in automatic control may be cast as optimization programs subject to matrix inequality constraints. Here we investigate an approach which converts such problems into non-convex eigenvalue optimization programs and makes them amenable to non-smooth analysis techniques like bundle or cutting plane methods. We prove global convergence of a first-order bundle method for programs with non-convex maximum eigenvalue functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Apkarian, P., Noll, D., Tuan, H.D.: A prototype primal-dual LMI interior point algorithm for non-convex robust control problems. Rapport interne 01-08 (2002), UPS-MIP, CNRS UMR 5640. http://mip.ups-tlse.fr/publi/publi.html

  2. Apkarian, P., Noll, D., Tuan, H.D.: Fixed-order control design via an augmented Lagrangian method. Int. J. Robust and Nonlin. Control 13 (12), 1137–1148 (2003)

    Article  Google Scholar 

  3. Apkarian, P., Noll, D., Thevenet, J.-B., Tuan, H.D.: A spectral quadratic-SDP method with applications to fixed-order H2 and H∞ synthesis. European J. Control, 10, 527–538 (2004)

    Google Scholar 

  4. Apkarian, P., Noll, D.: Controller design via nonsmooth multi-directional search. SIAM J. on Control and Optimization, to appear (2005)

  5. Apkarian, P., Noll, D.: Nonsmooth H∞-synthesis. Submitted

  6. Apkarian, P., Tuan, H.D.: Concave programming in control theory. J. Global Optim. 15, 343–370 (1999)

    Article  Google Scholar 

  7. Apkarian, P., Tuan, H.D.: Robust control via concave minimization - local and global algorithms. IEEE Trans. Autom. Control 45, 299–305 (2000)

    Article  Google Scholar 

  8. Beran, E.B., Vandenberghe, L., Boyd, S.: A global BMI algorithm based on the generalized Benders decomposition. Proc. European Control Conf., Belgium, 1997

  9. Boyd, S., ElGhaoui, L., Feron, E., Balakrishnan, V.: Linear Matrix Inequalities in System and Control Theory. vol. 15 of Studies in Appl. Math. SIAM, Philadelphia, 1994

  10. Boyd, S., Vandenberghe, L.: Semidefinite programming relaxations of non-convex problems in control and combinatorial optimization

  11. Clarke, F.H.: Optimization and Nonsmooth Analysis. John Wiley, New York, 1983

  12. Cullum, J., Donath, W.E., Wolfe, P.: The minimization of certain nondifferential sums of eigenvalues of symmetric matrices. Math. Progr. Stud. 3, 35–55 (1975)

    Google Scholar 

  13. ElGhaoui, L., Balakrishnan, V.: Synthesis of fixed-structure controllers via numerical optimization. Proc. IEEE Conf. Dec. Contr., December 1994

  14. Fares, B., Noll, D., Apkarian, P.: An augmented Lagrangian method for a class of LMI-constrained problems in robust control theory. Intern. J. Control 74, 348–360 (2001)

    Article  Google Scholar 

  15. Fares, B., Noll, D., Apkarian, P.: Robust control via successive semidefinite programming. SIAM J. Control Optim. 40, 1791–1820 (2002)

    Article  Google Scholar 

  16. Fletcher, R.: Semidefinite constraints in optimization. SIAM J. Control Optim. 23, 493–513 (1985)

    Article  Google Scholar 

  17. Fletcher, R., Leyffer, S.: A bundle filter method for nonsmooth nonlinear optimization. University of Dundee, Report NA/195, 1999

  18. Fuduli, A., Gaudioso, M., Giallombardo, G.: Minimizing nonconvex nonsmooth functions via cutting planes and proximity control. SIAM J. Optim. 14 (3), 743–756 (2004)

    Article  Google Scholar 

  19. Fuduli, A., Gaudioso, M., Giallombardo, G.: A DC piecewise affine model and a bundling technique in nonconvex nonsmooth minimization. Optimization Methods and Software 19 (1), 89–102 (2004)

    Google Scholar 

  20. Fukuda, M., Kojima, M.: Branch-and-cut algorithms for the bilinear matrix inequality eigenvalue problem. Research Report on Mathematical and Computational Sciences. Series B: Operations Research, 1999

  21. Grigoriadis, K.M., Skelton, R.E.: Low-order control design for LMI problems using alternating projection methods. Automatica 32, 1117–1125 (1996)

    Article  Google Scholar 

  22. Goh, K.-C., Safonov, M.G., Papavassilopoulos, G.P.: Global optimization for the biaffine matrix inequality problem. J. Global Optim. 7, 365–380 (1995)

    Article  MathSciNet  Google Scholar 

  23. Helmberg, C.: A cutting plane algorithm for large scale semidefinite relaxations. ZIB-Report ZR-01-26. Padberg Festschrift. The sharpest cut. SIAM, 2001

  24. Helmberg, C., Kiwiel, K.C.: A spectral bundle method with bounds. Math. Programming 93 (2), 173–194 (2002)

    Article  Google Scholar 

  25. Helmberg, C., Oustry, F.: Bundle methods to minimize the maximum eigenvalue function. In: L. Vandenberghe, R. Saigal, H. Wolkowitz, (eds.), Handbook of Semidefinite Programming. Theory, Algorithms and Applications, vol. 27, Int. Series in Oper. Res. and Management Sci. Kluwer Acad. Publ., 2000

  26. Helmberg, C., Rendl, F.: A spectral bundle method for semidefinite programming. SIAM J. Optim. 10 (3), 673–696 (2000)

    Article  Google Scholar 

  27. Helmberg, C., Rendl, F.: Solving quadratic (0,1)-problems by semidefinite programs and cutting planes. Math. Programming 82, 291–315 (1998)

    Google Scholar 

  28. Helton, J.W., Merino, O.: Coordinate optimization for bi-convex matrix inequalities. Proc. IEEE Conf. Decision and Control, San Diego, 1997, pp. 3609–3613

  29. Hiriart-Urruty, J.-B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms, part II. Springer Verlag, Berlin, 1993

  30. Hol, C.W.J., Scherer, C.W., van der Meché, E.G., Bosgra, O.H.: A nonlinear SDP approach to fixed-order controller synthesis and comparison with two other methods applied to an active suspension system. submitted, 2002

  31. Iwasaki, T.: The dual iteration for fixed order control. Proc. Am. Control Conf., 1997, pp. 62–66

  32. Kiwiel, K.C.: Methods of descent for nondifferentiable optimization. Lect. Notes in Math. vol. 1133, Springer Verlag, Berlin, 1985

  33. Kiwiel, K.C.: A linearization algorithm for optimizing control systems subject to singular value inequalities. IEEE Trans. Autom. Control AC-31, 1986, pp. 595–602

    Google Scholar 

  34. Kiwiel, K.C.: A constraint linearization method for nondifferentiable convex minimization. Numerische Mathematik 51, 395–414 (1987)

    Article  Google Scholar 

  35. Kiwiel, K.C.: Proximity control in bundle methods for convex nondifferentiable optimization. Math. Programming 46, 105–122 (1990)

    Article  Google Scholar 

  36. Kiwiel, K.C.: Restricted-step and Levenberg-Marquardt techniques in proximal bundle methods for nonconvex nondifferentiable optimization. SIAM J. Optim. 6, 227–249 (1996)

    Article  Google Scholar 

  37. Krishnan, K., Mitchell, J.E.: Cutting plane methods for semidefinite programming. Submitted, 2002

  38. Lemaréchal, C.: Extensions diverses des méthodes de gradient et applications. Thèse d'Etat, Paris, 1980

  39. Lemaréchal, C.: Bundle methods in nonsmooth optimization. In: Nonsmooth Optimization, Proc. IIASA Workshop 1977, C. Lemaréchal, R. Mifflin (eds.), Pergamon Press, 1978

  40. Lemaréchal, C.: Nondifferentiable Optimization. chapter VII In: Handbooks in Operations Research and Management Science, vol. 1, Optimization, G.L. Nemhauser, A.H.G. Rinnooy Kan, M.J. Todd (eds.), North Holland, 1989

  41. Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Programming 69, 111–147 (1995)

    Google Scholar 

  42. Lemaréchal, C., Oustry, F.: Nonsmooth algorithms to solve semidefinite programs. In: L. El Ghaoui, L. Niculescu (eds.), Advances in LMI methods in Control, SIAM Advances in design and Control Series, 2000

  43. Miller, S.A., Smith, R.S.: Solving large structured semidefinite programs using an inexact spectral bundle method. Proc. IEEE Conf. Dec. Control, 2000, pp. 5027–5032

  44. Miller, S.A., Smith, R.S.: A bundle method for efficiently solving large structured linear matrix inequalities. Proc. Am. Control Conf., Chicago, 2000

  45. Nemirovskii, A.: Several NP-Hard problems arising in robust stability analysis. Mathematics of Control, Signals, and Systems. 6 (1), 99–105 (1994)

    Google Scholar 

  46. Noll, D., Apkarian, P.: Spectral bundle methods for nonconvex maximum eigenvalue functions: second-order methods. Math. Programming, Series B 104, 729–747 2005

  47. Noll, D., Torki, M., Apkarian, P.: Partially augmented Lagrangian method for matrix inequalities. SIAM J. on Optimization 15 (1), 161–184 (2004)

    Article  Google Scholar 

  48. Overton, M.L.: On minimizing the maximum eigenvalue of a symmetric matrix. SIAM J. Matrix Anal. Appl. 9 (2), 256–268 (1988)

    Article  Google Scholar 

  49. Overton, M.L.: Large-scale optimization of eigenvalues. SIAM J. Optim. 2 (1), 88–120 (1992)

    Article  Google Scholar 

  50. Overton, M.L., Womersley, R.S.: Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices. Math. Programming 62, 321–357 (1993)

    Article  Google Scholar 

  51. Overton, M.L., Womersley, R.S.: Second derivatives for optimizing eigenvalues of symmetric matrices. SIAM J. Matrix Anal. Appl. 16 (3), 697–718 (1995)

    Article  Google Scholar 

  52. Haeberly, J.-P.A., Overton, M.L.: A hybrid algorithm for optimizing eigenvalues of symmetric definite pencils. SIAM J. Matrix Anal. Appl. 15, 1141–1156 (1994)

    Article  Google Scholar 

  53. Oustry, F.: Vertical development of a convex function. J. Convex Analysis 5, 153–170 (1998)

    Google Scholar 

  54. Oustry, F.: The -Lagrangian of the maximum eigenvalue function. SIAM J. Optim. 9 (2), 526–549 (1999)

    Article  Google Scholar 

  55. Oustry, F.: A second-order bundle method to minimize the maximum eigenvalue function. Math. Programming Series A 89 (1), 1–33 (2000)

    Google Scholar 

  56. Polak, E.: On the mathematical foundations of nondifferentiable optimization in engineering design. SIAM Review 29, 21–89 (1987)

    Article  Google Scholar 

  57. Polak, E.: Optimization: Algorithms and consistent approximations. Springer Verlag, Applied Mathematical Sciences, 1997

  58. Polak, E., Wardi, Y.: Nondifferentiable optimization algorithm for designing control systems having singular value inequalities. Automatica 18 (3), 267–283 (1982)

    Article  Google Scholar 

  59. Rotunno, M., de Callafon, R.A.: A bundle method for solving the fixed order control problem. Proc. IEEE Conf. Dec. Control, Las Vegas, 2002, pp. 3156–3161

  60. Scherer, C.: A full block S-procedure with applications. Proc. IEEE Conf. on decision and Control, San Diego, USA, 1997, pp. 2602–2607

  61. Scherer, C.: Robust mixed control and linear parameter-varying control with full block scalings. SIAM Advances in Linear Matrix Inequality Methods in Control Series, L. El Ghaoui, S.I. Niculescu (eds.), 2000

  62. Schramm, H., Zowe, J.: A version of the bundle method for minimizing a nonsmooth function: conceptual idea, convergence analysis, numerical results. SIAM J. Optim. 2, 121–152 (1992)

    Article  Google Scholar 

  63. Shapiro, A.: Extremal problems on the set of nonnegative matrices. Lin. Algebra Appl. 67, 7–18 (1985)

    Article  Google Scholar 

  64. Shapiro, A.: First and second order analysis of nonlinear semidefinite programs. Math. Programming Ser. B 77, 301–320 (1997)

    Article  Google Scholar 

  65. Shapiro, A., Fan, M.K.H.: On eigenvalue optimization. SIAM J. Optim. 5(30), 552–569 (1995)

    Article  Google Scholar 

  66. Steward, G.W., Ji-guang Sun: Matrix Perturbation Theory. Computer Science and Scientific Computing, Academic Press, 1990

  67. Thevenet, J.-B., Apkarian, P., Noll, D.: Description and user's guide of specSDP, a Matlab-interfaced Fortran code to solve linear and nonlinear SDP constrained optimization programs. Technical Report, UPS-MIP, 2005

  68. Thevenet, J.-B., Apkarian, P., Noll, D.: Reduced-order output feedback control design with specSDP, a code for linear and nonlinear SDP problems. ICCA05, Budapest, 2005

  69. Thevenet, J.-B., Noll, D., Apkarian, P.: Nonlinear spectral SDP method for BMI-constrained problems: Applications to control design. Informatics in Systems and Control, Automation and Robotics, Kluwer Academic Publishers, 2004

  70. Torki, M.: Second order directional derivative of all eigenvalues of a symmetric matrix. Nonlin. Analysis, Theory, Methods and Appl. 46, 1133–1500 (2001)

    Google Scholar 

  71. Tuan, H.D., Apkarian, P., Nakashima, Y.: A new Lagrangian dual global optimization algorithm for solving bilinear matrix inequalities. Proc. Amer. Control Conf., 1999

  72. Whidborne, J.F., Wu, J., Istepanian, R.H.: Finite word length stability issues in an ℓ1 Framework. Int. J. Control 73 (2), 166–176 (2000)

    Google Scholar 

  73. Wolfe, Ph.: A method of conjugate subgradients for minimizing nondifferentiable convex functions. Math. Programming Studies 3, 145–173 (1975)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut de Mathématiques, Université Paul Sabatier, 118, route de Narbonne, 31062, Toulouse, France

    Dominikus Noll

  2. Control System Department, CERT-ONERA, 2, avenue Edouard Bélin, 31055, Toulouse, France

    Pierre Apkarian

Authors
  1. Dominikus Noll
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pierre Apkarian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dominikus Noll.

Additional information

Dedicated to R. T. Rockafellar on the occasion of his 70th anniversary

Rights and permissions

Reprints and permissions

About this article

Cite this article

Noll, D., Apkarian, P. Spectral bundle methods for non-convex maximum eigenvalue functions: first-order methods. Math. Program. 104, 701–727 (2005). https://doi.org/10.1007/s10107-005-0634-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-005-0634-z

Keywords

Search

Navigation