Skip to main content
Springer Nature Link
Log in

A Simple Method for Convex Optimization in the Oracle Model

  • Conference paper
  • First Online:
Integer Programming and Combinatorial Optimization (IPCO 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13265))

  • 1037 Accesses

Abstract

We give a simple and natural method for computing approximately optimal solutions for minimizing a convex function f over a convex set K given by a separation oracle. Our method utilizes the Frank–Wolfe algorithm over the cone of valid inequalities of K and subgradients of f. Under the assumption that f is L-Lipschitz and that K contains a ball of radius r and is contained inside the origin centered ball of radius R, using \(O(\frac{(RL)^2}{\varepsilon ^2} \cdot \frac{R^2}{r^2})\) iterations and calls to the oracle, our main method outputs a point \(x \in K\) satisfying \(f(x) \le \varepsilon + \min _{z \in K} f(z)\).

Our algorithm is easy to implement, and we believe it can serve as a useful alternative to existing cutting plane methods. As evidence towards this, we show that it compares favorably in terms of iteration counts to the standard LP based cutting plane method and the analytic center cutting plane method, on a testbed of combinatorial, semidefinite and machine learning instances.

This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement QIP–805241).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/christopherhojny/supplement_simple-iterative-methods-linopt-convex-sets.

  2. 2.

    https://github.com/christopherhojny/supplement_simple-iterative-methods-linopt-convex-sets.

References

  1. Atkinson, D.S., Vaidya, P.M.: A cutting plane algorithm for convex programming that uses analytic centers. Math. Program. 69(1), 1–43 (1995)

    MathSciNet  MATH  Google Scholar 

  2. Badenbroek, R., de Klerk, E.: An analytic center cutting plane method to determine complete positivity of a matrix. INFORMS J. Comput. 34(2), 1115–1125 (2021)

    Google Scholar 

  3. Beck, A.: First-Order Methods in Optimization. Society for Industrial and Applied Mathematics (2017). https://doi.org/10.1137/1.9781611974997

  4. Belloni, A., Freund, R.M., Vempala, S.: An efficient rescaled perceptron algorithm for conic systems. Math. Oper. Res. 34(3), 621–641 (2009)

    Article  MathSciNet  Google Scholar 

  5. Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization. Society for Industrial and Applied Mathematics (2001). https://doi.org/10.1137/1.9780898718829

  6. Betke, U.: Relaxation, new combinatorial and polynomial algorithms for the linear feasibility problem. Discrete Comput. Geom. 32(3), 317–338 (2004). https://doi.org/10.1007/s00454-004-2878-4

    Article  MathSciNet  MATH  Google Scholar 

  7. Chekuri, C., Quanrud, K.: Approximating the held-karp bound for metric TSP in nearly-linear time. In: 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS). IEEE, October 2017. https://doi.org/10.1109/focs.2017.78

  8. Chekuri, C., Quanrud, K.: Near-linear time approximation schemes for some implicit fractional packing problems. In: Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics, January 2017. https://doi.org/10.1137/1.9781611974782.51

  9. Cheney, E.W., Goldstein, A.A.: Newton’s method for convex programming and Tchebycheff approximation. Numer. Math. 1(1), 253–268 (1959)

    Article  MathSciNet  Google Scholar 

  10. Chubanov, S.: A strongly polynomial algorithm for linear systems having a binary solution. Math. Program. 134(2), 533–570 (2011). https://doi.org/10.1007/s10107-011-0445-3

    Article  MathSciNet  MATH  Google Scholar 

  11. Chubanov, S.: A polynomial algorithm for linear feasibility problems given by separation oracles. Optim. Online (2017)

    Google Scholar 

  12. Color02 - computational symposium: Graph coloring and its generalizations (2002). http://mat.gsia.cmu.edu/COLOR02

  13. Dadush, D., Hojny, C., Huiberts, S., Weltge, S.: A simple method for convex optimization in the oracle model. arXiv:2011.08557 (2021). https://arxiv.org/abs/2011.08557

  14. Dadush, D., Végh, L.A., Zambelli, G.: Rescaling algorithms for linear conic feasibility. Math. Oper. Res. 45(2), 732–754 (2020). https://doi.org/10.1287/moor.2019.1011

    Article  MathSciNet  MATH  Google Scholar 

  15. Dantzig, G.B.: Converting a converging algorithm into a polynomially bounded algorithm. Technical report, Stanford University, 1992. 5.6, 6.1, 6.5 (1991)

    Google Scholar 

  16. Demiriz, A., Bennett, K.P., Shawe-Taylor, J.: Linear programming boosting via column generation. Mach. Learn. 46(1), 225–254 (2002)

    Article  Google Scholar 

  17. Dunagan, J., Vempala, S.: A simple polynomial-time rescaling algorithm for solving linear programs. Math. Program. 114(1), 101–114 (2007). https://doi.org/10.1007/s10107-007-0095-7

    Article  MathSciNet  MATH  Google Scholar 

  18. Edmonds, J.: Maximum matching and a polyhedron with 0,1-vertices. J. Res. Natl. Bur. Stand. 69B(1–2), 125–130 (1964)

    MathSciNet  MATH  Google Scholar 

  19. Frank, M., Wolfe, P.: An algorithm for quadratic programming. Naval Res. Logist. Q. 3(1–2), 95–110 (1956)

    Article  MathSciNet  Google Scholar 

  20. Garg, N., Könemann, J.: Faster and simpler algorithms for multicommodity flow and other fractional packing problems. SIAM J. Comput. 37(2), 630–652 (2007). https://doi.org/10.1137/s0097539704446232

    Article  MathSciNet  MATH  Google Scholar 

  21. Goemans, M.X., Williamson, D.P.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM 42(6), 1115–1145 (1995). https://doi.org/10.1145/227683.227684

    Article  MathSciNet  MATH  Google Scholar 

  22. Gomory, R.E.: Outline of an algorithm for integer solutions to linear programs. Bull. Amer. Math. Soc. 64, 275–278 (1958)

    Article  MathSciNet  Google Scholar 

  23. Grötschel, M., Lovász, L., Schrijver, A.: Geometric Algorithms and Combinatorial Optimization, vol. 2. Springer, Heidelberg (1988). https://doi.org/10.1007/978-3-642-78240-4

    Book  MATH  Google Scholar 

  24. Jaggi, M.: Revisiting Frank-Wolfe: projection-free sparse convex optimization. In: Proceedings of Machine Learning Research, vol. 28, pp. 427–435. PMLR, Atlanta, 17–19 June 2013. http://proceedings.mlr.press/v28/jaggi13.html

  25. Jiang, H., Lee, Y.T., Song, Z., Wong, S.C.W.: An improved cutting plane method for convex optimization, convex-concave games, and its applications. In: Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, STOC 2020, pp. 944–953. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3357713.3384284

  26. Kelley, J.E., Jr.: The cutting-plane method for solving convex programs. J. Soc. Ind. Appl. Math. 8(4), 703–712 (1960)

    Article  MathSciNet  Google Scholar 

  27. Khachiyan, L.G.: A polynomial algorithm in linear programming (in russian). Doklady Akademiia Nauk SSSR 224 224, 1093–1096 (1979). English Translation: Soviet Mathematics Doklady 20, 191–194

    Google Scholar 

  28. Lee, Y.T., Sidford, A., Wong, S.C.: A faster cutting plane method and its implications for combinatorial and convex optimization. In: 2015 IEEE 56th Annual Symposium on Foundations of Computer Science, pp. 1049–1065 (2015). https://doi.org/10.1109/FOCS.2015.68

  29. Nemirovsky, A., Yudin, D.: Informational complexity and efficient methods for solution of convex extremal problems. Ékon. Math. Metody 12 (1983)

    Google Scholar 

  30. Nesterov, Y.: Cutting plane algorithms from analytic centers: efficiency estimates. Math. Program. 69(1), 149–176 (1995)

    Article  MathSciNet  Google Scholar 

  31. Plotkin, S.A., Shmoys, D.B., Tardos, É.: Fast approximation algorithms for fractional packing and covering problems. Math. Oper. Res. 20(2), 257–301 (1995). https://doi.org/10.1287/moor.20.2.257

    Article  MathSciNet  MATH  Google Scholar 

  32. Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386–408 (1958). https://doi.org/10.1037/h0042519

    Article  Google Scholar 

  33. Shahrokhi, F., Matula, D.W.: The maximum concurrent flow problem. J. ACM 37(2), 318–334 (1990). https://doi.org/10.1145/77600.77620, http://doi.acm.org/10.1145/77600.77620

  34. Sonnevend, G.: New algorithms in convex programming based on a notion of “centre” (for systems of analytic inequalities) and on rational extrapolation. In: Hoffmann, K.H., Zowe, J., Hiriart-Urruty, J.B., Lemarechal, C. (eds.) Trends in Mathematical Optimization: 4th French-German Conference on Optimization, pp. 311–326. Birkhäuser Basel, Basel (1988)

    Google Scholar 

  35. UC Irvine Machine Learning Repository. https://archive-beta.ics.uci.edu/ml/datasets. Accessed 3 Sept 2021

  36. Vaidya, P.M.: A new algorithm for minimizing convex functions over convex sets. Math. Program. 73(3), 291–341 (1996). https://doi.org/10.1007/bf02592216

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

We would like to thank Robert Luce and Sebastian Pokutta for their very valuable feedback on our work.

Author information

Authors and Affiliations

  1. Centrum Wiskunde and Informatica, Amsterdam, The Netherlands

    Daniel Dadush & Sophie Huiberts

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Christopher Hojny

  3. Technical University of Munich, Munich, Germany

    Stefan Weltge

Authors
  1. Daniel Dadush
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher Hojny
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sophie Huiberts
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefan Weltge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefan Weltge .

Editor information

Editors and Affiliations

  1. Delft University of Technology, Delft, The Netherlands

    Karen Aardal

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Laura Sanità

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dadush, D., Hojny, C., Huiberts, S., Weltge, S. (2022). A Simple Method for Convex Optimization in the Oracle Model. In: Aardal, K., Sanità, L. (eds) Integer Programming and Combinatorial Optimization. IPCO 2022. Lecture Notes in Computer Science, vol 13265. Springer, Cham. https://doi.org/10.1007/978-3-031-06901-7_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06901-7_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06900-0

  • Online ISBN: 978-3-031-06901-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics