Abstract
The perceptron algorithm, developed mainly in the machine learning literature, is a simple greedy method for finding a feasible solution to a linear program (alternatively, for learning a threshold function). In spite of its exponential worst-case complexity, it is often quite useful, in part due to its noise-tolerance and also its overall simplicity. In this paper, we show that a randomized version of the perceptron algorithm along with periodic rescaling runs in polynomial-time. The resulting algorithm for linear programming has an elementary description and analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agmon S. (1954). The relaxation method for linear inequalities. Can. J. Math. 6(3): 382–392
Bertsimas D. and Vempala S. (2004). Solving convex programs by random walks. J. ACM 51(4): 540–556
Blum, A., Dunagan, J.: Smoothed Analysis of the Perceptron Algorithm for Linear Programming. In: Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 905–914 (2002)
Blum A., Frieze A., Kannan R. and Vempala S. (1998). A polynomial-time algorithm for learning noisy linear threshold functions. Algorithmica 22(1): 35–52
Bylander, T.: Learning linear threshold functions in the presence of classification noise. In: Proceedings of the Workshop on Computational Learning Theory (1994)
Cohen, E.: Learning noisy perceptrons by a perceptron in polynomial time. In: Proceedings of the Annual IEEE Symposium on the Foundations of Computer Science, pp. 514–523 (1997)
Cucker F. and Cheung D. (2001). A new condition number for linear programming. Math. Program. Ser. A 91(1): 163–174
Chvatal V. (1983). Linear Programming. W.H. Freeman, San Francisco
Dunagan, J., Teng, S., Spielman, D.A.: Smoothed analysis of Renegar’s condition number for linear programming. In: SIAM Conference on Optimization (2002)
Freund R.M. and Mizuno S. (2000). Interior point methods: current status and future directions. In: Frenk, H. (eds) High Performance Optimization., pp 441–466. Kluwer, Dordrecht
Freund R.M. and Vera J.R. (1999). Some characterizations and properties of the distance to ill-posedness and the condition measure of a conic linear system. Math. Program. 86: 225–260
Forster, J.: A linear lower bound on the unbounded error probabilistic communication complexity. In: 16th Annual IEEE Conference on Computational Complexity (2001). http:// citeseer.nj.nec.com/forster01linear.html
Goffin, J.-L.: On the finite convergence of the relaxation method for solving systems of inequalities. Ph.D Thesis (1971). Also research report of the Operations Research Center, University of California at Berkeley
Goffin J.-L. (1980). The relaxation method for solving systems of linear inequalities. Math. Oper. Res. 5(3): 388–414
Grötchel L., Lovasz L. and Schrijver A. (1988). Geometric algorithms and Combinatorial Optimization. Springer, Berlin
Karmarkar N. (1984). A new polynomial-time algorithm for linear programming. Combinatorica 4: 373–396
Khachiyan, L.G.: A polynomial algorithm in linear programming, (in Russian), Doklady Akedamii Nauk SSSR, vol. 244, pp. 1093–1096 (1979) (English translation: Soviet Mathematics Doklady, 20, 191–194 1979)
Minsky, M.L., Papert, S.A.: Perceptrons: sn introduction to computational geometry (1969)
Renegar J. (1995). Incorporating condition measures into the complexity theory of linear programming. SIAM J Optim 5(3): 506–524
Rosenblatt F. (1962). Principles of Neurodynamics. Spartan Books, New York
Schrijver A. (1998). Theory of Linar and Integer Programming. Wiley, New York
Servedio R. (2002). On PAC learning using perceptron, winnow and a perceptron-like algorithm. SIAM J. Comput. 31(5): 1358–1369
Servedio, R.: Smooth boosting and learning with malicious noise. In: 14th Annual Conference on Computational Learning Theory, pp. 473–489 (2001)
Shor N.Z. (1977). Cut-off method with space extension in convex programming problems. Cybernetics 13: 94–96
Shor, N.Z.: Utilization of the operation of space dilation in the minimization of convex functions. Kibernetika 1, 6–12 (1970) English translation: Cybernetics, 1, 7–15
Vaidya P.M. (1996). A new algorithm for minimizing convex functions over convex sets. Math Program 73: 291–341
Yudin, D.B., Nemirovski, A.S.: Evaluation of the information complexity of mathematical programming problems (in Russian). Ekonomika i Matematicheskie Metody 12, 128–142, (1976) (English translation: Matekon 13(2), 3–45 1976)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dunagan, J., Vempala, S. A simple polynomial-time rescaling algorithm for solving linear programs. Math. Program. 114, 101–114 (2008). https://doi.org/10.1007/s10107-007-0095-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-007-0095-7