Abstract
The problem of minimizing the number of misclassified points by a plane, attempting to separate two point sets with intersecting convex hulls inn-dimensional real space, is formulated as a linear program with equilibrium constraints (LPEC). This general LPEC can be converted to an exact penalty problem with a quadratic objective and linear constraints. A Frank-Wolfe-type algorithm is proposed for the penalty problem that terminates at a stationary point or a global solution. Novel aspects of the approach include: (i) A linear complementarity formulation of the step function that “counts” misclassifications, (ii) Exact penalty formulation without boundedness, nondegeneracy or constraint qualification assumptions, (iii) An exact solution extraction from the sequence of minimizers of the penalty function for a finite value of the penalty parameter for the general LPEC and an explicitly exact solution for the LPEC with uncoupled constraints, and (iv) A parametric quadratic programming formulation of the LPEC associated with the misclassification minimization problem.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
G. Anandalingam and T.L. Friesz (1992), Hierarchical optimization: An introduction,Annals of Operations Research 34: 1–11.
G. Anandalingam and D.J. White (1990), A solution method for the linear static stackelberg problem using penalty functions,IEEE Transactions on Automatic Control 35(10): 1170–1173.
K.P. Bennett and O.L. Mangasarian (1992), Neural network training via linear programming, in P. M. Pardalos (ed.),Advances in Optimization and Parallel Computing, pp. 56–67, North Holland, Amsterdam.
K.P. Bennett and O.L. Mangasarian (1992), Robust linear programming discrimination of two linearly inseparable sets,Optimization Methods and Software 1: 23–34.
K.P. Bennett and O.L. Mangasarian (1993), Bilinear separation of two sets in n-space,Computational Optimization & Applications 2: 207–227.
C. Berge and A. Ghouila-Houri (1965),Programming, Games and Transportation Networks, Wiley, New York.
H.D. Block and S.A. Levin (1970), On the boundedness of an iterative procedure for solving a system of linear inequalities,Proceedings of the American Mathematical Society 26: 229–235.
J.E. Dennis and R.B. Schnabel (1983),Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, N.J.
R. O. Duda and P. E. Hart (1973),Pattern Classification and Scene Analysis, John Wiley & Sons, New York.
M. Frank and P. Wolfe (1956), An algorithm for quadratic programming,Naval Research Logistics Quarterly 3: 95–110.
P.T. Harker and J.-S. Pang (1988), Existence of optimal solutions to mathematical programs with equilibrium constraints,Operations Research Letters 7: 61–64.
J. Hertz, A. Krogh, and R. G. Palmer (1991),Introduction to the Theory of Neural Computation, Addison-Wesley, Redwood City, California.
Z.-Q. Luo, J.-S. Pang, D. Ralph, and S.-Q. Wu (1993), Exact penalization and stationarity conditions of mathematical programs with equilibrium constraints, Technical Report 275, Communications Research Laboratory, McMaster University, Hamilton, Ontario, L8S 4K1, Canada.
O.L. Mangasarian (1965), Linear and nonlinear separation of patterns by linear programming.Operations Research 13: 444–452.
O.L. Mangasarian (1969),Nonlinear Programming, McGraw-Hill, New York.
O.L. Mangasarian (1978), Characterization of linear complementarity problems as linear programs,Mathematical Programming Study 7: 74–87.
O.L. Mangasarian (1986), Some applications of penalty functions in mathematical programming, in R. Conti, E. De Giorgi, and F. Giannessi (eds.),Optimization and Related Fields, pp. 307–329. Springer-Verlag, Heidelberg. Lecture Notes in Mathematics 1190.
O.L. Mangasarian, R. Setiono, and W.H. Wolberg (1989), Pattern recognition via linear programming: Theory and application to medical diagnosis, in T. F Coleman and Y. Li (eds.),Large-Scale Numerical Optimization, pp. 22–31, Philadelphia, Pennsylvania. SIAM. Proceedings of the Workshop on Large-Scale Numerical Optimization, Cornell University, Ithaca, New York, October 19–20.
S. Murthy, S. Kasif, S. Salzberg, and R. Beigel (1993), OC1: Randomized induction of oblique decision trees, inProceedings of the Eleventh National Conference on Artificial Intelligence, pp. 322–327, Cambridge, MA 02142. The AAAI Press/The MIT Press.
F. Rosenblatt (1957), The perceptron — a perceiving and recognizing automaton. Technical Report 85-460-1, Cornell Aeronautical Laboratory, Ithaca, New York.
W. H. Wolberg and O.L. Mangasarian (1990), Multisurface method of pattern separation for medical diagnosis applied to breast cytology,Proceedings of the National Academy of Sciences, U.S.A. 87: 9193–9196.
Author information
Authors and Affiliations
Additional information
This material is based on research supported by Air Force Office of Scientific Research Grant F49620-94-1-0036 and National Science Foundation Grants CCR-9101801 and CDA-9024618.
Rights and permissions
About this article
Cite this article
Mangasarian, O.L. Misclassification minimization. J Glob Optim 5, 309–323 (1994). https://doi.org/10.1007/BF01096681
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF01096681