Abstract
The NP-complete problem of determining whether two disjoint point sets in then-dimensional real spaceR n can be separated by two planes is cast as a bilinear program, that is minimizing the scalar product of two linear functions on a polyhedral set. The bilinear program, which has a vertex solution, is processed by an iterative linear programming algorithm that terminates in a finite number of steps a point satisfying a necessary optimality condition or at a global minimum. Encouraging computational experience on a number of test problems is reported.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
F.A. Al-Khayyal and J.E. Falk, “Jointly constrained biconvex programming,” Math. Oper. Res., 8-2, pp. 273–286, 1983.
E.B. Baum, Private communication, 1992.
E.B. Baum and D. Haussler, “What size net gives valid generalization,” Advances in Neural Information Processing Systems I (D.S. Touretzky, ed.), Morgan Kaufmann: San Mateo, CA, 1989, pp. 81–90.
K.P. Bennett, “Decision tree construction via linear programming,” in Proc. of the 4th Midwest Artificial Intelligence and Cognitive Science Society Conf. (M. Evans, ed.), 1992, pp. 97–101.
K.P. Bennett and O.L. Mangasarian, “Neural network training via linear programming,” Advances in Optimization and Parallel Computing (P.M. Pardalos, ed.), North Holland, Amsterdam, 1992, pp. 56–67.
K.P. Bennett and O.L. Mangasarian, “Robust linear programming discrimination of two linearly inseparable sets,” Optimization Methods and Software 1, Gordon and Breach Science Publishers: Reading, England, 1992, pp. 23–34.
C. Berge and A. Ghouila-Houri, Programming, Games and Transportation Networks, Wiley: New York, 1965.
A. Blum and R.L. Rivest, “Training a 3-node neural network is np-complete,” Advances in Neural Information Processing System I (D.S. Touretzky, ed.), Morgan Kaufmann: San Mateo, CA, 1989, pp. 494–501.
A. Charnes, “Some fundamental theorems of perceptron theory and their geometry,” Computer and Information Sciences (J.T. Lou and R.H. Wilcox, eds.), Spartan Books: Washington, 1964, pp. 67–74.
M. Frank and P. Wolfe, “An algorithm for quadratic programming,” Nav. Res. Logistics Q. 3, pp. 95–110, 1956.
J. Hertz, A. Krogh, and R.G. Palmer, Introduction to the Theory of Neural Computation, Addison-Wesley: Redwood City, CA, 1991.
W.H. Highleyman, “A note on linear separation,” IEEE Trans. Electronic Comput., vol. 10, pp. 777–778, 1961.
H. Konno and T. Kuno, “Linear multiplicative programming,” Math. Prog., vol. 56, pp. 51–64, 1992.
O.L. Mangasarian, “Linear and nonlinear separation of patterns by linear programming,” Oper. Res. vol. 13, pp. 444–452, 1965.
O.L. Mangasarian, “Multi-surface method of pattern separation,” IEEE Trans. Information Theory, vol. IT-14, pp. 801–807, 1968.
O.L. Mangasarian, Nonlinear Programming, McGraw-Hill: New York, 1969.
O.L. Mangasarian, “Characterization of linear complementarity problems as linear programs,” Math. Prog. Study, vol. 7, pp. 74–87, 1978.
O.L. Mangasarian, R. Setiono, and W.H. Wolberg, “Pattern recognition via linear programming: Theory and application to medical diagnosis,” in Proc. Workshop on Large-Scale Numerical Optimization, (T.F. Coleman and Y. Li, eds.), Cornell University, Ithaca, New York, 1989, pp. 22–31.
J.L. McClelland and D.E. Rummelhart, Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises, MIT Press: Cambridge, MA, 1987.
N. Megiddo, “On the complexity of polyhedral separability,” Discrete and Comput. Geometry, vol. 3, pp. 325–337, 1988.
M. Minsky and S. Papert, Perceptrons: An Introduction to Computational Geometry, MIT Press: Cambridge, MA, 1969.
B.A. Murtagh and M.A. Saunders, “MINOS 5.0 user's guide,” Technical Report SOL 83.20, Stanford University: Stanford, CA, 1983.
K.G. Murty, Linear Programming, John Wiley & Sons: New York, 1983.
P.M. Pardalos, “Polynomial time algorithm for some classes of constrained non-convex quadratic problems,” Optimization, vol. 21, pp. 843–853, 1990.
R.T. Rockafellar, Convex Analysis, Princeton University Press: Princeton, NJ, 1970.
F. Rosenblatt, Principles of Neurodynamics, Spartan Books: New York, 1959.
D.E. Rumelhart, G.E. Hinton, and J.L. McClelland, “Learning internal representations,” Parallel Distributed Processing (D.E. Rumelhart and J.L. McClelland, eds.), MIT Press: Cambridge, MA, 1986, pp. 318–362.
F.W. Smith, “Pattern classifier design by linear programming,” IEEE Trans. Comput., vol. C-17, pp. 367–372, 1968.
T.V. Thieu, “A note on the solution of bilinear programming problems by reduction to concave minimization,” Math. Prog., vol. 41, pp. 249–260, 1988.
D.J. White, “A linear programming approach to solving bilinear programs,” Math. Prog., vol. 56, pp. 45–50, 1992.
W.H. Wolberg and O.L. Mangasarian, “Multisurface method of pattern separation for medical diagnosis applied to breast cytology,” in Proc. Nat. Acad. Sci., vol. 87, 1990, pp. 9193–9196.
Y. Yajima and H. Konno, “Efficient algorithms for solving rank two and rank three bilinear programming problems,” J. Global Optimization, vol. 1, pp. 155–171, 1991.
Author information
Authors and Affiliations
Additional information
This material is based on research supported by Air Force Office of Scientific Research grant AFOSR-89-0410, National Science Foundation grant CCR-9101801, and Air Force Laboratory Graduate Fellowship SSN 531-56-2969.
Rights and permissions
About this article
Cite this article
Bennett, K.P., Mangasarian, O.L. Bilinear separation of two sets inn-space. Comput Optim Applic 2, 207–227 (1993). https://doi.org/10.1007/BF01299449
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01299449