Introduction
Classification is a fundamental machine learning task whereby rules are developed for the allocation of independent observations to groups. Classic examples of applications include medical diagnosis – the allocation of patients to disease classes on the basis of symptoms and laboratory tests – and credit screening – the acceptance or rejection of credit applications on the basis of applicant data. Data are collected concerning observations with known group membership. These training data are used to develop rules for the classification of future observations with unknown group membership.
In this introduction, we briefly describe some terminologies related to classification, and provide a brief description of the organization of this chapter.
Pattern Recognition, Discriminant Analysis, and Statistical Pattern Classification
Cognitive science is the science of learning, knowing, and reasoning. Pattern recognitionis a broad field within...
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abad PL, Banks WJ (1993) New LP based heuristics for the classification problem. Eur J Oper Res 67:88–100
Anderson JA (1969) Constrained discrimination between k populations. J Roy Statist Soc Ser B (Methodological) 31(1):123–139
Asparoukhov OK, Stam A (1997) Mathematical programming formulations for two-group classification with binary variables. Ann Oper Res 74:89–112
Atamturk A (1998) Conflict graphs and flow models for mixed-integer linear optimization problems. PhD thesis, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta
Bajgier SM, Hill AV (1982) An experimental comparison of statistical and linear programming approaches to the discriminant problem. Decis Sci 13:604–618
Banks WJ, Abad PL (1991) An efficient optimal solution algorithm for the classification problem. Decis Sci 22:1008–1023
Banks WJ, Abad PL (1994) On the performance of linear programming heuristics applied on a quadratic transformation in the classification problem. Eur J Oper Res 74:23–28
Bennett KP (1992) Decision tree construction via linear programming. In: Evans M (ed) Proceedings of the 4th Midwest Artificial Intelligence and Cognitive Science Society Conference, pp 97–101
Bennett KP, Mangasarian OL (1992) Robust linear programming discrimination of two linearly inseparable sets. Optim Methods Softw 1:23–34
Bennett KP, Mangasarian OL (1994) Multicategory discrimination via linear programming. Optim Methods Softw 3:27–39
Bixby RE, Lee EK (1998) Solving a truck dispatching scheduling problem using branch-and-cut. Oper Res 46:355–367
Borndörfer R (1997) Aspects of set packing, partitioning and covering. PhD thesis, Technischen Universität Berlin, Berlin
Bradley PS, Mangasarian OL (2000) Massive data discrimination via linear support vector machines. Optim Methods Softw 13(1):1–10
Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and Regression Trees. Wadsworth and Brooks/Cole Advanced Books and Software, Pacific Grove
Brock GJ, Huang TH, Chen CM, Johnson KJ (2001) A novel technique for the identification of CpG islands exhibiting altered methylation patterns (ICEAMP). Nucleic Acids Res 29:e123
Brooks JP, Wright A, Zhu C, Lee EK (2007) Discriminant analysis of motility and morphology data from human lung carcinoma cells placed on purified extracellular matrix proteins. Ann Biomed Eng, Submitted
Brooks JP, Lee EK (2006) Solving a mixed-integer programming formulation of a multi-category constrained discrimination model. In: Proceedings of the 2006 INFORMS Workshop on Artificial Intelligence and Data Mining, Pittsburgh
Brooks JP, Lee EK (2007) Analysis of the consistency of a mixed integer programming-based multi-category constrained discriminant model. Submitted
Brooks JP, Lee EK (2007) Mixed integer programming constrained discrimination model for credit screening. In: Proceedings of the 2007 Spring Simulation Multiconference, Business and Industry Symposium, pp 1–6, Norfolk, VA, March ACM Digital Library
Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Mining Knowl Discov 2:121–167
Chen C, Mangasarian OL (1996) Hybrid misclassification minimization. Adv Comput Math 5:127–136
Chevion M, Berenshtein E, Stadtman ER (2000) Human studies related to protein oxidation: protein carbonyl content as a marker of damage. Free Radical Res 33(Suppl):S99–S108
Costello JF, Fruhwald MC, Smiraglia DJ, Rush LJ, Robertson GP, Gao X, Wright FA, Feramisco JD, Peltomaki P, Lang JC, Schuller DE, Yu L, Bloomfield CD, Caligiuri MA, Yates A, Nishikawa R, Su HH, Petrelli NJ, Zhang X, O'Dorisio MS, Held WA, Cavenee WK, Plass C (2000) Aberrant CpG-island methylation has non-random and tumour-type-specific patterns. Nat Genet 24:132–138
Costello JF, Plass C, Cavenee WK (2000) Aberrant methylation of genes in low-grade astrocytomas. Brain Tumor Pathol 17:49–56
Duda RO, Hart PE, Stork DG (2001) Pattern Classification. Wiley, New York
Easton T, Hooker K, Lee EK (2003) Facets of the independent set plytope. Math Program Ser B 98:177–199
Erenguc SS, Koehler GJ (1990) Survey of mathematical programming models and experimental results for linear discriminant analysis. Managerial Decis Econ 11:215–225
Feltus FA, Lee EK, Costello JF, Plass C, Vertino PM (2003) Predicting aberrant CpG island methylation. Proc Natl Acad Sci USA 100:12253–12258
Feltus FA, Lee EK, Costello JF, Plass C, Vertino PM (2006) DNA signatures associated with CpG island methylation states. Genomics 87:572–579
Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Eugenics 7:179–188
Freed N, Glover F (1981) A linear programming approach to the discriminant problem. Decis Sci 12:68–74
Freed N, Glover F (1981) Simple but powerful goal programming models for discriminant problems. Eur J Oper Res 7:44–60
Freed N, Glover F (1986) Evaluating alternative linear programming models to solve the two-group discriminant problem. Decis Sci 17:151–162
Freed N, Glover F (1986) Resolving certain difficulties and improving the classification power of LP discriminant analysis formulations. Decis Sci 17:589–595
Fruhwald MC, O'Dorisio MS, Rush LJ, Reiter JL, Smiraglia DJ, Wenger G, Costello JF, White PS, Krahe R, Brodeur GM, Plass C (2000) Gene amplification in NETs/medulloblastomas: mapping of a novel amplified gene within the MYCN amplicon. J Med Genet 37:501–509
Fung GM, Mangasarian OL (2001) Proximal support vector machine classifiers. In Proceedings KDD-2001, San Francisco
Fung GM, Mangasarian OL (2002) Incremental support vector machine classification. In: Grossman R, Mannila H, Motwani R (eds) Proceedings of the Second SIAM International Conference on Data Mining. SIAM, Philadelphia, pp 247–260
Fung GM, Mangasarian OL (2005) Multicategory proximal support vector machine classifiers. Mach Learn 59:77–97
Gallagher RJ, Lee EK, Patterson DA (1996) An optimization model for constrained discriminant analysis and numerical experiments with iris, thyroid, and heart disease datasets. In: Proceedings of the 1996 American Medical Informatics Association
Gallagher RJ, Lee EK, Patterson DA (1997) Constrained discriminant analysis via 0/1 mixed integer programming. Ann Oper Res 74:65–88
Gehrlein WV (1986) General mathematical programming formulations for the statistical classification problem. Oper Res Lett 5(6):299–304
Glen JJ (1999) Integer programming methods for normalisation and variable selection in mathematical programming discriminant analysis models. J Oper Res Soc 50:1043–1053
Glen JJ (2004) Dichotomous categorical variable formation in mathematical programming discriminant analysis models. Naval Res Logist 51:575–596
Glover F (1990) Improved linear programming models for discriminant analysis. Decis Sci 21:771–785
Glover F, Keene S, Duea B (1988) A new class of models for the discriminant problem. Decis Sci 19:269–280
Gochet W, Stam A, Srinivasan V, Chen S (1997) Multigroup discriminant analysis using linear programming. Oper Res 45(2):213–225
Hand DJ (1981) Discrimination and classification. Wiley, New York
Horton P, Nakai K (1996) A probablistic classification system for predicting the cellular localization sites of proteins. In: Proceedings of the Fourth International Conference on Intelligent Systems for Molecular Biology, St. Louis, USA, pp 109–115
Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Networks 13(2):415–425
Joachimsthaler EA, Stam A (1990) Mathematical programming approaches for the classification problem in two-group discriminant analysis. Multivariate Behavior Res 25(4):427–454
Koehler GJ (1989) Characterization of unacceptable solutions in LP discriminant analysis. Decis Sci 20:239–257
Koehler GJ (1989) Unacceptable solutions and the hybrid discriminant model. Decis Sci 20:844–848
Koehler GJ (1994) A response to Xiao's “necessary and sufficient conditions of unacceptable solutions in LP discriminant analysls”: Something is amiss. Decis Sci 25:331–333
Koehler GJ, Erenguc SS (1990) Minimizing misclassifications in linear discriminant analysis. Decis Sci 21:63–85
Lee EK (1993) Solving a truck dispatching scheduling problem using branch-and-cut. PhD thesis, Computational and Applied Mathematics, Rice University, Houston
Lee EK, Fung AYC, Brooks JP, Zaider M (2002) Automated planning volume definition in soft-tissue sarcoma adjuvant brachytherapy. Biol Phys Med 47:1891–1910
Lee EK, Gallagher RJ, Campbell AM, Prausnitz MR (2004) Prediction of ultrasound-mediated disruption of cell membranes using machine learning techniques and statistial analysis of acoustic spectra. IEEE Trans Biomed Eng 51:1–9
Lee EK, Maheshwary S (2006) Conflict hypergraphs in integer programming. Technical report, Georgia Institute of Technology, submitted
Lee EK (2007) Optimization-based predictive models in medicine and biology. Optimization in Medicine. Springer Netherlands. Springer Series in Optimization and Its Application 12:127–151
Lee EK (2007) Large-scale optimization-based classification models in medicine and biology. Ann Biomed Eng Syst Biol Bioinformat 35(6):1095–1109
Lee EK, Easton T, Gupta K (2006) Novel evolutionary models and applications to sequence alignment problems. Ann Oper Res Oper Res Medic – Comput Optim Medic Life Sci 148:167–187
Lee EK, Fung AYC, Zaider M (2001) Automated planning volume contouring in soft-tissue sarcoma adjuvant brachytherapy treatment. Int J Radiat Oncol Biol Phys 51:391
Lee EK, Gallagher RJ, Patterson DA (2003) A linear programming approach to discriminant analysis with a reserved-judgment region. INFORMS J Comput 15(1):23–41
Lee EK, Jagannathan S, Johnson C, Galis ZS (2006) Fingerprinting native and angiogenic microvascular networks through pattern recognition and discriminant analysis of functional perfusion data. submitted
Lee EK, Wu TL, Ashfaq S, Jones DP, Rhodes SD, Weintrau WS, Hopper CH, Vaccarino V, Harrison DG, Quyyumi AA (2007) Prediction of early atherosclerosis in healthy adults via novel markers of oxidative stress and d-ROMs. Working paper
Lee Y, Lin Y, Wahba G (2004) Multicategory support vector machines: Theory and application to the classification of microarray data and satellite radiance data. J Am Stat Assoc 99:67–81
Lee YJ, Mangasarian OL (2001) RSVM: Reduced support vector machines. In: Proceedings of the SIAM International Conference on Data Mining, Chicago, April 5–7
Lee YJ, Mangasarian OL (2001) SSVM: A smooth support vector machine for classification. Comput Optim Appl 20(1):5–22
Lee YJ, Mangasarian OL, Wolberg WH (2000) Breast cancer survival and chemotherapy: A support vector machine analysis. In: DIMACS Series in Discrete Mathematical and Theoretical Computer Science, vol 55. American Mathematical Society, Providence, pp 1–10
Lee YJ, Mangasarian OL, Wolberg WH (2003) Survival-time classification of breast cancer patients. Comput Optim Appl 25:151–166
Loucopoulos C, Pavur R (1997) Computational characteristics of a new mathematical programming model for the three-group discriminant problem. Comput Oper Res 24(2):179–191
Loucopoulos C, Pavur R (1997) Experimental evaluation of the classificatory performance of mathematical programming approaches to the three-group discriminant problem: The case of small samples. Ann Oper Res 74:191–209
Luedi PP, Hartemink AJ, Jirtle RL (2005) Genome-wide prediction of imprinted murine genes. Genome Res 15:875–884
Mangasarian OL (1965) Linear and nonlinear separation of patterns by linear programming. Oper Res 13:444–452
Mangasarian OL (1968) Multi-surface method of pattern separation. IEEE Trans Inform Theory 14(6):801–807
Mangasarian OL (1994) Misclassification minimization. J Global Optim 5:309–323
Mangasarian OL (1996) Machine learning via polyhedral concave minimization. In: Fischer H, Riedmueller B, Schaeffler S (eds) Applied Mathematics and Parallel computing – Festschrift for Klaus Ritter. Physica-Verlag, Heidelberg, pp 175–188
Mangasarian OL (1999) Arbitrary-norm separating plane. Oper Res Lett 24:15–23
Mangasarian OL (2000) Generalized support vector machines. In: Smola AJ, Bartlett P, Schökopf B, Schuurmans D (eds) Advances in Large Margin Classifiers. MIT Press, Cambridge, pp 135–146
Mangasarian OL (2003) Data mining via support vector machines. In: Sachs EW, Tichatschke R (eds) System Modeling and Optimization XX. Kluwer, Boston, pp 91–112
Mangasarian OL (2005) Support vector machine classification via parameterless robust linear programming. Optim Methods Softw 20:115–125
Mangasarian OL, Musicant DR (1999) Successive overrelaxation for support vector machines. IEEE Trans Neural Networks 10:1032–1037
Mangasarian OL, Musicant DR (2001) Data discrimination via nonlinear generalized support vector machines. In: Ferris MC, Mangasarian OL, Pang JS (eds) Complementarity: Applications, Algorithms and Extensions. Kluwer, Boston, pp 233–251
Mangasarian OL, Musicant DR (2001) Lagrangian support vector machines. J Mach Learn Res 1:161–177
Mangasarian OL, Setiono R, Wolberg WH (1990) Pattern recognition via linear programming: Theory and application to medical diagnosis. In: Coleman TF, Li Y (eds) Large-Scale Numerical Optimization. SIAM, Philadelphia, pp 22–31
Mangasarian OL, Street WN, Wolberg WH (1995) Breast cancer diagnosis and prognosis via linear programming. Oper Res 43(4):570–577
Markowski EP, Markowski CA (1985) Some difficulties and improvements in applying linear programming formulations to the discriminant problem. Decis Sci 16:237–247
McCord JM (2000) The evolution of free radicals and oxidative stress. Am J Med 108:652–659
McLachlan GJ (1992) Discriminant analysis and statistical pattern recognition. Wiley, New York
Müller KR, Mika S, Rätsch G, Tsuda K, Schölkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Networks 12(2):181–201
Murphy PM, Aha DW (1994) UCI Repository of machine learning databases http://www.ics.uci.edu/~mlearn/MLRepository.html. Department of Information and Computer Science, University of California, Irvine
O'Hagan A (1994) Kendall's Advanced Theory of Statistics: Bayesian Inference, vol 2B. Halsted Press, New York
Pavur R (1997) Dimensionality representation of linear discriminant function space for the multiple-group problem: An MIP approach. Ann Oper Res 74:37–50
Pavur R (2002) A comparative study of the effect of the position of outliers on classical and nontraditional approaches to the two-group classification problem. Eur J Oper Res 136:603–615
Pavur R, Loucopoulos C (2001) Evaluating the effect of gap size in a single function mathematical programming model for the three-group classification problem. J Oper Res Soc 52:896–904
Pavur R, Wanarat P, Loucopoulos C (1997) Examination of the classificatory performance of MIP models with secondary goals for the two-group discriminant problem. Ann Oper Res 74:173–189
Raz A, Ben-Zéev A (1987) Cell contact and architecture of malignant cells and their relationship to metastasis. Cancer Metastasis Rev 6:3–21
Rencher AC (1998) Multivariate Statistical Inference and Application. Wiley, New York
Rubin PA (1990) A comparison of linear programming and parametric approaches to the two-group discriminant problem. Decis Sci 21:373–386
Rubin PA (1991) Separation failure in linear programming discriminant models. Decis Sci 22:519–535
Rubin PA (1997) Solving mixed integer classification problems by decomposition. Ann Oper Res 74:51–64
Rush LJ, Dai Z, Smiraglia DJ, Gao X, Wright FA, Fruhwald M, Costello JF, Held WA, Yu L, Krahe R, Kolitz JE, Bloomfield CD, Caligiuri MA, Plass C (2001) Novel methylation targets in de novo acute myeloid leukemia with prevalence of chromosome 11 loci. Blood 97:3226–3233
Sies H (1985) Oxidative stress: introductory comments. In: Sies H (ed) Oxidative Stress. Academic Press, London, pp 1–8
Duarte Silva AP, Stam A (1994) Second order mathematical programming formulations for discriminant analysis. Eur J Oper Res 72:4–22
Duarte Silva AP, Stam A (1997) A mixed integer programming algorithm for minimizing the training sample misclassification cost in two-group classification. Ann Oper Res 74:129–157
Smith CAB (1947) Some examples of discrimination. Ann Eugenics 13:272–282
Stam A (1997) Nontraditional approaches to statistical classification: Some perspectives on l p -norm methods. Ann Oper Res 74:1–36
Stam A, Joachimsthaler EA (1989) Solving the classification problem in discriminant analysis via linear and nonlinear programming methods. Decis Sci 20:285–293
Stam A, Joachimsthaler EA (1990) A comparison of a robust mixed-integer approach to existing methods for establishing classification rules for the discriminant problem. Eur J Oper Res 46:113–122
Stam A, Ungar DR (1995) RAGNU: A microcomputer package for two-group mathematical programming‐based nonparametric classification. Eur J Oper Res 86:374–388
Tahara S, Matsuo M, Kaneko T (2001) Age-related changes in oxidative damage to lipids and DNA in rat skin. Mechan Ageing Develop 122:415–426
Vapnik V (1995) The Nature of Statistical Learning Theory. Springer, New York
Wanarat P, Pavur R (1996) Examining the effect of second-order terms in mathematical programming approaches to the classification problem. Eur J Oper Res 93:582–601
Xiao B (1993) Necessary and sufficient conditions of unacceptable solutions in LP discriminant analysis. Decis Sci 24:699–712
Xiao B (1994) Decision power and solutions of LP discriminant models: Rejoinder. Decis Sci 25:335–336
Xiao B, Feng Y (1997) Alternative discriminant vectors in LP models and a regularization method. Ann Oper Res 74:113–127
Yan PS, Chen CM, Shi H, Rahmatpanah F, Wei SH, Caldwell CW, Huang TH (2001) Dissecting complex epigenetic alterations in breast cancer using CpG island microarrays. Cancer Res 61:8375–8380
Yan PS, Perry MR, Laux DE, Asare AL, Caldwell CW, Huang TH (2000) CpG island arrays: an application toward deciphering epigenetic signatures of breast cancer. Clin Cancer Res 6:1432–1438
Yanev N, Balev S (1999) A combinatorial approach to the classification problem. Eur J Oper Res 115:339–350
Zimmermann A, Keller HU (1987) Locomotion of tumor cells as an element of invasion and metastasis. Biomed Pharmacotherapy 41:337–344
Zopounidis C, Doumpos M (2002) Multicriteria classification and sorting methods: A literature review. Eur J Oper Res 138:229–246
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag
About this entry
Cite this entry
K. Lee, E., Wu, TL. (2008). Disease Diagnosis: Optimization-Based Methods . In: Floudas, C., Pardalos, P. (eds) Encyclopedia of Optimization. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-74759-0_133
Download citation
DOI: https://doi.org/10.1007/978-0-387-74759-0_133
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-74758-3
Online ISBN: 978-0-387-74759-0
eBook Packages: Mathematics and StatisticsReference Module Computer Science and Engineering