Skip to main content
SpringerLink
Log in

Hyper-box Classification Model Using Mathematical Programming

  • Conference paper
  • First Online:
Learning and Intelligent Optimization (LION 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14286))

Included in the following conference series:

  • 544 Accesses

Abstract

Classification constitutes focal topic of study within the machine learning research community. Interpretable machine learning algorithms have been gaining ground against black box models because people want to understand the decision-making process. Mathematical programming based classifiers have received attention because they can compete with state-of-the-art algorithms in terms of accuracy and interpretability. This work introduces a single-level hyper-box classification approach, which is formulated mathematically as Mixed Integer Linear Programming model. Its objective is to identify the patterns of the dataset using a hyper-box representation. Hyper-boxes enclose as many samples of the corresponding class as possible. At the same time, they are not allowed to overlap with hyper-boxes of different class. The interpretability of the approach stems from the fact that IF-THEN rules can easily be generated. Towards the evaluation of the performance of the proposed method, its prediction accuracy is compared to other state-of-the-art interpretable approaches in a number of real-world datasets. The results provide evidence that the algorithm can compare favourably against well-known counterparts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aghaei, S., Gomez, A., Vayanos, P.: Learning optimal classification trees: strong max-flow formulations (2020). https://doi.org/10.48550/arXiv.2002.09142

  2. Bertsimas, D., Dunn, J.: Optimal classification trees. Mach. Learn. 106, 1039–1082 (2017). https://doi.org/10.1007/s10994-017-5633-9

    Article  MathSciNet  MATH  Google Scholar 

  3. Bixby, R.E.: A brief history of linear and mixed-integer programming computation. Doc. Math. 1, 107–121 (2012)

    MathSciNet  MATH  Google Scholar 

  4. Blanco, V., Japón, A., Puerto, J.: Optimal arrangements of hyperplanes for SVM-based multiclass classification. Adv. Data Anal. Classif. 14, 175–199 (2020). https://doi.org/10.1007/s11634-019-00367-6

    Article  MathSciNet  MATH  Google Scholar 

  5. Blanco, V., Japón, A., Puerto, J.: A mathematical programming approach to SVM-based classification with label noise. Comput. Ind. Eng. 172, 108611 (2022). https://doi.org/10.1016/j.cie.2022.108611

    Article  Google Scholar 

  6. Blanquero, R., Carrizosa, E., Molero-Río, C., Morales, D.R.: Sparsity in optimal randomized classification trees. Eur. J. Oper. Res. 284, 255–272 (2020). https://doi.org/10.1016/j.ejor.2019.12.002

    Article  MathSciNet  MATH  Google Scholar 

  7. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Taylor & Francis, Milton Park (1984). https://doi.org/10.1201/9781315139470

  8. Busygin, S., Prokopyev, O.A., Pardalos, P.M.: An optimization-based approach for data classification. Optim. Meth. Softw. 22, 3–9 (2007). https://doi.org/10.1080/10556780600881639

    Article  MathSciNet  MATH  Google Scholar 

  9. Carrizosa, E., Morales, D.R.: Supervised classification and mathematical optimization. Comput. Oper. Res. 40, 150–165 (2013). https://doi.org/10.1016/j.cor.2012.05.015

    Article  MathSciNet  MATH  Google Scholar 

  10. Dua, D., Graff, C.: UCI machine learning repository. https://archive.ics.uci.edu/ml/index.php (2017)

  11. GAMS Development Corporation: General Algebraic Model System (GAMS) (2022). Release 41.5.0, Washington, DC, USA

    Google Scholar 

  12. Gehrlein, W.V.: General mathematical programming formulations for the statistical classification problem. Oper. Res. Lett. 5, 299–304 (1986). https://doi.org/10.1016/0167-6377(86)90068-4

  13. Gurobi Optimization, LLC: Gurobi Optimizer Reference Manual (2023). https://www.gurobi.com

  14. Interpretable AI, LLC: Interpretable AI Documentation (2023). https://www.interpretable.ai

  15. Maskooki, A.: Improving the efficiency of a mixed integer linear programming based approach for multi-class classification problem. Comput. Ind. Eng. 66, 383–388 (2013). https://doi.org/10.1016/j.cie.2013.07.005

    Article  Google Scholar 

  16. Müller, T.T., Lio, P.: Peclides neuro: a personalisable clinical decision support system for neurological diseases. Front. Artif. Intell. 3, 23 (2020). https://doi.org/10.3389/frai.2020.00023

    Article  Google Scholar 

  17. Nasseri, A.A., Tucker, A., Cesare, S.D.: Quantifying stockTwits semantic terms’ trading behavior in financial markets: an effective application of decision tree algorithms. Expert Syst. Appl. 42, 9192–9210 (2015). https://doi.org/10.1016/j.eswa.2015.08.008

    Article  Google Scholar 

  18. Papageorgiou, L.G., Rotstein, G.E.: Continuous-domain mathematical models for optimal process plant layout. Ind. Eng. Chem. Res. 37, 3631–3639 (1998). https://doi.org/10.1021/ie980146v

    Article  Google Scholar 

  19. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011). https://doi.org/10.48550/arXiv.1201.0490

  20. Quinlan, J.R.: Improved use of continuous attributes in c4.5. J. Artif. Intell. Res. 4, 77–90 (1996). https://doi.org/10.1613/jair.279

  21. Simpson, P.: Fuzzy min-max neural networks. I. classification. IEEE Transa. Neural Netw. 3, 776–786 (1992). https://doi.org/10.1109/72.159066. https://ieeexplore.ieee.org/document/159066/

  22. Sueyoshi, T.: Mixed integer programming approach of extended DEA-discriminant analysis. Eur. J. Oper. Res. 152, 45–55 (2004). https://doi.org/10.1016/S0377-2217(02)00657-4

    Article  MathSciNet  MATH  Google Scholar 

  23. Verwer, S., Zhang, Y.: Learning optimal classification trees using a binary linear program formulation. In: 33rd Conference on Artificial Intelligence (2019). https://doi.org/10.1609/aaai.v33i01.33011624

  24. Xu, G., Papageorgiou, L.G.: A mixed integer optimisation model for data classification. Comput. Ind. Eng. 56, 1205–1215 (2009). https://doi.org/10.1016/j.cie.2008.07.012

    Article  Google Scholar 

  25. Yang, L., Liu, S., Tsoka, S., Papageorgiou, L.G.: Sample re-weighting hyper box classifier for multi-class data classification. Comput. Ind. Eng. 85, 44–56 (2015). https://doi.org/10.1016/j.cie.2015.02.022

    Article  Google Scholar 

  26. Yoo, I., et al.: Data mining in healthcare and biomedicine: a survey of the literature. J. Med. Syst. 36, 2431–2448 (2012). https://doi.org/10.1007/s10916-011-9710-5

    Article  Google Scholar 

  27. Zibanezhad, E., Foroghi, D., Monadjemi, A.: Applying decision tree to predict bankruptcy. In: IEEE International Conference on Computer Science and Automation Engineering, vol. 4, pp. 165–169 (2011). https://doi.org/10.1109/CSAE.2011.5952826

  28. Üney, F., Türkay, M.: A mixed-integer programming approach to multi-class data classification problem. Eur. J. Oper. Res. 173, 910–920 (2006). https://doi.org/10.1016/j.ejor.2005.04.049

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

Authors gratefully acknowledge the financial support from Engineering and Physical Sciences Research Council (EPSRC) under the project EP/V051008/1.

Author information

Authors and Affiliations

  1. The Sargent Centre for Process Systems Engineering, Department of Chemical Engineering, UCL (University College London), Torrington Place, London, WC1E 7JE, UK

    Georgios I. Liapis & Lazaros G. Papageorgiou

Authors
  1. Georgios I. Liapis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lazaros G. Papageorgiou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lazaros G. Papageorgiou .

Editor information

Editors and Affiliations

  1. InsideOpt, Dover, DE, USA

    Meinolf Sellmann

  2. Bielefeld University, Bielefeld, Germany

    Kevin Tierney

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liapis, G.I., Papageorgiou, L.G. (2023). Hyper-box Classification Model Using Mathematical Programming. In: Sellmann, M., Tierney, K. (eds) Learning and Intelligent Optimization. LION 2023. Lecture Notes in Computer Science, vol 14286. Springer, Cham. https://doi.org/10.1007/978-3-031-44505-7_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-44505-7_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-44504-0

  • Online ISBN: 978-3-031-44505-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation