Skip to main content
SpringerLink
Log in
Book cover

Conference of the Spanish Association for Artificial Intelligence

CAEPIA 2021: Advances in Artificial Intelligence pp 54–64Cite as

STree: A Single Multi-class Oblique Decision Tree Based on Support Vector Machines

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12882))

Abstract

We propose a new oblique decision tree algorithm based on support vector machines. Our algorithm produces a single model for a multi-class target variable. On the contrary to previous works that manage the multi-class problem by using clustering at each split, we test all the one-vs-rest labels at each split, choosing the one which minimizes an impurity measure. The experimental evaluation carried out over 49 datasets shows that our algorithm is ranked before those used for comparison, and significantly outperforms all of them when the SVM hyperparameters are carefully tuned.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    In [9] and [14] ensemble methods are proposed. In this paper we compare against the proposed base ODT classifiers.

  2. 2.

    The code can be found in https://git.io/J3jkQ.

References

  1. Arias, J., Cózar, J.: ExReport: fast, reliable and elegant reproducible research (2016). https://cran.r-project.org/web/packages/exreport/index.html. Accessed 04 Aug 2021

  2. Bennett, K.P., Blue, J.A.: A support vector machine approach to decision trees. In: 1998 IEEE International Joint Conference on Neural Networks Proceedings, vol. 3, pp. 2396–2401 (1998)

    Google Scholar 

  3. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)

    MATH  Google Scholar 

  4. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, COLT 1992, pp. 144–152 (1992)

    Google Scholar 

  5. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth and Brooks, Monterey (1984)

    MATH  Google Scholar 

  6. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). https://doi.org/10.1023/A:1010933404324

    Article  MATH  Google Scholar 

  7. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

  8. Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml

  9. Ganaie, M., Tanveer, M., Suganthan, P.: Oblique decision tree ensemble via twin bounded SVM. Expert Syst. Appl. 143, 113072 (2020)

    Article  Google Scholar 

  10. García, S., Herrera, F.: An extension on “statistical comparisons of classifiers over multiple data sets’’ for all pairwise comparisons. J. Mach. Learn. Res. 9, 2677–2694 (2008)

    MATH  Google Scholar 

  11. Kontschieder, P., Fiterau, M., Criminisi, A., Bulò, S.R.: Deep neural decision forests (2015)

    Google Scholar 

  12. Kotsiantis, S.B.: Decision trees: a recent overview. Artif. Intell. Rev. 39(4), 261–283 (2013). https://doi.org/10.1007/s10462-011-9272-4

    Article  Google Scholar 

  13. Manwani, N., Sastry, P.S.: Geometric decision tree. IEEE Trans. Syst. Man Cybern. Part B 42(1), 181–192 (2012)

    Article  Google Scholar 

  14. Menkovski, V., Christou, I.T., Efremidis, S.: Oblique decision trees using embedded support vector machines in classifier ensembles (2008)

    Google Scholar 

  15. Murthy, S.K., Kasif, S., Salzberg, S.: A system for induction of oblique decision trees. J. Artif. Intell. Res. 2(1), 1–32 (1994)

    MATH  Google Scholar 

  16. Pelleg, D., Moore, A.W.: X-means: extending k-means with efficient estimation of the number of clusters. In: Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000), pp. 727–734 (2000)

    Google Scholar 

  17. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco (1993)

    Google Scholar 

  18. Rivera-Lopez, R., Canul-Reich, J., Gámez, J.A., Puerta, J.M.: OC1-DE: a differential evolution based approach for inducing oblique decision trees. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2017. LNCS (LNAI), vol. 10245, pp. 427–438. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59063-9_38

    Chapter  Google Scholar 

  19. Shotton, J., et al.: Efficient human pose estimation from single depth images. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2821–2840 (2013)

    Article  Google Scholar 

  20. Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995). https://doi.org/10.1007/978-1-4757-2440-0

    Book  MATH  Google Scholar 

  21. Wu, X., et al.: Top 10 algorithms in data mining. Knowl. Inf. Syst. 14(1), 1–37 (2007). https://doi.org/10.1007/s10115-007-0114-2

    Article  Google Scholar 

  22. Yang, B.B., Shen, S.Q., Gao, W.: Weighted oblique decision trees. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 5621–5627 (2019)

    Google Scholar 

  23. Zhang, L., Suganthan, P.N.: Oblique decision tree ensemble via multisurface proximal support vector machine. IEEE Trans. Cybern. 45(10), 2165–2176 (2015)

    Article  Google Scholar 

  24. Zhang, L., Zhou, W., Su, T., Jiao, L.: Decision tree support vector machine. Int. J. Artif. Intell. Tools 16(1), 1–16 (2007)

    Article  Google Scholar 

Download references

Acknowledgements

We are indebted to the authors of [9, 14] and [22] because of providing us with the code of their implementations. This work has been partially funded by FEDER funds, the JCCM Government and the Spanish Goverment through the projects SBPLY/17/180501/ 000493 and PID2019-106758GB-C33.

Author information

Authors and Affiliations

  1. Computing Systems Department, University of Castilla-La Mancha, Albacete, Spain

    Ricardo Montañana, Jose A. Gámez & Jose M. Puerta

Authors
  1. Ricardo Montañana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jose A. Gámez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jose M. Puerta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ricardo Montañana .

Editor information

Editors and Affiliations

  1. University of Malaga, Málaga, Spain

    Enrique Alba

  2. University of Malaga, Málaga, Spain

    Gabriel Luque

  3. University of Malaga, Málaga, Spain

    Francisco Chicano

  4. University of Malaga, Málaga, Spain

    Carlos Cotta

  5. Technical University of Madrid, Madrid, Spain

    David Camacho

  6. University of Malaga, Málaga, Spain

    Manuel Ojeda-Aciego

  7. University of Oviedo, Oviedo, Spain

    Susana Montes

  8. Pablo de Olavide University, Seville, Spain

    Alicia Troncoso

  9. University of Seville, Seville, Spain

    José Riquelme

  10. University of Malaga, Málaga, Spain

    Rodrigo Gil-Merino

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Montañana, R., Gámez, J.A., Puerta, J.M. (2021). STree: A Single Multi-class Oblique Decision Tree Based on Support Vector Machines. In: Alba, E., et al. Advances in Artificial Intelligence. CAEPIA 2021. Lecture Notes in Computer Science(), vol 12882. Springer, Cham. https://doi.org/10.1007/978-3-030-85713-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85713-4_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85712-7

  • Online ISBN: 978-3-030-85713-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation