Skip to main content
SpringerLink
Log in

DRCW-FRkNN-OVO: distance-based related competence weighting based on fixed radius k nearest neighbour for one-vs-one scheme

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The one-versus-one (OVO) binarization decomposition scheme is considered as one of the most effective techniques to deal with multi-class classification problems. Its inherent mechanism is to use the “divide-and-conquer” strategy to decompose the multi-class classification problem into as many pairs of easier-to-solve binary sub-problems as possible. One common issue in the OVO scheme is that of non-competent classifiers. In this study, we proposed a novel OVO scheme strategy, named DRCW-FRkNN-OVO, to reduce the negative effect of non-competent classifiers. Specifically, we focused on the definition of region of competence, which plays a crucial role in managing the non-competent classifiers. To overcome the issue of skew and sparse distribution during the management of non-competent classifiers, we developed a relative competence weighting combination method via the fixed radius nearest neighbour search to find the local region within each class for the query sample. Our proposed DRCW-FRkNN-OVO is tested on 30 real-world multi-class datasets compared with several well-known related works. Experimental results supported by thorough statistical analysis confirmed the effectiveness and robustness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. http://archive.ics.uci.edu/ml/datasets.php.

References

  1. Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66

    Google Scholar 

  2. AL-Sharuee MT, Liu F, Pratama M (2018) Sentiment analysis: an automatic contextual analysis and ensemble clustering approach and comparison. Data Knowl Eng 115:194–213

    Article  Google Scholar 

  3. Alcalá-Fdez J, Sanchez L, Garcia S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM et al (2009) KEEL: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13(3):307–318

    Article  Google Scholar 

  4. Allwein EL, Schapire RE, Singer Y (2000) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141

    MathSciNet  MATH  Google Scholar 

  5. Alqudah AM (2020) AOCT-NET: a convolutional network automated classification of multiclass retinal diseases using spectral-domain optical coherence tomography images. Med Biol Eng Comput 58(1):41–53

    Article  MathSciNet  Google Scholar 

  6. Bouazizi M, Ohtsuki T (2019) Multi-class sentiment analysis on twitter: classification performance and challenges. Big Data Min Anal 2(3):181–194

    Article  Google Scholar 

  7. Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and regression trees. CRC Press, Boca Raton

    MATH  Google Scholar 

  8. Cerf L, Gay D, Selmaoui-Folcher N, Crémilleux B, Boulicaut JF (2013) Parameter-free classification in multi-class imbalanced data sets. Data Knowl Eng 87:109–129

    Article  Google Scholar 

  9. Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):1–27

    Article  Google Scholar 

  10. Clark P, Boswell R (1991) Rule induction with CN2: some recent improvements. European working session on learning. Springer, Berlin, pp 151–163

    Google Scholar 

  11. Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46

    Article  Google Scholar 

  12. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  13. Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18

    Article  Google Scholar 

  14. Dietterich TG, Bakiri G (1994) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286

    Article  Google Scholar 

  15. Don DR, Iacob IE (2020) DCSVM: fast multi-class classifcation using support vector machines. Int J Mach Learn Cybern 11:433–447

    Article  Google Scholar 

  16. Fernandes ER, de Carvalho AC (2019) Evolutionary inversion of class distribution in overlapping areas for multi-class imbalanced learning. Inf Sci 494:141–154

    Article  Google Scholar 

  17. Fernández A, Calderón M, Barrenechea E, Bustince H, Herrera F (2010) Solving multi-class problems with linguistic fuzzy rule based classification systems based on pairwise learning and preference relations. Fuzzy Sets Syst 161(23):3064–3080

    Article  MathSciNet  Google Scholar 

  18. Friedman J (1996) Another approach to polychotomous classification. Department of Statistics, Stanford University, Tech. rep

  19. Galar M, Fernández A, Barrenechea E, Bustince H, Herrera F (2011) An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Pattern Recogn 44(8):1761–1776

    Article  Google Scholar 

  20. Galar M, Fernández A, Barrenechea E, Bustince H, Herrera F (2013) Dynamic classifier selection for one-vs-one strategy: avoiding non-competent classifiers. Pattern Recogn 46(12):3412–3424

    Article  Google Scholar 

  21. Galar M, Fernández A, Barrenechea E, Herrera F (2015) DRCW-OVO: distance-based relative competence weighting combination for one-vs-one strategy in multi-class problems. Pattern Recogn 48(1):28–42

    Article  Google Scholar 

  22. García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180(10):2044–2064

    Article  Google Scholar 

  23. García S, Zhang ZL, Altalhi A, Alshomrani S, Herrera F (2018) Dynamic ensemble selection for multi-class imbalanced datasets. Inf Sci 445:22–37

    Article  MathSciNet  Google Scholar 

  24. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. ACM SIGKDD Explor Newsl 11(1):10–18

    Article  Google Scholar 

  25. Hühn JC, Hüllermeier E (2008) FR3: a fuzzy rule learner for inducing reliable classifiers. IEEE Trans Fuzzy Syst 17(1):138–149

    Article  Google Scholar 

  26. Hüllermeier E, Brinker K (2008) Learning valued preference structures for solving classification problems. Fuzzy Sets Syst. 159(18):2337–2352

    Article  MathSciNet  Google Scholar 

  27. Hüllermeier E, Vanderlooy S (2010) Combining predictions in pairwise classification: an optimal adaptive voting strategy and its relation to weighted voting. Pattern Recogn 43(1):128–142

    Article  Google Scholar 

  28. Islam MM, Kim JM (2019) Reliable multiple combined fault diagnosis of bearings using heterogeneous feature models and multiclass support vector machines. Reliab Eng Syst Saf 184:55–66

    Article  Google Scholar 

  29. Karagoz GN, Yazici A, Dokeroglu T, Cosar A (2021) A new framework of multi-objective evolutionary algorithms for feature selection and multi-label classification of video data. Int J Mach Learn Cybern 12(1):53–71

    Article  Google Scholar 

  30. Knerr S, Personnaz L, Dreyfus G (1990) Single-layer learning revisited: a stepwise procedure for building and training a neural network. Neurocomputing: Algorithms. Archit Appl 68:41–50

    Google Scholar 

  31. Liu Y, Bi JW, Fan ZP (2017) Multi-class sentiment classification: the experimental comparisons of feature selection and machine learning algorithms. Expert Syst Appl 80:323–339

    Article  Google Scholar 

  32. Nayak DR, Dash R, Majhi B (2020) Automated diagnosis of multi-class brain abnormalities using mri images: a deep convolutional neural network based method. Pattern Recogn Lett 138:385–391

    Article  Google Scholar 

  33. Orlovsky S (1978) Decision-making with a fuzzy preference relation. Fuzzy Sets Syst 1(3):155–167

    Article  MathSciNet  Google Scholar 

  34. Pang Y, He Q, Jiang G, Xie P (2020) Spatio-temporal fusion neural network for multi-class fault diagnosis of wind turbines based on scada data. Renew Energy 161:510–524

    Article  Google Scholar 

  35. Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv Large Margin Classif 10(3):61–74

    Google Scholar 

  36. Ramzan F, Khan MUG, Rehmat A, Iqbal S, Saba T, Rehman A, Mehmood Z (2020) A deep learning approach for automated diagnosis and multi-class classification of Alzheimer’s disease stages using resting-state fmri and residual neural networks. J Med Syst 44(2):37

    Article  Google Scholar 

  37. Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  38. Villacampa-Calvo C, Zaldivar B, Garrido-Merchán EC, Hernández-Lobato D (2020) Multi-class gaussian process classification with noisy inputs

  39. Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83

    Article  Google Scholar 

  40. Wilson DR, Martinez TR (1997) Improved heterogeneous distance functions. J Artif Intell Res 6(1):1–34

    MathSciNet  MATH  Google Scholar 

  41. Wu S, Xia X, Liu T, Han B, Gong M, Wang N, Liu H, Niu G (2020) Multi-class classification from noisy-similarity-labeled data. arXiv preprint arXiv:200206508

  42. Xu F, Pan Z, Xia R (2020) E-commerce product review sentiment classification based on a na\(\ddot{i}\)ve bayes continuous learning framework. Inf Process Manag 57(5):102221

    Article  Google Scholar 

  43. Zhang X, Ding S, Sun T (2016) Multi-class LSTMSVM based on optimal directed acyclic graph and shuffled frog leaping algorithm. Int J Mach Learn Cybern 7:241–251

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the (anonymous) reviewers for their constructive comments. This research was supported by Zhejiang Provincial Natural Science Foundation of China under Grant No. LZ20G010001, the National Science Foundation of China under Grant Nos. 71801065, 71831006, 71771070, and 71932005, as well as the Hangzhou 2019 Philosophy and Social Science Planning Project of China under Grant No. Z19JC112.

Author information

Authors and Affiliations

  1. School of Management, Hangzhou Dianzi University, Hangzhou, 310018, China

    Zhong-Liang Zhang, Xing-Gang Luo & Qing Zhou

Authors
  1. Zhong-Liang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xing-Gang Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qing Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xing-Gang Luo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, ZL., Luo, XG. & Zhou, Q. DRCW-FRkNN-OVO: distance-based related competence weighting based on fixed radius k nearest neighbour for one-vs-one scheme. Int. J. Mach. Learn. & Cyber. 13, 1441–1459 (2022). https://doi.org/10.1007/s13042-021-01458-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01458-7

Keywords

Search

Navigation