Abstract
In supervised learning, the underlying skewed distribution of multiple classes poses extreme difficulties for learning good models. A common scheme to deal with the multi-class imbalanced problem is to decompose an original dataset into several binary-class subsets and incorporate some imbalanced learning techniques. This paper presents our empirical study on the state-of-the-art multi-class imbalanced learning algorithms which are based on One-versus-One (OVO) decomposition. We implemented six algorithms in literature, including SMOTEBagging, UnderBagging, OVO plus OVA, OVO plus SMOTE, One-Against-Higher-Order, and DynamicOVO, and evaluate their performance in terms of multi-class Area Under the ROC (MAUC) on eighteen datasets with different characteristics. Experimental results show that the OVO plus SMOTE algorithm is superior to other algorithms and it is quite stable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yang, Q., Wu, X.D.: 10 challenging problems in data mining research. Int. J. Inf. Technol. Decis. Mak. 5(04), 597–604 (2006)
He, H.B., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009)
Zhou, Z.H., Liu, X.Y.: On multi-class cost-sensitive learning. Nat. Conf. Artif. Intell. 26(3), 567–572 (2006)
Anand, R., Mehrotra, K., Mohan, C.K., Ranka, S.: Efficient classification for multiclass problems using modular neural networks. IEEE Trans. Neural Netw. 6(1), 117–124 (1995)
Fernández, A., López, V., Galar, M., Del Jesus, M.J., Herrera, F.: Analysing the classification of imbalanced data-sets with multiple classes: binarization techniques and ad-hoc approaches. Knowl.-Based Syst. 42(2), 97–110 (2013)
Galar, M., Ndez, A., Barrenechea, E., Bustince, H., Herrera, F.: An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Pattern Recogn. 44(8), 1761–1776 (2011)
Galar, M., Fernández, A., Ndez, A., Barrenechea, E., Bustince, H., Herrera, F.: Dynamic classifier selection for one-vs-one strategy: avoiding non-competent classifiers. Pattern Recogn. 46(12), 3412–3424 (2013)
Lorena, A.C., Carvalho, A.C., Gama, J.M.: A review on the combination of binary classifiers in multi-class problems. Artif. Intell. Rev. 30(1–4), 19–37 (2008)
Fernández, A., del Jesus, M.J., Herrera, F.: Multi-class imbalanced data-sets with linguistic fuzzy rule based classification systems based on pairwise learning. In: Hüllermeier, E., Kruse, R., Hoffmann, F. (eds.) IPMU 2010. LNCS (LNAI), vol. 6178, pp. 89–98. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14049-5_10
Krawczyk, B.: Combining one-vs-one decomposition and ensemble learning for multi-class imbalanced data. In: Burduk, R., Jackowski, K., Kurzyński, M., Woźniak, M., Żołnierek, A. (eds.) Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015. AISC, vol. 403, pp. 27–36. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-26227-7_3
Ghanem, A.S., Venkatesh, S., West, G.: Multi-class pattern classification in imbalanced data. In: IEEE 2010 International Conference on Pattern Recognition, pp. 2881–2884 (2010)
Murphey, Y.L., Wang, H., Ou, G., Feldkamp, L.A.: OAHO: an effective algorithm for multi-class learning from imbalanced data. In: IEEE 2007 International Joint Conference on Neural Networks, pp. 406–411 (2007)
Tan, A.C., Gilbert, D., Deville, Y.: Multi-class protein fold classification using a new ensemble machine learning approach. Genome Inf. 14, 206–217 (2011)
Vluymans, S., Fernández, A., Saeys, Y., Cornelis, C., Herrera, F.: Dynamic affinity-based classification of multi-class imbalanced data with one-versus-one decomposition: a fuzzy rough set approach. Knowl. Inf. Syst. 1, 1–30 (2017)
Cerf, L., Gay, D., Selmaoui-Folcher, N., Milleux, B., Boulicaut, J.F.: Editorial: parameter-free classification in multi-class imbalanced data sets. Data Knowl. Eng. 87(9), 109–129 (2013)
Zhang, Z., Krawczyk, B., Garcìa, S., Rosales-Pérez, A., Herrera, F.: Empowering one-vs-one decomposition with ensemble learning for multi-class imbalanced data. Knowl. Based Syst. 106(C), 251–263 (2016)
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over–sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)
Wang, S., Yao, X.: Diversity analysis on imbalanced data sets by using ensemble models. In: IEEE 2009 Symposium on Computational Intelligence and Data Mining, vol. 1, no. 5, pp. 324–331 (2009)
Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern.-Part C 42(4), 463–484 (2012)
Garcia-Pedrajas, N., Ortiz-Boyer, D.: Improving multi-class pattern recognition by the combination of two strategies. IEEE Trans. Pattern Anal. Mach. Intell. 28(6), 1001–1006 (2006)
Martino, M.D., Fernández, A., Iturralde, P.: Novel classifier scheme for imbalanced problems. Pattern Recogn. Lett. 34(10), 1146–1151 (2013)
Hand, D.J., Till, R.J.: A simple generalisation of the area under the roc curve for multiple class classification problems. Mach. Learn. 45(2), 171–186 (2001)
Krawczyk, B.: Learning from imbalanced data: open challenges and future directions. Prog. rtif. Intell. 5(4), 1–12 (2016)
Acknowledgment
This research has been supported by the National Natural Science Foundation of China under grant 61603186, the Natural Science Foundation of Jiangsu Province, China, under grant BK20160843, the China Postdoctoral Science Foundation under grants 2017T100370 and 2016M590457, the Postdoctoral Science Foundation of Jiangsu Province, China, under grant 1601199C, the Science Foundation (for Youth) of the Science and Technology Commission of the Central Military Commission (CMC), the national key research and development program under grant 2016YFE0108000, the CERNET next generation internet technology innovation project under grant NGII20160122, the project of ZTE cooperation research under grant 2016ZTE04-11, and Jiangsu Province Key Research and Development Program under grants BE2017739 and BE2017100.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Song, Y., Zhang, J., Yan, H., Li, Q. (2018). Multi-class Imbalanced Learning with One-Versus-One Decomposition: An Empirical Study. In: Sun, X., Pan, Z., Bertino, E. (eds) Cloud Computing and Security. ICCCS 2018. Lecture Notes in Computer Science(), vol 11065. Springer, Cham. https://doi.org/10.1007/978-3-030-00012-7_56
Download citation
DOI: https://doi.org/10.1007/978-3-030-00012-7_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00011-0
Online ISBN: 978-3-030-00012-7
eBook Packages: Computer ScienceComputer Science (R0)