Abstract
Large-scale classification is one of the most significant topics in machine learning. However, previous classification methods usually require the assumption that the data has a balanced class distribution. Thus, when dealing with imbalanced data, these methods often present performance degradation. In order to seek the better performance in large-scale classification, we propose a novel Cost-Sensitive Alternating Direction Method of Multipliers method (CSADMM) to deal with imbalanced data in this paper. CSADMM derives the problem into a series of subproblems efficiently solved by a dual coordinate descent method in parallel. In particular, CSADMM incorporates different classification costs for large-scale imbalanced classification by cost-sensitive learning. Experimental results on several large-scale imbalanced datasets show that compared with distributed random forest and fuzzy rule based classification system, CSADMM obtains better classification performance, with the training time is significantly reduced. Moreover, compared with single-machine methods, CSADMM also shows promising results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Forero, P.A., Cano, A., Giannakis, G.B.: Consensus-based distributed support vector machines. J. Mach. Learn. Res. 11, 1663–1707 (2010)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends\(\textregistered \) Mach. Learn. 3(1), 1–122 (2011)
López, V., del Río, S., Benítez, J.M., Herrera, F.: Cost-sensitive linguistic fuzzy rule based classification systems under the mapreduce framework for imbalanced big data. Fuzzy Sets Syst. 258, 5–38 (2015)
del Río, S., López, V., Benítez, J.M., Herrera, F.: On the use of mapreduce for imbalanced big data using random forest. Inf. Sci. 285, 112–137 (2014)
Kumar, N.S., Rao, K.N., Govardhan, A., Reddy, K.S., Mahmood, A.M.: Undersampled k-means approach for handling imbalanced distributed data. Progress Artif. Intell. 3(1), 29–38 (2014)
Veropoulos, K., Campbell, C., Cristianini, N., et al.: Controlling the sensitivity of support vector machines. In: Proceedings of the International Joint Conference on AI, pp. 55–60 (1999)
Batuwita, R., Palade, V.: FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans. Fuzzy Syst. 18(3), 558–571 (2010)
Cao, P., Zhao, D., Zaiane, O.: An optimized cost-sensitive SVM for imbalanced data learning. In: Pei, J., Tseng, V.S., Cao, L., Motoda, H., Xu, G. (eds.) PAKDD 2013. LNCS, vol. 7819, pp. 280–292. Springer, Heidelberg (2013). doi:10.1007/978-3-642-37456-2_24
Goldfarb, D., Ma, S., Scheinberg, K.: Fast alternating linearization methods for minimizing the sum of two convex functions. Math. Program. 141(1–2), 349–382 (2013)
Zhang, C., Lee, H., Shin, K.G.: Efficient distributed linear classification algorithms via the alternating direction method of multipliers. In: International Conference on Artificial Intelligence and Statistics, pp. 1398–1406 (2012)
Tao, Q., Gao, Q.K., Chu, D.J., Wu, G.W.: Stochastic learning via optimizing the variational inequalities. IEEE Trans. Neural Netw. Learn. Syst. 25(10), 1769–1778 (2014)
Hsieh, C.J., Chang, K.W., Lin, C.J., Keerthi, S.S., Sundararajan, S.: A dual coordinate descent method for large-scale linear SVM. In: Proceedings of the 25th International Conference on Machine Learning, pp. 408–415. ACM (2008)
Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: Liblinear: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)
Zhou, Z.H., Liu, X.Y.: Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans. Knowl. Data Eng. 18(1), 63–77 (2006)
Owusu, E., Zhan, Y., Mao, Q.R.: An SVM-AdaBoost facial expression recognition system. Appl. Intell. 40(3), 536–545 (2014)
Acknowledgements
The work was supported by National Natural Science Foundation of China (61403208, 61673203), and Young Elite Scientists Sponsorship Program by CAST (YESS 20160035).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Wang, H., Shi, Y., Chen, X., Gao, Y. (2017). Cost-Sensitive Alternating Direction Method of Multipliers for Large-Scale Classification. In: Yin, H., et al. Intelligent Data Engineering and Automated Learning – IDEAL 2017. IDEAL 2017. Lecture Notes in Computer Science(), vol 10585. Springer, Cham. https://doi.org/10.1007/978-3-319-68935-7_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-68935-7_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68934-0
Online ISBN: 978-3-319-68935-7
eBook Packages: Computer ScienceComputer Science (R0)