Abstract
Learning from imbalanced data is a challenging task in a wide range of applications, which attracts significant research efforts from machine learning and data mining community. As a natural approach to this issue, oversampling balances the training samples through replicating existing samples or synthesizing new samples. In general, synthesization outperforms replication by supplying additional information on the minority class. However, the additional information needs to follow the same normal distribution of the training set, which further constrains the new samples within the predefined range of training set. In this paper, we present the Wiener process oversampling (WPO) technique that brings the physics phenomena into sample synthesization. WPO constructs a robust decision region by expanding the attribute ranges in training set while keeping the same normal distribution. The satisfactory performance of WPO can be achieved with much lower computing complexity. In addition, by integrating WPO with ensemble learning, the WPOBoost algorithm outperformsmany prevalent imbalance learning solutions.
Similar content being viewed by others
References
Zhou Z H, Liu X Y. Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Transactions on Knowledge and Data Engineering, 2006, 18(1): 63–77
Liu X Y, Zhou Z H. The influence of class imbalance on cost-sensitive learning: An empirical study. In: Proceedings of the 6th International Conference on Data Mining. 2006, 970–974
Yu L, Wang S, Lai K K. Developing an svm-based ensemble learning system for customer risk identification collaborating with customer relationship management. Frontiers of Computer Science in China, 2010, 4(2): 196–203
Liu E, Zhao H, Guo F, Liang J, Tian J. Fingerprint segmentation based on an adaboost classifier. Frontiers of Computer Science in China, 2011, 5(2): 148–157
Han H, Wang W, Mao B. Over-sampling algorithm based on adaboost in unbalanced data set. Computer Engineering, 2007, 33(10): 207–209 (in Chinese)
Chawla N V, Lazarevic A, Hall L O, Bowyer K W. Smoteboost: improving prediction of the minority class in boosting. Lecture Notes in Computer Science, 2003, 2838: 107–119
Mease D, Wyner A J, Buja A. Boosted classification trees and class probability/quantile estimation. The Journal of Machine Learning Research, 2007, 8: 409–439
Batista G E, Prati R C, Monard M C. A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explorations Newsletter, 2004, 6(1): 20–29
Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C. Safe-levelsmote: safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem. In: Proceedings of Advances in Knowledge Discovery and Data Mining. 2009, 475–482
Chawla N V, Bowyer K W, Hall L O, Kegelmeyer W P. Smote: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 2002, 321–357
Yuan B, Liu W. Measure oriented training: a targeted approach to imbalanced classification problems. Frontiers of Computer Science, 2012, 6(5): 489–497
Kang P, Cho S. EUS SVMS: ensemble of under-sampled svms for data imbalance problems. In: Proceedings of Neural Information Processing. 2006, 837–846
Japkowicz N. The class imbalance problem: significance and strategies. In: Proceedings of International Conference on Artificial Intelligence. 2000
Galar M, Fernandez A, Barrenechea E, Bustince H, Herrera F. A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Transactions on Systems, Man, and Cybernetics, 2012, 42(4): 463–484
Yuan B, Ma X. Sampling+ reweighting: boosting the performance of adaboost on imbalanced datasets. In: Proceedings of International Joint Conference on Neural Networks. 2012, 1–6
Hida T. Brownian motion. Springer US, 1980, 11(5): 44–113
Dietterich T G. Ensemble methods in machine learning. In: Proceedings of Multiple classifier systems. 2000, 1–15
Maloof M A. Learning when data sets are imbalanced and when costs are unequal and unknown. In: Proceedings of ICML-2003 Workshop on Learning from Imbalanced Data Sets II. 2003
Chawla N V, Japkowicz N, Kotcz A. Editorial: special issue on learning from imbalanced data sets. ACM SIGKDD Explorations Newsletter, 2004, 6(1): 1–6
Han H, Wang WY, Mao B H. Borderline-smote: A new over-sampling method in imbalanced data sets learning. In: Proceedings of Advances in Intelligent Computing. 2005, 878–887
Liu X Y, Wu J, Zhou Z H. Exploratory undersampling for classimbalance learning. IEEE Transactions on Systems, Man, and Cybernetics, 2009, 39(2): 539–550
Schapire R E. The boosting approach to machine learning: an overview. Nonlinear Estimation and Classification, 2003, 149–171
Schapire R E, Singer Y. Boostexter: a boosting-based system for text categorization. Machine Learning, 2000, 39(2–3): 135–168
Li X, Wang L, Sung E. Adaboost with svm-based component classifiers. Engineering Applications of Artificial Intelligence, 2008, 21(5): 785–795
Boyd S, Vandenberghe L. Convex Optimization. Cambridge: Cambridge University Press, 2004
Asuncion A, Newman D. UCI machine learning repository. http://www.ics.uci.edu/mlearn/MLRepository.html, 2007
Breiman L, Friedman J, Stone C J, Olshen R A. Classification and Regression Trees. Belmont: Wadsworth International Group, 1984
Lewis D D. Naive (Bayes) at forty: the independence assumption in information retrieval. In: Proceedings of Machine learning: ECML-98. 1998, 4–15
Keller J M, Gray M R, Givens J A. A fuzzy k-nearest neighbor algorithm. IEEE Transactions on Systems, Man and Cybernetics, 1985 (4): 580–585
Breiman L. Bagging predictors. Machine Learning, 1996, 24(2): 123–140
Acknowledgements
This research was partially supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (XDA06030200), the National Natural Science Foundation of China (Grant Nos. M1552006, 61403369, 61272427, and 61363030), Xinjiang Uygur Autonomous Region Science and Technology Project (201230123), Beijing Key Lab of Intelligent Telecommunication Software, Multimedia (ITSM201502), Guangxi Key Laboratory of Trusted Software (kx201418).
Author information
Authors and Affiliations
Corresponding author
Additional information
Qian Li received her MS at Shandong University of Computer Software and Theory, China. Now she is a PhD student of Institute of Information Engineering, Chinese Academy of Sciences, China. Her main research interests include machine learning, data mining and services computing.
Gang Li is currently a senior lecturer in the School of Information Technology at Deakin University, Australia. His research interest are in the area of data mining, machine learning and multimedia analysis. He served on the Program Committee for over 40 international conferences in artificial intelligence, data mining and machine learning, tourism and hospitality management.
Wenjia Niu is an associate professor in the Institute of Information Engineering, Chinese Academy of Sciences, China. His research interests include Web services, agent, sensor network and data mining. He has served as a regular reviewer for Journal of Network and Computer Applications (JNCA), Knowledge and Information Systems (KAIS), and Journal of Computer Science and Technology (JCST).
Yanan Cao is an associate professor in the Institute of Information Engineering, Academy of Sciences, China. She obtained her PhD in the Institute of Computing Technology in 2012. Her research interests include data mining methodologies, machine learning algorithms and knowledge graph.
Liang Chang received his PhD in computer science from the Institute of Computing Technology, Chinese Academy of Sciences, China in 2008. He is currently a professor in the School of Computer Science and Engineering, Guilin University of Electronic Technology, China. His research interests include knowledge representation and reasoning, formal methods, trusted software and intelligent planning.
Jianlong Tan is a researcher in the Institute of Information Engineering, Chinese Academy of Sciences, China. He is also the chairman of the Intelligent Information Processing Research Center, Institute of Information Engineering, Chinese Academy of Sciences. His research interests are string matching algorithm, algorithm security and information security.
Li Guo is a researcher in the Institute of Information Engineering, Chinese Academy of Sciences, China. Her research interests include data stream management systems and information security.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Li, Q., Li, G., Niu, W. et al. Boosting imbalanced data learning with Wiener process oversampling. Front. Comput. Sci. 11, 836–851 (2017). https://doi.org/10.1007/s11704-016-5250-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11704-016-5250-y