Skip to main content
SpringerLink
Log in

An Innovative AdaBoost Process Using Flexible Soft Labels on Imbalanced Big Data

  • Conference paper
  • First Online:
Big Data and Security (ICBDS 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1796))

Included in the following conference series:

  • 306 Accesses

Abstract

DNNs (Deep Neural Networks) have been proved to be a successful technique in many areas. Understanding the reasons behind DNN is, however, quite important in assessing trust, which is fundamental if one plans to take action based on leaning results, or when choosing whether to deploy a new model. Lack of insights into the model can be an obstacle that hinders the development and application of the DNN. AdaBoost is a well-known boosting learning technique with better interpretability, combining many relatively weak and inaccurate rules to increase the performance. And as it modified sample weights in the training process, it shows excellent adaptability even in complex cases as it can alleviate overfitting. In the paper, we propose a method training the nets by the AdaBoost based on the soft label, which we call flexibly soft-labeling AdaBoost (FSL AdaBoost). Our soft labels are made in a novel and sequential way, adding further interpretability and adaptability to the learning process. Experimental results on several well-known datasets have validated the effectiveness and novelty of FSL AdaBoost.

The authors extend their appreciation to National Key Research and Development Program of China (International Technology Cooperation Project No. 2021YFE014400) and National Science Foundation of China (No. 42175194) for funding this work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2(4), 303–314 (1989). https://doi.org/10.1007/BF02551274

    Article  MathSciNet  MATH  Google Scholar 

  2. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016). http://www.deeplearningbook.org

  3. Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530 (2016)

  4. Krueger, D., et al.: Deep nets don’t learn via memorization (2017)

    Google Scholar 

  5. Arpit, D., et al.: A closer look at memorization in deep networks. In: Proceedings of the 34th International Conference on Machine Learning-vol. 70, pp. 233–242 (2017)

    Google Scholar 

  6. Lee, W., Jun, C.H., Lee, J.S.: Instance categorization by support vector machines to adjust weights in AdaBoost for imbalanced data classification. Inf. Sci. (Ny) 381, 92–103 (2017)

    Article  Google Scholar 

  7. Lima, N.H.C., Neto, A.D.D., De Melo, J.D.: Creating an ensemble of diverse support vector machines using Adaboost. In: International Joint Conference on Neural Networks (2009)

    Google Scholar 

  8. Gan, Y., Chen, J., Xu, L.: Facial expression recognition boostedby soft label with a diverse ensemble. Pattern Recogn. Lett. 125, 105–112 (2019)

    Article  Google Scholar 

  9. Li , S., Deng, W., Du, J.: Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. Computer Vision and Pattern Recognition (CVPR). In: 2017 IEEE Conference on IEEE, pp. 2584–2593 (2017)

    Google Scholar 

  10. Zhao, M., Zhang, Z., Chow, T.W.S., Li, B.: Soft label based linear discriminant analysis for image recognition and retrieval. Comput. Vis. Image Underst. 121, 86–99 (2014). https://doi.org/10.1016/j.cviu.2014.01.008

    Article  Google Scholar 

  11. Liu, T., Wang, K., Chang, B., Sui, Z.: A soft-label method for noise-tolerant distantly supervised relation extraction. In: Proceedings of the 2017 Conference on Empirical Methods (2017)

    Google Scholar 

  12. Nguyen, Q., Valizadega, H., Hauskrecht, M.: Learning classification models with soft-label information. J. Am. Med. Inform. Assoc. 21, 501–508 (2014)

    Article  Google Scholar 

  13. Algan, G., Ulusoy, I.: Meta soft label generation for noisy labels. In: 2020 25th International Conference on Pattern Recognition (ICPR) (2021)

    Google Scholar 

  14. Schapire, R.E.: The strength of weak learnability. Mach. Learn. 5, 197–227 (1990)

    Article  Google Scholar 

  15. Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Machine Learning: Proceedings of the Thirteenth International Conference, Morgan Kauman, SanFrancisco, pp. 148–156 (1996)

    Google Scholar 

  16. Lienhart, R., Kuranov, A., Pisarevsky, V.: Empirical analysis of detection cascades of boosted classifiers for rapid object detection. In: Michaelis, B., Krell, G. (eds.) DAGM 2003. LNCS, vol. 2781, pp. 297–304. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45243-0_39

    Chapter  Google Scholar 

  17. Collins, M., Schapire, R.E., Singer, Y.: Logistic regression, adaboost and bregman distances. In: Thirteenth Conference on Computational (2000)

    Google Scholar 

  18. Domingo, C.: MadaBoost: A modification of AdaBoost. In: Proceedings of the 13th Conference on Computational Learning Theory (2013)

    Google Scholar 

  19. Wang, W., Sun, D.: The improved AdaBoost algorithms for imbalanced data classification. Inform. Sci. 563, 358–374 (2021)

    Article  MathSciNet  Google Scholar 

  20. Cao, J., Kwong, S., Wang, R.: A noise-detection based AdaBoost algorithm for mislabeled data. Pattern Recog. 45(12), 4451–4465 (2012)

    Article  MATH  Google Scholar 

  21. Ji, Z., Hui, Z., Rosset, S., Hastie, T.: Multi-class AdaBoost. Stat. Its Interface 2, 349–360 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  22. Taherkhani, A., Cosma, G., McGinnity, T.M.: AdaBoost-CNN: an adaptive boosting algorithm for convolutional neural networks to classify multi-class imbalanced datasets using transfer learning. Neurocomputing 404, 351–366 (2020)

    Article  Google Scholar 

  23. Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8228, pp. 117–124. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-42051-1_16

    Chapter  Google Scholar 

  24. Geng, X., Xia, Y.: Head pose estimation based on multivariate label distribution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1837–1842 (2014)

    Google Scholar 

  25. Xu, L., Chen, J., Gan, Y.: Head pose estimation with soft labels using regularized convolutional neural network. Neurocomputing 337, 339–353 (2019)

    Article  Google Scholar 

  26. Wang, X., Wang, X., Ni, Y.: Unsupervised domain adaptation for facial expression recognition using generative adversarial networks. Comput. Intell. Neurosci. 2018, 1–10 (2018). https://doi.org/10.1155/2018/7208794

    Article  Google Scholar 

  27. Barsoum, E., Zhang, C., Ferrer, C.C., Zhang, Z.: Training deep networks for facial expression recognition with crowd-sourced label distribution. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction, ACM, pp. 279–283 (2016)

    Google Scholar 

  28. Ho, T.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20(8), 832–844 (1998)

    Article  Google Scholar 

  29. Shirai, S., Kudo, M., Nakamura, A.: Comparison of bagging and boosting algorithms on sample and feature weighting. In: Benediktsson, J.A., Kittler, J., Roli, F. (eds.) MCS 2009. LNCS, vol. 5519, pp. 22–31. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02326-2_3

    Chapter  Google Scholar 

  30. García-Pedrajas, N., Ortiz-Boyer, D.: Boosting random subspace method. Neural Netw. 21(9), 1344–1362 (2008)

    Article  MATH  Google Scholar 

  31. Rao, Q., Yu, B., He, K., Feng, B.: Regularization and Iterative Initialization of Softmax for Fast Training of Convolutional Neural Networks. In: 2019 International Joint Conference on Neural Networks (IJCNN), 30 Sep 2019

    Google Scholar 

  32. Masko, D., Hensman. P.: The Impact of Imbalanced Training Data for Convolutional Neural Networks. Bachelor thesis, KTH, Sch. Comput. Sci. Commun. (2015)

    Google Scholar 

  33. Hu, W., Maybank, S.: AdaBoost-based algorithm for network intrusion detection. In: IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 1 Apr 2008

    Google Scholar 

  34. Ahfock, D., McLachlan, G.J.: Harmless label noise and informative soft-labels in supervised classification. Comput. Stat. Data Anal. 161, 107253 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  35. Ochiuddin Miah, M., Shahriar Khan, S., Shatabda, S., Farid, D.M.: Improving detection accuracy for imbal-anced network intrusion classification using cluster-based under-sampling with random forests. In: 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT), pp. 1–5. IEEE (2019)

    Google Scholar 

  36. Bennin, K.E., Keung, J., Phannachitta, P., Monden, A., Mensah, S.: Mahakil: diversity based oversampling approach to alleviate the class imbalance issue in software defect prediction. IEEE Trans. Software Eng. 44(6), 534–550 (2017)

    Article  Google Scholar 

  37. Song, Q., Guo, Y., Shepperd, M.: A comprehensive investigation of the role of imbalanced learning for software defect prediction. IEEE Trans. Software Eng. 45(12), 1253–1269 (2018)

    Article  Google Scholar 

  38. Cruz-Roa, A.A., Ovalle, J.E.A., Madabhushi, A., Osorio, F.A.G.: A deep learning architecture for image representation, visual interpretability and automated basal-cell carcinoma cancer detection. In: Salinesi, C., Norrie, M.C., Pastor, Ó. (eds.) Advanced Information Systems Engineering: 25th International Conference, CAiSE 2013, Valencia, Spain, June 17-21, 2013. Proceedings, pp. 403–410. Springer Berlin Heidelberg, Berlin, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40763-5_50

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Nanjing University of Information Science and Technology (NUIST), Nanjing, China

    Jinke Wang, Biao Song & Xinchang Zhang

  2. Nanjing Institute of Technology (NJIT), Nanjing, China

    Yuan Tian

  3. Cyberspace Institute Advanced Technology, Guangzhou University, Guangzhou, China

    Ran Guo

Authors
  1. Jinke Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Biao Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinchang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuan Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ran Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Biao Song .

Editor information

Editors and Affiliations

  1. Nanjing Institute of Technology, Nanjing, China

    Yuan Tian

  2. Nanjing University of Information Science and Technology, Nanjing, Jiangsu, China

    Tinghuai Ma

  3. Chinese Academy of Sciences, Shenzhen, China

    Qingshan Jiang

  4. Nanjing University of Information Science and Technology, Nanjing, China

    Qi Liu

  5. King Saud University, Riyadh, Saudi Arabia

    Muhammad Khurram Khan

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, J., Song, B., Zhang, X., Tian, Y., Guo, R. (2023). An Innovative AdaBoost Process Using Flexible Soft Labels on Imbalanced Big Data. In: Tian, Y., Ma, T., Jiang, Q., Liu, Q., Khan, M.K. (eds) Big Data and Security. ICBDS 2022. Communications in Computer and Information Science, vol 1796. Springer, Singapore. https://doi.org/10.1007/978-981-99-3300-6_13

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-3300-6_13

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-3299-3

  • Online ISBN: 978-981-99-3300-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation