Skip to main content
SpringerLink
Log in

Biggest Margin Tree for the Multi-class Classification

  • Conference paper
  • First Online:
Future Data and Security Engineering. Big Data, Security and Privacy, Smart City and Industry 4.0 Applications (FDSE 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1925))

Included in the following conference series:

  • 418 Accesses

Abstract

In this paper, we propose the random forest algorithm of biggest margin trees \(RF{-}BMT\) for the multi-class classification. The novel algorithm enhances the classification of chest X-ray images, specifically for distinguishing between normal, covid-19, edema, mass-nodule, and pneumothorax cases. Our approach combines contrastive learning with our proposed algorithm to improve performance and address the limitation of labeled data by leveraging a large amount of unlabeled data for learning features. We propose training the \(RF{-}BMT\) algorithm on the features extracted from the linear fine-tuned model of Momentum Contrast (MoCo), which is trained on Resnet50 architecture. The \(RF{-}BMT\) algorithm plays a role as a replacement for softmax in deep networks. Based on the empirical results, our proposed \(RF{-}BMT\) algorithm demonstrates substantial improvement compared to solely fine-tuning the linear layer both the ImageNet pretrained model and the MoCo pretrained model, reaching an impressive accuracy rate of 88.4%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abadi, M., et al.: Tensorflow: large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016)

  2. Afshar, P., Heidarian, S., Naderkhani, F., Oikonomou, A., Plataniotis, K.N., Mohammadi, A.: COVID-caps: a capsule network-based framework for identification of COVID-19 cases from x-ray images. Pattern Recogn. Lett. 138, 638–643 (2020)

    Article  Google Scholar 

  3. Akkus, Z., Galimzianova, A., Hoogi, A., Rubin, D.L., Erickson, B.J.: Deep learning for brain MRI segmentation: state of the art and future directions. J. Digit. Imaging 30, 449–459 (2017)

    Article  Google Scholar 

  4. Bhandary, A., et al.: Deep-learning framework to detect lung abnormality-a study with chest x-ray and lung CT scan images. Pattern Recogn. Lett. 129, 271–278 (2020)

    Article  Google Scholar 

  5. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  6. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.: Classification and Regression Trees. Wadsworth International (1984)

    Google Scholar 

  7. Castiglioni, I., et al.: Ai applications to medical images: from machine learning to deep learning. Physica Med. 83, 9–24 (2021)

    Article  Google Scholar 

  8. Chakraborty, S., Murali, B., Mitra, A.K.: An efficient deep learning model to detect COVID-19 using chest x-ray images. Int. J. Environ. Res. Public Health 19(4), 2013 (2022)

    Article  Google Scholar 

  9. Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016)

    Google Scholar 

  10. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709 (2020)

  11. Chen, T., Kornblith, S., Swersky, K., Norouzi, M., Hinton, G.: Big self-supervised models are strong semi-supervised learners. arXiv preprint arXiv:2006.10029 (2020)

  12. Chen, X., Fan, H., Girshick, R., He, K.: Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 (2020)

  13. Chlap, P., Min, H., Vandenberg, N., Dowling, J., Holloway, L., Haworth, A.: A review of medical image data augmentation techniques for deep learning applications. J. Med. Imaging Radiat. Oncol. 65(5), 545–563 (2021)

    Article  Google Scholar 

  14. Chouhan, V., et al.: A novel transfer learning based approach for pneumonia detection in chest x-ray images. Appl. Sci. 10(2), 559 (2020)

    Article  Google Scholar 

  15. Cohen, J.P., Morrison, P., Dao, L., Roth, K., Duong, T.Q., Ghassemi, M.: COVID-19 image data collection: Prospective predictions are the future. arXiv preprint arXiv:2006.11988 (2020)

  16. Do, T.N., Le, V.T., Doan, T.H.: SVM on top of deep networks for COVID-19 detection from chest x-ray images. J. Inf. Commun. Converg. Eng. (2022)

    Google Scholar 

  17. Do, T., Lenca, P., Lallich, S.: Classifying many-class high-dimensional fingerprint datasets using random forest of oblique decision trees. Vietnam J. Comput. Sci. 2(1), 3–12 (2015)

    Article  Google Scholar 

  18. Dorogush, A.V., Ershov, V., Gulin, A.: CatBoost: gradient boosting with categorical features support. arXiv preprint arXiv:1810.11363 (2018)

  19. Haghanifar, A., Majdabadi, M.M., Ko, S.: COVID-19 chest x-ray image repository (2021). https://figshare.com/articles/dataset/COVID-19/_Chest/_X-Ray/_Image/_Repository/12580328

  20. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)

    Google Scholar 

  21. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  22. Hemdan, E.E.D., Shouman, M.A., Karar, M.E.: COVIDX-Net: a framework of deep learning classifiers to diagnose COVID-19 in x-ray images. arXiv preprint arXiv:2003.11055 (2020)

  23. Irvin, J., et al.: CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison (2019)

    Google Scholar 

  24. Ke, G., et al.: LightGBM: a highly efficient gradient boosting decision tree. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  25. Ker, J., Wang, L., Rao, J., Lim, T.: Deep learning applications in medical image analysis. IEEE Access 6, 9375–9389 (2017)

    Article  Google Scholar 

  26. Kermany, D.S., et al.: Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5), 1122–1131 (2018)

    Article  Google Scholar 

  27. Kesim, E., Dokur, Z., Olmez, T.: X-ray chest image classification by a small-sized convolutional neural network. In: 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), pp. 1–5. IEEE (2019)

    Google Scholar 

  28. Liang, C.H., Liu, Y.C., Wu, M.T., Garcia-Castro, F., Alberich-Bayarri, A., Wu, F.Z.: Identifying pulmonary nodules or masses on chest radiography using deep learning: external validation and strategies to improve clinical practice. Clin. Radiol. 75(1), 38–45 (2020)

    Article  Google Scholar 

  29. Nguyen, H.Q., et al.: VinDR-CXR: an open dataset of chest x-rays with radiologist’s annotations. Sci. Data 9(1), 429 (2022)

    Article  Google Scholar 

  30. Ozturk, T., Talo, M., Yildirim, E.A., Baloglu, U.B., Yildirim, O., Acharya, U.R.: Automated detection of COVID-19 cases using deep neural networks with x-ray images. Comput. Biol. Med. 121, 103792 (2020)

    Article  Google Scholar 

  31. Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  32. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA (1993)

    Google Scholar 

  33. Rajpurkar, P., et al.: Deep learning for chest radiograph diagnosis: a retrospective comparison of the cheXNeXt algorithm to practicing radiologists. PLoS Med. 15(11), e1002686 (2018)

    Article  Google Scholar 

  34. Saltz, J., et al.: Stony brook university COVID-19 positive cases. Cancer Imaging Archive 4 (2021)

    Google Scholar 

  35. Shiraishi, J., et al.: Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. Am. J. Roentgenol. 174(1), 71–74 (2000)

    Article  Google Scholar 

  36. Sowrirajan, H., Yang, J., Ng, A.Y., Rajpurkar, P.: MoCo pretraining improves representation and transferability of chest x-ray models. In: Medical Imaging with Deep Learning, pp. 728–744. PMLR (2021)

    Google Scholar 

  37. Vapnik, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer, Berlin (2000)

    Book  MATH  Google Scholar 

  38. Vayá, M.D.L.I., et al.: BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patients. arXiv preprint arXiv:2006.01174 (2020)

  39. Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2097–2106 (2017)

    Google Scholar 

  40. Winther, H., et al.: Dataset: COVID-19 Image Repository. Hannover Medical School, Hannover, Germany (2020)

    Google Scholar 

  41. Woźniak, M., Połap, D., Capizzi, G., Sciuto, G.L., Kośmider, L., Frankiewicz, K.: Small lung nodules detection based on local variance analysis and probabilistic neural network. Comput. Meth. Programs Biomed. 161, 173–180 (2018)

    Article  Google Scholar 

  42. Yadav, S.S., Jadhav, S.M.: Deep convolutional neural network based medical image classification for disease diagnosis. J. Big Data 6(1), 1–18 (2019)

    Article  Google Scholar 

  43. Zawacki, A., et al.: SIIM-ACR pneumothorax segmentation (2019). https://kaggle.com/competitions/siim-acr-pneumothorax-segmentation

Download references

Acknowledgements

This work has been funded by the STARWARS project (STormwAteR and WastewAteR networkS heterogeneous data AI-driven management). The authors would like to particularly thank for the STARWARS’ support.

Author information

Authors and Affiliations

  1. College of Information Technology Can Tho University, Cantho, 92000, Vietnam

    Tri-Thuc Vo & Thanh-Nghi Do

  2. UMI UMMISCO 209 (IRD/UPMC) Sorbonne University, Pierre and Marie Curie University - Paris 6, Paris, France

    Thanh-Nghi Do

Authors
  1. Tri-Thuc Vo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thanh-Nghi Do
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tri-Thuc Vo .

Editor information

Editors and Affiliations

  1. Ho Chi Minh City University of Industry and Trade, Ho Chi Minh City, Vietnam

    Tran Khanh Dang

  2. Johannes Kepler University of Linz, Linz, Austria

    Josef Küng

  3. Sungkyunkwan University, Suwon-si, Korea (Republic of)

    Tai M. Chung

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vo, TT., Do, TN. (2023). Biggest Margin Tree for the Multi-class Classification. In: Dang, T.K., Küng, J., Chung, T.M. (eds) Future Data and Security Engineering. Big Data, Security and Privacy, Smart City and Industry 4.0 Applications. FDSE 2023. Communications in Computer and Information Science, vol 1925. Springer, Singapore. https://doi.org/10.1007/978-981-99-8296-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8296-7_3

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8295-0

  • Online ISBN: 978-981-99-8296-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation