Skip to main content

Advertisement

SpringerLink
Log in

PatchResNet: Multiple Patch Division–Based Deep Feature Fusion Framework for Brain Tumor Classification Using MRI Images

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Modern computer vision algorithms are based on convolutional neural networks (CNNs), and both end-to-end learning and transfer learning modes have been used with CNN for image classification. Thus, automated brain tumor classification models have been proposed by deploying CNNs to help medical professionals. Our primary objective is to increase the classification performance using CNN. Therefore, a patch-based deep feature engineering model has been proposed in this work. Nowadays, patch division techniques have been used to attain high classification performance, and variable-sized patches have achieved good results. In this work, we have used three types of patches of different sizes (32 × 32, 56 × 56, 112 × 112). Six feature vectors have been obtained using these patches and two layers of the pretrained ResNet50 (global average pooling and fully connected layers). In the feature selection phase, three selectors—neighborhood component analysis (NCA), Chi2, and ReliefF—have been used, and 18 final feature vectors have been obtained. By deploying k nearest neighbors (kNN), 18 results have been calculated. Iterative hard majority voting (IHMV) has been applied to compute the general classification accuracy of this framework. This model uses different patches, feature extractors (two layers of the ResNet50 have been utilized as feature extractors), and selectors, making this a framework that we have named PatchResNet. A public brain image dataset containing four classes (glioblastoma multiforme (GBM), meningioma, pituitary tumor, healthy) has been used to develop the proposed PatchResNet model. Our proposed PatchResNet attained 98.10% classification accuracy using the public brain tumor image dataset. The developed PatchResNet model obtained high classification accuracy and has the advantage of being a self-organized framework. Therefore, the proposed method can choose the best result validation prediction vectors and achieve high image classification performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

The data used in this study were downloaded from https://www.kaggle.com/datasets/navoneel/brain-mri-images-for-brain-tumor-detection.

References

  1. Thau L, Reddy V, Singh P: Anatomy, Central Nervous System, 2019

  2. Talo M, Baloglu UB, Yıldırım Ö, Acharya UR: Application of deep transfer learning for automated brain abnormality classification using MR images. Cognitive Systems Research 54:176-188, 2019

    Article  Google Scholar 

  3. Shoeibi A, et al.: Diagnosis of brain diseases in fusion of neuroimaging modalities using deep learning: A review. Information Fusion, 2022

  4. Nayak DR, Dash R, Majhi B, Acharya UR: Application of fast curvelet Tsallis entropy and kernel random vector functional link network for automated detection of multiclass brain abnormalities. Computerized Medical Imaging and Graphics 77:101656, 2019

    Article  PubMed  Google Scholar 

  5. Arvanitis CD, Ferraro GB, Jain RK: The blood–brain barrier and blood–tumour barrier in brain tumours and metastases. Nature Reviews Cancer 20:26-41, 2020

    Article  CAS  PubMed  Google Scholar 

  6. Lapointe S, Perry A, Butowski NA: Primary brain tumours in adults. The Lancet 392:432-446, 2018

    Article  Google Scholar 

  7. Raghavendra U, Acharya UR, Adeli H: Artificial intelligence techniques for automated diagnosis of neurological disorders. European neurology 82:41-64, 2019

    Article  CAS  PubMed  Google Scholar 

  8. Liu K-W, Pajtler KW, Worst BC, Pfister SM, Wechsler-Reya RJ: Molecular mechanisms and therapeutic targets in pediatric brain tumors. Science signaling 10:eaaf7593, 2017

  9. Jones DT, et al.: Molecular characteristics and therapeutic vulnerabilities across paediatric solid tumours. Nature Reviews Cancer 19:420-438, 2019

    Article  CAS  PubMed  Google Scholar 

  10. Abd-Ellah MK, Awad AI, Khalaf AAM, Hamed HFA: A review on brain tumor diagnosis from MRI images: Practical implications, key achievements, and lessons learned. Magnetic resonance imaging 61:300-318, 2019

    Article  PubMed  Google Scholar 

  11. Herholz K, Langen K-J, Schiepers C, Mountz JM: Brain tumors: City, 2012 Year

  12. Pellico J, Gawne PJ, de Rosales RTM: Radiolabelling of nanomaterials for medical imaging and therapy. Chemical Society Reviews 50:3355-3423, 2021

    Article  CAS  PubMed  Google Scholar 

  13. Qayyum A, Qadir J, Bilal M, Al-Fuqaha A: Secure and robust machine learning for healthcare: A survey. IEEE Reviews in Biomedical Engineering 14:156-180, 2020

    Article  Google Scholar 

  14. Fujita H: AI-based computer-aided diagnosis (AI-CAD): the latest review to read first. Radiological physics and technology 13:6-19, 2020

    Article  PubMed  Google Scholar 

  15. Gudigar A, Raghavendra U, Hegde A, Kalyani M, Ciaccio EJ, Acharya UR: Brain pathology identification using computer aided diagnostic tool: A systematic review. Computer Methods and Programs in Biomedicine 187:105205, 2020

    Article  PubMed  Google Scholar 

  16. Chan HP, Hadjiiski LM, Samala RK: Computer‐aided diagnosis in the era of deep learning. Medical physics 47:e218-e227, 2020

    Article  PubMed  Google Scholar 

  17. Ullah Z, Usman M, Jeon M, Gwak J: Cascade multiscale residual attention cnns with adaptive roi for automatic brain tumor segmentation. Information sciences 608:1541-1556, 2022

    Article  Google Scholar 

  18. Tiwari A, Srivastava S, Pant M: Brain tumor segmentation and classification from magnetic resonance images: Review of selected methods from 2014 to 2019. Pattern Recognition Letters 131:244-260, 2020

    Article  Google Scholar 

  19. Gudigar A, Raghavendra U, San TR, Ciaccio EJ, Acharya UR: Application of multiresolution analysis for automated detection of brain abnormality using MR images: A comparative study. Future Generation Computer Systems 90:359-367, 2019

    Article  Google Scholar 

  20. Harvard Medical School Data, http://www.med.harvard.edu/AANLIB/.

  21. Talo M, Yildirim O, Baloglu UB, Aydin G, Acharya UR: Convolutional neural networks for multi-class brain disease detection using MRI images. Computerized Medical Imaging and Graphics 78:101673, 2019

    Article  PubMed  Google Scholar 

  22. Khan MA, et al.: Multimodal brain tumor classification using deep learning and robust feature selection: A machine learning application for radiologists. Diagnostics 10:565, 2020

    Article  PubMed  PubMed Central  Google Scholar 

  23. Menze BH, et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE transactions on medical imaging 34:1993-2024, 2014

    Article  PubMed  PubMed Central  Google Scholar 

  24. Bakas S, et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific data 4:1-13, 2017

    Article  Google Scholar 

  25. Bakas S, et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv preprint arXiv:181102629, 2018

  26. Ghassemi N, Shoeibi A, Rouhani M: Deep neural network with generative adversarial networks pre-training for brain tumor classification based on MR images. Biomedical Signal Processing and Control 57:101678, 2020

    Article  Google Scholar 

  27. Cheng J, et al.: Enhanced performance of brain tumor classification via tumor region augmentation and partition. PloS one 10:e0140381, 2015

    Article  PubMed  PubMed Central  Google Scholar 

  28. Raghavendra U, et al.: Feature‐versus deep learning‐based approaches for the automated detection of brain tumor with magnetic resonance images: A comparative study. International Journal of Imaging Systems and Technology 32:501-516, 2022

    Article  Google Scholar 

  29. Clark K, et al.: The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. Journal of digital imaging 26:1045-1057, 2013

    Article  PubMed  PubMed Central  Google Scholar 

  30. Ahmad B, Sun J, You Q, Palade V, Mao Z: Brain Tumor Classification Using a Combination of Variational Autoencoders and Generative Adversarial Networks. Biomedicines 10:223, 2022

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Nayak DR, Padhy N, Mallick PK, Zymbler M, Kumar S: Brain Tumor Classification Using Dense Efficient-Net. Axioms 11:34, 2022

    Article  Google Scholar 

  32. Zahoor MM, et al.: A New Deep Hybrid Boosted and Ensemble Learning-Based Brain Tumor Analysis Using MRI. Sensors 22:2726, 2022

    Article  PubMed  PubMed Central  Google Scholar 

  33. Brain MRI Images for Brain Tumor Detection. Available at https://www.kaggle.com/datasets/navoneel/brain-mri-images-for-brain-tumor-detection. Accessed 19/06/2022 2022.

  34. Shaik NS, Cherukuri TK: Multi-level attention network: application to brain tumor classification. Signal, Image and Video Processing 16:817-824, 2022

    Article  Google Scholar 

  35. Raza A, et al.: A Hybrid Deep Learning-Based Approach for Brain Tumor Classification. Electronics 11:1146, 2022

    Article  Google Scholar 

  36. Neelima G, Chigurukota DR, Maram B, Girirajan B: Optimal DeepMRSeg based tumor segmentation with GAN for brain tumor classification. Biomedical Signal Processing and Control 74:103537, 2022

    Article  Google Scholar 

  37. Öksüz C, Urhan O, Güllü MK: Brain tumor classification using the fused features extracted from expanded tumor region. Biomedical Signal Processing and Control 72:103356, 2022

    Article  Google Scholar 

  38. Ahuja S, Panigrahi BK, Gandhi TK: Enhanced performance of Dark-Nets for brain tumor classification and segmentation using colormap-based superpixel techniques. Machine Learning with Applications 7:100212, 2022

    Article  Google Scholar 

  39. He K, Zhang X, Ren S, Sun J: Deep residual learning for image recognition: City, 2016 Year

  40. Dosovitskiy A, et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:201011929, 2020

  41. Tolstikhin IO, et al.: Mlp-mixer: An all-mlp architecture for vision. Advances in Neural Information Processing Systems 34, 2021

  42. Trockman A, Kolter JZ: Patches are all you need? arXiv preprint arXiv:220109792, 2022

  43. Allen-Zhu Z, Li Y: What can resnet learn efficiently, going beyond kernels? Advances in Neural Information Processing Systems 32, 2019

  44. Koonce B, Koonce B: ResNet 50. Convolutional Neural Networks with Swift for Tensorflow: Image Recognition and Dataset Categorization:63–72, 2021

  45. Dogan A, et al.: PrimePatNet87: prime pattern and tunable q-factor wavelet transform techniques for automated accurate EEG emotion recognition. Computers in Biology and Medicine 138:104867, 2021

    Article  PubMed  Google Scholar 

  46. Goldberger J, Hinton GE, Roweis S, Salakhutdinov RR: Neighbourhood components analysis. Advances in neural information processing systems 17:513-520, 2004

    Google Scholar 

  47. Liu H, Setiono R: Chi2: Feature selection and discretization of numeric attributes. Proc. Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence: Herndon, VA, USA, 1995

  48. Urbanowicz RJ, Meeker M, La Cava W, Olson RS, Moore JH: Relief-based feature selection: Introduction and review. Journal of biomedical informatics 85:189-203, 2018

    Article  PubMed  PubMed Central  Google Scholar 

  49. Peterson LE: K-nearest neighbor. Scholarpedia 4:1883, 2009

    Article  Google Scholar 

  50. Kang J, Gwak J: Deep Learning-Based Brain Tumor Classification in MRI images using Ensemble of Deep Features. Journal of the Korea Society of Computer and Information 26:37-44, 2021

    Google Scholar 

  51. Brain Tumor Classification (MRI). Available at https://www.kaggle.com/sartajbhuvaji/brain-tumor-classification-mri/discussion.

  52. Tummala S, Kadry S, Bukhari SAC, Rauf HT: Classification of Brain Tumor from Magnetic Resonance Imaging Using Vision Transformers Ensembling. Current Oncology 29:7498-7511, 2022

    Article  PubMed  PubMed Central  Google Scholar 

  53. Kaldera H, Gunasekara SR, Dissanayake MB: MRI based Glioma segmentation using Deep Learning algorithms: City, 2019 Year

  54. Musallam AS, Sherif AS, Hussein MK: A New Convolutional Neural Network Architecture for Automatic Detection of Brain Tumors in Magnetic Resonance Imaging Images. IEEE Access 10:2775-2782, 2022

    Article  Google Scholar 

  55. Rasool M, et al.: A Hybrid Deep Learning Model for Brain Tumour Classification. Entropy 24:799, 2022

    Article  PubMed  PubMed Central  Google Scholar 

  56. Aurna NF, Yousuf MA, Taher KA, Azad AKM, Moni MA: A classification of MRI brain tumor based on two stage feature level ensemble of deep CNN models. Computers in Biology and Medicine 146:105539, 2022

    Article  PubMed  Google Scholar 

  57. Ullah N, et al.: An Effective Approach to Detect and Identify Brain Tumors Using Transfer Learning. Applied Sciences 12:5645, 2022

    Article  CAS  Google Scholar 

  58. Kang J, Ullah Z, Gwak J: Mri-based brain tumor classification using ensemble of deep features and machine learning classifiers. Sensors 21:2222, 2021

    Article  PubMed  PubMed Central  Google Scholar 

  59. Senan EM, Jadhav ME, Rassem TH, Aljaloud AS, Mohammed BA, Al-Mekhlafi ZG: Early Diagnosis of Brain Tumour MRI Images Using Hybrid Techniques between Deep and Machine Learning. Computational and Mathematical Methods in Medicine 2022, 2022

  60. Gupta RK, Bharti S, Kunhare N, Sahu Y, Pathik N: Brain Tumor Detection and Classification Using Cycle Generative Adversarial Networks. Interdisciplinary Sciences: Computational Life Sciences:1–18, 2022

  61. Alanazi MF, et al.: Brain tumor/mass classification framework using magnetic-resonance-imaging-based isolated and developed transfer deep-learning model. Sensors 22:372, 2022

    Article  PubMed  PubMed Central  Google Scholar 

  62. Kibriya H, Masood M, Nawaz M, Nazir T: Multiclass classification of brain tumors using a novel CNN architecture. Multimedia Tools and Applications:1–17, 2022

  63. Loh HW, Ooi CP, Seoni S, Barua PD, Molinari F, Acharya UR: Application of explainable artificial intelligence for healthcare: A systematic review of the last decade (2011–2022). Computer Methods and Programs in Biomedicine:107161, 2022

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Engineering, Munzur University, Tunceli, Turkey

    Taha Muezzinoglu

  2. Department of Computer Engineering, Faculty of Engineering, Erzurum Technical University, Erzurum, Turkey

    Nursena Baygin

  3. Interior Ministry, Elazig, Turkey

    Ilknur Tuncer

  4. School of Management & Enterprise, University of Southern Queensland, Toowoomba, Australia

    Prabal Datta Barua

  5. Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo, Australia

    Prabal Datta Barua

  6. Department of Computer Engineering, Faculty of Engineering, Ardahan University, Ardahan, Turkey

    Mehmet Baygin

  7. Department of Digital Forensics Engineering, College of Technology, Firat University, Elazig, Turkey

    Sengul Dogan & Turker Tuncer

  8. Centre of Clinical Genetics, Sydney Children’s Hospitals Network, Randwick, 2031, Australia

    Elizabeth Emma Palmer

  9. School of Women’s and Children’s Health, University of New South Wales, Randwick, 2031, Australia

    Elizabeth Emma Palmer

  10. Science, Mathematics and Technology Cluster, Singapore University of Technology and Design, Singapore, S487372, Singapore

    Kang Hao Cheong

  11. Department of Electronics and Computer Engineering, Ngee Ann Polytechnic, Singapore, 599489, Singapore

    U. Rajendra Acharya

  12. Department of Biomedical Engineering, School of Science and Technology, SUSS University, Singapore, Singapore

    U. Rajendra Acharya

  13. Department of Biomedical Informatics and Medical Engineering, Asia University, Taichung, Taiwan

    U. Rajendra Acharya

Authors
  1. Taha Muezzinoglu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nursena Baygin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ilknur Tuncer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Prabal Datta Barua
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mehmet Baygin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sengul Dogan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Turker Tuncer
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Elizabeth Emma Palmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Kang Hao Cheong
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. U. Rajendra Acharya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sengul Dogan.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Muezzinoglu, T., Baygin, N., Tuncer, I. et al. PatchResNet: Multiple Patch Division–Based Deep Feature Fusion Framework for Brain Tumor Classification Using MRI Images. J Digit Imaging 36, 973–987 (2023). https://doi.org/10.1007/s10278-023-00789-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-023-00789-x

Keywords

Search

Navigation