Skip to main content
SpringerLink
Log in

Robust application of new deep learning tools: an experimental study in medical imaging

  • 1176: Artificial Intelligence and Deep Learning for Biomedical Applications
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Nowadays medical imaging plays a vital role in diagnosing the various types of diseases among patients across the healthcare system. Robust and accurate analysis of medical data is crucial to achieving a successful diagnosis from physicians. Traditional diagnostic methods are highly time-consuming and prone to handmade errors. Cost is reduced and performance is improved by adopting computer-aided diagnosis methods. Usually, the performance of traditional machine learning (ML) classification methods much depends on both feature extraction and selection methods that are sensitive to colors, shapes, and sizes, which conveys a complex solution when facing classification tasks in medical imaging. Currently, deep learning (DL) tools have become an alternative solution to overcome the drawbacks of traditional methods that make use of handmade features. In this paper, a new DL approach based on a hybrid deep convolutional neural network model is proposed for the automatic classification of several different types of medical images. Specifically, gradient vanishing and over-fitting issues have been properly addressed in the proposed model in order to improve its robustness by means of different tested techniques involving residual links, global average pooling layers, dropout layers, and data augmentation. Additionally, we employed the idea of parallel convolutional layers with the aim of achieving better feature representation by adopting different filter sizes on the same input and then concatenated as a result. The proposed model is trained and tested on the ICIAR 2018 dataset to classify hematoxylin and eosin-stained breast biopsy images into four categories: invasive carcinoma, in situ carcinoma, benign tumors, and normal tissue. As the experimental results show, our proposed method outperforms several of the state-of-the-art methods by achieving rate values of 93.2% and 89.8% for both image- and patch-wise image classification tasks, respectively. Moreover, we fine-tuned our model to classify foot images into two classes in order to test its robustness by considering normal and abnormal diabetic foot ulcer (DFU) image datasets. In this case, the model achieved an F1 score value of 94.80% on the public DFU dataset and 97.3% on the private DFU dataset. Lastly, transfer learning (TL) has been adopted to validate the proposed model with multiple classes with the aim of classifying six different wound types. This approach significantly improves the accuracy rate from a rate of 76.92% when trained from scratch to 87.94% when TL was considered. Our proposed model has proven its suitability and robustness by addressing several medical imaging tasks dealing with complex and challenging scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, Santamaría J, Fadhel MA, Al-Amidie M, Farhan L (2021) Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data 8(1):1–74

  2. Alzubaidi L, Al-Shamma O, Fadhel MA, Farhan L, Zhang J, Duan Y (2020) Optimizing the performance of breast Cancer classification by employing the same domain transfer learning from hybrid deep convolutional neural network model. Electronics 9(3):445

    Article  Google Scholar 

  3. Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Duan Y (2020) Deep learning models for classification of red blood cells in microscopy images to aid in sickle cell Anemia diagnosis. Electronics 9(3):427

    Article  Google Scholar 

  4. Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Santamaría J, Duan Y, Oleiwi SR (2020) Towards a better understanding of transfer learning for medical imaging: a case study. Appl Sci 10(13):4523

    Article  Google Scholar 

  5. Alzubaidi L, Fadhel MA, Oleiwi SR, al-Shamma O, Zhang J (2020) DFU_QUTNet: diabetic foot ulcer classification using novel deep convolutional neural network. Multimed Tools Appl 79:15655–15677. https://doi.org/10.1007/s11042-019-07820-w

    Article  Google Scholar 

  6. Alzubaidi L, Hasan RI, Awad FH, Fadhel MA, Alshamma O, Zhang J (2019) Multi-class breast Cancer classification by a novel two-branch deep convolutional neural network architecture. In proceedings of the 12th international conference on developments in eSystems engineering (DeSE), pp. 268–273

  7. Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A, Campilho A (2017) Classification of breast cancer histology images using convolutional neural networks. PLoS One 12(6):e0177544

    Article  Google Scholar 

  8. Aresta G, Araújo T, Kwok S, Chennamsetty SS, Safwan M, Alex V, … Fernandez G (2019) Bach: grand challenge on breast cancer histology images. Med Image Anal 56:122–139

    Article  Google Scholar 

  9. Awan R; Koohbanani NA; Shaban M; Lisowska A; Rajpoot N (2018) Context-aware learning using transferable features for classification of breast cancer histology images. In proceedings of the international conference on image analysis and recognition, springer, Cham, June 2018; pp. 788–795

  10. Barker J, Hoogi A, Depeursinge A, Rubin DL (2016) Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles. Med Image Anal 30:60–71

    Article  Google Scholar 

  11. Belsare A, Mushrif M, Pangarkar M, Meshram N (2015) Classification of breast cancer histopathology images using texture feature analysis. In proceedings of the 10th TENCON conference. Macao, China

  12. Bi WL, Hosny A, Schabath MB, Giger ML, Birkbak NJ, Mehrtash A, Allison T, Arnaout O, Abbosh C, Dunn IF, Mak RH (2019) Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J Clin 69(2):127–157

  13. Cruz-Roa A, Basavanhally A, Gonzalez F, Gilmore H, Feldman M, Ganesan S, Shih N, Tomaszewski J, Madabhushi A (2014) Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks. In proceedings of SPIE medical imaging conference. San Diego, California, USA

  14. Cui Y, Zhou F, Wang J, Liu X, Lin Y, Belongie S (2017) Kernel pooling for convolutional neural networks. In proceedings of the IEEE conference on computer vision and pattern recognition. Honolulu

  15. Dahl GE, Sainath TN, Hinton GE (2013) Improving deep neural networks for LVCSR using rectified linear units and dropout. In proceedings of the international conference on acoustics, Speech and Signal Processing, Vancouver

  16. DeSantis CE, Ma J, Goding Sauer A, Newman LA, Jemal A (2017) Breast cancer statistics, 2017, racial disparity in mortality by state. CA Cancer J Clin 67(6):439–448

  17. Doyle S, Agner S, Madabhushi A, Feldman M, Tomaszewski J (2008) Automated grading of breast cancer histopathology using spectral clustering with textural and architectural image features. In proceedings of the international symposium on biomedical imaging: from Nano to macro. Paris, France

  18. Esteva A, Robicquet A, Ramsundar B, Kuleshov V, DePristo M, Chou K, Cui C, Corrado G, Thrun S, Dean J (2019) A guide to deep learning in healthcare. Nat Med 25(1):24–29

    Article  Google Scholar 

  19. Ferreira CA, Melo T, Sousa P, Meyer MI, Shakibapour E, Costa P, Campilho A (2018) Classification of breast cancer histology images through transfer learning using a pre-trained inception ResNet v2. In proceedings of the international conference on image analysis and recognition. Springer, Cham, pp 763–770

    Google Scholar 

  20. Filipczuk P, Fevens T, Krzyżak A, Monczak R (2013) Computer-aided breast cancer diagnosis based on the analysis of cytological images of fine needle biopsies. IEEE Trans Med Imaging 32(12):2169–2178

    Article  Google Scholar 

  21. George YM, Zayed HH, Roushdy MI, Elbagoury BM (2013) Remote computer-aided breast cancer detection and diagnosis system based on cytological images. IEEE Syst J 8(3):949–964

    Article  Google Scholar 

  22. Golatkar A, Anand D, Sethi A (2018) Classification of breast cancer histology using deep learning. In International Conference Image Analysis and Recognition. Springer: Cham, Switzerland, pp 837–844

  23. Google-images-medetec-combined:https://github.com/mlaradji/deep-learning-for-wound-care/tree/master/data/google-images-medetec-combined (n.d.) (accessed on 7 April 2020)

  24. Goyal M, Reeves ND, Davison AK, Rajbhandari S, Spragg J, Yap MH (2018) DFUNet: Convolutional neural networks for diabetic foot ulcer classification. IEEE Trans Emerg Topics Comput Intell:1–12

  25. Guo Y, Dong H, Song F, Zhu C, Liu J (2018) Breast Cancer histology image classification based on deep neural networks. In: Proceedings of the International Conference Image Analysis and Recognition. Springer, Cham, Switzerland, pp 827–836

  26. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp 770–778

  27. Herent P, Schmauch B, Jehanno P, Dehaene O, Saillard C, Balleyguier C, Arfi-Rouche J, Jégou S (2019) Detection and characterization of MRI breast lesions using deep learning. Diagn Interv Imaging 100(4):219–225

  28. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Honolulu, HI, USA, pp 4700–4708

  29. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning. PMLR, pp 448–456

  30. Kassani SH, Kassani PH, Wesolowski MJ, Schneider KA, Deters R (2019) Breast cancer diagnosis with transfer learning and global pooling. In: 2019 International Conference on Information and Communication Technology Convergence (ICTC). IEEE, Jeju, Korea (South), pp 519–524

  31. Ker J, Wang L, Rao J, Lim T (2017) Deep learning applications in medical image analysis. IEEE Access 6:9375–9389

    Article  Google Scholar 

  32. Khan A, Sohail A, Zahoora U, Qureshi AS (2019) A survey of the recent architectures of deep convolutional neural networks. Artif Intell Rev, 1–62

  33. Kothari S, Phan JH, Young AN, Wang MD (2013) Histological image classification using biologically interpretable shape-based features. BMC Med Imaging 13(1):9

    Article  Google Scholar 

  34. Kowal M, Filipczuk P, Obuchowicz A, Korbicz J, Monczak R (2013) Computer-aided diagnosis of breast cancer based on fine needle biopsy microscopic images. Comput Biol Med 43(10):1563–1572

  35. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, pp 1097–1105.

  36. Lateef F, Ruichek Y (2019) Survey on semantic segmentation using deep learning techniques. Neurocomputing 338:321–348

    Article  Google Scholar 

  37. Lu L, Wang X, Carneiro G, Yang L (2019) Deep learning and convolutional neural networks for medical imaging and clinical informatics. Springer, Berlin/Heidelberg, Germany, p 201

  38. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  39. Li Y, Huang C, Ding L, Li Z, Pan Y, Gao X (2019) Deep learning in bioinformatics: introduction, application, and perspective in the big data era. Methods 166:4–21

    Article  Google Scholar 

  40. Li C, Wang X, Liu W, Latecki LJ, Wang B, Huang J (2019) Weakly supervised mitosis detection in breast histopathology images using concentric loss. Med Image Anal 53:165–178

    Article  Google Scholar 

  41. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–68

  42. Lv E, Wang X, Cheng Y, Yu Q (2019) Deep ensemble network based on multi-path fusion. Artif Intell Rev 52(1):151–168

  43. Maier A, Syben C, Lasser T, Riess C (2019) A gentle introduction to deep learning in medical image processing. Z Med Phys 29(2):86–101

    Article  Google Scholar 

  44. Medetec Wound Database (2020) http://www.medetec.co.uk/files/medetec-image-databases.html. Accessed 7 April

  45. Mohanty SP, Hughes DP, Salathé M (2016) Using deep learning for image-based plant disease detection. Front Plant Sci 7:1419

  46. Nawaz W, Ahmed S, Tahir A, Khan HA (2018) Classification of breast cancer histology images using AlexNet. In: Proceedings of the International Conference on Image Analysis and Recognition. Springer, Cham, Switzerland, pp 869–876

  47. Raghu M, Zhang C, Kleinberg J, Bengio S (2019) Transfusion: understanding transfer learning for medical imaging. In: Advances in Neural Information Processing Systems, pp 3347–3357

  48. Roy K, Banik D, Bhattacharjee D, Nasipuri M (2019) Patch-based system for classification of breast histology images using deep learning. Comput Med Imaging Graph 71:90–103

    Article  Google Scholar 

  49. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

  50. Sarker MI, Kim H, Tarasov D, Akhmetzanov D (2019) Inception architecture and residual connections in classification of breast Cancer histology images. arXiv 2019, arXiv:1912.04619. Available online https://arxiv.org/abs/1912.04619. Accessed on 28 December 2019

  51. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  52. Sivaranjini S, Sujatha CM (2019) Deep learning based diagnosis of Parkinson’s disease using convolutional neural network. Multimedia tools and applications, 1-13

  53. Spanhol FA, Oliveira LS, Petitjean C, Heutte L (2016) Breast cancer histopathological image classification using convolutional neural networks. In: Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada, pp 2560–2567

  54. Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2017) Inception-v4 inception-ResNet and the impact of residual connections on learning. In: Proceedings of the 31th AAAI Conference on Artificial Intelligence, vol 31, no 1

  55. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, ..., Rabinovich A (2015) Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9

  56. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2818–2826

  57. Targ S; Almeida D; Lyman K (2016) ResNet in ResNet: generalizing residual architectures, arXiv 2016, arXiv:1603.08029. Available online: https://arxiv.org/abs/1603.08029 (accessed on 2 January 2020)

  58. Vang YS, Chen Z, Xie X (2018) Deep learning framework for multi-class breast cancer histology image classification. In proceedings of the international conference image analysis and recognition. Springer, Cham, pp 914–922

    Google Scholar 

  59. Vedaldi A, Lenc K (2015) Matconvnet: convolutional neural networks for MATLAB. In proceedings of the 23rd ACM international conference on multimedia. Brisbane

  60. Wang Z, Dong N, Dai W, Rosario SD, Xing EP (2018) Classification of breast cancer histopathological images using convolutional neural networks with hierarchical loss and global pooling. In proceedings of the international conference on image analysis and recognition. Springer, Cham, pp 745–753

    Google Scholar 

  61. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3156–3164

  62. Wang D, Khosla A, Gargeya R, Irshad H, Beck AH (2016) Deep learning for identifying metastatic breast cancer. Cornell University Library, New York (NY)

  63. Ward EM, DeSantis CE, Lin CC, Kramer JL, Jemal A, Kohler B, Brawley OW, Gansler T (2015) Cancer statistics: breast cancer in situ. CA Cancer J Clin 65(6):481–495

  64. Yap MH, Goyal M, Osman F, Ahmad E, Marti R, Denton E, Juette A, Zwiggelaar R (2018) End-to-end breast ultrasound lesions recognition with a deep learning approach. In: Medical imaging 2018: Biomedical applications in molecular, structural, and functional imaging, vol 10578. International Society for Optics and Photonics, p 1057819

  65. Yu Z, Jiang X, Zhou F, Qin J, Ni D, Chen S, … Wang T (2018) Melanoma recognition in dermoscopy images via aggregated deep convolutional features. IEEE Trans Biomed Eng 66(4):1006–1016

    Article  Google Scholar 

  66. Zagoruyko S; Komodakis N (2016) Wide residual networks, arXiv 2016, arXiv:1605.07146. Available online: https://arxiv.org/abs/1605.07146 (accessed on 2 January 2020)

  67. Zhang B (2011) Breast cancer diagnosis from biopsy images by serial fusion of random subspace ensembles. In proceedings of the 4th international conference on biomedical engineering and informatics (BMEI2011). Shanghai, China

Download references

Acknowledgments

We would like to express our gratitude to Dr.Sameer R. Oleiwi from Nursing College, Muthanna University, for his effort in confirming the labels of wound images. We would also like to thank the Queensland University of Technology for supporting us.

Author information

Authors and Affiliations

  1. School of Computer Science, Queensland University of Technology, Brisbane, QLD, Australia

    Laith Alzubaidi & Jinglan Zhang

  2. University of Information Technology & Communications, Baghdad, Iraq

    Laith Alzubaidi & Omran Al-Shamma

  3. College of Computer Science and Information Technology, University of Sumer, Rifai, Thi Qar, Iraq

    Mohammed A. Fadhel

  4. Department of Computer Science, University of Jaén, Jaén, Spain

    J. Santamaría

  5. Faculty of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO, USA

    Ye Duan

Authors
  1. Laith Alzubaidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammed A. Fadhel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Omran Al-Shamma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinglan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. J. Santamaría
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ye Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laith Alzubaidi.

Ethics declarations

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alzubaidi, L., Fadhel, M.A., Al-Shamma, O. et al. Robust application of new deep learning tools: an experimental study in medical imaging. Multimed Tools Appl 81, 13289–13317 (2022). https://doi.org/10.1007/s11042-021-10942-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10942-9

Keywords

Search

Navigation