Skip to main content

Advertisement

SpringerLink
Log in

Breast Tumor Classification in Ultrasound Images by Fusion of Deep Convolutional Neural Network and Shallow LBP Feature

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Breast cancer is one of the most dangerous and common cancers in women which leads to a major research topic in medical science. To assist physicians in pre-screening for breast cancer to reduce unnecessary biopsies, breast ultrasound and computer-aided diagnosis (CAD) have been used to distinguish between benign and malignant tumors. In this study, we proposed a CAD system for tumor diagnosis using a multi-channel fusion method and feature extraction structure based on multi-feature fusion on breast ultrasound (BUS) images. In the pre-processing stage, the multi-channel fusion method completed the color conversion of the BUS image to make it contain richer information. In the feature extraction stage, the pre-trained ResNet50 network was selected as the basic network, and three levels of features were combined based on adaptive spatial feature fusion (ASFF), and finally, the shallow local binary pattern (LBP) texture features were fused. Support vector machine (SVM) was used for comparative analysis. A retrospective analysis was carried out, and 1615 breast tumor images (572 benign and 1043 malignant) confirmed by pathological examinations were collected. After data processing and augmentation, for an independent test set consisting of 874 breast ultrasound images (457 benign and 417 malignant), the accuracy, precision, recall, specificity, F1 score, and AUC of our method were 96.91%, 98.75%, 94.72%, 98.91%, 0.97, and 0.991, respectively. The results show that the integration of shallow LBP texture features and multi-level depth features can more effectively improve the comprehensive performance of breast tumor diagnosis, and has strong clinical application value. Compared with the past methods, our proposed method is expected to realize the automatic diagnosis of breast tumors and provide an auxiliary tool for radiologists to accurately diagnose breast diseases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Erratum: Global Burden of Disease Cancer Collaboration. Global, regional, and national cancer incidence, mortality, years of life lost, years lived with disability, and disability-adjusted life-years for 29 cancer groups, 1990 to 2017: a systemic analysis, (2020)

  2. Stimpfel, M., Virant-Klun, I.: Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012. J. Cancer Stem Cell Res. (2016). https://doi.org/10.14343/jcscr.2016.4e1003

  3. Qi, X., Zhang, L., Chen, Y., Pi, Y., Chen, Y., Lv, Q., Yi, Z.: Automated diagnosis of breast ultrasonography images using deep neural networks. Med. Image Anal. 52, 185–198 (2019). https://doi.org/10.1016/j.media.2018.12.006

    Article  PubMed  Google Scholar 

  4. Evans, A.: Breast imaging reporting and data systems. The Breast. (1994). https://doi.org/10.1016/0960-9776(94)90019-1

    Article  Google Scholar 

  5. Hizukuri, A., Nakayama, R.: Computer-aided diagnosis scheme for determining histological classification of breast lesions on ultrasonographic images using convolutional neural network. Diagnostics. (2018). https://doi.org/10.3390/diagnostics8030048

    Article  PubMed  PubMed Central  Google Scholar 

  6. Ekici, S., Jawzal, H.: Breast cancer diagnosis using thermography and convolutional neural networks. Med. Hypotheses. (2020). https://doi.org/10.1016/j.mehy.2019.109542

    Article  PubMed  Google Scholar 

  7. Daoud, M.I., Abdel-Rahman, S., Bdair, T.M., Al-Najar, M.S., Al-Hawari, F.H., Alazrai, R.: Breast tumor classification in ultrasound images using combined deep and handcrafted features. Sensors (Switzerland). (2020). https://doi.org/10.3390/s20236838

    Article  PubMed Central  Google Scholar 

  8. Shia, W.C., Chen, D.R.: Classification of malignant tumors in breast ultrasound using a pretrained deep residual network model and support vector machine. Comput. Med. Imaging Graph. 87, 101829 (2021). https://doi.org/10.1016/j.compmedimag.2020.101829

    Article  PubMed  Google Scholar 

  9. Kriti, Virmani, J., Agarwal, R.: Deep feature extraction and classification of breast ultrasound images. Multimed. Tools Appl. (2020). https://doi.org/10.1007/s11042-020-09337-z

  10. Nica, R.E., Șerbănescu, M.S., Florescu, L.M., Camen, G.C., Streba, C.T., Gheonea, I.A.: Deep learning: a promising method for histological class prediction of breast tumors in mammography. J. Digit. Imaging. (2021). https://doi.org/10.1007/s10278-021-00508-4

    Article  PubMed  PubMed Central  Google Scholar 

  11. Liu, Y., Ren, L., Cao, X., Tong, Y.: Breast tumors recognition based on edge feature extraction using support vector machine. Biomed. Signal Process. Control. (2020). https://doi.org/10.1016/j.bspc.2019.101825

    Article  Google Scholar 

  12. Nemat, H., Fehri, H., Ahmadinejad, N., Frangi, A.F., Gooya, A.: Classification of breast lesions in ultrasonography using sparse logistic regression and morphology-based texture features. Med. Phys. (2018). https://doi.org/10.1002/mp.13082

    Article  PubMed  Google Scholar 

  13. Gupta, V., Vasudev, M., Doegar, A., Sambyal, N.: Breast cancer detection from histopathology images using modified residual neural networks. Biocybern. Biomed. Eng. (2021). https://doi.org/10.1016/j.bbe.2021.08.011

    Article  Google Scholar 

  14. Zhuang, Z., Yang, Z., Raj, A.N.J., Wei, C., Jin, P., Zhuang, S.: Breast ultrasound tumor image classification using image decomposition and fusion based on adaptive multi-model spatial feature fusion. Comput. Methods Programs Biomed. 208, 106221 (2021). https://doi.org/10.1016/j.cmpb.2021.106221

    Article  PubMed  Google Scholar 

  15. Alom, M.Z., Yakopcic, C., Nasrin, M.S., Taha, T.M., Asari, V.K.: Breast cancer classification from histopathological images with inception recurrent residual convolutional neural network. J. Digit. Imaging. (2019). https://doi.org/10.1007/s10278-019-00182-7

    Article  PubMed  PubMed Central  Google Scholar 

  16. Zhang, H., Han, L., Chen, K., Peng, Y., Lin, J.: Diagnostic efficiency of the breast ultrasound computer-aided prediction model based on convolutional neural network in breast cancer. J. Digit. Imaging. (2020). https://doi.org/10.1007/s10278-020-00357-7

    Article  PubMed  PubMed Central  Google Scholar 

  17. Chi, J., Walia, E., Babyn, P., Wang, J., Groot, G., Eramian, M.: Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network. J. Digit. Imaging. 30, 477–486 (2017). https://doi.org/10.1007/s10278-017-9997-y

    Article  PubMed  PubMed Central  Google Scholar 

  18. Zhou, H., Jin, Y., Dai, L., Zhang, M., Qiu, Y., wang, K., Tian, J., Zheng, J.: Differential diagnosis of benign and malignant thyroid nodules using deep learning radiomics of thyroid ultrasound images. Eur. J. Radiol. 127, 108992 (2020). https://doi.org/10.1016/j.ejrad.2020.108992

  19. Ma, J., Wu, F., Zhu, J., Xu, D., Kong, D.: A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics. 73, 221–230 (2017). https://doi.org/10.1016/j.ultras.2016.09.011

    Article  PubMed  Google Scholar 

  20. Agarwal, R., Diaz, O., Lladó, X., Yap, M.H., Martí, R.: Automatic mass detection in mammograms using deep convolutional neural networks. J. Med. Imaging. 6, 1 (2019). https://doi.org/10.1117/1.jmi.6.3.031409

    Article  Google Scholar 

  21. Aswiga, R. V., Aishwarya, R., Shanthi, A.P.: Augmenting transfer learning with feature extraction techniques for limited breast imaging datasets. J. Digit. Imaging. (2021). https://doi.org/10.1007/s10278-021-00456-z

    Article  Google Scholar 

  22. Tanaka, H., Chiu, S.-W., Watanabe, T., Kaoku, S., Yamaguchi, T.: Computer-aided diagnosis system for breast ultrasound images using deep learning. Ultrasound Med. Biol. (2019). https://doi.org/10.1016/j.ultrasmedbio.2019.07.426

    Article  PubMed  Google Scholar 

  23. Sudharson, S., Kokil, P.: An ensemble of deep neural networks for kidney ultrasound image classification. Computer Methods and Programs in Biomedicine. 197, (2020). https://doi.org/10.1016/j.cmpb.2020.105709

  24. Liu, T., Xie, S., Yu, J., Niu, L., Sun, W.: Classification of thyroid nodules in ultrasound images using deep model based transfer learning and hybrid features. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings (2017)

  25. Xiao, T., Liu, L., Li, K., Qin, W., Yu, S., Li, Z.: Comparison of transferred deep neural networks in ultrasonic breast masses discrimination. 2018, (2018)

  26. Liao, W.X., He, P., Hao, J., Wang, X.Y., Yang, R.L., An, D., Cui, L.G.: Automatic identification of breast ultrasound image based on supervised block-based region segmentation algorithm and features combination migration deep learning model. IEEE J. Biomed. Heal. Informatics. (2020). https://doi.org/10.1109/JBHI.2019.2960821

    Article  Google Scholar 

  27. Moon, W.K., Lee, Y.W., Ke, H.H., Lee, S.H., Huang, C.S., Chang, R.F.: Computer‐aided diagnosis of breast ultrasound images using ensemble learning from convolutional neural networks. Comput. Methods Programs Biomed. (2020). https://doi.org/10.1016/j.cmpb.2020.105361

    Article  PubMed  Google Scholar 

  28. Makandar, A., Halalli, B.: Pre-processing of mammography image for early detection of breast cancer. Int. J. Comput. Appl. (2016). https://doi.org/10.5120/ijca2016910153

    Article  Google Scholar 

  29. Makandar, A., Halalli, B.: Breast cancer image enhancement using median filter and CLAHE. Int. J. Sci. Eng. Res. (2015)

  30. Singh, P., Mukundan, R., De Ryke, R.: Feature enhancement in medical ultrasound videos using contrast-limited adaptive histogram equalization. J. Digit. Imaging. (2020). https://doi.org/10.1007/s10278-019-00211-5

    Article  PubMed  PubMed Central  Google Scholar 

  31. Byra, M., Galperin, M., Ojeda-Fournier, H., Olson, L., O’Boyle, M., Comstock, C., Andre, M.: Breast mass classification in sonography with transfer learning using a deep convolutional neural network and color conversion. Med. Phys. (2019). https://doi.org/10.1002/mp.13361

    Article  PubMed  PubMed Central  Google Scholar 

  32. Acharya, T.: Image processing: principles and applications. J. Electron. Imaging. (2006). https://doi.org/10.1117/1.2348895

    Article  Google Scholar 

  33. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. (1986). https://doi.org/10.1109/TPAMI.1986.4767851

    Article  PubMed  Google Scholar 

  34. Olivas, E.S., Guerrero, J.D.M., Martinez Sober, M., Magdalena Benedito, J.R., Serrano López, A.J.: Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques. (2009)

  35. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017. 2017-Janua, 2261–2269 (2017). https://doi.org/10.1109/CVPR.2017.243

  36. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-ResNet and the impact of residual connections on learning. In: 31st AAAI Conference on Artificial Intelligence, AAAI 2017 (2017)

  37. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2016-Decem, 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90

  38. Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., Le, Q. V.: Mnasnet: platform-aware neural architecture search for mobile. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2019)

  39. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  Google Scholar 

  40. Sarwinda, D., Paradisa, R.H., Bustamam, A., Anggia, P.: Deep learning in image classification using residual network (ResNet) variants for detection of colorectal cancer. Procedia Comput. Sci. 179, 423–431 (2021). https://doi.org/10.1016/j.procs.2021.01.025

    Article  Google Scholar 

  41. Jiao, Z., Gao, X., Wang, Y., Li, J.: A deep feature based framework for breast masses classification. Neurocomputing. (2016). https://doi.org/10.1016/j.neucom.2016.02.060

    Article  Google Scholar 

  42. Liu, S., Huang, D., Wang, Y.: Learning Spatial Fusion for Single-Shot Object Detection. (2019)

  43. Ojala, T., Pietikäinen, M., Harwood, D.: Performance evaluation of texture measures with classification based on Kullback discrimination of distributions. In: Proceedings - International Conference on Pattern Recognition (1994)

  44. Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. (2002). https://doi.org/10.1109/TPAMI.2002.1017623

    Article  Google Scholar 

  45. He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. (2009). https://doi.org/10.1109/TKDE.2008.239

    Article  Google Scholar 

  46. Van Der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. (2008)

  47. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis. (2020). https://doi.org/10.1007/s11263-019-01228-7

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Yanshan University, Qinhuangdao, 066004, China

    Hua Chen, Minglun Ma, Gang Liu, Zhihao Jin & Chong Liu

  2. The Second Hospital of Hebei Medical University, Shijiazhuang, 050000, China

    Ying Wang

Authors
  1. Hua Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minglun Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhihao Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Hua Chen and Gang Liu conceived the idea; material preparation, data collection, and analysis were performed by Minglun Ma, Ying Wang, Zhihao Jin, and Chong Liu. The first draft of the manuscript was written by Hua Chen, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Gang Liu.

Ethics declarations

Ethics Approval and Consent to Participate

This study was approved by the ethics committee of the relevant institutions. Informed consent was obtained from all individual participants included in the study.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, H., Ma, M., Liu, G. et al. Breast Tumor Classification in Ultrasound Images by Fusion of Deep Convolutional Neural Network and Shallow LBP Feature. J Digit Imaging 36, 932–946 (2023). https://doi.org/10.1007/s10278-022-00711-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-022-00711-x

Keywords

Search

Navigation