Skip to main content

Advertisement

SpringerLink
Log in

Parameter Transfer Deep Neural Network for Single-Modal B-Mode Ultrasound-Based Computer-Aided Diagnosis

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Elastography ultrasound (EUS) imaging has shown its effectiveness for diagnosis of tumors by providing additional information about tissue stiffness to the conventional B-mode ultrasound (BUS). However, due to the lack of EUS devices and experienced sonologists, EUS is not widely used, especially in rural areas. It is still a challenging task to improve the performance of the single-modal BUS-based computer-aided diagnosis (CAD) for tumors. In this work, we propose a novel transfer learning (TL)–based deep neural network (DNN) algorithm, named CW-PM-DNN, for the BUS-based CAD by transferring diagnosis knowledge from EUS during model training. CW-PM-DNN integrates both the feature-level and classifier-level knowledge transfer into a unified framework. In the feature-level TL, a bichannel DNN is learned by the cross-weight-based multimodal DL (MDL-CW) algorithm to transfer informative features from EUS to BUS. In the classifier-level TL, a projective model (PM)–based classifier is then embedded to the pretrained bichannel DNN to implement the parameter transfer in the classifier model at the second stage. The back-propagation procedure is then applied to optimize the whole CW-PM-DNN to further improve its performance. Experimental results on two bimodal ultrasound tumor datasets demonstrate that the proposed CW-PM-DNN achieves the best classification accuracy, sensitivity, and specificity of 89.02 ± 1.54%, 88.37 ± 4.72%, and 89.63 ± 4.06%, respectively, for the breast ultrasound dataset, and the corresponding values of 80.57 ± 3.41%, 76.67 ± 3.85%, and 83.94 ± 3.95%, respectively, for the prostate ultrasound dataset. The proposed two-stage TL-based CW-PM-DNN algorithm outperforms all the compared algorithms. It is also proved that the performance of the BUS-based CAD can be significantly improved by transferring the knowledge of EUS. It suggests that CW-PM-DNN has the potential for more applications in the field of medical image–based CAD.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Mitterberger M, Horningera W, Aignerb F, Pinggeraa G, Steppanb I, Rehdera P, et al. Ultrasound of the prostate. Cancer Imaging. 2010;10:40–8.

    Google Scholar 

  2. Hernaez R, Lazo M, Bonekamp S, Kamel I, Brancati FL, Guallar E, et al. Diagnostic accuracy and reliability of ultrasonography for the detection of fatty liver: a meta-analysis. Hepatology. 2011;54:1082–90.

    Google Scholar 

  3. Huang Q, Zhang F, Li X. Machine learning in ultrasound computer-aided diagnostic systems: a survey. Biomed Res Int. 2018;2018:1–10.

    Google Scholar 

  4. Liu S, Wang Y, Yang X, Lei B, Liu L, Li S, et al. Deep learning in medical ultrasound analysis: a review. Engineering. 2019;5:261–75.

    Google Scholar 

  5. Sigrist R, Liau J, Kaffas A, Chammas M, Willmann J. Ultrasound elastography: review of techniques and clinical applications. Theranostics. 2017;7:1303–29.

    Google Scholar 

  6. Gatos I, Tsantis S, Spiliopoulos S. A new computer aided diagnosis system for evaluation of chronic liver disease with ultrasound shear wave elastography imaging. Med Phys. 2016;43:1428–36.

    Google Scholar 

  7. Zhang Q, Xiao Y, Dai W, Suo J, Wang C, Shi J, et al. Deep learning based classification of breast tumors with shear-wave elastography. Ultrasonics. 2016;72:150–7.

    Google Scholar 

  8. Ara S, Alam F, Rahman M, Akhter S, Awwal R, Hasan M. Bimodal multiparameter-based approach for benign-malignant classification of breast tumors. Ultrasound Med Biol. 2015;41:2022–38.

    Google Scholar 

  9. Zhang Q, Song S, Xiao Y, Chen S, Shi J, Zhen H. Dual-modal artificially intelligent diagnosis of breast tumors on both shear-wave elastography and B-mode ultrasound using deep polynomial networks. Med Eng Phys. 2019;64:1–6.

    Google Scholar 

  10. Robicquet A, Ramsundar B, Kuleshov V, Depristo M, Chou K, Cui C, et al. A guide to deep learning in healthcare. Nat Med. 2019;25:24–9.

    Google Scholar 

  11. Shi J, Xue Z, Dai Y, Peng B, Dong Y, Zhang Q, et al. Cascaded multi-column RVFL+ classifier for single-modal neuroimaging-based diagnosis of Parkinson’s disease. IEEE Trans Biomed Eng. 2019;66:2362–71.

    Google Scholar 

  12. Pan S, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng. 2010;22:1345–59.

    Google Scholar 

  13. Weiss K, Khoshgoftaar T, Wang D. A survey of transfer learning. J Big Data. 2016;3:9.

    Google Scholar 

  14. Cheplygina V, Bruijne M, Pluim J. Not-so-supervised a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med Image Anal. 2019;54:280–96.

    Google Scholar 

  15. Chao C, Jiang B. Parameter transfer extreme learning machine based on projective model. In: International Joint Conference on Neural Network; 2018. p. 1–8.

    Google Scholar 

  16. Huang G, Zhu Q, Siew C. Extreme learning machine: theory and applications. Neurocomputing. 2006;70:489–501.

    Google Scholar 

  17. Huang G, Huang G, Song S, You K. Trends in extreme learning machines: a review. Neural Netw. 2015;61:32–48.

    MATH  Google Scholar 

  18. Xia J, Chen H, Li Q, Zhou M, Chen L, Cai Z, et al. Ultrasound-based differentiation of malignant and benign thyroid nodules: an extreme learning machine approach. Comput Methods Prog Biomed. 2017;147:37–49.

    Google Scholar 

  19. Salaken S, Khosravi A, Nguyen T, Nahavandi S. Extreme learning machine based transfer learning algorithms: a survey. Neurocomputing. 2017;267:516–24.

    Google Scholar 

  20. Zhou S, Shi J, Zhu J, Cai Y, Wang R. Shearlet-based texture feature extraction for classification of breast tumor in ultrasound image. Biomed Signal Process Control. 2013;8:688–96.

    Google Scholar 

  21. Shi J, Zheng X, Wu J, Gong B, Zhang Q, Ying S. Quaternion grassmann average network for learning representation of histopathological image. Pattern Recogn. 2019;89:67–76.

    Google Scholar 

  22. Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw. 2015;61:85–117.

    Google Scholar 

  23. Tang J, Deng C, Huang G. Extreme learning machine for multilayer perceptron. IEEE Trans Neur Net Learn Syst. 2015;27:809–21.

    MathSciNet  Google Scholar 

  24. Tan C, Sun F, Kong T, Zhang W, Yang C, Liu C. A survey on deep transfer learning. In: International Conference on Artificial Neural Networks; 2018. p. 270–9.

    Google Scholar 

  25. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Computer Vision and Pattern Recognition; 2014.

    Google Scholar 

  26. Krizhevsky A, Sutskever I, Hinton G. ImageNet classification with deep convolutional neural networks. Neural Inf Process Syst. 2012;141:1097–105.

    Google Scholar 

  27. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: Computer Vision and Pattern Recognition; 2015. p. 1–9.

    Google Scholar 

  28. Long M, Cao Y, Wang J, Jordan M. Learning transferable features with deep adaptation networks. In: International Conference on Machine Learning; 2015. p. 97–105.

    Google Scholar 

  29. Rastegar S, Soleymani M, Rabiee H, Shojaee S. MDL-CW: a multimodal deep learning framework with cross weights. In: Computer Vision and Pattern Recognition; 2016. p. 2601–9.

    Google Scholar 

  30. Ramachandram D, Taylor G. Deep multimodal learning: a survey on recent advances and trends. IEEE Signal Proc Mag. 2017;34:96–108.

    Google Scholar 

  31. Guo W, Wang J, Wang S. Deep multimodal representation learning: a survey. IEEE Access. 2019;7:63373–94.

    Google Scholar 

  32. Tommasi T, Orabona F, Caputo B. Learning categories from few examples with multi model knowledge transfer. IEEE Trans Pattern Anal Mach Intell. 2014;36:928–41.

    Google Scholar 

  33. Ding C, Zhou D, He X, Zha H. R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In: International Conference on Machine Learning; 2006. p. 281–8.

    Google Scholar 

  34. Nemat H, Fehri H, Ahmadinejad N, Frangi A, Gooya A. Classification of breast lesions in ultrasonography using sparse logistic regression and morphology-based texture features. Med Phys. 2018;45:4112–24.

    Google Scholar 

  35. Huang Q, Huang Y, Luo Y, Yuan F, Li X. Segmentation of breast ultrasound image with semantic classification of superpixels. Med Image Anal. 2020;61:101657.

    Google Scholar 

  36. Zhang Q, Xiao Y, Suo J, Shi J, Yu J, Guo Y, et al. Sonoelastomics for breast tumor classification: a radiomics approach with clustering-based feature selection on sonoelastography. Ultrasound Med Biol. 2017;43:1058–69.

    Google Scholar 

  37. Livni R, Shalevshwartz S, Shamir O. An algorithm for training polynomial networks. arXiv: Learning. 2013.

    Google Scholar 

  38. Li W, Duan L, Xu D, Tsang I. Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation. IEEE Trans Pattern Anal Mach Intell. 2014;36:1134–48.

    Google Scholar 

  39. Zhang L, Zhang D. Robust visual knowledge transfer via extreme learning machine based domain adaptation. IEEE Trans Image Process. 2016;25:4959–73.

    MathSciNet  MATH  Google Scholar 

  40. Zhang P. A new learning paradigm for random vector functional-link network: RVFL+. Neural Netw. 2020;122:94–105.

    Google Scholar 

  41. Sohn K, Shang W, Lee H. Improved multimodal deep learning with variation of information. In: Neural Information Processing Systems; 2014. p. 2141–9.

    Google Scholar 

  42. Li Y, Meng F, Shi J. Learning using privileged information improves neuroimaging-based CAD of Alzheimer’s disease: a comparative study. Med Biol Eng Comput. 2019;57(7):1605–16.

    Google Scholar 

  43. Andrew G, Arora R, Bilmes J, Livescu K. Deep canonical correlation analysis. In: International Conference on Machine Learning; 2013. p. 1247–55.

    Google Scholar 

  44. Huang Q, Zhang F, Li X. Machine learning in ultrasound computer-aided diagnostic systems: a survey. Biomed Res Int. 2018;2018:1–10.

    Google Scholar 

  45. Huang Q, Chen Y, Liu L, Tao D, Li X. On combining biclustering mining and adaBoost for breast tumor classification. IEEE Trans Knowl Data Eng. 2020;32(4):728–38.

    Google Scholar 

  46. Huang Q, Hu B, Zhang F. Evolutionary optimized fuzzy reasoning with mined diagnostic patterns for classification of breast tumors in ultrasound. Inf Sci. 2019;502:525–36.

    Google Scholar 

Download references

Funding

This work is supported by the National Natural Science Foundation of China (81830058, 81627804), the Science and Technology Commission of Shanghai Municipality (17411953400, 18010500600, and18411967400), the 111 Project (D20031), and the Nanjing Science and Technology Planning Project (201803027).

Author information

Authors and Affiliations

  1. Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Joint International Research Laboratory of Specialty Fiber Optics and Advanced Communication, School of Communication and Information Engineering, Shanghai University, Shanghai, China

    Xiaoyan Fei, Lu Shen, Qi Zhang & Jun Shi

  2. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai, China

    Xiaoyan Fei, Qi Zhang & Jun Shi

  3. Department of Mathematics, School of Science, Shanghai University, Shanghai, China

    Shihui Ying

  4. Department of Ultrasound, Huashan Hospital, Fudan University, Shanghai, China

    Yehua Cai

  5. Department of Ultrasound, The Affiliated Drum Tower Hospital of Nanjing University Medical School, Nanjing, China

    Wentao Kong & Weijun Zhou

  6. Department of Ultrasound, The First Affiliated Hospital of Anhui Medical University, No. 218, Jixi Road, Hefei, China

    Weijun Zhou

Authors
  1. Xiaoyan Fei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lu Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shihui Ying
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yehua Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wentao Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Weijun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jun Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Weijun Zhou or Jun Shi.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fei, X., Shen, L., Ying, S. et al. Parameter Transfer Deep Neural Network for Single-Modal B-Mode Ultrasound-Based Computer-Aided Diagnosis. Cogn Comput 12, 1252–1264 (2020). https://doi.org/10.1007/s12559-020-09761-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-020-09761-1

Keywords

Search

Navigation