Skip to main content

Advertisement

Springer Nature Link
Log in

DMs-MAFM+EfficientNet: a hybrid model for predicting dysthyroid optic neuropathy

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Thyroid-associated ophthalmopathy (TAO) is a very common autoimmune orbital disease. Approximately 4%–8% of TAO patients will deteriorate and develop the most severe dysthyroid optic neuropathy (DON). According to the current data provided by clinical experts, there is still a certain proportion of suspected DON patients who cannot be diagnosed, and the clinical evaluation has low sensitivity and specificity. There is an urgent need for an efficient and accurate method to assist physicians in identifying DON. This study proposes a hybrid deep learning model to accurately identify suspected DON patients using computed tomography (CT). The hybrid model is mainly composed of the double multiscale and multi attention fusion module (DMs-MAFM) and a deep convolutional neural network. The DMs-MAFM is the feature extraction module proposed in this study, and it contains a multiscale feature fusion algorithm and improved channel attention and spatial attention, which can capture the features of tiny objects in the images. Multiscale feature fusion is combined with an attention mechanism to form a multilevel feature extraction module. The multiscale fusion algorithm can aggregate different receptive field features, and then fully obtain the channel and spatial correlation of the feature map through the multiscale channel attention aggregation module and spatial attention module, respectively. According to the experimental results, the hybrid model proposed in this study can accurately identify suspected DON patients, with Accuracy reaching 96%, Specificity reaching 99.5%, Sensitivity reaching 94%, Precision reaching 98.9% and F1-score reaching 96.4%. According to the evaluation by experts, the hybrid model proposed in this study has some enlightening significance for the diagnosis and prediction of clinically suspect DON.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Bartley GB (1994) The epidemiologic characteristics and clinical course of ophthalmopathy associated with autoimmune thyroid disease in olmsted county, minnesota. Trans Am Ophthalmol Soc 92(92):477

    CAS  PubMed  PubMed Central  Google Scholar 

  2. Neigel JM, Rootman J, Belkin RI, Nugent RA, Spinelli JA (1988) Dysthyroid optic neuropathy. the crowded orbital apex syndrome. Ophthalmology 95(11):1515–1521

    Article  CAS  Google Scholar 

  3. McKeag D, Lane C, Lazarus JH, Baldeschi L, Boboridis K, Dickinson AJ, Hullo AL, Kahaly G, Krassas G, Marcocci C et al (2007) Clinical features of dysthyroid optic neuropathy: a european group on graves’ orbitopathy (EUGOGO) survey. Br J Ophthalmol 91(4):455–458

  4. Saeed P, Rad ST, Peter HLT (2018) Bisschop: Dysthyroid optic neuropathy. Ophthalmic Plastic Reconstructive Surgery 34(4S):S60–S67

    Article  Google Scholar 

  5. Victores AJ, Takashima M (2016) Thyroid eye disease: optic neuropathy and orbital decompression. Int Ophthalmol Clin 56(1):69–79

    Article  Google Scholar 

  6. Jeon C, Shin JH, Woo KI, Kim Y-D (2012) Clinical profile and visual outcomes after treatment in patients with dysthyroid optic neuropathy. Korean J Ophthalmol 26(2):73–79

    Article  Google Scholar 

  7. Blandford AD, Zhang D, Chundury RV, Perry JD (2017) Dysthyroid optic neuropathy: update on pathogenesis, diagnosis, and management. Expert Review of Ophthalmology 12(2):111–121

    Article  CAS  Google Scholar 

  8. Giaconi JAA, Kazim M, Rho T, Pfaff C (2002) Ct scan evidence of dysthyroid optic neuropathy. Ophthalmic Plastic Reconstructive Surgery 18(3):177–182

    Article  Google Scholar 

  9. da Rocha Lima B, Perry JD (2013) Superior ophthalmic vein enlargement and increased muscle index in dysthyroid optic neuropathy. Ophthalmic Plastic Reconstructive Surgery 29(3):147–149

    Article  Google Scholar 

  10. Rong G, Mendez A, Assi EB, Bo Z, Sawan M (2020) Artificial intelligence in healthcare: review and prediction case studies. Engineering 6(3):291–301

    Article  Google Scholar 

  11. Vaid S, Kalantar R, Bhandari M (2020) Deep learning covid-19 detection bias: accuracy through artificial intelligence. Int Orthop 44(8):1539–1542

    Article  Google Scholar 

  12. Apostolopoulos ID, Mpesiana TA (2020) Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks. Physical and Engineering Sciences in Medicine 43(2):635–640

    Article  Google Scholar 

  13. Bakator M, Radosav D (2018) Deep learning and medical diagnosis: A review of literature. Multimodal Technologies and Interaction 2(3):47

    Article  Google Scholar 

  14. Ravi D, Wong C, Deligianni F, Berthelot M, Andreu-Perez J, Lo B, Yang GZ (2017) Deep learning for health informatics. IEEE Journal of Biomedical Health Informatics 21(1):4–21

    Article  Google Scholar 

  15. Ozturk T, Talo M, Yildirim EA, Baloglu UB, Yildirim O, Rajendra Acharya U (2020) Automated detection of covid-19 cases using deep neural networks with x-ray images. Comput Biol Med 121:103792

    Article  CAS  Google Scholar 

  16. Lin C, Song X, Li L, Li Y, Jiang M, Sun R, Zhou H, Fan X (2021) Detection of active and inactive phases of thyroid-associated ophthalmopathy using deep convolutional neural network. BMC Ophthalmol 21(1):1–9

    Article  CAS  Google Scholar 

  17. Alom Z, Rahman MM, Nasrin S, Taha TM, Asari VK (2020) Covid_mtnet: Covid-19 detection with multi-task deep learning approaches. arXiv:2004.03747

  18. Wang D, Khosla A, Gargeya R, Irshad H, Beck AH (2016) Deep learning for identifying metastatic breast cancer. arXiv:1606.05718

  19. Xuan G, Xia X, Zhu W, Zhang B, Doermann D, Zhuo L (2021) Deformable gabor feature networks for biomedical image classification. In Proceedings of the IEEE/CVF Winter Conf Appl Comp Vision pp 4004–4012

  20. Wei J, Suriawinata A, Ren B, Liu X, Lisovsky M, Vaickus L, Brown C, Baker M, Nasir-Moin M, Tomita N, Torresani L (2021) Learn like a pathologist: curriculum learning by annotator agreement for histopathology image classification. In Proc IEEE/CVF Winter Conf Appl Compt Vis pp 2473–2483

  21. Zhang J, Liu A, Gao M, Chen X, Xu Z, Chen X (2020) Ecg-based multi-class arrhythmia detection using spatio-temporal attention-based convolutional recurrent neural network. Artif Intell Med 106:101856

    Article  Google Scholar 

  22. Liu L, Kurgan L, Wu F-X, Wang J (2020) Attention convolutional neural network for accurate segmentation and quantification of lesions in ischemic stroke disease. Med Image Anal 65:101791

    Article  Google Scholar 

  23. Zhang B, Tan J, Cho K, Chang G, Deniz CM (2020) Attention-based cnn for kl grade classification: data from the osteoarthritis initiative. In2020 IEEE 17th international symposium on biomedical imaging (ISBI). IEEE, pp 731–735

  24. Hu H, Li Q, Zhao Y, Ye Z (2020) Parallel deep learning algorithms with hybrid attention mechanism for image segmentation of lung tumors. IEEE Trans Indust Inform 17(4):2880–2889

    Article  Google Scholar 

  25. He A, Li T, Li N, Wang K, Fu H (2020) Cabnet: category attention block for imbalanced diabetic retinopathy grading. IEEE Trans Med Imaging 40(1):143–153

    Article  Google Scholar 

  26. Yushkevich PA, Gerig G (2017) Itk-snap: an intractive medical image segmentation tool to meet the need for expert-guided segmentation of complex medical images. IEEE pulse 8(4):54–57

    Article  Google Scholar 

  27. Siddhartha M, Santra A (2020) Covidlite: A depth-wise separable deep neural network with white balance and clahe for detection of covid-19. arXiv:2006.13873

  28. Yu W, Yang K, Yao H, Sun X, Xu P (2017) Exploiting the complementary strengths of multi-layer cnn features for image retrieval. Neurocomputing 237:235–241

    Article  Google Scholar 

  29. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In Proc IEEE Conf Comput Vis Pattern Recognit pp 7132–7141

  30. Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) ECA-Net: efficient channel attention for deep convolutional neural networks. In Proc IEEE/CVF Conf Comp Vis Pattern Recognit pp 13–19

  31. Woo S, Park J, Lee JY, Kweon IS (2018) Cbam: Convolutional block attention module. Springer, Cham

    Google Scholar 

  32. Tan M, Le Q (2019) Efficientnet: rethinking model scaling for convolutional neural networks. In Int Conf Machine Learning. PMLR, pp 6105–6114

  33. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In Proc IEEE Conf Comput Vis Pattern Recognit pp 4510–4520

  34. Kobak D, Berens P (2019) The art of using t-sne for single-cell transcriptomics. Nat Commun 10(1):1–14

    Article  CAS  Google Scholar 

  35. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  36. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Pro IEEE Conf Comput Vis Pattern Recognit pp 770– 778

  37. Park J, Woo S, Lee J-Y, Kweon IS (2018) Bam: Bottleneck attention module. arXiv:1807.06514

  38. Qin Z, Zhang P, Wu P,  Li X (2021) Fcanet: frequency channel attention networks. In Proc IEEE/CVF Int Conf Comput Vis pp 783–792

  39. Huang Z, Wang X, Huang L, Huang C, Wei Y, Liu W (2019) Ccnet: criss-cross attention for semantic segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pp 603–612

  40. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In Proc IEEE Conference Computer Vis Pattern Recognit pp 1–9

  41. Cao Y, Xu Y, Lin S, Wei F, Hu H (2019) Gcnet: Non-local networks meet squeeze-excitation networks and beyond. In Proc IEEE/CVF Int Conf Comput Vis Workshops

  42. Hou Q, Zhou D, Feng J (2021) Coordinate attention for efficient mobile network design. In Proc IEEE/CVF Conference Computer Vis Pattern Recognit pp 13713–13722

  43. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI Conf Artif Intell

Download references

Acknowledgements

Thanks to Union Hospital Tongji Medical College Huazhong University of Science and Technology for providing data and medical background support for this study.

Author information

Authors and Affiliations

  1. School of Computer Science, Hubei University of Technology, Nanli Street 28, Wuhan, 430068, China

    Cong Wu, Shijun Li & Xiao Liu

  2. Union Hosptial Tongji Medical College Huazhong University of Science and Technology, Zhongshan Park, Wuhan, 430022, China

    Fagang Jiang & Bingjie Shi

Authors
  1. Cong Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Shijun Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xiao Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Fagang Jiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Bingjie Shi
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Shijun Li conceived and practiced the study, and Cong Wu put forward relevant guidance, Xiao Liu and Bingjie Shi collected and analyzed the data set, Professor Fagang Jiang provided medical theoretical support for this study by analyzing the actual situation, and all the authors contributed to the writing and approval of this study.

Corresponding author

Correspondence to Cong Wu.

Ethics declarations

Ethics approval

This study met the requirements of The Code of Ethics of the World Medical Association, and the data were used with the consent of the hospital and the patients themselves.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, C., Li, S., Liu, X. et al. DMs-MAFM+EfficientNet: a hybrid model for predicting dysthyroid optic neuropathy. Med Biol Eng Comput 60, 3217–3230 (2022). https://doi.org/10.1007/s11517-022-02663-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02663-4

Keywords