Skip to main content

Advertisement

Log in

Joint grading of diabetic retinopathy and diabetic macular edema using an adaptive attention block and semisupervised learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The early screening and treatment of diabetic retinopathy (DR) and diabetic macular edema (DME) can prevent the risk of blindness for most diabetic patients. Joint grading of DR and DME can reduce the screening cost and improve screening efficiency. Most grading methods only focus on grading one disease and lack generality. In this paper, we propose an adaptive attention block (AAB) to better extract features and improve model performance, which can be adaptively adjusted according to different grading tasks. We propose the AABNet to handle multiple grading tasks, consisting of a lightweight architecture MobileNetv2 as a backbone for feature extraction and two independent branches, the DR branch and the DME branch, for grading prediction. To alleviate the lack of labeled data, we propose an adaptive teacher-student model as a semisupervised learning method to train the AABNet with additional unlabeled data, dependent on fewer model parameters, whose foundation is consistency regularization, which is added to the loss to provide additional supervised signals for training to obtain a more accurate model without destroying the image details and strictly requiring the quality of unlabeled images. Extensive experiments are conducted, including performance comparisons with state-of-the-art methods and ablation studies of attention mechanisms and semisupervised learning mechanisms. Experiments verify that AABNet can bring a satisfactory improvement in the multiple DR grading, DME grading and joint DR&DME grading tasks. AABNet outperforms other state-of-the-art methods and achieves better results in multiple grading tasks on the Messidor and DDR datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Dogan S, Barua PD, Kutlu H, Baygin M, Fujita H, Tuncer T, Acharya UR (2022) Automated accurate fire detection system using ensemble pretrained residual network. Expert Syst Appl 203:117407

    Article  Google Scholar 

  2. Tuncer T, Ertam F, Dogan S, Aydemir E, Pławiak P (2020) Ensemble residual network-based gender and activity recognition method with signals. J Supercomputing 76(3):2119–2138

    Article  Google Scholar 

  3. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929

  4. Tolstikhin IO, Houlsby N, Kolesnikov A, Beyer L, Zhai X, Unterthiner T, Yung J, Steiner A, Keysers D, Uszkoreit J et al (2021) Mlp-mixer: an all-mlp architecture for vision. Adv Neural Inf Process Syst 34:24261–24272

    Google Scholar 

  5. Karadal CH, Kaya MC, Tuncer T, Dogan S, Acharya UR (2021) Automated classification of remote sensing images using multileveled mobilenetv2 and dwt techniques. Expert Syst Appl 185:115659

    Article  Google Scholar 

  6. Din MS, Gurbuz S, Akbal E, Dogan S, Durak Ma, Yildirim IO, Tuncer T (2022) Exemplar deep and hand-modeled features based automate and accurate cerebral hemorrhage classification method. Med Eng Phys, pp 103819

  7. Frazao LB, Theera-Umpon N, Auephanwiriyakul S (2019) Diagnosis of diabetic retinopathy based on holistic texture and local retinal features. Inf Sci 475:44–66

    Article  Google Scholar 

  8. Al-Bander B, Al-Nuaimy W, Al-Taee MA, Williams BM, Zheng Y (2016) Diabetic macular edema grading based on deep neural networks, pp 121–128

  9. Li T, Gao Y, Wang K, Guo S, Liu H, Kang H (2019) Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening. Inf Sci 501:511–522

    Article  Google Scholar 

  10. He A, Li T, Li N, Wang K, Fu H (2020) Cabnet: category attention block for imbalanced diabetic retinopathy grading. IEEE Trans Med Imaging 40(1):143–153

    Article  Google Scholar 

  11. Li X, Hu X, Yu L, Zhu L, Fu C-W, Heng P-A (2019) Canet: cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading. IEEE Trans Med Imaging 39(5):1483–1493

    Article  Google Scholar 

  12. Bai W, Oktay O, Sinclair M, Suzuki H, Rajchl M, Tarroni G, Glocker B, King A, Matthews PM, Rueckert D (2017) Semi-supervised learning for network-based cardiac mr image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 253–260

  13. Laibacher T, Anjos A (2019) On the evaluation and real-world usage scenarios of deep vessel segmentation for funduscopy. Technical report, Idiap

  14. Cui W, Liu Y, Li Y, Guo M, Li Y, Li X, Wang T, Zeng X, Ye C (2019) Semi-supervised brain lesion segmentation with an adapted mean teacher model. In: International conference on information processing in medical imaging. Springer, pp 554–565

  15. Zhao Z, Zhang X, Chen C, Li W, Peng S, Wang J, Yang X, Zhang L, Zeng Z (2019) Semi-supervised self-taught deep learning for finger bones segmentation. In: 2019 IEEE EMBS international conference on biomedical & health informatics (BHI). IEEE, pp 1–4

  16. Quellec G, Charrière K, Boudi Y, Cochener B, Lamard M (2017) Deep image mining for diabetic retinopathy screening. Med Image Anal 39:178–193

    Article  Google Scholar 

  17. Antony M, Brggemann S (2015) Kaggle diabetic retinopathy detection team o O solution. Competition report Github. https://github.com/sveitser/kaggle_diabetic.... Accessed Sept 2020

  18. Esfahani MT, Ghaderi M, Kafiyeh R (2018) Classification of diabetic and normal fundus images using new deep learning method. Leonardo Electron. J. Pract. Technol 17:233–248

    Google Scholar 

  19. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778

  20. Zhang W, Zhong J, Yang S, Gao Z, Hu J, Chen Y, Yi Z (2019) Automated identification and grading system of diabetic retinopathy using deep neural networks. Knowl-Based Syst 175:12–25

    Article  Google Scholar 

  21. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  22. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  23. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251– 1258

  24. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligence

  25. Wang Z, Yin Y, Shi J, Fang W, Li H, Wang X (2017) Zoom-in-net: deep mining lesions for diabetic retinopathy detection. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 267–275

  26. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316(22):2402–2410

    Article  Google Scholar 

  27. Jiang H, Yang K, Gao M, Zhang D, Ma H, Qian W (2019) An interpretable ensemble deep learning model for diabetic retinopathy disease classification. In: 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 2045–2048

  28. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  29. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  30. Mo J, Zhang L, Feng Y (2018) Exudate-based diabetic macular edema recognition in retinal images using cascaded deep residual networks. Neurocomputing 290:161–171

    Article  Google Scholar 

  31. Tarvainen A, Valpola H (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Adv Neural Inform Process Syst, p 30

  32. Xie Q, Dai Z, Hovy E, Luong T, Le Q (2020) Unsupervised data augmentation for consistency training. Adv Neural Inf Process Syst 33:6256–6268

  33. Berthelot D, Carlini N, Goodfellow I, Papernot N, Oliver A, Raffel CA (2019) Mixmatch: a holistic approach to semi-supervised learning. Adv Neural Inform Process Syst, p 32

  34. Sohn K, Berthelot D, Carlini N, Zhang Z, Zhang H, Raffel CA, Cubuk ED, Kurakin A, Li C -L (2020) Fixmatch: simplifying semi-supervised learning with consistency and confidence. Adv Neural Inform Process Syst 33:596–608

    Google Scholar 

  35. Zhang B, Wang Y, Hou W, Wu H, Wang J, Okumura M, Shinozaki T (2021) Flexmatch: boosting semi-supervised learning with curriculum pseudo labeling. Adv Neural Inf Process Syst 34:18408–18419

    Google Scholar 

  36. Woo S, Park J, Lee J -Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 3–19

  37. Decencière E, Zhang X, Cazuguel G, Lay B, Cochener B, Trone C, Gain P, Ordonez R, Massin P, Erginay A, Charton B, Klein J-C (2014) Feedback on a publicly distributed database: the messidor database. Image Anal Stereology 33(3):231–234. https://doi.org/10.5566/ias.1155

    Article  MATH  Google Scholar 

  38. Kaggle dataset (2015) https://kaggle.com/c/diabetic-retinopat. Accessed Sept 2020

  39. APTOS dataset (2019) https://www.kaggle.com/c/aptos2019-blindness-detection/overviewhttps://www.kaggle.com/c/aptos2019-blindness-detection/overview. Accessed Sept 2020

  40. Decencière E, Zhang X, Cazuguel G, Lay B, Cochener B, Trone C, Gain P, Ordonez R, Massin P, Erginay A et al (2014) Feedback on a publicly distributed image database: the messidor database. Image Anal Stereology 33(3):231–234

    Article  MATH  Google Scholar 

  41. Shi L, Kusko R, Wolfinger RD, Haibe-Kains B, Fischer M, Sansone S-A, Mason CE, Furlanello C, Jones WD, Ning B et al (2017) The international maqc society launches to enhance reproducibility of high-throughput technologies. Nat Biotechnol 35(12):1127–1128

    Article  Google Scholar 

  42. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch

  43. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520

  44. Pires R, Avila S, Jelinek HF, Wainer J, Valle E, Rocha A (2015) Beyond lesion-based diabetic retinopathy: a direct approach for referral. IEEE J Biomed Health Informa 21(1):193–200

    Article  Google Scholar 

  45. Vo HH, Verma A (2016) New deep neural nets for fine-grained diabetic retinopathy recognition on hybrid color space. In: 2016 IEEE international symposium on multimedia (ISM). IEEE, pp 209–215

  46. Seoud L, Hurtut T, Chelbi J, Cheriet F, Langlois JP (2015) Red lesion detection using dynamic shape features for diabetic retinopathy screening. IEEE Trans Med Imaging 35(4):1116–1126

    Article  Google Scholar 

  47. Kobat SG, Baygin N, Yusufoglu E, Baygin M, Barua PD, Dogan S, Yaman O, Celiker U, Yildirim H, Tan R-S et al (2022) Automated diabetic retinopathy detection using horizontal and vertical patch division-based pre-trained densenet with digital fundus images. Diagnostics 12(8):1975

    Article  Google Scholar 

  48. Tomaszewski JE (2021) Overview of the role of artificial intelligence in pathology: the computer as a pathology digital assistant. In: Artificial intelligence and deep learning in pathology. Elsevier, pp 237–262

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant 82071995, the Key Research and Development Program of Jilin Province under Grant 20220201141GX and the Natural Science Foundation of Jilin Province under Grant 20200201292JC.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xiaoxin Guo.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflicts of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, X., Li, X., Lin, Q. et al. Joint grading of diabetic retinopathy and diabetic macular edema using an adaptive attention block and semisupervised learning. Appl Intell 53, 16797–16812 (2023). https://doi.org/10.1007/s10489-022-04295-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04295-5

Keywords

Navigation