Abstract
The early screening and treatment of diabetic retinopathy (DR) and diabetic macular edema (DME) can prevent the risk of blindness for most diabetic patients. Joint grading of DR and DME can reduce the screening cost and improve screening efficiency. Most grading methods only focus on grading one disease and lack generality. In this paper, we propose an adaptive attention block (AAB) to better extract features and improve model performance, which can be adaptively adjusted according to different grading tasks. We propose the AABNet to handle multiple grading tasks, consisting of a lightweight architecture MobileNetv2 as a backbone for feature extraction and two independent branches, the DR branch and the DME branch, for grading prediction. To alleviate the lack of labeled data, we propose an adaptive teacher-student model as a semisupervised learning method to train the AABNet with additional unlabeled data, dependent on fewer model parameters, whose foundation is consistency regularization, which is added to the loss to provide additional supervised signals for training to obtain a more accurate model without destroying the image details and strictly requiring the quality of unlabeled images. Extensive experiments are conducted, including performance comparisons with state-of-the-art methods and ablation studies of attention mechanisms and semisupervised learning mechanisms. Experiments verify that AABNet can bring a satisfactory improvement in the multiple DR grading, DME grading and joint DR&DME grading tasks. AABNet outperforms other state-of-the-art methods and achieves better results in multiple grading tasks on the Messidor and DDR datasets.








Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Dogan S, Barua PD, Kutlu H, Baygin M, Fujita H, Tuncer T, Acharya UR (2022) Automated accurate fire detection system using ensemble pretrained residual network. Expert Syst Appl 203:117407
Tuncer T, Ertam F, Dogan S, Aydemir E, Pławiak P (2020) Ensemble residual network-based gender and activity recognition method with signals. J Supercomputing 76(3):2119–2138
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929
Tolstikhin IO, Houlsby N, Kolesnikov A, Beyer L, Zhai X, Unterthiner T, Yung J, Steiner A, Keysers D, Uszkoreit J et al (2021) Mlp-mixer: an all-mlp architecture for vision. Adv Neural Inf Process Syst 34:24261–24272
Karadal CH, Kaya MC, Tuncer T, Dogan S, Acharya UR (2021) Automated classification of remote sensing images using multileveled mobilenetv2 and dwt techniques. Expert Syst Appl 185:115659
Din MS, Gurbuz S, Akbal E, Dogan S, Durak Ma, Yildirim IO, Tuncer T (2022) Exemplar deep and hand-modeled features based automate and accurate cerebral hemorrhage classification method. Med Eng Phys, pp 103819
Frazao LB, Theera-Umpon N, Auephanwiriyakul S (2019) Diagnosis of diabetic retinopathy based on holistic texture and local retinal features. Inf Sci 475:44–66
Al-Bander B, Al-Nuaimy W, Al-Taee MA, Williams BM, Zheng Y (2016) Diabetic macular edema grading based on deep neural networks, pp 121–128
Li T, Gao Y, Wang K, Guo S, Liu H, Kang H (2019) Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening. Inf Sci 501:511–522
He A, Li T, Li N, Wang K, Fu H (2020) Cabnet: category attention block for imbalanced diabetic retinopathy grading. IEEE Trans Med Imaging 40(1):143–153
Li X, Hu X, Yu L, Zhu L, Fu C-W, Heng P-A (2019) Canet: cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading. IEEE Trans Med Imaging 39(5):1483–1493
Bai W, Oktay O, Sinclair M, Suzuki H, Rajchl M, Tarroni G, Glocker B, King A, Matthews PM, Rueckert D (2017) Semi-supervised learning for network-based cardiac mr image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 253–260
Laibacher T, Anjos A (2019) On the evaluation and real-world usage scenarios of deep vessel segmentation for funduscopy. Technical report, Idiap
Cui W, Liu Y, Li Y, Guo M, Li Y, Li X, Wang T, Zeng X, Ye C (2019) Semi-supervised brain lesion segmentation with an adapted mean teacher model. In: International conference on information processing in medical imaging. Springer, pp 554–565
Zhao Z, Zhang X, Chen C, Li W, Peng S, Wang J, Yang X, Zhang L, Zeng Z (2019) Semi-supervised self-taught deep learning for finger bones segmentation. In: 2019 IEEE EMBS international conference on biomedical & health informatics (BHI). IEEE, pp 1–4
Quellec G, Charrière K, Boudi Y, Cochener B, Lamard M (2017) Deep image mining for diabetic retinopathy screening. Med Image Anal 39:178–193
Antony M, Brggemann S (2015) Kaggle diabetic retinopathy detection team o O solution. Competition report Github. https://github.com/sveitser/kaggle_diabetic.... Accessed Sept 2020
Esfahani MT, Ghaderi M, Kafiyeh R (2018) Classification of diabetic and normal fundus images using new deep learning method. Leonardo Electron. J. Pract. Technol 17:233–248
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778
Zhang W, Zhong J, Yang S, Gao Z, Hu J, Chen Y, Yi Z (2019) Automated identification and grading system of diabetic retinopathy using deep neural networks. Knowl-Based Syst 175:12–25
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826
Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251– 1258
Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligence
Wang Z, Yin Y, Shi J, Fang W, Li H, Wang X (2017) Zoom-in-net: deep mining lesions for diabetic retinopathy detection. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 267–275
Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316(22):2402–2410
Jiang H, Yang K, Gao M, Zhang D, Ma H, Qian W (2019) An interpretable ensemble deep learning model for diabetic retinopathy disease classification. In: 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 2045–2048
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141
Mo J, Zhang L, Feng Y (2018) Exudate-based diabetic macular edema recognition in retinal images using cascaded deep residual networks. Neurocomputing 290:161–171
Tarvainen A, Valpola H (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Adv Neural Inform Process Syst, p 30
Xie Q, Dai Z, Hovy E, Luong T, Le Q (2020) Unsupervised data augmentation for consistency training. Adv Neural Inf Process Syst 33:6256–6268
Berthelot D, Carlini N, Goodfellow I, Papernot N, Oliver A, Raffel CA (2019) Mixmatch: a holistic approach to semi-supervised learning. Adv Neural Inform Process Syst, p 32
Sohn K, Berthelot D, Carlini N, Zhang Z, Zhang H, Raffel CA, Cubuk ED, Kurakin A, Li C -L (2020) Fixmatch: simplifying semi-supervised learning with consistency and confidence. Adv Neural Inform Process Syst 33:596–608
Zhang B, Wang Y, Hou W, Wu H, Wang J, Okumura M, Shinozaki T (2021) Flexmatch: boosting semi-supervised learning with curriculum pseudo labeling. Adv Neural Inf Process Syst 34:18408–18419
Woo S, Park J, Lee J -Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 3–19
Decencière E, Zhang X, Cazuguel G, Lay B, Cochener B, Trone C, Gain P, Ordonez R, Massin P, Erginay A, Charton B, Klein J-C (2014) Feedback on a publicly distributed database: the messidor database. Image Anal Stereology 33(3):231–234. https://doi.org/10.5566/ias.1155
Kaggle dataset (2015) https://kaggle.com/c/diabetic-retinopat. Accessed Sept 2020
APTOS dataset (2019) https://www.kaggle.com/c/aptos2019-blindness-detection/overviewhttps://www.kaggle.com/c/aptos2019-blindness-detection/overview. Accessed Sept 2020
Decencière E, Zhang X, Cazuguel G, Lay B, Cochener B, Trone C, Gain P, Ordonez R, Massin P, Erginay A et al (2014) Feedback on a publicly distributed image database: the messidor database. Image Anal Stereology 33(3):231–234
Shi L, Kusko R, Wolfinger RD, Haibe-Kains B, Fischer M, Sansone S-A, Mason CE, Furlanello C, Jones WD, Ning B et al (2017) The international maqc society launches to enhance reproducibility of high-throughput technologies. Nat Biotechnol 35(12):1127–1128
Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520
Pires R, Avila S, Jelinek HF, Wainer J, Valle E, Rocha A (2015) Beyond lesion-based diabetic retinopathy: a direct approach for referral. IEEE J Biomed Health Informa 21(1):193–200
Vo HH, Verma A (2016) New deep neural nets for fine-grained diabetic retinopathy recognition on hybrid color space. In: 2016 IEEE international symposium on multimedia (ISM). IEEE, pp 209–215
Seoud L, Hurtut T, Chelbi J, Cheriet F, Langlois JP (2015) Red lesion detection using dynamic shape features for diabetic retinopathy screening. IEEE Trans Med Imaging 35(4):1116–1126
Kobat SG, Baygin N, Yusufoglu E, Baygin M, Barua PD, Dogan S, Yaman O, Celiker U, Yildirim H, Tan R-S et al (2022) Automated diabetic retinopathy detection using horizontal and vertical patch division-based pre-trained densenet with digital fundus images. Diagnostics 12(8):1975
Tomaszewski JE (2021) Overview of the role of artificial intelligence in pathology: the computer as a pathology digital assistant. In: Artificial intelligence and deep learning in pathology. Elsevier, pp 237–262
Acknowledgments
This work was supported by the National Natural Science Foundation of China under Grant 82071995, the Key Research and Development Program of Jilin Province under Grant 20220201141GX and the Natural Science Foundation of Jilin Province under Grant 20200201292JC.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no conflicts of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Guo, X., Li, X., Lin, Q. et al. Joint grading of diabetic retinopathy and diabetic macular edema using an adaptive attention block and semisupervised learning. Appl Intell 53, 16797–16812 (2023). https://doi.org/10.1007/s10489-022-04295-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-022-04295-5