Abstract
Macular edema is a type of eye disease commonly associated with diabetes and treated with three injections namely Ranibizumab, Aflibercept, and Bevacizumab. Often, the prescription is majorly dependent on the decision of a physician supported by optical coherence tomography (OCT) images that capture the progress or failure of treatments. Nevertheless, comparative clinical effectiveness of these treatments for the management of macular edema remains unclear. Additionally, each of these treatments are driven by cost-to-patient, as well as medical side effects. Thus, there is a high demand for artificial intelligence (AI)-driven decision assistance to avoid ineffective treatments and unnecessary costs. Therefore, this work aims at proposing a machine learning (ML)-based decision support system for physicians. The results illustrate that the transformer-based model is capable of predicting if the respective treatment will have a positive, or negative impact for the patient with an accuracy higher than a skillful physician (acc of 89.07). In addition, we have shown that the combination of metadata from patient clinical records with OCT images shows 12% increase in accuracy as compared to the models trained only using OCT images. In conclusion, the proposed ML model provides comparative effectiveness probabilities for the three types of injections which can assist physicians in macular edema treatment after conducting proper follow-up and clinical trial studies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alryalat, S.A.: Deep learning prediction of response to anti-vegf among diabetic macular edema patients: Treatment response analyzer system (tras). Diagnostics 12(2) (2022)
Bates, D.W., Singh, H.: Two decades since to err is human: an assessment of progress and emerging priorities in patient safety. Health Aff. 37(11), 1736–1743 (2018)
Breiman, L.: Random forests. Machine Learning Springer 45(1), 5–32 (2001)
Bryan, M., Heyer, G., Philipp, N., Rehak, M., Wiedemann, P.: Convolutional attention on images for locating macular edema. In: Zheng, Y., Williams, B.M., Chen, K. (eds.) MIUA 2019. CCIS, vol. 1065, pp. 391–398. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-39343-4_33
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Li, F.F.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE CCVPR, pp. 248–255. IEEE (2009)
Djulbegovic, B., et al.: How do physicians decide to treat: an empirical evaluation of the threshold model. BMC Med. Inform. Decis. Mak. 14(1), 1–10 (2014)
Donaldson, M.S., Corrigan, J.M., Kohn, L.T., et al.: To err is human: building a safer health system (2000)
Fang, L., Wang, C., Li, S., Rabbani, H., Chen, X., Liu, Z.: Attention to lesion: Lesion-aware convolutional neural network for retinal optical coherence tomography image classification. IEEE Trans. Med. Imaging 38(8), 1959–1970 (2019)
Gessert, N., Nielsen, M., Shaikh, M., Werner, R., Schlaefer, A.: Skin lesion classification using ensembles of multi-resolution efficientnets with meta data. MethodsX 7, 100864 (2020)
Gorgi Zadeh, S., et al.: CNNs enable accurate and fast segmentation of drusen in optical coherence tomography. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 65–73. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_8
Guan, Q., et al.: Deep convolutional neural network vgg-16 model for differential diagnosing of papillary thyroid carcinomas in cytological images: a pilot study. J. Cancer 10(20), 4876 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR, abs/ arXiv: 1512.03385 (2015)
Honavar, S.G.: To err is human, but errors can be prevented. Indian J. Ophthalmol. 67(10), 1517 (2019)
Hwang, D.-K., et al.: Artificial intelligence-based decision-making for age-related macular degeneration. Theranostics 9, 232–245 (2019)
Lu, W., Tong, Y., Yu, Y., Xing, Y., Chen, C., She, Y.: Deep learning-based automated classification of multi-categorical abnormalities from optical coherence tomography images. Translational Vis. Sci. Technol. 7(6), 41–41 (2018)
Moazemi, S., et al.: Decision-support for treatment with 177 Lu-PSMA: machine learning predicts response with high accuracy based on PSMA-PET/CT and clinical parameters. Annal. Translational Med. 9(9), 818–818 (2021)
Moazemi, S., Essler, M., Schultz, T., Bundschuh, R.A.: Predicting treatment response in prostate cancer patients based on multimodal pet/ct for clinical decision support. In: Syeda-Mahmood, T., et al. (eds.) ML-CDS 2021. LNCS, vol. 13050, pp. 22–35. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-89847-2_3
Morais, C., Yung, K.L., Johnson, K., Moura, R., Beer, M., Patelli, E.: Identification of human errors and influencing factors: A machine learning approach. Safety Sci. 146, 105528 (2022)
Prahs, P., et al.: Oct-based deep learning algorithm for the evaluation of treatment indication with anti-vascular endothelial growth factor medications. Graefes Arch. Clin. Exp. Ophthalmol. 256(1), 91–98 (2018)
Prati, F., et al.: Angiography alone versus angiography plus optical coherence tomography to guide decision-making during percutaneous coronary intervention: the centro per la lotta contro l’infarto-optimisation of percutaneous coronary intervention (cli-opci) study. EuroIntervention 8(7), 823–829 (2012)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: Mobilenetv 2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)
Tan, M., Le, Q.V.: Efficientnet: Rethinking model scaling for convolutional neural networks. CoRR, abs/ arXiv: 1905.11946 (2019)
Tomkins-Netzer, O., Niederer, R., Lightman, S.: The role of statins in diabetic retinopathy. Trends Cardiovascular Med. (2022). ISSN 1050–1738
Acknowledgement
We acknowledge the support of the IntelliLung (10105743) a HORIZON-HLTH-2021-DISEASE EU project, E-Vita (101016453) EU project, and the ScaDS.AI competence centre (IS18026A-F) for providing high performance computing clusters for evaluations.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Vahdati, S., Nagaraj, D.B., Bryan, M., Moazemi, S., Gründer-Fahrer, S., Martin, M. (2023). Utilizing Transformers on OCT Imagery and Metadata for Treatment Response Prediction in Macular Edema Patients. In: Gade, R., Felsberg, M., Kämäräinen, JK. (eds) Image Analysis. SCIA 2023. Lecture Notes in Computer Science, vol 13886. Springer, Cham. https://doi.org/10.1007/978-3-031-31438-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-31438-4_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31437-7
Online ISBN: 978-3-031-31438-4
eBook Packages: Computer ScienceComputer Science (R0)
