Skip to main content
SpringerLink
Log in

End-to-End Deep Learning Vector Autoregressive Prognostic Models to Predict Disease Progression with Uneven Time Intervals

  • Conference paper
  • First Online:
Medical Image Understanding and Analysis (MIUA 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12722))

Included in the following conference series:

Abstract

We propose an end-to-end deep learning method combining implicit feature extraction and an autoregressive model to predict the future course of a disease or condition. By merging the feature extraction and autoregression into one deep learning model, we can simultaneously train both models together. Our novel approach begins by fine-tuning a pretrained convolutional neural network to extract features from previously obtained images of patients. A trainable autoregression mechanism then predicts the features of the future image and a fully connected layer gives a prognosis based on the predicted features. We utilize a novel time interval scaling, allowing the model to account for uneven time intervals and allowing us to choose the final time point that we wish to predict. Experiments on the Age-Related Eye Disease Study give a testing area under the receiver operating characteristic curve, sensitivity, and specificity of 0.966 (95% CI: 0.947, 0.984), 0.878 (0.810, 0.945), and 0.930 (0.914, 0.947), respectively. This shows that the model can predict progression with good performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abadi, M., et al.: Tensorflow: Large-scale machine learning on heterogeneous systems (2015). Software available from tensorflow.org

    Google Scholar 

  2. Age-Related Eye Disease Study Research Group: The Age-Related Eye Disease Study (AREDS): design implications. AREDS report no. 1. Control. Clin. Trials 20(6), 573–600 (1999). https://doi.org/10.1016/s0197-2456(99)00031-8

  3. Arcadu, F., Benmansour, F., Maunz, A., Willis, J., Haskova, Z., Prunotto, M.: Deep learning algorithm predicts diabetic retinopathy progression in individual patients. NPJ Digit. Med. 2, 92 (2019). https://doi.org/10.1038/s41746-019-0172-3

    Article  Google Scholar 

  4. Arenja, N., et al.: Prognostic value of novel imaging parameters derived from standard cardiovascular magnetic resonance in high risk patients with systemic light chain amyloidosis. J. Cardiovasc. Magn. Reson. 21(1), 53 (2019). https://doi.org/10.1186/s12968-019-0564-1

    Article  Google Scholar 

  5. Babenko, B., et al.: Predicting progression of age-related macular degeneration from fundus images using deep learning (2019). arXiv preprint arXiv:1904.05478

  6. Banerjee, I., et al.: A deep-learning approach for prognosis of age-related macular degeneration disease using SD-OCT imaging biomarkers (2019). arXiv preprint arXiv:1902.10700

  7. Bridge, J., Harding, S., Zheng, Y.: Development and validation of a novel prognostic model for predicting amd progression using longitudinal fundus images. BMJ Open Ophthalmol. 5(1) (2020). https://doi.org/10.1136/bmjophth-2020-000569

  8. Brier, G.W.: Verification of forecasts expressed in terms of probability. Monthly Weather Rev. 78(1), 1–3 (1950)

    Article  Google Scholar 

  9. Choi, H., Jin, K.H.: Predicting cognitive decline with deep learning of brain metabolism and amyloid imaging. Behav. Brain Res. 344, 103–109 (2018). https://doi.org/10.1016/j.bbr.2018.02.017

    Article  Google Scholar 

  10. Chollet, F., et al.: Keras (2015). https://keras.io

  11. Cox, D.R.: Regression models and life-tables. J. Roy. Stat. Soc. Ser. B (Methodol.) 34(2), 187–220 (1972). www.jstor.org/stable/2985181

  12. DeLong, E.R., DeLong, D.M., Clarke-Pearson, D.L.: Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44(3), 837–845 (1988)

    Article  Google Scholar 

  13. Du, Z., Hao, Y.: reportROC: An Easy Way to Report ROC Analysis (2019). https://CRAN.R-project.org/package=reportROC, R package version 3.4

  14. Grassmann, F., et al.: A deep learning algorithm for prediction of age-related eye disease study severity scale for age-related macular degeneration from color fundus photography. Ophthalmology 125(9), 1410–1420 (2018). https://doi.org/10.1016/j.ophtha.2018.02.037

    Article  Google Scholar 

  15. Gregor, K., Danihelka, I., Mnih, A., Blundell, C., Wierstra, D.: Deep autoregressive networks (2013). arXiv preprint arXiv:1310.8499

  16. Harrell, F.E., Califf, R.M., Pryor, D.B., Lee, K.L., Rosati, R.A.: Evaluating the yield of medical tests. Jama 247(18), 2543–2546 (1982). https://jamanetwork.com/journals/jama/article-abstract/372568

  17. Hilario, A., et al.: A prognostic model based on preoperative MRI predicts overall survival in patients with diffuse gliomas. Am. J. Neuroradiol. 35(6), 1096 (2014). https://doi.org/10.3174/ajnr.A3837

    Article  Google Scholar 

  18. Keane, P.A., Patel, P.J., Liakopoulos, S., Heussen, F.M., Sadda, S.R., Tufail, A.: Evaluation of age-related macular degeneration with optical coherence tomography. Surv. Ophthalmol 57(5), 389–414 (2012). https://doi.org/10.1016/j.survophthal.2012.01.006

    Article  Google Scholar 

  19. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014). arXiv preprint arXiv:1412.6980

  20. Kundu, S., Aulchenko, Y.S., Janssens, A.C.J.: PredictABEL: Assessment of Risk Prediction Models (2014). https://CRAN.R-project.org/package=PredictABEL, R package version 1.2-2

  21. Leening, M.J., Vedder, M.M., Witteman, J.C., Pencina, M.J., Steyerberg, E.W.: Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician’s guide. Ann. Internal Med. 160(2), 122–131 (2014). https://doi.org/10.7326/m13-1522

    Article  Google Scholar 

  22. Owen, C.G., Jarrar, Z., Wormald, R., Cook, D.G., Fletcher, A.E., Rudnicka, A.R.: The estimated prevalence and incidence of late stage age related macular degeneration in the uk. Brit. J. Ophthalmol. 96(5), 752 (2012). https://doi.org/10.1136/bjophthalmol-2011-301109

    Article  Google Scholar 

  23. Priestley, M.: Non-linear and Non-stationary Time Series Analysis. Academic Press, Cambridge (1988)

    Google Scholar 

  24. R Core Team: R: A language and environment for statistical computing. R Foundation for Statistical Computing (2019). https://www.R-project.org/

  25. Robin, X., et al.: pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinf. 12(1), 77 (2011). https://doi.org/10.1186/1471-2105-12-77

    Article  Google Scholar 

  26. Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  27. de Sisternes, L., Simon, N., Tibshirani, R., Leng, T., Rubin, D.L.: Quantitative SD-OCT imaging biomarkers as indicators of age-related macular degeneration progression. Invest. Ophthalmol. Vision Sci. 55(11), 7093–103 (2014). https://doi.org/10.1167/iovs.14-14918

    Article  Google Scholar 

  28. Sjoberg, D.D.: dca: Decision Curve Analysis (2020). R package version 0.1.0.9000

    Google Scholar 

  29. Steyerberg, E.W., et al.: Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med. 10(2), e1001381–e1001381 (2013)

    Google Scholar 

  30. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826

    Google Scholar 

  31. Turkowski, K.: Filters for common resampling tasks (1990)

    Google Scholar 

  32. Uria, B., Côté, M.A., Gregor, K., Murray, I., Larochelle, H.: Neural autoregressive distribution estimation. J. Mach. Learn. Res. 17(1), 7184–7220 (2016)

    MathSciNet  MATH  Google Scholar 

  33. Van Rossum, G., Drake Jr, F.L.: Python tutorial. Centrum voor Wiskunde en Informatica Amsterdam (1995)

    Google Scholar 

  34. Vickers, A.J., Elkin, E.B.: Decision curve analysis: a novel method for evaluating prediction models. Med. Decis. Mak. 26(6), 565–574 (2006). https://doi.org/10.1177/0272989x06295361

    Article  Google Scholar 

  35. Williams, B.M., Burgess, P.I., Zheng, Y.: Chapter 13 - Drusen and macular degeneration, pp. 245–272. Academic Press, Cambridge (2019). https://doi.org/10.1016/B978-0-08-102816-2.00013-7

  36. Yan, Q., et al.: Deep-learning-based prediction of late age-related macular degeneration progression. medRxiv, p. 19006171 (2019). https://doi.org/10.1101/19006171

  37. Youden, W.J.: Index for rating diagnostic tests. Cancer 3(1), 32–35 (1950)

    Article  Google Scholar 

  38. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Eye and Vision Science, Institute of Life Course and Medical Sciences, University of Liverpool, Liverpool, UK

    Joshua Bridge, Simon Harding & Yalin Zheng

Authors
  1. Joshua Bridge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Harding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yalin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joshua Bridge .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Bartłomiej W. Papież

  2. Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates

    Mohammad Yaqub

  3. University of Oxford, Oxford, UK

    Jianbo Jiao

  4. University of Oxford, Oxford, UK

    Ana I. L. Namburete

  5. University of Oxford, Oxford, UK

    J. Alison Noble

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bridge, J., Harding, S., Zheng, Y. (2021). End-to-End Deep Learning Vector Autoregressive Prognostic Models to Predict Disease Progression with Uneven Time Intervals. In: Papież, B.W., Yaqub, M., Jiao, J., Namburete, A.I.L., Noble, J.A. (eds) Medical Image Understanding and Analysis. MIUA 2021. Lecture Notes in Computer Science(), vol 12722. Springer, Cham. https://doi.org/10.1007/978-3-030-80432-9_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-80432-9_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-80431-2

  • Online ISBN: 978-3-030-80432-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation