Skip to main content

Continuous-Time Deep Glioma Growth Models

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 (MICCAI 2021)

Abstract

The ability to estimate how a tumor might evolve in the future could have tremendous clinical benefits, from improved treatment decisions to better dose distribution in radiation therapy. Recent work has approached the glioma growth modeling problem via deep learning and variational inference, thus learning growth dynamics entirely from a real patient data distribution. So far, this approach was constrained to predefined image acquisition intervals and sequences of fixed length, which limits its applicability in more realistic scenarios. We overcome these limitations by extending Neural Processes, a class of conditional generative models for stochastic time series, with a hierarchical multi-scale representation encoding including a spatio-temporal attention mechanism. The result is a learned growth model that can be conditioned on an arbitrary number of observations, and that can produce a distribution of temporally consistent growth trajectories on a continuous time axis. On a dataset of 379 patients, the approach successfully captures both global and finer-grained variations in the images, exhibiting superior performance compared to other learned growth models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Image domain refers to the fact that the observations are entire images. The original Neural Processes [5, 6] work on images by treating individual pixels as observations.

References

  1. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805 [cs] (2018)

  2. Drozdzal, M., Vorontsov, E., Chartrand, G., Kadoury, S., Pal, C.: The importance of skip connections in biomedical image segmentation. In: Carneiro, G., et al. (eds.) LABELS/DLMIA -2016. LNCS, vol. 10008, pp. 179–187. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46976-8_19

    Chapter  Google Scholar 

  3. Eslami, S.M.A., et al.: Neural scene representation and rendering. Science 360(6394), 1204–1210 (2018)

    Article  Google Scholar 

  4. Ezhov, I., et al.: Neural parameters estimation for brain tumor growth modeling. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 787–795. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_87

    Chapter  Google Scholar 

  5. Garnelo, M., et al.: Conditional neural processes. In: International Conference on Machine Learning, pp. 1704–1713 (2018)

    Google Scholar 

  6. Garnelo, M., et al.: Neural processes. In: International Conference on Machine Learning – Workshop on Theoretical Foundations and Applications of Deep Generative Models (2018)

    Google Scholar 

  7. Higgins, I., et al.: \(\beta \)-VAE: learning basic visual concepts with a constrained variational framework. In: International Conference on Learning Representations (2017)

    Google Scholar 

  8. Kia, S., Marquand, A.: Neural processes mixed-effect models for deep normative modeling of clinical neuroimaging data. In: International Conference on Medical Imaging with Deep Learning (MIDL)(2019)

    Google Scholar 

  9. Kickingereder, P., et al.: Automated quantitative tumour response assessment of MRI in neuro-oncology with artificial neural networks: a multicentre, retrospective study. Lancet Oncol. 20(5), 728–740 (2019)

    Article  Google Scholar 

  10. Kim, H., et al.: Attentive neural processes. In: International Conference on Learning Representations (2019)

    Google Scholar 

  11. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)

    Google Scholar 

  12. Kitaev, N., Kaiser, L., Levskaya, A.: Reformer: the efficient transformer. In: International Conference on Learning Representations (2020)

    Google Scholar 

  13. Kumar, A., et al.: Consistent jumpy predictions for videos and scenes. In: Advances in Neural Information Processing Systems – Bayesian Deep Learning Workshop (2018)

    Google Scholar 

  14. Lipková, J., et al.: Personalized radiotherapy design for glioblastoma: integrating mathematical tumor models, multimodal scans, and Bayesian inference. IEEE Trans. Med. Imaging 38(8), 1875–1884 (2019)

    Article  MathSciNet  Google Scholar 

  15. Lê, M., et al.: Personalized radiotherapy planning based on a computational tumor growth model. IEEE Trans. Med. Imaging 36(3), 815–825 (2017)

    Article  Google Scholar 

  16. Mang, A., Bakas, S., Subramanian, S., Davatzikos, C., Biros, G.: Integrated biophysical modeling and image analysis: application to neuro-oncology. Annu. Rev. Biomed. Eng. 22(1), 309–341 (2020)

    Article  Google Scholar 

  17. Menze, B.H., Stretton, E., Konukoglu, E., Ayache, N.: Image-based modeling of tumor growth in patients with glioma. Technical report (2011)

    Google Scholar 

  18. Petersen, J., et al.: Deep probabilistic modeling of glioma growth. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 806–814. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_89

    Chapter  Google Scholar 

  19. Rosenbaum, D., Besse, F., Viola, F., Rezende, D.J., Eslami, S.M.A.: Learning models for visual 3D localization with implicit mapping. In: Advances in Neural Information Processing Systems - Bayesian Deep Learning Workshop (2018)

    Google Scholar 

  20. Subramanian, S., Scheufele, K., Himthani, N., Biros, G.: Multiatlas calibration of biophysical brain tumor growth models with mass effect. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 551–560. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_53

    Chapter  Google Scholar 

  21. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

    Google Scholar 

  22. Wang, S., Li, B.Z., Khabsa, M., Fang, H., Ma, H.: Linformer: self-attention with linear complexity. arXiv:2006.04768 [cs, stat] (2020)

  23. Wick, W., et al.: Lomustine and bevacizumab in progressive glioblastoma. N. Engl. J. Med. 377(20), 1954–1963 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

Part of this work was funded by the Helmholtz Imaging Platform (HIP), a platform of the Helmholtz Incubator on Information and Data Science.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jens Petersen .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Petersen, J. et al. (2021). Continuous-Time Deep Glioma Growth Models. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12903. Springer, Cham. https://doi.org/10.1007/978-3-030-87199-4_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87199-4_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87198-7

  • Online ISBN: 978-3-030-87199-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics