Abstract
The ability to estimate how a tumor might evolve in the future could have tremendous clinical benefits, from improved treatment decisions to better dose distribution in radiation therapy. Recent work has approached the glioma growth modeling problem via deep learning and variational inference, thus learning growth dynamics entirely from a real patient data distribution. So far, this approach was constrained to predefined image acquisition intervals and sequences of fixed length, which limits its applicability in more realistic scenarios. We overcome these limitations by extending Neural Processes, a class of conditional generative models for stochastic time series, with a hierarchical multi-scale representation encoding including a spatio-temporal attention mechanism. The result is a learned growth model that can be conditioned on an arbitrary number of observations, and that can produce a distribution of temporally consistent growth trajectories on a continuous time axis. On a dataset of 379 patients, the approach successfully captures both global and finer-grained variations in the images, exhibiting superior performance compared to other learned growth models.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805 [cs] (2018)
Drozdzal, M., Vorontsov, E., Chartrand, G., Kadoury, S., Pal, C.: The importance of skip connections in biomedical image segmentation. In: Carneiro, G., et al. (eds.) LABELS/DLMIA -2016. LNCS, vol. 10008, pp. 179–187. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46976-8_19
Eslami, S.M.A., et al.: Neural scene representation and rendering. Science 360(6394), 1204–1210 (2018)
Ezhov, I., et al.: Neural parameters estimation for brain tumor growth modeling. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 787–795. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_87
Garnelo, M., et al.: Conditional neural processes. In: International Conference on Machine Learning, pp. 1704–1713 (2018)
Garnelo, M., et al.: Neural processes. In: International Conference on Machine Learning – Workshop on Theoretical Foundations and Applications of Deep Generative Models (2018)
Higgins, I., et al.: \(\beta \)-VAE: learning basic visual concepts with a constrained variational framework. In: International Conference on Learning Representations (2017)
Kia, S., Marquand, A.: Neural processes mixed-effect models for deep normative modeling of clinical neuroimaging data. In: International Conference on Medical Imaging with Deep Learning (MIDL)(2019)
Kickingereder, P., et al.: Automated quantitative tumour response assessment of MRI in neuro-oncology with artificial neural networks: a multicentre, retrospective study. Lancet Oncol. 20(5), 728–740 (2019)
Kim, H., et al.: Attentive neural processes. In: International Conference on Learning Representations (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)
Kitaev, N., Kaiser, L., Levskaya, A.: Reformer: the efficient transformer. In: International Conference on Learning Representations (2020)
Kumar, A., et al.: Consistent jumpy predictions for videos and scenes. In: Advances in Neural Information Processing Systems – Bayesian Deep Learning Workshop (2018)
Lipková, J., et al.: Personalized radiotherapy design for glioblastoma: integrating mathematical tumor models, multimodal scans, and Bayesian inference. IEEE Trans. Med. Imaging 38(8), 1875–1884 (2019)
Lê, M., et al.: Personalized radiotherapy planning based on a computational tumor growth model. IEEE Trans. Med. Imaging 36(3), 815–825 (2017)
Mang, A., Bakas, S., Subramanian, S., Davatzikos, C., Biros, G.: Integrated biophysical modeling and image analysis: application to neuro-oncology. Annu. Rev. Biomed. Eng. 22(1), 309–341 (2020)
Menze, B.H., Stretton, E., Konukoglu, E., Ayache, N.: Image-based modeling of tumor growth in patients with glioma. Technical report (2011)
Petersen, J., et al.: Deep probabilistic modeling of glioma growth. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 806–814. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_89
Rosenbaum, D., Besse, F., Viola, F., Rezende, D.J., Eslami, S.M.A.: Learning models for visual 3D localization with implicit mapping. In: Advances in Neural Information Processing Systems - Bayesian Deep Learning Workshop (2018)
Subramanian, S., Scheufele, K., Himthani, N., Biros, G.: Multiatlas calibration of biophysical brain tumor growth models with mass effect. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 551–560. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_53
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wang, S., Li, B.Z., Khabsa, M., Fang, H., Ma, H.: Linformer: self-attention with linear complexity. arXiv:2006.04768 [cs, stat] (2020)
Wick, W., et al.: Lomustine and bevacizumab in progressive glioblastoma. N. Engl. J. Med. 377(20), 1954–1963 (2017)
Acknowledgements
Part of this work was funded by the Helmholtz Imaging Platform (HIP), a platform of the Helmholtz Incubator on Information and Data Science.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Petersen, J. et al. (2021). Continuous-Time Deep Glioma Growth Models. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12903. Springer, Cham. https://doi.org/10.1007/978-3-030-87199-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-87199-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87198-7
Online ISBN: 978-3-030-87199-4
eBook Packages: Computer ScienceComputer Science (R0)