Abstract
MRI-guided radiotherapy systems enable real-time 2D cine acquisitions for target monitoring, but cannot provide volumetric information due to spatio-temporal constraints. Hence, respiratory motion models coupled with a temporal predictive mechanism are a suitable solution to enable ahead-of-time 3D tumor and anatomy tracking in combination with real-time online plan adaptation. We propose a novel subject-specific probabilistic model to enable 3D+t predictions from image-based surrogates during radiotherapy treatments. The model is trained end-to-end to simultaneously capture and learn a distribution of realistic motion fields over a population dataset. Furthermore, the distribution is conditioned on a sequence of partial observations, which can be extrapolated in time using a \( {seq2seq}\)-inspired mechanism allowing for scalable predictive horizon. Based on the generative properties of conditional variational autoencoders, it integrates anatomical features and temporal information to construct an interpretable latent space with respiratory phase discrimination. The choice of a probabilistic framework allows improving uncertainty estimation during the volume generation phase. Experimental validation on 25 subjects demonstrates the potential of the proposed model, which achieves a mean landmark error of \(1.4 \pm 1.1\) mm, yielding statistically significant improvements over state-of-the-art methods.
Supported by NSERC research grant (CRDPJ-517413-17) and by the Canada First Research Excellence Fund through the TransMedTech Institute.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdi, A.H., Pesteie, M., Prisman, E., Abolmaesumi, P., Fels, S.: Variational shape completion for virtual planning of jaw reconstructive surgery. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 227–235. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_26
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019)
Ballas, N., Yao, L., Pal, C., Courville, A.C.: Delving deeper into convolutional networks for learning video representations. In: ICLR (Poster) (2016)
Ehrhardt, J., Lorenz, C., et al.: 4D Modeling and Estimation of Respiratory Motion for Radiation Therapy. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36441-9
Fayad, H.J., Buerger, C., Tsoumpas, C., Cheze-Le-Rest, C., Visvikis, D.: A generic respiratory motion model based on 4D MRI imaging and 2D image navigators. In: 2012 IEEE Nuclear Science Symposium and Medical Imaging Conference Record (NSS/MIC), pp. 4058–4061. IEEE (2012)
Garau, N., et al.: A ROI-based global motion model established on 4DCT and 2D cine-MRI data for MRI-guidance in radiation therapy. Phys. Med. Biol. 64(4), 045002 (2019)
Giger, A., et al.: Respiratory motion modelling using cGANs. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 81–88. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_10
Giger, A.T., et al.: Liver-ultrasound based motion modelling to estimate 4D dose distributions for lung tumours in scanned proton therapy. Phys. Med. Biol. 65(23), 235050 (2020)
Harris, W., Yin, F.F., Cai, J., Ren, L.: Volumetric cine magnetic resonance imaging (VC-MRI) using motion modeling, free-form deformation and multi-slice undersampled 2D cine MRI reconstructed with spatio-temporal low-rank decomposition. Quant. Imaging Med. Surg. 10(2), 432 (2020)
Henke, L., et al.: Phase I trial of stereotactic MR-guided online adaptive radiation therapy (SMART) for the treatment of oligometastatic or unresectable primary malignancies of the abdomen. Radiother. Oncol. 126(3), 519–526 (2018)
Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: Advances in Neural Information Processing Systems, pp. 2017–2025 (2015)
Jud, C., Preiswerk, F., Cattin, P.C.: Respiratory motion compensation with topology independent surrogates. In: Workshop on Imaging and Computer Assistance in Radiation Therapy (2015)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. CoRR arXiv:1312.6114 (2013)
Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.: elastix: A toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29(1), 196–205 (2009)
McClelland, J.R., Hawkes, D.J., Schaeffter, T., King, A.P.: Respiratory motion models: a review. Med. Image Anal. 17(1), 19–42 (2013)
Mezheritsky, T., Romaguera, L.V., Kadoury, S.: 3D ultrasound generation from partial 2D observations using fully convolutional and spatial transformation networks. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 1808–1811. IEEE (2020)
Paganelli, C., et al.: Time-resolved volumetric MRI in MRI-guided radiotherapy: an in silico comparative analysis. Phys. Med. Biol. 64(18), 185013 (2019)
Paganelli, C., et al.: Feasibility study on 3D image reconstruction from 2D orthogonal cine-MRI for MRI-guided radiotherapy. J. Med. Imaging Radiat. Oncol. 62(3), 389–400 (2018)
Pham, J., Harris, W., Sun, W., Yang, Z., Yin, F.F., Ren, L.: Predicting real-time 3D deformation field maps (DFM) based on volumetric cine MRI (VC-MRI) and artificial neural networks for on-board 4D target tracking: a feasibility study. Phys. Med. Biol. 64(16), 165016 (2019)
Preiswerk, F., et al.: Model-guided respiratory organ motion prediction of the liver from 2D ultrasound. Med. Image Anal. 18(5), 740–751 (2014)
Qin, C., et al.: Joint learning of motion estimation and segmentation for cardiac MR image sequences. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 472–480. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_53
Romaguera, L.V., Plantefève, R., Romero, F.P., Hébert, F., Carrier, J.F., Kadoury, S.: Prediction of in-plane organ deformation during free-breathing radiotherapy via discriminative spatial transformer networks. Med. Image Anal. 64, 101754 (2020)
Seregni, M., Paganelli, C., Kipritidis, J., Baroni, G., Riboldi, M.: Out-of-plane motion correction in orthogonal cine-MRI registration. Radiother. Oncol. 123, S147–S148 (2017)
von Siebenthal, M., Szekely, G., Gamper, U., Boesiger, P., Lomax, A., Cattin, P.: 4D MR imaging of respiratory organ motion and its variability. Phys. Med. Biol. 52(6), 1547 (2007)
Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. In: Advances in Neural Information Processing Systems, pp. 3483–3491 (2015)
Stemkens, B., Tijssen, R.H., De Senneville, B.D., Lagendijk, J.J., Van Den Berg, C.A.: Image-driven, model-based 3D abdominal motion estimation for MR-guided radiotherapy. Phys. Med. Biol. 61(14), 5335 (2016)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. arXiv preprint arXiv:1409.3215 (2014)
Tanner, C., et al.: In vivo validation of spatio-temporal liver motion prediction from motion tracked on MR thermometry images. Int. J. Comput. Assist. Radiol. Surg. 11(6), 1143–1152 (2016)
Zhang, Y., Yin, F.F., Pan, T., Vergalasova, I., Ren, L.: Preliminary clinical evaluation of a 4D-CBCT estimation technique using prior information and limited-angle projections. Radiother. Oncol. 115(1), 22–29 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Vázquez Romaguera, L., Mezheritsky, T., Kadoury, S. (2021). Personalized Respiratory Motion Model Using Conditional Generative Networks for MR-Guided Radiotherapy. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12904. Springer, Cham. https://doi.org/10.1007/978-3-030-87202-1_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-87202-1_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87201-4
Online ISBN: 978-3-030-87202-1
eBook Packages: Computer ScienceComputer Science (R0)
