Abstract
Image monitoring and guidance during medical examinations can aid both diagnosis and treatment. However, the sampling frequency is often too low, which creates a need to estimate the missing images. We present a probabilistic motion model for sequential medical images, with the ability to both estimate motion between acquired images and forecast the motion ahead of time. The core is a low-dimensional temporal process based on a linear Gaussian state-space model with analytically tractable solutions for forecasting, simulation, and imputation of missing samples. The results, from two experiments on publicly available cardiac datasets, show reliable motion estimates and an improved forecasting performance using patient-specific adaptation by online learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Angelini, E.D., Laine, A.F., Takuma, S., et al.: LV volume quantification via spatiotemporal analysis of real-time 3-D echocardiography. IEEE Transactions on Medical Imaging 20(6), 457–469 (2001)
Arsigny, V., Commowick, O., Pennec, X., Ayache, N.: A log-euclidean framework for statistics on diffeomorphisms. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI: 9th International Conference, Copenhagen, Denmark, October. Proceedings, Part I 9. Springer (2006)
Åström, K.J., Murray, R.: Feedback systems: an introduction for scientists and engineers. Princeton university press (2021)
Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C.: Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical image analysis 12(1), 26–41 (2008)
Avants, B.B., Tustison, N., Song, G., et al.: Advanced normalization tools (ANTS). Insight j 2(365), 1–35 (2009)
Beg, M.F., Miller, M.I., Trouvé, A., Younes, L.: Computing large deformation metric mappings via geodesic flows of diffeomorphisms. International journal of computer vision 61, 139–157 (2005)
Bernard, O., Lalande, A., Zotti, C., et al.: Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Transactions on Medical Imaging 37(11), 2514–2525 (2018)
Chen, C., Qin, C., Qiu, H., et al.: Deep learning for cardiac image segmentation: a review. Frontiers in Cardiovascular Medicine 7, 25 (2020)
Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Medical image analysis 57, 226–236 (2019)
Fraccaro, M., Kamronn, S., Paquet, U., Winther, O.: A disentangled recognition and nonlinear dynamics model for unsupervised learning. Advances in neural information processing systems 30 (2017)
Gu, A., Goel, K., Ré, C.: Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396 (2021)
Gunnarsson, N., Sjölund, J., Kimstrand, P., Schön, T.B.: Unsupervised dynamic modeling of medical image transformations. In: 2022 25th International Conference on Information Fusion (FUSION). pp. 01–07. IEEE (2022)
Jöhl, A., Ehrbar, S., Guckenberger, M., et al.: Performance comparison of prediction filters for respiratory motion tracking in radiotherapy. Medical physics 47(2), 643–650 (2020)
Kalman, R.E.: A new approach to linear filtering and prediction problems. Journal of basic Engineering 82(1), 35–45 (1960)
Keall, P.J., Brighi, C., Glide-Hurst, C., et al.: Integrated MRI-guided radiotherapy-opportunities and challenges. Nature Reviews Clinical Oncology 19(7), 458–470 (2022)
Krebs, J., Delingette, H., Ayache, N., Mansi, T.: Learning a generative motion model from image sequences based on a latent motion matrix. IEEE Transactions on Medical Imaging 40(5), 1405–1416 (2021)
Krebs, J., Delingette, H., Mailhé, B., et al.: Learning a probabilistic model for diffeomorphic registration. IEEE Transactions on Medical Imaging 38(9), 2165–2176 (2019)
Lombardo, E., Dhont, J., Page, D., et al.: Real-time motion management in MRI-guided radiotherapy: Current status and AI-enabled prospects. Radiotherapy and Oncology p. 109970 (2023)
Lombardo, E., Rabe, M., Xiong, Y., et al.: Offline and online LSTM networks for respiratory motion prediction in MR-guided radiotherapy. Physics in Medicine & Biology 67(9), 095006 (2022)
Mattingley, J., Boyd, S.: Real-Time Convex Optimization in Signal Processing. IEEE Signal Processing Magazine 27(3), 50–61 (2010)
Modersitzki, J.: Numerical methods for image registration. OUP Oxford (2003)
Oktay, O., Schlemper, J., Folgoc, L.L., et al.: Attention U-Net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 (2018)
Ouyang, D., He, B., Ghorbani, A., et al.: Video-based AI for beat-to-beat assessment of cardiac function. Nature 580(7802), 252–256 (2020)
Paganelli, C., Whelan, B., Peroni, M., et al.: MRI-guidance for motion management in external beam radiotherapy: current status and future challenges. Physics in Medicine & Biology 63(22), 22TR03 (2018)
Raaymakers, B.W., Lagendijk, J., Overweg, J., et al.: Integrating a 1.5 T MRI scanner with a 6 MV accelerator: proof of concept. Physics in Medicine & Biology 54(12), N229 (2009)
Rauch, H.E., Tung, F., Striebel, C.T.: Maximum likelihood estimates of linear dynamic systems. AIAA journal 3(8), 1445–1450 (1965)
Romaguera, L.V., Mezheritsky, T., Mansour, R., et al.: Probabilistic 4D predictive model from in-room surrogates using conditional generative networks for image-guided radiotherapy. Medical image analysis 74, 102250 (2021)
Romaguera, L.V., Plantefève, R., Romero, F.P., et al.: Prediction of in-plane organ deformation during free-breathing radiotherapy via discriminative spatial transformer networks. Medical image analysis 64, 101754 (2020)
Sharp, G.C., Jiang, S.B., Shimizu, S., Shirato, H.: Prediction of respiratory tumour motion for real-time image-guided radiotherapy. Physics in Medicine & Biology 49(3), 425 (2004)
Ye, M., Yang, D., Huang, Q., et al.: SequenceMorph: A Unified Unsupervised Learning Framework for Motion Tracking on Cardiac Image Sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(08), 10409–10426 (2023)
Acknowledgments
This research was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by Knut and Alice Wallenberg Foundation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gunnarsson, N., Sjölund, J., Kimstrand, P., Schön, T.B. (2024). Online Learning in Motion Modeling for Intra-interventional Image Sequences. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15002. Springer, Cham. https://doi.org/10.1007/978-3-031-72069-7_66
Download citation
DOI: https://doi.org/10.1007/978-3-031-72069-7_66
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72068-0
Online ISBN: 978-3-031-72069-7
eBook Packages: Computer ScienceComputer Science (R0)
