Skip to main content
SpringerLink
Log in

A multi-view assisted registration network for MRI registration pre- and post-therapy

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Image registration of magnetic resonance imaging (MRI) pre- and post-therapy is an important part of evaluating the effect of therapy in tumor patients. The accuracy of evaluation results heavily relies on the alignment of the MRI image after registration. Although recent advancements have been made in medical image registration, applying these methods to MRI registration pre- and post-therapy remains challenging. Existing methods typically utilize single-view data for registration. However, when applied to MRI data where some slices are clear while others are blurred, these methods can be misled by erroneous spatial information in the blurred regions, leading to poor registration outcomes. To mitigate the interference caused by erroneous spatial information in single-view data, this paper proposes a multi-stream fusion-assisted registration network that incorporates different-view MRIs of the same patient at the same site. Additionally, a cross-attention guided fusion module is designed within the network to effectively utilize accurate spatial information from multi-view data. The proposed approach was evaluated on clinical data, and the experimental results demonstrated that incorporating multiple view data as auxiliary information significantly enhances the accuracy of MRI image registration before and after radiotherapy.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Thirion J (1998) Image matching as a diffusion process: an analogy with Maxwell’s demons. Medical Image Anal 2(3):243–260. https://doi.org/10.1016/S1361-8415(98)80022-4

    Article  CAS  Google Scholar 

  2. Klein S, Staring M, Pluim JPW (2007) Evaluation of optimization methods for nonrigid medical image registration using mutual information and b-splines. IEEE Trans Image Process 16(12):2879–2890. https://doi.org/10.1109/TIP.2007.909412

    Article  PubMed  Google Scholar 

  3. Klein S, Staring M, Murphy K, Viergever MA, Pluim JPW (2010) elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging 29(1):196–205

    Article  PubMed  Google Scholar 

  4. Beg MF, Miller MI, Trouvé A, Younes L (2005) Computing large deformation metric mappings via geodesic flows of diffeomorphisms. Int J Comput Vis 61(2):139–157. https://doi.org/10.1023/B:VISI.0000043755.93987.aa

    Article  Google Scholar 

  5. Avants BB, Tustison NJ, Song G, Cook PA, Klein A, Gee JC (2011) A reproducible evaluation of ants similarity metric performance in brain image registration. Neuroimage 54(3):2033–2044. https://doi.org/10.1016/j.neuroimage.2010.09.025

    Article  PubMed  Google Scholar 

  6. Vercauteren T, Pennec X, Perchant A, Ayache N (2009) Diffeomorphic demons: efficient non-parametric image registration. Neuroimage 45(1):S61–S72. https://doi.org/10.1016/j.neuroimage.2008.10.040

    Article  PubMed  Google Scholar 

  7. Lorenzi M, Ayache N, Frisoni GB, Pennec X (2013) LCC-demons: a robust and accurate symmetric diffeomorphic registration algorithm. Neuroimage 81:470–483. https://doi.org/10.1016/j.neuroimage.2013.04.114

    Article  CAS  PubMed  Google Scholar 

  8. Yang T, Bai X, Cui X, Gong Y, Li L (2023) DAU-net: an unsupervised 3d brain MRI registration model with dual-attention mechanism. Int J Imaging Syst Technol 33(1):217–229. https://doi.org/10.1002/ima.22801

    Article  Google Scholar 

  9. Jaderberg M, Simonyan K, Zisserman A, Kavukcuoglu K (2015) Spatial transformer networks. CoRR abs/1506.02025. http://arxiv.org/abs/1506.02025

  10. Balakrishnan G, Zhao A, Sabuncu MR, Guttag JV, Dalca AV (2018a) An unsupervised learning model for deformable medical image registration. In: 2018 IEEE Conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, computer vision foundation / IEEE computer society, pp 9252–9260

  11. Kuang D, Schmah T (2019) FAIM - A convnet method for unsupervised 3d medical image registration. In: Suk H, Liu M, Yan P, Lian C (eds) Machine learning in medical imaging - 10th international workshop, MLMI 2019, held in conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, proceedings, Springer, Lecture Notes in Computer Science, vol 11861, pp 646–654. https://doi.org/10.1007/978-3-030-32692-0_74

  12. de Vos BD, Berendsen FF, Viergever MA, Staring M, Isgum I (2017) End-to-end unsupervised deformable image registration with a convolutional neural network. CoRR abs/1704.06065. http://arxiv.org/abs/1704.06065

  13. Zhang J (2018) Inverse-consistent deep networks for unsupervised deformable image registration. CoRR abs/1809.03443. http://arxiv.org/abs/1809.03443

  14. Mok TCW, Chung ACS (2020a) Fast symmetric diffeomorphic image registration with convolutional neural networks. CoRR abs/2003.09514. https://arxiv.org/abs/2003.09514

  15. Han R, Jones CK, Lee J, Wu P, Vagdargi P, Uneri A, Helm PA, Luciano M, Anderson WS, Siewerdsen JH (2022) Deformable MR-CT image registration using an unsupervised, dual-channel network for neurosurgical guidance. Medical Image Anal 75:102292. https://doi.org/10.1016/j.media.2021.102292

  16. Fechter T, Baltas D (2020) One-shot learning for deformable medical image registration and periodic motion tracking. IEEE Trans Medical Imaging 39(7):2506–2517. https://doi.org/10.1109/TMI.2020.2972616

    Article  PubMed  Google Scholar 

  17. Mok TCW, Chung ACS (2021) Conditional deformable image registration with convolutional neural network. In: de Bruijne M, Cattin PC, Cotin S, Padoy N, Speidel S, Zheng Y, Essert C (eds) Medical image computing and computer assisted intervention - MICCAI 2021 - 24th international conference, Strasbourg, France, September 27 - October 1, 2021, Proceedings, Part IV, Springer, Lecture Notes in Computer Science, vol 12904, pp 35–45. https://doi.org/10.1007/978-3-030-87202-1_4

  18. Hering A, van Ginneken B, Heldmann S (2019) mlvirnet: multilevel variational image registration network. In: Shen D, Liu T, Peters TM, Staib LH, Essert C, Zhou S, Yap P, Khan AR (eds) Medical image computing and computer assisted intervention - MICCAI 2019 - 22nd international conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part VI, Springer, Lecture Notes in Computer Science, vol 11769, pp 257–265. https://doi.org/10.1007/978-3-030-32226-7_29

  19. He Z, He Y, Cao W (2023) Deformable image registration with attention-guided fusion of multi-scale deformation fields. Appl Intell 53(3):2936–2950. https://doi.org/10.1007/s10489-022-03659-1

    Article  Google Scholar 

  20. Mok TCW, Chung ACS (2022) Unsupervised deformable image registration with absent correspondences in pre-operative and post-recurrence brain tumor MRI scans. In: Wang L, Dou Q, Fletcher PT, Speidel S, Li S (eds) Medical image computing and computer assisted intervention - MICCAI 2022 - 25th International Conference, Singapore, September 18-22, 2022, proceedings, Part VI, Springer, Lecture Notes in Computer Science, vol 13436, pp 25–35. https://doi.org/10.1007/978-3-031-16446-0_3

  21. Hu X, Kang M, Huang W, Scott MR, Wiest R, Reyes M (2019) Dual-stream pyramid registration network. CoRR abs/1909.11966. http://arxiv.org/abs/1909.11966

  22. Mok TCW, Chung ACS (2020b) Large deformation diffeomorphic image registration with Laplacian pyramid networks. CoRR abs/2006.16148. https://arxiv.org/abs/2006.16148

  23. Li C, Zhou Y, Li Y, Yang S (2021) A coarse-to-fine registration method for three-dimensional MR images. Medical Biol Eng Comput 59(2):457–469. https://doi.org/10.1007/s11517-021-02317-x

    Article  Google Scholar 

  24. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. CoRR abs/1505.04597

  25. de Vos BD, Berendsen FF, Viergever MA, Sokooti H, Staring M, Isgum I (2018) A deep learning framework for unsupervised affine and deformable image registration. CoRR abs/1809.06130. http://arxiv.org/abs/1809.06130

  26. Balakrishnan G, Zhao A, Sabuncu MR, Guttag JV, Dalca AV (2018b) An unsupervised learning model for deformable medical image registration. CoRR abs/1802.02604. http://arxiv.org/abs/1802.02604

  27. Kim B, Kim J, Lee J, Kim DH, Park SH, Ye JC (2019) Unsupervised deformable image registration using cycle-consistent CNN. CoRR abs/1907.01319. http://arxiv.org/abs/1907.01319

  28. Berthilsson R (1998) Affine correlation. In: Jain AK, Venkatesh S, Lovell BC (eds) Fourteenth International Conference on Pattern Recognition, ICPR 1998, Brisbane, Australia, 16-20 August, 1998, IEEE Computer Society, pp 1458–1460. https://doi.org/10.1109/ICPR.1998.711979

  29. Shu Y, Wang H, Xiao B, Bi X, Li W (2021) Medical image registration based on uncoupled learning and accumulative enhancement. In: de Bruijne M, Cattin PC, Cotin S, Padoy N, Speidel S, Zheng Y, Essert C (eds) Medical Image Computing and Computer Assisted Intervention - MICCAI 2021 - 24th International Conference, Strasbourg, France, September 27 - October 1, 2021, Proceedings, Part IV, Springer, Lecture Notes in Computer Science, vol 12904, pp 3–13. https://doi.org/10.1007/978-3-030-87202-1_1

Download references

Funding

This work was supported by Natural Science Foundation of China (No.82272617) and Pearl River S &T Nova Program of Guangzhou (201710010162).

Author information

Authors and Affiliations

  1. School of Software Engineering, South China University of Technology, Guangzhou, Guangdong, 510006, China

    Yanxia Liu, Xiaozhen Li & Rui Li

  2. Sun Yat-sen University Cancer Center, Guangzhou, Guangdong, 510060, China

    SiJuan Huang & Xin Yang

  3. State Key Laboratory of Oncology in South China, Guangzhou, Guangdong, 510060, China

    SiJuan Huang & Xin Yang

  4. Collaborative Innovation Center for Cancer Medicine, Guangzhou, Guangdong, 510060, China

    SiJuan Huang & Xin Yang

  5. Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou, Guangdong, 510060, China

    SiJuan Huang & Xin Yang

Authors
  1. Yanxia Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaozhen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. SiJuan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to SiJuan Huang or Xin Yang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Li, X., Li, R. et al. A multi-view assisted registration network for MRI registration pre- and post-therapy. Med Biol Eng Comput 61, 3181–3191 (2023). https://doi.org/10.1007/s11517-023-02949-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02949-1

Keywords

Search

Navigation