Skip to main content
SpringerLink
Log in

AEAU-Net: an unsupervised end-to-end registration network by combining affine transformation and deformable medical image registration

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

A Correction to this article was published on 18 August 2023

This article has been updated

Abstract

Deformable medical image registration plays an essential role in clinical diagnosis and treatment. However, due to the large difference in image deformation, unsupervised convolutional neural network (CNN)-based methods cannot extract global features and local features simultaneously and cannot capture long-distance dependencies to solve the problem of excessive deformation. In this paper, an unsupervised end-to-end registration network is proposed for 3D MRI medical image registration, named AEAU-Net, which includes two-stage operations, i.e., an affine transformation and a deformable registration. These two operations are implemented by an affine transformation subnetwork and a deformable registration subnetwork, respectively. In the deformable registration subnetwork, termed as EAU-Net, we designed an efficient attention mechanism (EAM) module and a recursive residual path (RSP) module. The EAM module is embedded in the bottom layer of the EAU-Net to capture long-distance dependencies. The RSP model is used to obtain effective features by fusing deep and shallow features. Extensive experiments on two datasets, LPBA40 and Mindboggle101, were conducted to verify the effectiveness of the proposed method. Compared with baseline methods, this proposed method could obtain better registration performance. The ablation study further demonstrated the reasonability and validity of the designed architecture of the proposed method.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Change history

Notes

  1. https://github.com/voxelmorph/voxelmorph

  2. https://github.com/dykuang/Medical-image-registration

  3. https://github.com/boahK/MEDIA_CycleMorph

  4. https://github.com/SLKaMiHi/TD-Net-unsupervised-medical-image-registration-network-based-on-Transformer-and-CNN

References

  1. Lin SY, Chen YS (2015) L.F. Chen. In: Su FC, Wang SH, Yeh ML (eds) 1st Global Conference on Biomedical Engineering & 9th Asian-Pacific Conference on Medical and Biological Engineering. Springer International Publishing, Cham, pp 128–131

    Chapter  Google Scholar 

  2. Avants BB, Epstein CL, Grossman M, Gee JC (2008) Medical Image Analysis 12(1):26

    Article  CAS  PubMed  Google Scholar 

  3. Avants B, Tustison NJ, Song G, Cook PA, Klein A, Gee JC (2011) Neuroimage 54(3):2033

    Article  PubMed  Google Scholar 

  4. V. Jaouen, P.H. Conze, G. Dardenne, J. Bert, D. Visvikis, arXiv preprint http://arxiv.org/abs/2111.15509arXiv:2111.15509 (2021)

  5. A. Hering, L. Hansen, T.C. Mok, A.C. Chung, H. Siebert, S. Häger, A. Lange, S. Kuckertz, S. Heldmann, W. Shao, et al., IEEE Transactions on Medical Imaging (2022)

  6. G. Balakrishnan, A. Zhao, M.R. Sabuncu, J. Guttag, A.V. Dalca, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

  7. Miao S, Wang ZJ, Rui L (2016) IEEE Transactions on Medical Imaging 35(5):1

    Article  Google Scholar 

  8. Yang X, Kwitt R, Styner M, Niethammer M (2017) Neuroimage 158:378

    Article  PubMed  Google Scholar 

  9. Rohé MM, Datar M, Heimann T, Sermesant M (2017) X. Pennec. In: Descoteaux M, Maier-Hein L, Franz A, Jannin P, Collins DL, Duchesne S (eds) Medical image computing and computer assisted intervention - MICCAI 2017. Springer International Publishing, Cham, pp 266–274

    Google Scholar 

  10. Fan J, Cao X, Yap PT, Shen D (2019) Medical Image Analysis 54:193

  11. H. Sokooti, B.D. Vos, F. Berendsen, B. Lelieveldt, I. Išgum, M. Staring, Springer, Cham (2017)

  12. L. Han, H. Dou, Y. Huang, P.T. Yap, in Biomedical image registration, domain generalisation and out-of-distribution analysis: MICCAI 2021 challenges: MIDOG 2021, MOOD 2021, and Learn2Reg 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27–October 1, 2021, Proceedings (Springer, 2022), pp. 141–146

  13. M.C. Lee, O. Oktay, A. Schuh, M. Schaap, B. Glocker, in Medical image computing and computer assisted intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22 (Springer, 2019), pp. 337–345

  14. H. Siebert, L. Hansen, M.P. Heinrich, in Biomedical image registration, domain generalisation and out-of-distribution analysis: MICCAI 2021 challenges: MIDOG 2021, MOOD 2021, and Learn2Reg 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27–October 1, 2021, Proceedings (Springer, 2022), pp. 174–179

  15. H. Li, Y. Fan, in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) (2018), pp. 1075–1078. 10.1109/ISBI.2018.8363757

  16. Wu G, Kim M, Qian W, Munsell BC, Shen D (2016) Deep learning for medical image analysis 63(7):245

    Google Scholar 

  17. A.V. Dalca, G. Balakrishnan, J. Guttag, M.R. Sabuncu, in International conference on medical image computing and computer-assisted intervention (Springer, 2018), pp. 729–738

  18. B.D.d. Vos, F.F. Berendsen, M.A. Viergever, M. Staring, I. Išgum, in Deep learning in medical image analysis and multimodal learning for clinical decision support (Springer, 2017), pp. 204–212

  19. M. Jaderberg, K. Simonyan, A. Zisserman, et al., Advances in neural information processing systems 28 (2015)

  20. J. Long, E. Shelhamer, T. Darrell, in Proceedings of the IEEE conference on computer vision and pattern recognition (2015), pp. 3431–3440

  21. D. Kuang, T. Schmah, in International workshop on machine learning in medical imaging (Springer, 2019), pp. 646–654

  22. Hansen L, Heinrich MP (2021) IEEE Transactions on Medical Imaging 40(9):2246

    Article  PubMed  Google Scholar 

  23. N. Gunnarsson, J. Sjölund, T.B. Schön, in Segmentation, classification, and registration of multi-modality medical imaging data: MICCAI 2020 challenges, ABCs 2020, L2R 2020, TN-SCUI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, Proceedings 23 (Springer, 2021), pp. 80–86

  24. Fan J, Cao X, Wang Q, Yap PT, Shen D (2019) Medical Image Analysis 58:101545

  25. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, in Advances in neural information processing systems, vol. 27, ed. by Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, K. Weinberger (Curran Associates, Inc., 2014), vol. 27. https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf

  26. Kim B, Kim DH, Park SH, Kim J, Lee JG, Ye JC (2021) Medical Image Analysis 71:102036

    Article  PubMed  Google Scholar 

  27. S. Zhao, Y. Dong, E.I. Chang, Y. Xu, et al., in Proceedings of the IEEE/CVF International Conference on Computer Vision (2019), pp. 10,600–10,610

  28. Zhao S, Lau T, Luo J, Chang EIC, Xu Y (2020) IEEE Journal of Biomedical and Health Informatics 24(5):1394. https://doi.org/10.1109/JBHI.2019.2951024

    Article  PubMed  Google Scholar 

  29. Mok TCW (2020) A.C.S. Chung. In: Martel AL, Abolmaesumi P, Stoyanov D, Mateus D, Zuluaga MA, Zhou SK, Racoceanu D, Joskowicz L (eds) Medical image computing and computer assisted intervention - MICCAI 2020. Springer International Publishing, Cham, pp 211–221

    Google Scholar 

  30. Liu L, Hu X, Zhu L (2019) P.A. Heng. In: Shen D, Liu T, Peters TM, Staib LH, Essert C, Zhou S, Yap PT, Khan A (eds) Medical image computing and computer assisted intervention - MICCAI 2019. Springer International Publishing, Cham, pp 346–354

    Chapter  Google Scholar 

  31. W. Luo, Y. Li, R. Urtasun, R. Zemel, in Advances in neural information processing systems, vol. 29, ed. by D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, R. Garnett (Curran Associates, Inc., 2016), vol. 29. https://proceedings.neurips.cc/paper/2016/file/c8067ad1937f728f51288b3eb986afaa-Paper.pdf

  32. Z. Huang, X. Wang, L. Huang, C. Huang, Y. Wei, W. Liu, in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2019)

  33. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L.u. Kaiser, I. Polosukhin, in Advances in neural information processing systems, vol. 30, ed. by I. Guyon, U.V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Curran Associates, Inc., 2017), vol. 30. https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

  34. S.W. Zamir, A. Arora, S. Khan, M. Hayat, F.S. Khan, M.H. Yang, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), pp. 5728–5739

  35. L. Song, G. Liu, M. Ma, Applied intelligence pp. 1–9 (2022)

  36. de Vos BD, Berendsen FF, Viergever MA, Sokooti H, Staring M, Išgum I (2019) Medical Image Analysis 52:128

  37. Z. Zhu, Y. Cao, C. Qin, Y. Rao, D. Ni, Y. Wang, in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (2020), pp. 1355–1359. https://doi.org/10.1109/EMBC44109.2020.9176475

  38. O. Ronneberger, P. Fischer, T. Brox, in Medical image computing and computer-assisted intervention-MICCAI 2015, ed. by N. Navab, J. Hornegger, W.M. Wells, A.F. Frangi (Springer International Publishing, Cham, 2015), pp. 234–241

  39. Fu J, Liu J, Jiang J, Li Y, Bao Y, Lu H (2021) IEEE Transactions on Neural Networks and Learning Systems 32(6):2547. https://doi.org/10.1109/TNNLS.2020.3006524

    Article  PubMed  Google Scholar 

  40. Ibtehaz N, Rahman MS (2020) Neural Networks 121:74

    Article  PubMed  Google Scholar 

  41. C. Szegedy, S. Ioffe, V. Vanhoucke, A.A. Alemi, in Thirty-first AAAI conference on artificial intelligence (2017)

  42. J. Kim, J.K. Lee, K.M. Lee, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  43. Shattuck DW, Mirza M, Adisetiyo V, Hojatkashani C, Salamon G, Narr KL, Poldrack RA, Bilder RM, Toga AW (2008) Neuroimage 39(3):1064

    Article  PubMed  Google Scholar 

  44. A. Klein, J. Tourville, Frontiers in Neuroscience 6 (2012). https://doi.org/10.3389/fnins.2012.00171. https://www.frontiersin.org/articles/10.3389/fnins.2012.00171

  45. D.P. Kingma, J. Ba, arXiv preprint http://arxiv.org/abs/1412.6980arXiv:1412.6980 (2014)

  46. Kang M, Hu X, Huang W, Scott MR, Reyes M (2022) Medical Image Analysis 78:102379

  47. Klein A, Andersson J, Ardekani BA, Ashburner J, Avants B, Chiang MC, Christensen GE, Collins DL, Gee J, Hellier P et al (2009) Neuroimage 46(3):786

    Article  PubMed  Google Scholar 

  48. Dice LR (1945) Ecology 26(3):297

    Article  Google Scholar 

  49. Chen J, Frey EC, He Y, Segars WP, Li Y, Du Y (2022) Medical image analysis 82:102615

    Article  PubMed  Google Scholar 

  50. Hu B, Zhou S, Xiong Z, Wu F (2022) IEEE Transactions on Circuits and Systems for Video Technology 32(10):7269

    Article  Google Scholar 

Download references

Acknowledgements

The authors sincerely thank the anonymous reviewers for the insightful comments. This work was supported in part by the National Natural Science Foundation of China under Grant No. 62076209 and in part by the NHC Key Laboratory of Nuclear Technology Medical Transformation (Mianyang Central Hospital) under Grant No. 2022HYX012.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Laboratory for Brain Science and Medical Artificial Intelligence, Southwest University of Science and Technology, Mianyang, 621010, China

    Wei Qiu, Lianjin Xiong, Ning Li, Zhangrong Luo, Yaobin Wang & Yangsong Zhang

  2. NHC Key Laboratory of Nuclear Technology Medical Transformation (Mianyang Central Hospital), Mianyang, 621010, China

    Yangsong Zhang

  3. Key Laboratory of Testing Technology for Manufacturing Process, Ministry of Education, Southwest University of Science and Technology, Mianyang, Sichuan, 621010, China

    Yangsong Zhang

Authors
  1. Wei Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lianjin Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ning Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhangrong Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yaobin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yangsong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yangsong Zhang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The title text of the second row and first column in Tables 2 and 3 should be Method/Metrics in order to correspond one-to-one and in Table 3, Dice should be followed by an up arrow.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 491 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qiu, W., Xiong, L., Li, N. et al. AEAU-Net: an unsupervised end-to-end registration network by combining affine transformation and deformable medical image registration. Med Biol Eng Comput 61, 2859–2873 (2023). https://doi.org/10.1007/s11517-023-02887-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02887-y

Keywords

Search

Navigation