Abstract
Purpose
Image registration is a fundamental task in the area of image processing, and it is critical to many clinical applications, e.g., computer-assisted surgery. In this work, we attempt to design an effective framework that gains higher accuracy at a minimal cost of the invertibility of registration field.
Methods
A hierarchically aggregated transformation (HAT) module is proposed. Within each HAT module, we connect multiple convolutions in a hierarchical manner to capture the multi-scale context, enabling small and large displacements between a pair of images to be taken into account simultaneously during the registration process. Besides, an adaptive feature scaling (AFS) mechanism is presented to refine the multi-scale feature maps derived from the HAT module by rescaling channel-wise features in the global receptive field. Based on the HAT module and AFS mechanism, we establish an efficacious and efficient unsupervised deformable registration framework.
Results
The devised framework is validated on the dataset of SCARED and MICCAI Instrument Segmentation and Tracking Challenge 2015, and the experimental results demonstrate that our method achieves better registration accuracy with fewer number of folding pixels than three widely used baseline approaches of SyN, NiftyReg and VoxelMorph.
Conclusion
We develop a novel method for unsupervised deformable image registration by incorporating the HAT module and AFS mechanism into the framework, which provides a new way to obtain a desirable registration field between a pair of images.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Code availability
The source code will be publicly available upon the acceptance.
References
Haskins G, Kruger U, Yan P (2020) Deep learning in medical image registration: a survey. Mach Vis Appl 31(1):1–18
Rucker DC, Wu Y, Clements LW, Ondrake JE, Pheiffer TS, Simpson AL, Jarnagin WR, Miga MI (2013) A mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data. IEEE Trans Med Imaging 33(1):147–158
Plantefeve R, Peterlik I, Haouchine N, Cotin S (2016) Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann Biomed Eng 44(1):139–153
Hattab G, Riediger C, Weitz J, Speidel S (2020) A case study: impact of target surface mesh size and mesh quality on volume-to-surface registration performance in hepatic soft tissue navigation. Int J Comput Assist Radiol Surg 15(8):1235–1245
Klein S, Staring M, Murphy K, Viergever MA, Pluim JPW (2009) Elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging 29(1):196–205
Rueckert D, Sonoda LI, Hayes C, Hill DLG, Leach MO, Hawkes DJ (1999) Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans Med Imaging 18(8):712–721
Shen D, Davatzikos C (2002) Hammer: hierarchical attribute matching mechanism for elastic registration. IEEE Trans Med imaging 21(11):1421–1439
Avants BB, Epstein CL, Grossman M, Gee JC (2008) Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal 12(1):26–41
Miao S, Jane WZ, Liao R (2016) A CNN regression approach for real-time 2d/3d registration. IEEE Trans Med Imaging 35(5):1352–1363
Rohé MM, Datar M, Heimann T, Sermesant M, Pennec X (2017) SVF-net: learning deformable image registration using shape matching. In: International conference on medical image computing and computer-assisted intervention, pp 266–274. Springer
Cao X, Yang J, Zhang J, Nie D, Kim M, Wang Q, Shen D (2017) Deformable image registration based on similarity-steered CNN regression. In: International conference on medical image computing and computer-assisted intervention, pp 300–308. Springer
Zhu W, Myronenko A, Xu Z, Li W, Roth H, Huang Y, Milletari F, Xu D (2020) Neurreg: neural registration and its application to image segmentation. In: Proceedings of the winter conference on applications of computer vision, pp 3617–3626
Jaderberg M, Simonyan K, Zisserman A, Kavukcuoglu K (2015) Spatial transformer networks. In: Proceedings of the 28th international conference on neural information processing systems, vol 2, pp 2017–2025
Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV (2019) Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans Med Imaging 38(8):1788–1800
Dalca AV, Balakrishnan G, Guttag J, Sabuncu MR (2019) Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Med Image Anal 57:226–236
Zhu W, Huang Y, Xu D, Qian Z, Fan W, Xie X (2021) Test-time training for deformable multi-scale image registration. In: International conference on robotics and automation
Zhao S, Lau T, Luo J, Eric I, Chang C, Xu Y (2019) Unsupervised 3d end-to-end medical image registration with volume tweening network. IEEE J Biomed Health Inform 24(5):1394–1404
de Vos BD, Berendsen FF, Viergever MA, Sokooti H, Staring M, Išgum I (2019) A deep learning framework for unsupervised affine and deformable image registration. Med Image Anal 52:128–143
Teed Z, Deng J (2020) Raft: recurrent all-pairs field transforms for optical flow. In: Proceedings of the European conference on computer vision, pp 402–419. Springer
Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder–decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision
Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023
Allan M, Mcleod J, Wang CC, Rosenthal JC, Fu KX, Zeffiro T, Xia W, Zhanshi Z, Luo H, Zhang X (2021) Stereo correspondence and reconstruction of endoscopic data challenge. arXiv preprint arXiv:2101.01133
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241. Springer
Avants BB, Tustison NJ, Song G, Cook PA, Klein A, Gee JC (2011) A reproducible evaluation of ants similarity metric performance in brain image registration. Neuroimage 54(3):2033–2044
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9
Acknowledgements
This work was supported by the Key Research and Development Program of Shandong Province under Grant NO.2019JZZY011101, the National Natural Science Foundation of China under Grant 61620106012, the Shenzhen Peacock Plan under Grant KQTD2016112515134654, and the Special Funding for Top Talents of Shandong Province. The authors would like to thank the Editor and the anonymous reviewers for their suggestions which have improved the quality of the work.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interests
The authors declare no conflict of interests.
Ethics approval
This work does not involve studies with human participants.
Informed consent
This work does not involve studies with human participants.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Shao, S., Pei, Z., Chen, W. et al. A multi-scale unsupervised learning for deformable image registration. Int J CARS 17, 157–166 (2022). https://doi.org/10.1007/s11548-021-02511-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-021-02511-0