Abstract
Segmentation-assisted registration models can leverage few available labels in exchange for large performance gains by their complementarity. Recent related works independently build the prediction branches of deformation field and segmentation label without any information interaction except for the joint supervision. They ignore underlying relationship between the two tasks, thereby failing to fully exploit their complementary nature. To this end, we propose a ProGressively Coupling Network (PGCNet) that relies on segmentation to regularize the correct projecting of registration. Our overall framework is a multi-task learning paradigm in which features are extracted by one shared encoder and then separate prediction branches are built for segmentation and registration. In the prediction phase, we utilize the bidirectional deformation fields as bridges to warp the features of moving and fixed images to each other’s segmentation branches, thereby progressively and interactively supplementing additional context information at multiple levels for their segmentation. By establishing the entangled correspondence, segmentation supervision can indirectly regularize registration stream to accurately project semantic layout for segmentation branches. In addition, we design the position correlation calculation for registration to easier capture the spatial correlation of the images from the shared features. Experimental results on public 3D brain MRI datasets show that our work performs favorably against the state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
IXI dataset is available in https://brain-development.org/ixi-dataset/.
References
Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C.: Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12(1), 26–41 (2008)
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: An unsupervised learning model for deformable medical image registration. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 9252–9260 (2018)
Baxter, J.: A Bayesian/information theoretic model of learning to learn via multiple task sampling. Mach. Learn. 28(1), 7–39 (1997)
Beg, M.F., Miller, M.I., Trouve, A., Younes, L.: Computing large deformation metric mappings via geodesic flows. Int. J. Comput. Vision 61(2), 139–157 (2005)
Chen, J., Frey, E.C., He, Y., Segars, W.P., Li, Y., Du, Y.: TransMorph: transformer for unsupervised medical image registration. Med. Image Anal. 82, 102615 (2022)
Chen, X., Xia, Y., Ravikumar, N., Frangi, A.F.: A deep discontinuity-preserving image registration network. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 46–55. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_5
Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Med. Image Anal. 57, 226–236 (2019)
Estienne, T., et al.: U-ReSNet: ultimate coupling of registration and segmentation with deep nets. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 310–319. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_35
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
He, Y., et al.: Few-shot learning for deformable medical image registration with perception-correspondence decoupling and reverse teaching. IEEE J. Biomed. Health Inform. 26(3), 1177–1187 (2022). https://doi.org/10.1109/JBHI.2021.3095409
He, Y., et al.: Deep complementary joint model for complex scene registration and few-shot segmentation on medical images. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12363, pp. 770–786. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58523-5_45
Hu, B., Zhou, S., Xiong, Z., Wu, F.: Recursive decomposition network for deformable image registration. IEEE J. Biomed. Health Inform. 26(10), 5130–5141 (2022)
Jonschkowski, R., Stone, A., Barron, J.T., Gordon, A., Konolige, K., Angelova, A.: What matters in unsupervised optical flow. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 557–572. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_33
Kang, M., Hu, X., Huang, W., Scott, M.R., Reyes, M.: Dual-stream pyramid registration network. Med. Image Anal. 78, 102379 (2022)
Lv, J., et al.: Joint progressive and coarse-to-fine registration of brain MRI via deformation field integration and non-rigid feature fusion. IEEE Trans. Med. Imaging 41(10), 2788–2802 (2022). https://doi.org/10.1109/TMI.2022.3170879
Marcus, D.S., Wang, T.H., Parker, J., Csernansky, J.G., Morris, J.C., Buckner, R.L.: Open access series of imaging studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J. Cogn. Neurosci. 19(9), 1498–1507 (2007)
Mok, T.C.W., Chung, A.C.S.: Large deformation diffeomorphic image registration with Laplacian pyramid networks. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12263, pp. 211–221. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59716-0_21
Qin, C., et al.: Joint learning of motion estimation and segmentation for cardiac MR image sequences. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 472–480. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_53
Shu, Y., Wang, H., Xiao, B., Bi, X., Li, W.: Medical image registration based on uncoupled learning and accumulative enhancement. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 3–13. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_1
Vakalopoulou, M., et al.: AtlasNet: multi-atlas non-linear deep networks for medical image segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 658–666. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_75
Vandenhende, S., Georgoulis, S., Van Gansbeke, W., Proesmans, M., Dai, D., Van Gool, L.: Multi-task learning for dense prediction tasks: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(7), 3614–3633 (2021)
Xu, D., Ouyang, W., Wang, X., Sebe, N.: PAD-Net: multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 675–684 (2018)
Xu, Z., Niethammer, M.: DeepAtlas: joint semi-supervised learning of image registration and segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 420–429. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_47
Zhang, Z., Cui, Z., Xu, C., Jie, Z., Li, X., Yang, J.: Joint task-recursive learning for semantic segmentation and depth estimation. In: European Conference on Computer Vision, pp. 235–251 (2018)
Zhang, Z., Cui, Z., Xu, C., Yan, Y., Sebe, N., Yang, J.: Pattern-affinitive propagation across depth, surface normal and semantic segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4106–4115 (2019)
Zhao, A., Balakrishnan, G., Durand, F., Guttag, J.V., Dalca, A.V.: Data augmentation using learned transformations for one-shot medical image segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8543–8553 (2019)
Zhao, F., Wu, Z., Wang, L., Lin, W., Xia, S., Li, G.: A deep network for joint registration and parcellation of cortical surfaces. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 171–181. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_17
Zhao, X., Pang, Y., Zhang, L., Lu, H.: Joint learning of salient object detection, depth estimation and contour extraction. IEEE Trans. Image Process. 31, 7350–7362 (2022)
Acknowledgements
This work was supported by the National Natural Science Foundation of China #62276046, and the Liaoning Natural Science Foundation #2021-KF-12-10.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tan, Z., Zhang, H., Tian, F., Zhang, L., Sun, W., Lu, H. (2023). Progressively Coupling Network for Brain MRI Registration in Few-Shot Situation. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14229. Springer, Cham. https://doi.org/10.1007/978-3-031-43999-5_59
Download citation
DOI: https://doi.org/10.1007/978-3-031-43999-5_59
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43998-8
Online ISBN: 978-3-031-43999-5
eBook Packages: Computer ScienceComputer Science (R0)
