Skip to main content

Deformable Medical Image Registration Under Distribution Shifts with Neural Instance Optimization

  • Conference paper
  • First Online:
Machine Learning in Medical Imaging (MLMI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14348))

Included in the following conference series:

Abstract

Deep-learning deformable image registration methods often struggle if test-image characteristic shifts from the training domain, such as the large variations in anatomy and contrast changes with different imaging protocols. Gradient descent-based instance optimization is often introduced to refine the solution of deep-learning methods, but the performance gain is minimal due to the high degree of freedom in the solution and the absence of robust initial deformation. In this paper, we propose a new instance optimization method, Neural Instance Optimization (NIO), to correct the bias in the deformation field caused by the distribution shifts for deep-learning methods. Our method naturally leverages the inductive bias of the convolutional neural network, the prior knowledge learned from the training domain and the multi-resolution optimization strategy to fully adapt a learning-based method to individual image pairs, avoiding registration failure during the inference phase. We evaluate our method with gold standard, human cortical and subcortical segmentation, and manually identified anatomical landmarks to contrast NIO’s performance with conventional and deep-learning approaches. Our method compares favourably with both approaches and significantly improves the performance of deep-learning methods under distribution shifts with 1.5% to 3.0% and 2.3% to 6.2% gains in registration accuracy and robustness, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C.: Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12(1), 26–41 (2008)

    Article  Google Scholar 

  2. Bajcsy, R., Kovačič, S.: Multiresolution elastic matching. Comput. Vis. Graph. Image Process. 46(1), 1–21 (1989)

    Article  Google Scholar 

  3. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019)

    Article  Google Scholar 

  4. Castillo, R., et al.: A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Phys. Med. Biol. 54(7), 1849 (2009)

    Article  Google Scholar 

  5. Delmon, V., et al.: Registration of sliding objects using direction dependent b-splines decomposition. Phys. Med. Biol. 58(5), 1303 (2013)

    Article  Google Scholar 

  6. Eisenmann, M., et al.: Biomedical image analysis competitions: the state of current participation practice. arXiv preprint arXiv:2212.08568 (2022)

  7. Falta, F., Hansen, L., Heinrich, M.P.: Learning iterative optimisation for deformable image registration of lung CT with recurrent convolutional networks. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13436, pp. 301–309. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16446-0_29

    Chapter  Google Scholar 

  8. Fu, Y., et al.: Lungregnet: an unsupervised deformable image registration method for 4D-CT lung. Med. Phys. 47(4), 1763–1774 (2020)

    Article  Google Scholar 

  9. Heinrich, M.P., Hansen, L.: Voxelmorph++ going beyond the cranial vault with keypoint supervision and multi-channel instance optimisation. In: Hering, A., Schnabel, J., Zhang, M., Ferrante, E., Heinrich, M., Rueckert, D. (eds.) WBIR 2022. LNCS, vol. 13386, pp. 85–95. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-11203-4_10

    Chapter  Google Scholar 

  10. Hering, A., Hansen, L., Mok, T.C., et al.: Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning. IEEE Trans. Med. Imaging 42(3), 697–712 (2022)

    Article  Google Scholar 

  11. Hoffmann, M., Billot, B., Greve, D.N., Iglesias, J.E., Fischl, B., Dalca, A.V.: Synthmorph: learning contrast-invariant registration without acquired images. IEEE Trans. Med. Imaging 41(3), 543–558 (2021)

    Article  Google Scholar 

  12. Hofmanninger, J., Prayer, F., Pan, J., Röhrich, S., Prosch, H., Langs, G.: Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem. Eur. Radiol. Exp. 4(1), 1–13 (2020)

    Article  Google Scholar 

  13. Hoopes, A., Hoffmann, M., Fischl, B., Guttag, J., Dalca, A.V.: HyperMorph: amortized hyperparameter learning for image registration. In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds.) IPMI 2021. LNCS, vol. 12729, pp. 3–17. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78191-0_1

    Chapter  Google Scholar 

  14. Liu, R., Li, Z., et al.: Learning deformable image registration from optimization: perspective, modules, bilevel training and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 44(11), 7688–7704 (2022)

    Article  Google Scholar 

  15. Marcus, D.S., Wang, T.H., Parker, J., Csernansky, J.G., Morris, J.C., Buckner, R.L.: Open access series of imaging studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J. Cogn. Neurosci. 19(9), 1498–1507 (2007)

    Article  Google Scholar 

  16. Marcus, D.S., Wang, T.H., et al.: Oasis brains - open access series of imaging studies. https://www.oasis-brains.org/. Accessed 01 Mar 2021

  17. Mok, T.C., Chung, A.: Fast symmetric diffeomorphic image registration with convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4644–4653 (2020)

    Google Scholar 

  18. Mok, T.C.W., Chung, A.C.S.: Conditional deformable image registration with convolutional neural network. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 35–45. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_4

    Chapter  Google Scholar 

  19. Mok, T.C.W., Chung, A.C.S.: Large deformation diffeomorphic image registration with laplacian pyramid networks. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12263, pp. 211–221. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59716-0_21

    Chapter  Google Scholar 

  20. Mok, T.C., Chung, A.C.: Unsupervised deformable image registration with absent correspondences in pre-operative and post-recurrence brain tumor mri scans. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13436, pp. 25–35. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16446-0_3

    Chapter  Google Scholar 

  21. Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optic flow computation with theoretically justified warping. Int. J. Comput. Vision 67, 141–158 (2006)

    Article  Google Scholar 

  22. Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L., Leach, M.O., Hawkes, D.J.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 18(8), 712–721 (1999)

    Article  Google Scholar 

  23. Ruthotto, L., Modersitzki, J.: Non-linear image registration. In: Handbook of Mathematical Methods in Imaging: Volume 1, 2nd edn, pp. 2005–2051 (2015)

    Google Scholar 

  24. Shattuck, D.W., et al.: Construction of a 3D probabilistic atlas of human cortical structures. Neuroimage 39(3), 1064–1080 (2008)

    Article  Google Scholar 

  25. Shattuck, D.W., Mirza, M., et al.: LPBA40 atlases download. https://resource.loni.usc.edu/resources/atlases-downloads/. Accessed 01 Mar 2021

  26. Shu, Y., Wang, H., Xiao, B., Bi, X., Li, W.: Medical image registration based on uncoupled learning and accumulative enhancement. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 3–13. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_1

    Chapter  Google Scholar 

  27. Siebert, H., Hansen, L., Heinrich, M.P.: Fast 3D registration with accurate optimisation and little learning for Learn2Reg 2021. In: Aubreville, M., Zimmerer, D., Heinrich, M. (eds.) MICCAI 2021. LNCS, vol. 13166, pp. 174–179. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-97281-3_25

    Chapter  Google Scholar 

  28. Teed, Z., Deng, J.: RAFT: recurrent all-pairs field transforms for optical flow. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 402–419. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_24

    Chapter  Google Scholar 

  29. Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: CVPR 2011, pp. 1521–1528. IEEE (2011)

    Google Scholar 

  30. de Vos, B.D., Berendsen, F.F., Viergever, M.A., Sokooti, H., Staring, M., Išgum, I.: A deep learning framework for unsupervised affine and deformable image registration. Med. Image Anal. 52, 128–143 (2019)

    Article  Google Scholar 

  31. Zhu, W., Huang, Y., Xu, D., Qian, Z., Fan, W., Xie, X.: Test-time training for deformable multi-scale image registration. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 13618–13625. IEEE (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Tony C. W. Mok .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mok, T.C.W. et al. (2024). Deformable Medical Image Registration Under Distribution Shifts with Neural Instance Optimization. In: Cao, X., Xu, X., Rekik, I., Cui, Z., Ouyang, X. (eds) Machine Learning in Medical Imaging. MLMI 2023. Lecture Notes in Computer Science, vol 14348. Springer, Cham. https://doi.org/10.1007/978-3-031-45673-2_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-45673-2_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-45672-5

  • Online ISBN: 978-3-031-45673-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics