Skip to main content

On the Adaptability of Unsupervised CNN-Based Deformable Image Registration to Unseen Image Domains

  • Conference paper
  • First Online:
Machine Learning in Medical Imaging (MLMI 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11046))

Included in the following conference series:

Abstract

Deformable image registration is a fundamental problem in medical image analysis. During the last years, several methods based on deep convolutional neural networks (CNN) proved to be highly accurate to perform this task. These models achieved state-of-the-art accuracy while drastically reducing the required computational time, but mainly focusing on images of specific organs and modalities. To date, no work has reported on how these models adapt across different domains. In this work, we ask the question: can we use CNN-based registration models to spatially align images coming from a domain different than the one/s used at training time? We explore the adaptability of CNN-based image registration to different organs/modalities. We employ a fully convolutional architecture trained following an unsupervised approach. We consider a simple transfer learning strategy to study the generalisation of such model to unseen target domains, and devise a one-shot learning scheme taking advantage of the unsupervised nature of the proposed method. Evaluation on two publicly available datasets of X-Ray lung images and cardiac cine magnetic resonance sequences is provided. Our experiments suggest that models learned in different domains can be transferred at the expense of a decrease in performance, and that one-shot learning in the context of unsupervised CNN-based registration is a valid alternative to achieve consistent registration performance when only a pair of images from the target domain is available.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    A complete list of configuration parameters for Elastix can be found in http://elastix.bigr.nl/wiki/index.php/Parameter_file_database.

  2. 2.

    publicly available at https://github.com/mshunshin/SegNetCMR.

  3. 3.

    Fine-tuning for 50 iterations takes only 0.5 s on GPU. When training from scratch/fine-tuning until convergence, we update the model for 3000 iterations leading to about 30 s per registration case.

  4. 4.

    We used Python and Tensorflow for implementation. Experiments were run in a machine with CPU Intel Core i7-7700, 64GB of RAM and NVidia Titan XP GPU. In order to encourage reproducible research, the project source code and Elastix parameter files can be downloaded from: https://gitlab.com/eferrante/.

  5. 5.

    The CNN-based models take 0.06s on GPU and 0.08 s on CPU to register a pair of images, while Elastix 2.47s. In all the experiments we used Adam optimization, with LR = 1e-4 and \(\lambda _1 = \lambda _2\)=1e-6.

  6. 6.

    Elastix parameters were chosen by grid search using the training data and are available online in our project website.

  7. 7.

    We experimented with fine-tuning the model in whole or in part, but we found that fine-tuning the complete model achieved better results in general.

References

  1. Balakrishnan, G., et al.: An unsupervised learning model for deformable medical image registration. Accepted at CVPR 2018 (2018)

    Google Scholar 

  2. Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. ICML (2015)

    Google Scholar 

  3. Guerrero, R., et al.: White matter hyperintensity and stroke lesion segmentation and differentiation using cnns. NeuroImage: Clin. 17, 918–934 (2018)

    Google Scholar 

  4. Jaderberg, M., et al.: Spatial transformer networks. In: NIPS, pp. 2017–2025 (2015)

    Google Scholar 

  5. Kawaguchi, T., Harada, Y., Nagata, R., Miyake, H.: Image registration methods for contralateral subtraction of chest radiographs. In: IEE BMEI (2010)

    Google Scholar 

  6. Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.: Elastix: a toolbox for intensity-based medical image registration. IEEE TMI 29(1), 196–205 (2010)

    Google Scholar 

  7. Li, H., Fan, Y.: Non-rigid image registration using self-supervised fully convolutional networks without training data. Accepted at ISBI 2018 (2018)

    Google Scholar 

  8. Paragios, N.: (hyper)-graphical models in biomedical image analysis. Med. Image Anal. 33, 102–106 (2016)

    Article  Google Scholar 

  9. Phatak, N.S.: Strain measurement in the left ventricle during systole with deformable image registration. Med. Image Anal. 13(2), 354–361 (2009)

    Article  Google Scholar 

  10. Radau, P., Lu, Y., Connelly, K., Paul, G., et al.: Evaluation framework for algorithms segmenting short axis cardiac MRI. MIDAS J. 49 (2009)

    Google Scholar 

  11. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: MICCAI, pp. 234–241. Springer, Cham (2015)

    Google Scholar 

  12. Salakhutdinov, R., Tenenbaum, J., Torralba, A.: One-shot learning with a hierarchical nonparametric Bayesian model. In: ICML Workshop Proceedings (2012)

    Google Scholar 

  13. Shiraishi, J.: Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. Am. J. Roentgenol. 174(1), 71–74 (2000)

    Article  Google Scholar 

  14. Sokooti, Hessam, de Vos, Bob, Berendsen, Floris, Lelieveldt, Boudewijn P.F., Išgum, Ivana, Staring, Marius: Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks. In: Descoteaux, Maxime, Maier-Hein, Lena, Franz, Alfred, Jannin, Pierre, Collins, D.Louis, Duchesne, Simon (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 232–239. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_27

    Chapter  Google Scholar 

  15. Thirion, J.P.: Image matching as a diffusion process: an analogy with Maxwell’s demons. Med. Image Anal. 2(3), 243–260 (1998)

    Article  Google Scholar 

  16. de Vos, Bob D., Berendsen, Floris F., Viergever, Max A., Staring, Marius, Išgum, Ivana: End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network. In: Cardoso, M.Jorge, Arbel, Tal, Carneiro, Gustavo, Syeda-Mahmood, Tanveer, Tavares, João Manuel R.S., Moradi, Mehdi, Bradley, Andrew, Greenspan, Hayit, Papa, João Paulo, Madabhushi, Anant, Nascimento, Jacinto C., Cardoso, Jaime S., Belagiannis, Vasileios, Lu, Zhi (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 204–212. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_24

    Chapter  Google Scholar 

Download references

Acknowledgements

EF is beneficiary of an AXA Research Grant. We thank NVIDIA Corporation for the donation of the Titan X GPU used for this project.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Enzo Ferrante .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ferrante, E., Oktay, O., Glocker, B., Milone, D.H. (2018). On the Adaptability of Unsupervised CNN-Based Deformable Image Registration to Unseen Image Domains. In: Shi, Y., Suk, HI., Liu, M. (eds) Machine Learning in Medical Imaging. MLMI 2018. Lecture Notes in Computer Science(), vol 11046. Springer, Cham. https://doi.org/10.1007/978-3-030-00919-9_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-00919-9_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-00918-2

  • Online ISBN: 978-3-030-00919-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics