Skip to main content
SpringerLink
Log in

Probabilistic Image Registration via Deep Multi-class Classification: Characterizing Uncertainty

  • Conference paper
  • First Online:
Book cover Uncertainty for Safe Utilization of Machine Learning in Medical Imaging and Clinical Image-Based Procedures (CLIP 2019, UNSURE 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11840))

Abstract

We present a novel approach to probabilistic image registration that leverages the strengths of deep-learning for modeling agreement between images. We use a deep multi-class classifier trained on different classes of patch pairs, including unrelated, registered, and a collection of discrete displacements between patches. The displacement classes alleviate the need for registration-time optimization by gradient descent; instead, posterior probabilities are used to directly predict expected values of displacements on the lattice of sampled locations. These, in turn, are used to update transformation parameters and the process is iterated. We empirically demonstrate the accuracy of our proposed method on deformable cross-modality registrations of brain MRI, and show improved results compared to Mutual Information based method on challenging data that includes simulated resections. Our approach enables local predictions of registration uncertainty and diagnostics that can indicate areas that seem unrelated in the two images. Uncertainty estimates provide end-users with intuitively actionable information on the quality of registration in interventional and surgical settings.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE CVPR, pp. 9252–9260 (2018)

    Google Scholar 

  2. Cao, X., et al.: Deformable image registration based on similarity-steered CNN regression. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 300–308. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_35

    Chapter  Google Scholar 

  3. Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning for fast probabilistic diffeomorphic registration. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 729–738. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_82

    Chapter  Google Scholar 

  4. Fan, J., Cao, X., Yap, P., Shen, D.: BIRNet: brain image registration using dual-supervised fully convolutional networks. Med. Image Anal. 128–143 (2019)

    Google Scholar 

  5. Gerard, I., Kersten-Oertel, M., Petrecca, K., Sirhan, D., Hall, J., Collins, D.: Brain shift in neuronavigation of brain tumors: a review. Med. Image Anal. 35, 403–420 (2017)

    Article  Google Scholar 

  6. Glocker, B., Komodakis, N., Tziritas, G., Navab, N., Paragios, N.: Dense image registration through MRFs and efficient linear programming. Med. Image Anal. 12(6), 731–741 (2008)

    Article  Google Scholar 

  7. Glocker, B., Paragios, N., Komodakis, N., Tziritas, G., Navab, N.: Optical flow estimation with uncertainties through dynamic MRFs. In: IEEE CVPR, pp. 1–8 (2008)

    Google Scholar 

  8. Heinrich, M., Simpson, I., Papież, B., Brady, M., Schnabel, J.: Deformable image registration by combining uncertainty estimates from supervoxel belief propagation. Med. Image Anal. 27, 57–71 (2016)

    Article  Google Scholar 

  9. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE CVPR, pp. 4700–4708 (2017)

    Google Scholar 

  10. IXI: Information eXtraction from Images. http://brain-development.org/

  11. Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.: Elastix: a toolbox for intensity-based medical image registration. IEEE TMI 29(1), 196–205 (2010)

    Google Scholar 

  12. Krebs, J., et al.: Robust non-rigid registration through agent-based action learning. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 344–352. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_40

    Chapter  Google Scholar 

  13. Le Folgoc, L., Delingette, H., Criminisi, A., Ayache, N.: Quantifying registration uncertainty with sparse bayesian modelling. IEEE TMI 36(2), 607–617 (2016)

    Google Scholar 

  14. Luo, J., Golby, A., Sugiyama, M., Wells III, W., Frisken, S.: Pilot study on verifying the monotonic relationship between error and uncertainty in deformable registration for neurosurgery. arXiv:1908.07709v1

  15. Luo, J., et al.: A feature-driven active framework for ultrasound-based brain shift compensation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 30–38. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_4

    Chapter  Google Scholar 

  16. Risholm, P., Janoos, F., Norton, I., Golby, A., Wells, W.M.: Bayesian characterization of uncertainty in intra-subject non-rigid registration. Med. Image Anal. 17, 538–555 (2013)

    Article  Google Scholar 

  17. Risholm, P., Golby, A.J., Wells, W.M.: Multimodal image registration for preoperative planning and image-guided neurosurgical procedures. Neurosurg. Clin. 22(2), 197–206 (2011)

    Article  Google Scholar 

  18. Rohé, M.-M., Datar, M., Heimann, T., Sermesant, M., Pennec, X.: SVF-Net: learning deformable image registration using shape matching. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 266–274. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_31

    Chapter  Google Scholar 

  19. Sedghi, A., et al.: Semi-supervised image registration using deep learning. In: Proceedings of SPIE the International Society for Optical Engineering, vol. 10951, p. 109511G (2019)

    Google Scholar 

  20. Simonovsky, M., Gutiérrez-Becker, B., Mateus, D., Navab, N., Komodakis, N.: A deep metric for multimodal registration. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 10–18. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46726-9_2

    Chapter  Google Scholar 

  21. Simpson, I.J., Schnabel, J.A., Groves, A.R., Andersson, J.L., Woolrich, M.W.: Probabilistic inference of regularisation in non-rigid registration. NeuroImage 59(3), 2438–2451 (2012)

    Article  Google Scholar 

  22. Sokooti, H., de Vos, B., Berendsen, F., Lelieveldt, B.P.F., Išgum, I., Staring, M.: Nonrigid image registration using multi-scale 3D convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 232–239. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_27

    Chapter  Google Scholar 

  23. de Vos, B., Berendsen, F., Viergever, M., Sokooti, H., Staring, M., Isgum, I.: A deep learning framework for unsupervised affine and deformable image registration. Med. Image Anal. 52, 128–143 (2019)

    Article  Google Scholar 

  24. Popuri, K., Cobzas, D., Jägersand, M.: A variational formulation for discrete registration. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8151, pp. 187–194. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40760-4_24

    Chapter  Google Scholar 

  25. Wang, J., Wells, W.M., Golland, P., Zhang, M.: Efficient laplace approximation for bayesian registration uncertainty quantification. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 880–888. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_99

    Chapter  Google Scholar 

  26. Yang, X., Kwitt, R., Styner, M., Niethammer, M.: Quicksilver: fast predictive image registration-a deep learning approach. NeuroImage 158, 378–396 (2017)

    Article  Google Scholar 

  27. Zagoruyko, S., Komodakis, N.: Learning to compare image patches via convolutional neural networks. In: Proceedings of the IEEE CVPR, pp. 4353–4361 (2015)

    Google Scholar 

  28. Zhang, Y., Brady, M., Smith, S.: Segmentation of brain mr images through a hidden markov random field model and the expectation-maximization algorithm. IEEE TMI 20(1), 45–57 (2001)

    Google Scholar 

Download references

Acknowledgements

Research reported in this publication was supported by Natural Sciences and Engineering Research Council (NSERC) of Canada, the Canadian Institutes of Health Research (CIHR), Ontario Trillium Scholarship, NIH Grant P41EB015898.

Author information

Authors and Affiliations

  1. Medical Informatics Laboratory, Queen’s University, Kingston, ON, Canada

    Alireza Sedghi & Parvin Mousavi

  2. Radiology Department, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

    Tina Kapur, Jie Luo & William M. Wells III

  3. Graduate School of Frontier Sciences, The University of Tokyo, Tokyo, Japan

    Jie Luo

Authors
  1. Alireza Sedghi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tina Kapur
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Parvin Mousavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. William M. Wells III
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alireza Sedghi .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Hayit Greenspan

  2. University College London, London, UK

    Ryutaro Tanno

  3. Fraunhofer Singapore, Nanyang Technological University, Singapore, Singapore

    Marius Erdt

  4. McGill University, Montreal, QC, Canada

    Tal Arbel

  5. ETH Zürich, Zürich, Switzerland

    Christian Baumgartner

  6. Massachusetts Institute of Technology, Harvard Medical School, Cambridge, MA, USA

    Adrian Dalca

  7. University College London, King's College London, London, UK

    Carole H. Sudre

  8. Harvard Medical School, Boston, MA, USA

    William M. Wells

  9. Aachen University of Applied Sciences, Aachen, Germany

    Klaus Drechsler

  10. Children’s National Healthcare System, Washington, D.C., DC, USA

    Marius George Linguraru

  11. Fraunhofer IGD, Darmstadt, Germany

    Cristina Oyarzun Laura

  12. Children's National Healthcare System, Washington, D.C., DC, USA

    Raj Shekhar

  13. Fraunhofer IGD, Darmstadt, Germany

    Stefan Wesarg

  14. ICREA - Universitat Pompeu Fabra, Barcelona, Spain

    Miguel Ángel González Ballester

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sedghi, A., Kapur, T., Luo, J., Mousavi, P., Wells, W.M. (2019). Probabilistic Image Registration via Deep Multi-class Classification: Characterizing Uncertainty. In: Greenspan, H., et al. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging and Clinical Image-Based Procedures. CLIP UNSURE 2019 2019. Lecture Notes in Computer Science(), vol 11840. Springer, Cham. https://doi.org/10.1007/978-3-030-32689-0_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-32689-0_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-32688-3

  • Online ISBN: 978-3-030-32689-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation