Skip to main content
Springer Nature Link
Log in

Hessian-Based Similarity Metric for Multimodal Medical Image Registration

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops (MICCAI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14394))

  • 654 Accesses

Abstract

One of the fundamental elements of both traditional and certain deep learning medical image registration algorithms is measuring the similarity/dissimilarity between two images. In this work, we propose an analytical solution for measuring similarity between two different medical image modalities based on the Hessian of their intensities. First, assuming a functional dependence between the intensities of two perfectly corresponding patches, we investigate how their Hessians relate to each other. Secondly, we suggest a closed-form expression to quantify the deviation from this relationship, given arbitrary pairs of image patches. We propose a geometrical interpretation of the new similarity metric and an efficient implementation for registration. We demonstrate the robustness of the metric to intensity nonuniformities using synthetic bias fields. By integrating the new metric in an affine registration framework, we evaluate its performance for MRI and ultrasound registration in the context of image-guided neurosurgery using target registration error and computation time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arbel, T., et al.: Automatic non-linear MRI-ultrasound registration for the correction of intra-operative brain deformations. Comput. Aided Surg. 9(4), 123–136 (2004). https://doi.org/10.3109/10929080500079248

    Article  Google Scholar 

  2. Lee, D., et al.: Learning similarity measure for multi-modal 3D image registration. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 186–193. IEEE (2009). https://doi.org/10.1109/CVPR.2009.5206840

  3. De Nigris, D., et al.: Multi-modal image registration based on gradient orientations of minimal uncertainty. IEEE Trans. Med. Imag. 31(12), 2343–2354 (2012). https://doi.org/10.1109/TMI.2012.2218116

    Article  Google Scholar 

  4. Drouin, S., et al.: IBIS: an OR ready open-source platform for image-guided neurosurgery. Int. J. Comput. Assist. Radiol. Surg. 12(3), 363–378 (2017). https://doi.org/10.1007/s11548-016-1478-0

    Article  Google Scholar 

  5. Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel enhancement filtering. In: Wells, W.M., Colchester, A., Delp, S. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 130–137. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0056195

  6. Fuerst, B., et al.: Automatic ultrasound-MRI registration for neurosurgery using the 2D and 3D LC2 Metric. Med. Image Anal. 18(8), 1312–1319 (2014). https://doi.org/10.1016/j.media.2014.04.008

    Article  Google Scholar 

  7. Haber, E., Modersitzki, J.: Beyond mutual information: a simple and robust aternative. In: Bildverarbeitung für die Medizin 2005, pp. 350–354. Springer, Heidelberg (2005). https://doi.org/10.1007/3-540-26431-0_72

  8. Haskins, G., et al.: Learning deep similarity metric for 3D MR-TRUS image registration. Int. J. Comput. Assist. Radiol. Surg. 14(3), 417–425 (2019). https://doi.org/10.1007/s11548-018-1875-7

    Article  Google Scholar 

  9. Jiang, D., et al.: miLBP: a robust and fast modality-independent 3D LBP for multimodal deformable registration. Int. J. Comput. Assist. Radiol. Surg. 11(6), 997–1005 (2016). https://doi.org/10.1007/s11548-016-1407-2

    Article  MathSciNet  Google Scholar 

  10. Karacali, B.: Fully elastic multi-modality image registration using mutual information. In: 2004 2nd IEEE International Symposium on Biomedical Imaging: Macro to Nano (IEEE Cat No. 04EX821), pp. 1455–1458 IEEE (2004). https://doi.org/10.1109/ISBI.2004.1398823

  11. Loeckx, D., et al.: Nonrigid image registration using conditional mutual information. IEEE Trans. Med. Imag. 29(1), 19–29 (2010). https://doi.org/10.1109/TMI.2009.2021843

    Article  Google Scholar 

  12. Maes, F., et al.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imag. 16(2), 187–198 (1997). https://doi.org/10.1109/42.563664

    Article  Google Scholar 

  13. Mercier, L., et al.: Online database of clinical MR and ultrasound images of brain tumors. Med. Phys. 39(6Part1), 3253–3261 (2012). https://doi.org/10.1118/1.4709600

  14. Manera, A.L., et al.: CerebrA, registration and manual label correction of Mindboggle-101 atlas for MNI-ICBM152 template. Sci. Data 7(1), 237 (2020). https://doi.org/10.1038/s41597-020-0557-9

    Article  Google Scholar 

  15. Mezura-Montes, E., et al.: A comparative study of differential evolution variants for global optimization. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 485–492 ACM, New York (2006). https://doi.org/10.1145/1143997.1144086

  16. Oliveira, F.P.M., Tavares, J.M.R.S.: Medical image registration: a review. Comput. Methods Biomechan. Biomed. Eng. 17(2), 73–93 (2014). https://doi.org/10.1080/10255842.2012.670855

    Article  Google Scholar 

  17. Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Image registration by maximization of combined mutual information and gradient information. In: Delp, S.L., DiGoia, A.M., Jaramaz, B. (eds.) MICCAI 2000. LNCS, vol. 1935, pp. 452–461. Springer, Heidelberg (2000). https://doi.org/10.1007/978-3-540-40899-4_46

  18. Pradhan, S., Patra, D.: Enhanced mutual information based medical image registration. IET Image Proc. 10(5), 418–427 (2016). https://doi.org/10.1049/iet-ipr.2015.0346

    Article  Google Scholar 

  19. Rivaz, H., et al.: Nonrigid Registration of Ultrasound and MRI Using Contextual Conditioned Mutual Information. IEEE Trans. Med. Imaging 33(3), 708–725 (2014). https://doi.org/10.1109/TMI.2013.2294630

    Article  Google Scholar 

  20. Rivaz, H., et al.: Self-similarity weighted mutual information: a new nonrigid image registration metric. Med. Image Anal. 18(2), 343–358 (2014). https://doi.org/10.1016/j.media.2013.12.003

    Article  Google Scholar 

  21. Roche, A., et al.: The correlation ratio as a new similarity measure for multimodal image registration. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI’98: First International Conference Cambridge, October, pp. 1115–1124 (1998). https://doi.org/10.1007/BFb0056301

  22. Simonovsky, M., et al.: Presented at the A Deep Metric for Multimodal Registration (2016). https://doi.org/10.1007/978-3-319-46726-9_2

  23. Storn, R., Price, K.: Minimizing the real functions of the ICEC 1996 contest by differential evolution. In: Proceedings of IEEE International Conference on Evolutionary Computation, pp. 842–844. IEEE (1996). https://doi.org/10.1109/ICEC.1996.542711

  24. Viola, P., Wells, W.M.: Alignment by maximization of mutual information. Int. J. Comput. Vis. 24(2), 137–154 (1997). https://doi.org/10.1023/A:1007958904918

    Article  Google Scholar 

  25. Wachinger, C., Navab, N.: Entropy and Laplacian images: structural representations for multi-modal registration. Med. Image Anal. 16(1), 1–17 (2012). https://doi.org/10.1016/j.media.2011.03.001

    Article  Google Scholar 

  26. Wein, W., et al.: Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention. Med. Image Anal. 12(5), 577–585 (2008). https://doi.org/10.1016/j.media.2008.06.006

    Article  Google Scholar 

  27. Xiao, Y., et al.: REtroSpective evaluation of cerebral tumors (RESECT): a clinical database of pre-operative MRI and intra-operative ultrasound in low-grade glioma surgeries. Med. Phys. 44(7), 3875–3882 (2017). https://doi.org/10.1002/mp.12268

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, McGill University, Montreal, QC, Canada

    Mohammadreza Eskandari & D. Louis Collins

  2. McConnell Brain Imaging Center, Montreal Neurological Institute and Hospital, Montreal, QC, Canada

    Mohammadreza Eskandari, Houssem-Eddine Gueziri & D. Louis Collins

  3. Department of Neurology and Neurosurgery, McGill University, Montreal, QC, Canada

    D. Louis Collins

Authors
  1. Mohammadreza Eskandari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Houssem-Eddine Gueziri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. Louis Collins
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammadreza Eskandari .

Editor information

Editors and Affiliations

  1. Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA

    Jonghye Woo

  2. Radboud University Nijmegen Medical Center, Nijmegen, The Netherlands

    Alessa Hering

  3. The Netherlands Cancer Institute, Amsterdam, The Netherlands

    Wilson Silva

  4. Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA

    Xiang Li

  5. A*STAR, Institute of High Performance Computing, Singapore, Singapore

    Huazhu Fu

  6. Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA

    Xiaofeng Liu

  7. Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA

    Fangxu Xing

  8. University of Maryland Baltimore County, Baltimore, MD, USA

    Sanjay Purushotham

  9. National Institutes of Health, Bethesda, MD, USA

    Tejas S. Mathai

  10. National Institutes of Health, Bethesda, MD, USA

    Pritam Mukherjee

  11. Radboud University Nijmegen Medical Center, Nijmegen, The Netherlands

    Max De Grauw

  12. The Netherlands Cancer Institute, Amsterdam, The Netherlands

    Regina Beets Tan

  13. The Netherlands Cancer Institute, Amsterdam, The Netherlands

    Valentina Corbetta

  14. University Hospital Freiburg, Freiburg, Germany

    Elmar Kotter

  15. University of Bern, Bern, Switzerland

    Mauricio Reyes

  16. University of Tübingen, Tübingen, Germany

    Christian F. Baumgartner

  17. Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA

    Quanzheng Li

  18. University of Southern California, Los Angeles, CA, USA

    Richard Leahy

  19. Peking University, Beijing, China

    Bin Dong

  20. Hong Kong University of Science and Technology, Kowloon, Hong Kong

    Hao Chen

  21. Vanderbilt University, Nashville, TN, USA

    Yuankai Huo

  22. University of Sydney, Camperdown, NSW, Australia

    Jinglei Lv

  23. Institute of High Performance Computing, Singapore, Singapore

    Xinxing Xu

  24. Hong Kong University of Science and Technology, Hong Kong, China

    Xiaomeng Li

  25. Inception Institute of Artificial Intelligence, Abu Dhabi, United Arab Emirates

    Dwarikanath Mahapatra

  26. University of Alberta, Edmonton, AB, Canada

    Li Cheng

  27. LITIS, University of Rouen, Rouen, France

    Caroline Petitjean

  28. ImViA Laboratory, University of Burgundy, Dijon, France

    Benoît Presles

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 2179 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Eskandari, M., Gueziri, HE., Collins, D.L. (2023). Hessian-Based Similarity Metric for Multimodal Medical Image Registration. In: Woo, J., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops. MICCAI 2023. Lecture Notes in Computer Science, vol 14394. Springer, Cham. https://doi.org/10.1007/978-3-031-47425-5_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47425-5_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47424-8

  • Online ISBN: 978-3-031-47425-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships