Skip to main content

Advertisement

SpringerLink
Log in

ARENA: Inter-modality affine registration using evolutionary strategy

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Image fusion of different imaging modalities renders valuable information to clinicians. In this paper, we proposed an automatic multimodal registration method to register intra-operative ultrasound images (US) to preoperative magnetic resonance images (MRI) in the context of image-guided neurosurgery.

Methods

We employed refined correlation ratio as a similarity metric for our intensity-based image registration method. We deem MRI as the fixed image (\(I_\mathrm{f}\)) and US as the moving image (\(I_\mathrm{m}\)) and then transform \(I_\mathrm{m}\) to align with \(I_\mathrm{f}\). We utilized the covariance matrix adaptation evolutionary strategy to find the optimal affine transformation in registration of \(I_\mathrm{m}\) to \(I_\mathrm{f}\).

Results

We applied our method on the publicly available retrospective evaluation of cerebral tumors (RESECT) database and Montreal Neurological Institute’s brain images of tumors for evaluation (BITE) database. We validated the results qualitatively and quantitatively. Qualitative validation is conducted (by the three authors) through overlaying pre- and post-registration US and MRI to allow visual assessment of the alignment. Quantitative validation is performed by utilizing the corresponding landmarks in the databases for the preoperative MRI and the intra-operative US. Average mean target registration error (mTRE) has been reduced from \(5.40\pm 4.27\) to \(2.77\pm 1.13\) in 22 patients in the RESECT database and from \(4.12\pm 2.03\) to \(2.82\pm 0.72\) in the BITE database. A nonparametric statistical analysis performed using the Wilcoxon rank sum test shows that there is a significant difference between pre- and post-registration mTREs with a p value of \(0.0058\,(p<0.05)\) for the RESECT database and \(0.0483\,(p<0.05)\) for the BITE database.

Conclusions

The proposed fully automatic registration method significantly improved the alignment of MRI and US images and can therefore be used to reduce the misalignment of US and MRI caused by brain shift, calibration errors, and patient to MRI transformation matrix.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Damas S, Cordón O, Santamaría J (2011) Medical image registration using evolutionary computation: an experimental survey. IEEE Comput Intell Mag 6(4):26–42

    Article  Google Scholar 

  2. Ma J, Zhou H, Zhao J, Gao Y, Jiang J, Tian J (2015) Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans Geosci Remote Sens 53(12):6469–6481

    Article  Google Scholar 

  3. James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Inf Fusion 19:4–19

    Article  Google Scholar 

  4. Li S, Kang X, Fang L, Hu J, Yin H (2017) Pixel-level image fusion: a survey of the state of the art. Inf Fusion 33:100–112

    Article  Google Scholar 

  5. Yang Y, Que Y, Huang S, Lin P (2016) Multimodal sensor medical image fusion based on type-2 fuzzy logic in nsct domain. IEEE Sens J 16(10):3735–3745

    Article  Google Scholar 

  6. Golby AJ (2015) Image-guided neurosurgery. Academic Press, Cambridge

    Google Scholar 

  7. Besharati Tabrizi L, Mahvash M (2015) Augmented reality–guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg 123(1):206–211

    Article  PubMed  Google Scholar 

  8. Maurer CR, Fitzpatrick JM (1993) A review of medical image registration. Interact Image Guid Neurosurg 1:17–44

    Google Scholar 

  9. Gerard IJ, Kersten-Oertel M, Petrecca K, Sirhan D, Hall JA, Collins DL (2017) Brain shift in neuronavigation of brain tumors: a review. Med Image Anal 35:403–420

    Article  PubMed  Google Scholar 

  10. Nag S (2017) Image registration techniques: a survey. arXiv preprint arXiv:1712.07540

  11. Maintz JA, Viergever MA (1998) A survey of medical image registration. Med Image Anal 2(1):1–36

    Article  PubMed  CAS  Google Scholar 

  12. Gong M, Zhao S, Jiao L, Tian D, Wang S (2014) A novel coarse-to-fine scheme for automatic image registration based on sift and mutual information. IEEE Trans Geosci Remote Sens 52(7):4328–4338

    Article  Google Scholar 

  13. Johnson HJ, Christensen GE (2002) Consistent landmark and intensity-based image registration. IEEE Trans Med Imaging 21(5):450–461

    Article  PubMed  CAS  Google Scholar 

  14. Zitova B, Flusser J (2003) Image registration methods: a survey. Image Vis Comput 21(11):977–1000

    Article  Google Scholar 

  15. Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Comput Electr Eng 40(1):16–28

    Article  Google Scholar 

  16. Rueckert D, Aljabar P (2010) Nonrigid registration of medical images: theory, methods, and applications [applications corner]. IEEE Signal Process Mag 27(4):113–119

    Article  Google Scholar 

  17. Sotiras A, Davatzikos C, Paragios N (2013) Deformable medical image registration: a survey. IEEE Trans Med Imaging 32(7):1153–1190

    Article  PubMed  PubMed Central  Google Scholar 

  18. Yan CX, Goulet B, Pelletier J, Chen SJ-S, Tampieri D, Collins DL (2011) Towards accurate, robust and practical ultrasound-ct registration of vertebrae for image-guided spine surgery. Int J Comput Assist Radiol Surg 6(4):523–537

    Article  PubMed  Google Scholar 

  19. Gill S, Abolmaesumi P, Fichtinger G, Boisvert J, Pichora D, Borshneck D, Mousavi P (2012) Biomechanically constrained groupwise ultrasound to ct registration of the lumbar spine. Med Image Anal 16(3):662–674

    Article  PubMed  Google Scholar 

  20. Hacihaliloglu I, Rasoulian A, Rohling RN, Abolmaesumi P (2014) Local phase tensor features for 3-d ultrasound to statistical shape+ pose spine model registration. IEEE Trans Med Imaging 33(11):2167–2179

    Article  PubMed  Google Scholar 

  21. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV (2018) An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9252–9260

  22. Weistrand O, Svensson S (2015) The anaconda algorithm for deformable image registration in radiotherapy. Med Phys 42(1):40–53

    Article  PubMed  Google Scholar 

  23. Zhao B, Christensen GE, Hyun Song J, Pan Y, Gerard SE, Reinhardt JM, Du K, Patton T, Bayouth JM, Hugo GD (2016) Tissue-volume preserving deformable image registration for 4dct pulmonary images. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 41–49

  24. Maes F, Loeckx D, Vandermeulen D, Suetens P (2015) Image registration using mutual information. In: Paragios N, Duncan J, Ayache N (eds) Handbook of biomedical imaging. Springer, Boston, MA

    Google Scholar 

  25. Roche A, Malandain G, Ayache N, Pennec X (1998) Multimodal image registration by maximization of the correlation ratio. PhD thesis, INRIA

  26. Roche A, Pennec X, Rudolph M, Auer D, Malandain G, Ourselin S, Auer LM, Ayache N (2000) Generalized correlation ratio for rigid registration of 3d ultrasound with mr images. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 567–577

  27. Rivaz H, Collins DL (2015) Deformable registration of preoperative mr, pre-resection ultrasound, and post-resection ultrasound images of neurosurgery. Int J Comput Assist Radiol Surg 10(7):1017–1028

    Article  PubMed  Google Scholar 

  28. Masoumi N, Xiao Y, Rivaz H (2017) Marcel (inter-modality affine registration with correlation ratio): an application for brain shift correction in ultrasound-guided brain tumor resection. In: International MICCAI Brainlesion workshop. Springer, pp 55–63

  29. Rivaz H, Chen SJ-S, Collins DL (2015) Automatic deformable mr-ultrasound registration for image-guided neurosurgery. IEEE Trans Med Imaging 34(2):366–380

    Article  PubMed  Google Scholar 

  30. Hansen N, Ostermeier A (1996) Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation. In: Evolutionary computation, 1996., Proceedings of IEEE international conference on. IEEE, pp 312–317

  31. Xiao Y, Fortin M, Unsgård G, Rivaz H, Reinertsen I (2017) Retrospective evaluation of cerebral tumors (resect): a clinical database of pre-operative mri and intra-operative ultrasound in low-grade glioma surgeries. Med Phys 44:3875–3882

    Article  PubMed  Google Scholar 

  32. Machado I, Toews M, Luo J, Unadkat P, Essayed W, George E, Teodoro P, Carvalho H, Martins J, Golland P, Pieper S (2018) Non-rigid registration of 3d ultrasound for neurosurgery using automatic feature detection and matching. Int J Comput Assist Radiol Surg 13:1525–1538

    Article  PubMed  Google Scholar 

  33. Mercier L, Del Maestro RF, Petrecca K, Araujo D, Haegelen C, Collins DL (2012) Online database of clinical mr and ultrasound images of brain tumors. Med Phys 39(6 Part1):3253–3261

    Article  PubMed  Google Scholar 

  34. Klein S, Staring M, Pluim JP (2007) Evaluation of optimization methods for nonrigid medical image registration using mutual information and b-splines. IEEE Trans Image Process 16(12):2879–2890

    Article  PubMed  Google Scholar 

  35. Winter S, Brendel B, Pechlivanis I, Schmieder K, Igel C (2008) Registration of ct and intraoperative 3-d ultrasound images of the spine using evolutionary and gradient-based methods. IEEE Trans Evol Comput 12(3):284–296

    Article  Google Scholar 

  36. Gong RH, Abolmaesumi P (2008) 2d/3d registration with the cma-es method. In: Medical imaging 2008: visualization, image-guided procedures, and modeling. International Society for Optics and Photonics, vol 6918, p 69181M

  37. Otake Y, Armand M, Armiger RS, Kutzer MD, Basafa E, Kazanzides P, Taylor RH (2012) Intraoperative image-based multiview 2d/3d registration for image-guided orthopaedic surgery: incorporation of fiducial-based c-arm tracking and gpu-acceleration. IEEE Trans Med Imaging 31(4):948–962

    Article  PubMed  Google Scholar 

  38. Reinhard E, Heidrich W, Debevec P, Pattanaik S, Ward G, Myszkowski K (2010) High dynamic range imaging: acquisition, display, and image-based lighting. Morgan Kaufmann, Burlington

    Google Scholar 

  39. Fischer B, Modersitzki J (2008) Ill-posed medicine–an introduction to image registration. Inverse Probl 24(3):034008

    Article  Google Scholar 

  40. Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evol Comput 9(2):159–195

    Article  PubMed  CAS  Google Scholar 

  41. “Cma-es in matlab-yarpiz”

  42. Wein W, Ladikos A, Fuerst B, Shah A, Sharma K, Navab N (2013) Global registration of ultrasound to mri using the lc 2 metric for enabling neurosurgical guidance. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 34–41

  43. Heinrich MP, Jenkinson M, Papież BW, Brady M, Schnabel JA (2013) Towards realtime multimodal fusion for image-guided interventions using self-similarities. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 187–194

  44. Daga P, Winston G, Modat M, White M, Mancini L, Cardoso MJ, Symms M, Stretton J, McEvoy AW, Thornton J, Micallef C (2012) Accurate localization of optic radiation during neurosurgery in an interventional mri suite. IEEE Trans Med Imaging 31(4):882–891

    Article  PubMed  Google Scholar 

  45. Fitzpatrick JM (2009) Fiducial registration error and target registration error are uncorrelated. In: Medical imaging 2009: visualization, image-guided procedures, and modeling, International Society for Optics and Photonics, vol 7261, p 726102 (2009)

  46. Zhong X, Bayer S, Ravikumar N, Strobel N, Birkhold A, Kowarschik M, Fahrig R, Maier A (2018) Resolve intraoperative brain shift as imitation game. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 129–137

  47. Hong J, Park H (2018) Non-linear approach for mri to intra-operative us registration using structural skeleton. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 138–145

  48. Wein W (2018) Brain-shift correction with image-based registration and landmark accuracy evaluation. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 146–151

  49. Sun L, Zhang S (2018) Deformable mri-ultrasound registration using 3d convolutional neural network. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 152–158

  50. Heinrich MP (2018) Intra-operative ultrasound to mri fusion with a public multimodal discrete registration tool. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 159–164

  51. Machado I, Toews M, Luo J, Unadkat P, Essayed W, George E, Teodoro P, Carvalho H, Martins J, Golland P (2018) Deformable mri-ultrasound registration via attribute matching and mutual-saliency weighting for image-guided neurosurgery. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 165–171

  52. Drobny D, Vercauteren T, Ourselin S, Modat M (2018) Registration of mri and ius data to compensate brain shift using a symmetric block-matching based approach. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 172–178

  53. Shams R, Boucher M-A, Kadoury S (2018) Intra-operative brain shift correction with weighted locally linear correlations of 3dus and mri. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 179–184

  54. Gibbons JD, Chakraborti S (2011) Nonparametric statistical inference. In: Lovric M (ed) International encyclopedia of statistical science. Springer, Berlin, Heidelberg

  55. Avants BB, Tustison NJ, Song G, Cook PA, Klein A, Gee JC (2011) A reproducible evaluation of ants similarity metric performance in brain image registration. Neuroimage 54(3):2033–2044

    Article  PubMed  Google Scholar 

  56. Modat M, Cardoso MJ, Daga P, Cash D, Fox NC, Ourselin S (2012) Inverse-consistent symmetric free form deformation. In: International workshop on biomedical image registration. Springer, pp 79–88

  57. Xiao Y, Eikenes L, Reinertsen I, Rivaz H (2018) Nonlinear deformation of tractography in ultrasound-guided low-grade gliomas resection. Int J Comput Assist Radiol Surg 13(3):457–467

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This work is funded by Natural Science Engineering Council of Canada (NSERC) Grant RGPIN-2015-04136.

Author information

Authors and Affiliations

  1. PERFORM Centre, Concordia University, Montreal, Canada

    Nima Masoumi & Hassan Rivaz

  2. Department of Electrical and Computer Engineering, Concordia University, Montreal, Canada

    Nima Masoumi & Hassan Rivaz

  3. Robarts Research Institute, Western University, London, Canada

    Yiming Xiao

Authors
  1. Nima Masoumi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiming Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hassan Rivaz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nima Masoumi.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Ethical standard

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Masoumi, N., Xiao, Y. & Rivaz, H. ARENA: Inter-modality affine registration using evolutionary strategy. Int J CARS 14, 441–450 (2019). https://doi.org/10.1007/s11548-018-1897-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-018-1897-1

Keywords

Search

Navigation