Skip to main content
SpringerLink
Log in

COREG: a corner based registration technique for multimodal images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a COrner based REGistration technique for multimodal images (referred to as COREG). The proposed technique focuses on addressing large content and scale differences in multimodal images. Unlike traditional multimodal image registration techniques that rely on intensities or gradients for feature representation, we propose to use contour-based corners. First, curvature similarity between corners are for the first time explored for the purpose of multimodal image registration. Second, a novel local descriptor called Distribution of Edge Pixels Along Contour (DEPAC) is proposed to represent the edges in the neighborhood of corners. Third, a simple yet effective way of estimating scale difference is proposed by making use of geometric relationships between corner triplets from the reference and target images. Using a set of benchmark multimodal images and multimodal microscopic images, we will demonstrate that our proposed technique outperforms a state-of-the-art multimodal image registration technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. http://www.nikoninstruments.com/Learn-Explore/Techniques/Brightfield

  2. The Basics of MRI: http://www.cis.rit.edu/htbooks/mri/

References

  1. Awrangjeb M, Lu G (2008) An improved cuvature scale-space corner detector and a robust corner matching technique for transformed image identification. IEEE Trans Image Process (TIP) 17(12):2425–2441

    Article  MATH  Google Scholar 

  2. Awrangjeb M, Lu G (2008) Robust image corner detection based on the chord-to-point distance accumulation technique. IEEE Trans Multimedia (TMM) 10 (6):1059–1072

    Article  Google Scholar 

  3. Awrangjeb M, Lu G, Fraser CS (2012) Performance comparisons of contour-based corner detectors. IEEE Trans Image Process (TIP) 21(9):4167–4179

    Article  MathSciNet  MATH  Google Scholar 

  4. Awrangjeb M, Lu G, Fraser CS, Ravanbakhsh M (2009) A fast corner detector based on the chord-to-point distance accumulation technique International conference on digital image computing: technology and applications (DICTA), pp 519–525

    Google Scholar 

  5. Ba JL, Mnih V, Kavukcuoglu K (2014) Multiple object recognition with visual attention, arXiv:1412.7755

  6. Babenko A, Lempitsky V (2015) Aggregating local deep features for image retrieval IEEE international conference on computer vision (ICCV), pp 1269–1277

    Google Scholar 

  7. Bentoutou Y, Taleb N, et al (2005) An automatic image registration for applications in remote sensing. IEEE Trans Geosci Remote Sens (TGRS) 43(9):2127–2137

    Article  Google Scholar 

  8. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell (TPAMI) 8(6):679–698

    Article  Google Scholar 

  9. Chen J, Tian J (2009) Real-time multi-modal rigid registration based on a novel symmetric-sift descriptor. Prog Nat Sci (PNS) 19(5):643–651

    Article  Google Scholar 

  10. Chen J, Tian J, et al (2010) A partial intensity invariant feature descriptor for multimodal retinal image registration. IEEE Trans Biomed Eng (TBME) 57 (7):1707–1718

    Article  Google Scholar 

  11. Choi BS, Lee JW, Lee JJ, Park KT (2011) A hierarchical algorithm for indoor mobile robot localization using RFID sensor fusion. IEEE Trans Ind Electron (TIE) 58(6):2226–2235

    Article  Google Scholar 

  12. Cristhian A, Fernando B, et al (2012) Multispectral image feature points. Sensors 12(9):12661–12672

    Article  Google Scholar 

  13. Dollar P, Zitnick C (2015) Fast edge detection using structured forests. IEEE Trans Pattern Anal Mach Intell (TPAMI) 37(8):1558–1570

    Article  Google Scholar 

  14. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  15. Ghassabi Z, Shanbehzadeh J, et al (2013) An efficient approach for robust multimodal retinal image registration based on UR-SIFT features and PIIFD descriptors. EURASIP Journal of Image and Video Processing 1(25):1–16

    Google Scholar 

  16. Han JH, Poston TT (2001) Chord-to-point distance accumulation and planar curvature: a new approach to discrete curvature. Pattern Recognit Lett (PRL) 22 (10):1133–1144

    Article  MATH  Google Scholar 

  17. Harris C, Stephens M (1988) A combined corner and edge detector Alvey vision conference, pp 147–151

    Google Scholar 

  18. He XC, Yung NHC (2008) Corner detector based on global and local curvature properties. Opt Eng (OE) 47(5):1–12

    Google Scholar 

  19. Hopp T, Dietzel M, et al (2013) Automatic multimodal 2D/3D breast image registration using biomechanical FEM models and intensity-based optimization. Med Image Anal (MIA) 17(2):209– 218

    Article  Google Scholar 

  20. Hossain MT (2012) An effective technique for multi-modal image registration. PhD Thesis, Monash University

  21. Hossain MT, Lv G, Teng SW, Lu G, Lackmann M (2011) Improved symmetric-SIFT for multi-modal image registration International conference on digital image computing: technology and applications (DICTA), pp 197–202

    Google Scholar 

  22. Jonghye W, Maureen S, Prince JL (2015) Multimodal registration via mutual information incorporating geometric and spatial context. IEEE Trans Image Process (TIP) 24(2):757–769

    Article  Google Scholar 

  23. Kelman A, Sofka M, et al (2007) Keypoints descriptors for matching across multiple image modalities and non-linear intensity variations International conference on computer vision and pattern recognition (CVPR), pp 1–7

    Google Scholar 

  24. Klein S, Staring M, et al (2010) elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging (TMI) 29(1):196–205

    Article  Google Scholar 

  25. Lee JA, Cheng J, et al (2015) A low-dimensional step pattern analysis algorithm with application to multimodal retinal image registration International conference on computer vision and pattern recognition (CVPR), pp 1046–1053

    Google Scholar 

  26. Li Y, Stevenson R (2014) Incorporating global information in feature-based multimodal image registration. J Electron Imaging (JEI) 23(2):023013–1-14

    Google Scholar 

  27. Lindeberg T (1994) Scale-space theory: a basic tool for analyzing structures at different scales. J Appl Stat 21(1-2):225–270

    Article  Google Scholar 

  28. Liu Y, Sadowski SM, et al (2014) Patient specific tumor growth prediction using multimodal images. Med Image Anal (MIA) 18(3):555–566

    Article  Google Scholar 

  29. Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision (IJCV) 2(60):91–110

    Article  Google Scholar 

  30. Lv G (2015) Robust and effective techniques for multi-modal image registration. PhD Thesis, Monash University

  31. Lv G, Teng SW, Lu G, Lackmann M (2013) Detection of structural similarity for multimodal microscopic image registration International conference on digital image computing: techniques and applications (DICTA), pp 1–8

    Google Scholar 

  32. Lv G, Teng SW, Lu G, Lackmann M (2013) Maximizing structural similarity in multimodal biomedical microscopic images for effective registration IEEE international conference on multimedia and expo (ICME), pp 1–6

    Google Scholar 

  33. Lv G, Teng SW, Lu G (2014) A novel multi-modal image registration method based on corners International conference on digital image computing: technology and applications (DICTA), pp 1–8

    Google Scholar 

  34. Lv G, Teng SW, Lu G (2016) Enhancing SIFT-based image registration performance by building and selecting highly discriminating descriptors. Pattern Recognit Lett (PRL) 84:156–162

    Article  Google Scholar 

  35. Myronenko A, Song X (2010) Intensity-based image registration by minimizing residual complexity. IEEE Trans Med Imaging (TMI) 29(11):1882–1891

    Article  Google Scholar 

  36. Nie L, Wang M, Zha ZJ, Chua TS (2012) Oracle in image search: a content-based approach to performance prediction. ACM Trans Inf Syst (TOIS) 30 (2):13:1–13:23

    Article  Google Scholar 

  37. Nie L, Zhang L, et al (2015) Beyond doctors: future health prediction from multimedia and multimodal observations The 23rd ACM international conference on multimedia (MM), pp 591–600

    Google Scholar 

  38. Nigris DD, Collins DL, Arbel T (2012) Multi-modal image registration based on gradient orientations of minimal uncertainty. IEEE Trans Med Imaging (TMI) 31 (12):2343–2354

    Article  Google Scholar 

  39. Pan MS, Jiang JJ, et al (2014) A modified medical image registration. Multimedia Tools and Applications (MTAP) 70(3):1585–1615

    Article  MathSciNet  Google Scholar 

  40. Saleem S, Sablatnig R (2014) A robust SIFT descriptor for multispectral images. IEEE Signal Process Lett 21(4):400–403

    Article  Google Scholar 

  41. Sedaghat A, Ebadi H (2015) Remote sensing image matching based on adaptive binning SIFT descriptor. IEEE Trans Geosci Remote Sens (TGRS) 53(10):5283–5293

    Article  Google Scholar 

  42. Singh R, Khare A (2014) Fusion of multimodal medical images using Daubechies complex wavelet transform CA multiresolution approach. Information Fusion 19:49–60

    Article  Google Scholar 

  43. Sotiras A, Davatzikos C, Paragios N (2013) Deformable meical image registration: a survey. IEEE Trans Med Imaging (TMI) 32(7):1153–1190

    Article  Google Scholar 

  44. Staring M, Heide UA, et al (2009) Registration of cervical MRI using multifeature mutual information. IEEE Trans Med Imaging (TMI) 28(9):1412–1421

    Article  Google Scholar 

  45. Teng SW, Hossain MT, Lu G (2015) Multimodal image registration technique based on improved local feature descriptors. J Electron Imaging (JEI) 24(1):013013–1–17

    Article  Google Scholar 

  46. Teng SW, Sadat RMN, Lu G (2015) Effective and efficient contour-based corner detectors. Pattern Recognit (PR) 48(7):2185–2197

    Article  Google Scholar 

  47. Tsai CL, Li CY, et al (2010) The edge-driven dual-bootstrap iterative closest point algorithm for registration of multimodal fluorescein angiogram sequence. IEEE Trans Med Imaging (TMI) 29(3):636–649

    Article  Google Scholar 

  48. Wachinger C, Navab N (2012) Entropy and laplacian images: structural representations for multi-modal registration. Med Image Anal (MIA) 16(1):1–17

    Article  Google Scholar 

  49. Xia M, Liu B (2004) Image registration by ‘Super-Curves’. IEEE Trans Image Process (TIP) 13(5):720–732

    Article  MathSciNet  Google Scholar 

  50. Xie S, Tu Z (2015) Holistically-nested edge detection International conference on computer vision (ICCV), pp 1395–1403

    Google Scholar 

  51. Xu D, Kasparis T (2007) A hybrid and hierarchical approach to aerial image registration. Int J Pattern Recognit Artif Intell (IJPRAI) 21(3):573–590

    Article  Google Scholar 

  52. Yang G, Stewart CV, et al (2007) Registration of challenging image pairs: initialization, estimation, and decision. IEEE Trans Pattern Anal Mach Intell (TPAMI) 29(11):1973–1989

    Article  Google Scholar 

  53. Zheng Y, Cao Z, Xiao Y (2008) Multi-spectral remote image registration based on SIFT. Electron Lett 44(2):107–108

    Article  Google Scholar 

  54. Zitova B, Flusser J (2003) Image registration methods: a survey. Image Vision Comput (IVC) 21:977–1000

    Article  Google Scholar 

Download references

Acknowledgements

We thank Dr. Mary Vail from Department of Biochemistry & Molecular Biology of Monash University for providing valuable information to accurately describe how our test microscopic images were captured.

Author information

Authors and Affiliations

  1. School of Information, Qilu University of Technology, Jinan, China

    Guohua Lv

  2. Faculty of Science & Technology, Federation University Australia, Victoria, Australia

    Shyh Wei Teng & Guojun Lu

Authors
  1. Guohua Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shyh Wei Teng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guojun Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guohua Lv.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lv, G., Teng, S.W. & Lu, G. COREG: a corner based registration technique for multimodal images. Multimed Tools Appl 77, 12607–12634 (2018). https://doi.org/10.1007/s11042-017-4907-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4907-3

Keywords

Search

Navigation