Skip to main content
SpringerLink
Log in

Keypoint matching using salient regions and GMM in images with weak textures and repetitive patterns

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the keypoint matching task, most standard methods cannot extract uniform and effective keypoints in images with weak textures and repetitive patterns, nor can they design unique feature descriptors for them. In order to bypass the difficulties encountered in matching ambiguous images, we make some efforts from keypoint extraction and keypoint matching respectively. First, a keypoint extractor based on local energy maxima is designed, which can extract stable features in weak texture regions. Second, to guide keypoints matching based on the Gaussian mixture model framework, an adaptive search method to locate salient regions in images without training is introduced. This is accomplished by quantifying the similarity between the center point and the remaining keypoints. Finally, the iterative equations considering the guiding information is derived. These algorithms alleviate the impacts of weak texture and repetitive patterns on keypoints matching. Experiments on test data show that our method can extract stable and reproducible keypoints, in matching, the precision is more than 60% and the recall is more than 80% when images containing description ambiguities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Alcantarilla PF, Bartoli A, Davison AJ (2012) KAZE features, in: Eur. Springer, Conf. Comput. Vis, pp 214–227 KAZE features

    Google Scholar 

  2. Aldana-Iuit J, Mishkin D, Chum O, Matas J (2016) In the saddle: chasing fast and repeatable features, in: 2016 23rd Int. Conf. Pattern Recognit. ICPR, IEEE, 675–680

  3. Awrangjeb M, Lu G, Fraser CS (2012) Performance comparisons of contour-based corner detectors. IEEE Trans Image Process 21:4167–4179

    Article  MathSciNet  MATH  Google Scholar 

  4. Barroso-Laguna, E. Riba, D. Ponsa, K. (2019) Mikolajczyk, Key. net: Keypoint detection by handcrafted and learned cnn filters, in: Proc. IEEECVF Int. Conf. Comput. Vis. 5836–5844.

  5. Bay H, Ess A, Tuytelaars T, Van Gool (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110:346–359

    Article  Google Scholar 

  6. Beksi WJ, Papanikolopoulos N (2019) A topology-based descriptor for 3D point cloud modeling: theory and experiments. Image Vis Comput 88:84–95

    Article  Google Scholar 

  7. Bian J, Lin W-Y, Matsushita Y, Yeung S-K, Nguyen T-D, Cheng M-M (2017) Gms: Grid-based motion statistics for fast, ultra-robust feature correspondence, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 4181–4190.

  8. Cho M, Sun J, Duchenne O, Ponce J (2014) Finding matches in a haystack: A max-pooling strategy for graph matching in the presence of outliers, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2083–2090.

  9. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection, in: 2005 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. CVPR05, Ieee, pp. 886–893.

  10. Egozi A, Keller Y, Guterman H (2012) A probabilistic approach to spectral graph matching. IEEE Trans Pattern Anal Mach Intell 35:18–27

    Article  Google Scholar 

  11. Fan J, Wu Y, Li M, Liang W, Cao Y (2018) SAR and optical image registration using nonlinear diffusion and phase congruency structural descriptor. IEEE Trans Geosci Remote Sens 56:5368–5379

    Article  Google Scholar 

  12. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24:381–395

    Article  MathSciNet  Google Scholar 

  13. Harris CG, Stephens M (1988) A combined corner and edge detector., in: Alvey Vis. Conf. Citeseer. 10–5244

  14. Hirschmuller H, Scharstein D (2007) Evaluation of cost functions for stereo matching, in: 2007 IEEE Conf. Comput. Vis. Pattern Recognit, IEEE, pp 1–8

    Google Scholar 

  15. Jiang S, Jiang W, Li L, Wang L, Huang W (2020) Reliable and efficient UAV image matching via geometric constraints structured by Delaunay triangulation. Remote Sens 12:3390

    Article  Google Scholar 

  16. Kim H, Lee S (2010) A novel line matching method based on intersection context, in: 2010 IEEE Int. Conf. Robot. Autom, IEEE, pp 1014–1021

  17. Komorowski J, Czarnota K, Trzcinski T, Dabala L, Lynen S (2018) Interest point detectors stability evaluation on ApolloScape dataset, in: Proc. Eur. Conf. Comput. Vis. ECCV Workshop, pp. 0–0

  18. Kovesi P (1999) Image features from phase congruency. Videre J Comput Vis Res 1:1–26

    Google Scholar 

  19. Kovesi P (2000) Phase congruency: a low-level image invariant. Psychol Res 64:136–148

    Article  Google Scholar 

  20. Lawin FJ, Danelljan M, Khan FS, Forssén P-E, Felsberg M (2018) Density adaptive point set registration, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 3829–3837.

  21. Li Z, Mahapatra D, Tielbeek JA, Stoker J, van Vliet, Vos FM (2015) Image registration based on autocorrelation of local structure. IEEE Trans Med Imaging 35:63–75

    Article  Google Scholar 

  22. Lindeberg T (1998) Feature detection with automatic scale selection. Int J Comput Vis 30:79–116

    Article  Google Scholar 

  23. Livi L, Rizzi A (2013) The graph matching problem. Pattern Anal Appl 16:253–283

    Article  MathSciNet  MATH  Google Scholar 

  24. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  25. Ma J, Zhao J, Yuille AL (2015) Non-rigid point set registration by preserving global and local structures. IEEE Trans Image Process 25:53–64

    MathSciNet  MATH  Google Scholar 

  26. Matas J, Chum O, Urban M, Pajdla T (2004) Robust wide-baseline stereo from maximally stable extremal regions. Image Vis Comput 22:761–767

    Article  Google Scholar 

  27. Morago G, Bui Y (2015) Duan, an ensemble approach to image matching using contextual features. IEEE Trans Image Process 24:4474–4487

    Article  MathSciNet  MATH  Google Scholar 

  28. Mustafa A, Kim H, Hilton A (2018) MSFD: multi-scale segmentation-based feature detection for wide-baseline scene reconstruction. IEEE Trans Image Process 28:1118–1132

    Article  MathSciNet  Google Scholar 

  29. Myronenko X (2010) Song, point set registration: coherent point drift. IEEE Trans Pattern Anal Mach Intell 32:2262–2275

    Article  Google Scholar 

  30. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9:62–66

    Article  Google Scholar 

  31. Rocco M Cimpoi R Arandjelović A Torii T Pajdla JS (2018), Neighbourhood consensus networks, ArXiv Prepr. ArXiv181010510

  32. Rocco, Arandjelović R, Sivic J (2020) Efficient neighbourhood consensus networks via submanifold sparse convolutions, in: Eur. Conf. Comput. Vis. Springer, 605–621

  33. Rosten E, Porter R, Drummond T (2008) Faster and better: a machine learning approach to corner detection. IEEE Trans Pattern Anal Mach Intell 32:105–119

    Article  Google Scholar 

  34. Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF, in: 2011 Int. Conf. Comput. Vis, Ieee. 2564–2571

  35. Shi J (1994) Good features to track, in: 1994 proc. IEEE Conf. Comput. Vis. Pattern Recognit, IEEE, pp 593–600

  36. Suárez G, Sfeir JM, Buenaposada L (2020) Baumela, BEBLID: boosted efficient binary local image descriptor, pattern Recognit. Lett. 133:366–372

    Google Scholar 

  37. Sun J, Shen Z, Wang Y, Bao H, Zhou X (2021) LoFTR: Detector-Free Local Feature Matching with Transformers, ArXiv Prepr. ArXiv210400680

  38. Sun Y, Zhao L, Huang S, Yan L, Dissanayake G (2015) Line matching based on planar homography for stereo aerial images. ISPRS J Photogramm Remote Sens 104:1–17

    Article  Google Scholar 

  39. Trajković M, Hedley M (1998) Fast corner detection. Image Vis Comput 16:75–87

    Article  Google Scholar 

  40. Wang L, Chen B, Xu P, Ren H, Fang X, Wan S (2020) Geometry consistency aware confidence evaluation for feature matching. Image Vis Comput 103:103984

    Article  Google Scholar 

  41. Wang Z, Wu F, Hu Z (2009) MSLD: a robust descriptor for line matching. Pattern Recogn 42:941–953

    Article  Google Scholar 

  42. Wu Y, Zhang Q (2012) Zhu, integrated point and edge matching on poor textural images constrained by self-adaptive triangulations. ISPRS J Photogramm Remote Sens 68:40–55

    Article  Google Scholar 

  43. Yi KM, Trulls E, Lepetit V, Fua P (2016) Lift: learned invariant feature transform, in: Eur. Springer, Conf. Comput. Vis, pp 467–483

    Google Scholar 

  44. Yuan X, Chen S, Yuan W, Cai Y (2017) Poor textural image tie point matching via graph theory. ISPRS J Photogramm Remote Sens 129:21–31

    Article  Google Scholar 

  45. Zaafouri A, Sayadi M, Fnaiech F (2012) On the choice of filters bank for phase congruency computing in noisy images, in: 2012 16th IEEE Mediterr. Electrotech. Conf, IEEE, pp 129–132

  46. Zhang S, Yang Y, Yang K, Luo Y, Ong S-H (2017) Point set registration with global-local correspondence and transformation estimation, in: Proc. IEEE Int. Conf. Comput. Vis. 2669–2677.

  47. Zhang X, Qu Y, Yang D, Wang H, Kymer J (2015) Laplacian scale-space behavior of planar curve corners. IEEE Trans Pattern Anal Mach Intell 37:2207–2217

    Article  Google Scholar 

  48. Zhang Y, Li Y, Wu C, Liu B (2021) Attention-guided aggregation stereo matching network. Image Vis Comput 106:104088

    Article  Google Scholar 

  49. Zickler S, Efros A (2007) Detection of Multiple Deformable Objects using PCA-SIFT., in: AAAI. 1127–1133.

  50. Zitnick CL, Ramnath K (2011) Edge foci interest points, in: 2011 Int. Conf. Comput. Vis, IEEE, pp 359–366

Download references

Funding

This work was supported by [the National Natural Science Foundation of China] (Grant number [61673129]); [the Development Project of Ship Situational Intelligent Awareness System] (Grant number [MC-201920-X01]).

Author information

Authors and Affiliations

  1. College of Automation, Harbin Engineering University, Harbin, 150001, China

    Qidan Zhu, Feng Wang, Chengtao Cai & Renjie Qiao

  2. Shanghai Aerospace Control Technology Institute, Shanghai, China

    Haiyang Meng

Authors
  1. Qidan Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chengtao Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haiyang Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Renjie Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feng Wang.

Ethics declarations

Competing interests

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, Q., Wang, F., Cai, C. et al. Keypoint matching using salient regions and GMM in images with weak textures and repetitive patterns. Multimed Tools Appl 81, 23237–23257 (2022). https://doi.org/10.1007/s11042-022-12503-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12503-0

Keywords

Search

Navigation