Skip to main content
Springer Nature Link
Log in

On Image Matching and Feature Tracking for Embedded Systems: A State-of-the-Art

  • Chapter
Advances in Heuristic Signal Processing and Applications

Abstract

This chapter presents a state-of-the-art on image and feature matching in 2D and 3D. Only methods suitable for embedded or wearable real-time system implementation are considered. The implementation may be supported by a dedicated VLSI system. Heuristic guided predictive approaches to image matching are classified as area-based or feature-based. Correlation-based matching, Fourier matching, and mutual information approaches are area-based. Graph, series, and their combinations, including pyramidal or multiresolution algorithms, are feature-based. First, relaxation, maximal clique, tree search, region growing, and dynamic programming methods are briefly described. Next, the correlation-based methods, with a fixed size or adaptive sized window, pyramidal methods, the iterative closest point (ICP) algorithm, and probability (saliency)-based approaches are sketched. Some hardware architectures which support these methods offer new computational models for image matching and image processing. Methods for feature tracking are split into two classes: correlation-based methods and Bayesian methods. Kanade–Lucas–Tomassini (KLT), three-steps/new-three-steps, four-steps, diamond efficient search, and some of their new extensions with inertial data represent the first class, while Kalman and other filters, and their recent improvements represent the second class. The importance of matching is attested by the wide number of applications which include robot navigation, navigation assistance for impaired people, navigation in virtual systems, the processing of medical, satellite and urban imagery, human computer interaction, stereo vision, 3D reconstruction, multimodal fusion. processing, remote sensing, etc.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ackermann, F.: Digital image correlation: performance and potential application in photogrammetry. Photogramm. Rec. 64(11), 429–439 (1984)

    Google Scholar 

  2. Alvarez, L., Gomez, L., Sendra, J.R.: Algebraic lens distortion model estimation. Image Process. On Line (2010)

    Google Scholar 

  3. Ambler, A.P., Barrow, H.G., Brown, C.M., Burstall, R.M., Popplestone, R.J.: A versatile computer controlled assembly system. In: IJCAI, pp. 298–307 (1973)

    Google Scholar 

  4. Arulampalam, M.S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filter for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)

    Article  Google Scholar 

  5. Asada, H., Brady, M.: The curvature primal sketch. IEEE Trans. Pattern Anal. Mach. Intell. 8(1), 2–14 (1986)

    Article  Google Scholar 

  6. Baker, H.H., Binford, T.O.: Depth from edge- and intensity-based stereo. In: Proc. of JCAI, pp. 631–636 (1981)

    Google Scholar 

  7. Becker, F., Wieneke, B., Yuan, J., Schnorr, Ch.: Variational approach to adaptive correlation for motion estimation in particle image velocimetry. In: Rigoll, G. (ed.) Pattern Recognition. LNCS, vol. 5096, pp. 335–344 (2008)

    Chapter  Google Scholar 

  8. Bellman, R.: The theory of dynamic programming. Bull. Am. Math. Soc. 60, 503–516 (1954)

    Article  MATH  Google Scholar 

  9. Besl, P., McKay, N.: A method for registration of 3D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)

    Article  Google Scholar 

  10. Bigot, J., Gamboa, F., Vimond, M.: Estimation of translation, rotation, and scaling between noisy images using the Fourier–Mellin transform. SIAM J. Imaging Sci. 2(2), 614–645 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  11. Bouaziz, S., Pissaloux, E., Merigot, A., Devos, F.: Some hardware and software considerations for the multi-SIMD control strategy of massively parallel machines. In: Proc. of IEEE CompEuro 91 Conf., pp. 180–184 (1991)

    Google Scholar 

  12. Boykov: Fast approximate energy minimization via graph cuts. In: ICCV, pp. 377–384 (1999)

    Google Scholar 

  13. Broida, T.J., Chellapa, R.: Estimation of object motion parameters from a sequence of noisy images. IEEE Trans. Pattern Anal. Mach. Intell. 8(1), 90–99 (1986)

    Article  Google Scholar 

  14. Brown, D.C.: Decentering distortion of lenses. Photogramm. Eng. 7, 444–462 (1966)

    Google Scholar 

  15. Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24, 326–376 (1992)

    Article  Google Scholar 

  16. Bunke, H.: Recent developments in graph matching. In: IEEE ICPR (2000)

    Google Scholar 

  17. Burt, P., Adelson, E.: The Laplacian pyramid as a compact image code. In: IEEE COM-31, pp. 532–540 (1983)

    Google Scholar 

  18. Chen, R., Liu, J.S.: Mixture Kalman filters. J. R. Stat. Soc. B 62(3), 493–508 (2000)

    Article  MATH  Google Scholar 

  19. Chen, Y., Medioni, G.: Object modeling by registration of multiple range images. In: Proc. IEEE Conf. on Robotics and Automation (1991)

    Google Scholar 

  20. Christmas, W., Killter, J., Petrou, M.: Structural matching in computer vision using probabilistic relaxation. IEEE Trans. Pattern Anal. Mach. Intell. 8, 749–764 (1995)

    Article  Google Scholar 

  21. Cole-Rhodes, A., Johnson, K., Le Moigne, J.: Multiresolution registration of remote-sensing images using stochastic gradient. In: Aerosense (2002)

    Google Scholar 

  22. Conte, D., Foggia, P., Sansone, C., Vento, M.: Thirty years of graph matching in pattern recognition. Int. J. Pattern Recognit. Artif. Intell. 18(3), 265–298 (2004)

    Article  Google Scholar 

  23. Corke, P.: An inertial and visual sensing system for a small autonomous helicopter. J. Robot. Syst. 21(2), 43–51 (2004)

    Article  Google Scholar 

  24. Cormen, T.H., Leiserson, C.H., Rivest, R.L., Cliford, C.: Introduction to Algorithms, pp. 540–549. MIT Press and McGraw-Hill, Cambridge (2001)

    MATH  Google Scholar 

  25. Davis, L.: A survey of edge detection technique. Comput. Graph. Image Process. 4(3), 248–270 (1975)

    Article  Google Scholar 

  26. Davison, A.J., Molton, N.D.: MonoSlam: real-time monocamera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)

    Article  Google Scholar 

  27. De Castro, E.D., Morandi, C.: Registration of translated and rotated images using finite Fourier transform. IEEE Trans. Pattern Anal. Mach. Intell. 95, 700–703 (1987)

    Article  Google Scholar 

  28. Deriche, R., Blaszka, T.: Recovering and characterizing image features using an efficient model based approach. In: Proc. of ICCVPR, pp. 530–535 (1993)

    Google Scholar 

  29. Djabelkhir, F., Khamadja, M., Odet, C.: Level set constrained segmentation using local curvature. In: Proc. of ISPA, pp. 152–155 (2007)

    Google Scholar 

  30. Faugeras, O.: Tree-Dimensional Computer Vision. MIT Press, Cambridge (1993)

    Google Scholar 

  31. Forstner, W.: A feature based correspondence algorithm for image matching. Int. Arch. Photogramm. Remote Sens. 26, 150–166 (1986)

    Google Scholar 

  32. Fusiello, A., Roberto, V., Trucco, E.: Efficient stereo with multiple windowing. In: CVPR (1997)

    Google Scholar 

  33. Gao, D., Vasconcelos, N.: Discriminant saliency for visual recognition from cluttered scenes. In: Proc. of Neural Information Processing Systems (NIPS), pp. 481–488 (2004)

    Google Scholar 

  34. Giachetti, A.: Matching techniques to compute image motion. Int. J. Image Vis. Comput. 18(3), 245–258 (2000)

    Google Scholar 

  35. Grimson, E.: Object Recognition by Computer: The Role of Geometric Constraints. MIT Press, Cambridge (1990)

    Google Scholar 

  36. Hansen, D., Pece, A.: Iris tracking with feature free contours. In: Proc. of AMFG, pp. 208–214 (2003)

    Google Scholar 

  37. Haralick, R.M., Elliott, G.L.: Increasing tree search efficiency for constraint satisfaction problems. Artif. Intell. 14, 263–313 (1980)

    Article  Google Scholar 

  38. Harris, C., Stephens, M.: A combined corner and edge detector. In: Proc. of the 4th Alvey Vision Conference, pp. 147–151 (1988)

    Google Scholar 

  39. Hongjun, J., Li, Z.: A new cross diamond search algorithm for block motion estimation. In: Proc. of IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, vol. 3, pp. 357–360 (2004)

    Google Scholar 

  40. Hu, X., Ahuja, N.: Feature extraction and matching as signal detection. Int. J. Pattern Recognit. Artif. Intell. 8(6), 1343–1379 (1994)

    Article  Google Scholar 

  41. Hueckel, M.: An operator which locates edges in digitized pictures. J. Assoc. Comput. Mach. 18(1), 113–125 (1971)

    Article  MATH  Google Scholar 

  42. Itti, L., Koch, C., Niebur, E.: A model of saliency-based attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  43. Kanade, T., Okutomi, M.A.: Stereo matching algorithm with an adaptive window: theory and experiment. IEEE Trans. Pattern Anal. Mach. Intell. 16(9), 920–932 (1994)

    Article  Google Scholar 

  44. Ke, Y., Sukthankar, R.: PC-SIFT: a more distinctive representation for local image descriptors. In: Proc. IEEE CVPR, vol. 2, pp. 506–513 (2004)

    Google Scholar 

  45. Keller, Y., Averbuch, A., Miller, O.: Robust phase correlation. In: Proc. of ICPR, pp. 740–743 (2004)

    Google Scholar 

  46. Khan, J.F.: Empirical mode decomposition based interest point detector. In: IEEE ICASSP, pp. 1317–1320 (2008)

    Google Scholar 

  47. Kishore, M.S., Veerabhadra Rao, K.: Robust correlation tracker. Sâdhana 26(3), 227–236 (2001)

    Article  Google Scholar 

  48. Kitchen, L., Rosenfeld, A.: Gray-level corner detection. Pattern Recognit. Lett. 1(2), 54–69 (1982)

    Google Scholar 

  49. Koga, T., Linuma, K., Hirano, A., Iijima, Y., Ishiguro, T.: Motion-compensated interframe coding for video conferencing. In: Proc. of NTC, pp. C9.6.1–9.6.5 (1981)

    Google Scholar 

  50. Li, R., Zeng, B., Liou, M.: A new three-step search algorithm for block motion estimation. IEEE Trans. Circuits Syst. Video Technol. 4(4), 438–442 (1994)

    Article  Google Scholar 

  51. Liu, H., Guo, B., Feng, Z.: Pseudo-log-polar Fourier transform for image registration. Signal Process. Lett. 13(1), 17–20 (2006)

    Article  Google Scholar 

  52. Long, P., Giraudon, G.: Stereo matching based on contextual line-region primitives. In: Proc. of Int. Conference on Pattern Recognition, pp. 974–977 (1986)

    Google Scholar 

  53. Lotti, J.L., Giraudon, G.: Adaptive window algorithm for aerial image stereo. In: Proc. ICPR, pp. 701–703 (1994)

    Google Scholar 

  54. Lowe, D.: Distinctive image features from scale invariant key points. Int. J. Comput. Vis. 60, 91–110 (2004)

    Article  Google Scholar 

  55. Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. of the 7th Int. Joint Conference on Artificial Intelligence, pp. 674–679 (1981)

    Google Scholar 

  56. Manjunath, B.S., Shekar, C., Chellapa, R.: A new approach to image feature detection with applications. Pattern Recognit. 29, 627–640 (1996)

    Article  Google Scholar 

  57. Marr, D., Poggio, T.: Cooperative computation of stereo disparity. Science 194, 283–287 (1976)

    Article  Google Scholar 

  58. Martin, L., Hede, P., Leroux, Ch., Pissaloux, E.: Orientation filtering: a fast uncalibrated image matching method. In: Proc. of IEEE Int. Conf. on Signal Processing, pp. 1500–1504 (2006)

    Google Scholar 

  59. Martinez, F., Carbone, A., Pissaloux, F.: Radial symmetry guided particle filter for robust iris tracking. In: Proc. of IAPR CAIP (2011)

    Google Scholar 

  60. Masuda, T., Sakaue, K., Yokoya, N.: Registration and integration of multiple range images for 3D model construction. In: Proc. of CVPR (1996)

    Google Scholar 

  61. Maybank, S.: A Probabilistic definition of salient regions for image matching. Neurocomputing (2012, to appear)

    Google Scholar 

  62. Merigot, A.: Associative nets: a graph based parallel computing model. IEEE Trans. Comput. 46(5), 558–571 (1997)

    Article  MathSciNet  Google Scholar 

  63. Moravec, H.: Toward automatic visual obstacle avoidance. In: IJCAI, pp. 584–592 (1977)

    Google Scholar 

  64. Mudge, T.N., Turney, J.L., Volta, R.A.: Automatic generation of salient features for the recognition of partially occluded parts. Robotica 5, 117–127 (1987)

    Article  Google Scholar 

  65. Ni, Y., Merigot, A., Devos, F.: SPHINX-a VLSI processing element chip for pyramid computer. In: ASIC Seminar and Exhibit, 5-3/1-4 (1989)

    Google Scholar 

  66. Paris, S., Hasinoff, S.W., Kautz, J.: Local Laplacian filters: edge-aware image processing with a Laplacian pyramid. ACM Trans. Graph. 30(4) (2011)

    Google Scholar 

  67. Pelillo, M.: Matching free trees, maximal cliques and monotone game dynamics. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1535–1541 (2002)

    Article  Google Scholar 

  68. Pelillo, M., Siddiqi, K., Zucker, S.W.: Matching hierarchical structures using association graphs. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1105–1120 (1999)

    Article  Google Scholar 

  69. Pissaloux, E., Le Coat, F., Bonnin, P., Bezencenet, G., Durbin, F., Tissot, A.: A very fast dynamic programming based parallel algorithm for aerial image matching. In: SPIE ’s 11th Annual Int. Symp. on Aerospace/Defence Sensing, Simulation and Control, vol. 3069, pp. 354–360 (1997)

    Google Scholar 

  70. Pissaloux, E., Le Coat, F., Tissot, A., Durbin, F.: An adaptive parallel system dedicated to projective image matching. In: Proc. of IEEE ICIP (2000)

    Google Scholar 

  71. Pissaloux, E., Chen, Y., Velazquez, R.: Image matching optimization via vision and inertial data fusion: application to navigation of the visually impaired. Int. J. Image Graph. 10(4), 545–555 (2010)

    Article  Google Scholar 

  72. Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Mutual information based registration of medical images: a survey. IEEE Trans. Med. Imaging 22(8), 986–1004 (2003)

    Article  Google Scholar 

  73. Po, L., Ma, W.: A novel four-step search algorithm for block motion estimation. IEEE Trans. Circuits Syst. Video Technol. 6(3), 313–317 (1996)

    Article  Google Scholar 

  74. Pollard, S.B., Mayhew, J.E.W., Frisby, J.P.: PMF: a stereo correspondence algorithm using a disparity gradient constraint. Perception 14, 449–470 (1985)

    Article  Google Scholar 

  75. Quenot, G.M.: The orthogonal algorithm for optical flow detection using dynamic programming. In: IEEE ICASSP, vol. 3, pp. 249–252 (1992)

    Google Scholar 

  76. Ranade, S., Rosenfeld, A.: Point pattern matching by relaxation. Pattern Recognit. 12(4), 269–275 (1980)

    Article  Google Scholar 

  77. Reddy, B.S., Chatterji, B.N.: An FFT-based technique for fast image registration. IEEE Trans. Image Process. 5(8), 1266–1271 (1996)

    Article  Google Scholar 

  78. Remagnino, P., Brand, P., Mohr, R.: Correlation techniques in adaptive template matching with uncalibrated cameras. In: SPIE Int. Symp. on Photonic Sensors and Control for Commercial Applications, vol. III-2356, pp. 252–253 (1994)

    Google Scholar 

  79. Rosenfeld, A., Hummel, R.A., Zucker, S.W.: Scene labelling by relaxation operation. IEEE Trans. Syst. Man Cybern. 6(6), 420–433 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  80. Rusinkiewicz, S., Levoy, M.: Efficient variant of the ICP algorithm. In: Proc. of 3DIM, pp. 145–152 (2001)

    Google Scholar 

  81. Scharstein, D., Szeliski, R.: Stereo matching with nonlinear diffusion. Int. J. Comput. Vis. 28(2), 155–174 (1998)

    Article  Google Scholar 

  82. Schmid, C., Mohr, R., Bauckhage, C.: Evaluation of interest point detectors. Int. J. Comput. Vis. 37(2), 151–172 (2000)

    Article  MATH  Google Scholar 

  83. Shi, J., Tomasi, C.: Good feature to track. In: Proc. IEEE CVPR, pp. 593–600 (1994)

    Google Scholar 

  84. Sidibe, D., Montesinos, Ph., Janaqi, S.: Fast and robust image matching using contextual information and relaxation. In: Proc. of Int. Conference on Computer Vision Theory and Applications, pp. 68–75 (2007)

    Google Scholar 

  85. Smith, S.M., Brady, J.M.: SUSAN: a new approach to low level image processing. Int. J. Comput. Vis. 23(1), 45–78 (1997)

    Article  Google Scholar 

  86. Sojka, E.: A new approach to detecting the corners in digital images. In: IEEE ICIP, vol. 3, pp. 445–448 (2003)

    Google Scholar 

  87. Suveg, I., Vosselman, G.: Mutual information based evaluation of 3D building models. In: ICPR, vol. III, pp. 557–560 (2002)

    Google Scholar 

  88. Tanizaki, H.: Non-Gaussian state-space modeling of nonstationary time series. J. Am. Stat. Assoc. 82, 1032–1063 (1987)

    Google Scholar 

  89. Thevenaz, P., Unser, M.: Optimization of mutual information for multiresolution image registration. IEEE Trans. Image Process. 9(12) (2000)

    Google Scholar 

  90. Tissainayagam, P., Suter, D.: Assessing the performance of corner detectors for point feature tracking applications. Image Vis. Comput., 663–679 (2004)

    Google Scholar 

  91. Trujillo, L., Olague, G.: Automated design of image operators that detect interest points. Evol. Comput. 16(4), 483–507 (2008)

    Article  Google Scholar 

  92. Trujillo, L., Olague, G., de Vega, F.F., Lutton, E.: Evolutionary feature selection for probabilistic object recognition, novel object detection and object saliency estimation using gmms. In: Proc. 18th BMVC, vol. 2, pp. 630–639 (2007)

    Google Scholar 

  93. Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Found. Trends Comput. Graph. Vis. 3(3), 177–280 (2008)

    Article  Google Scholar 

  94. Tzimiropoulos, G., Argyriou, V., Stathaki, T.: Subpixel registration with gradient correlation. IEEE Trans. Image Process. 20(6), 1761–1767 (2011)

    Article  MathSciNet  Google Scholar 

  95. Vincent, E.: On feature point matching, in the calibrated and uncalibrated contexts, between widely and narrowly separated images. PhD thesis, Ottawa Carleton Institute for Computer Science (2004)

    Google Scholar 

  96. Weik, S.: Registration of 3-D partial surface models using luminance and depth information. In: Proc. of 3DIM (1997)

    Google Scholar 

  97. Weng, J., Cohen, P., Herniou, M.: Camera calibration with distortion models and accuracy evaluation. IEEE Trans. Pattern Anal. Mach. Intell. 14(10), 965–980 (1992)

    Article  Google Scholar 

  98. Wu, H., Kitagawa, Y., Wada, T., Kato, T., Chen, Q.: Tracking iris contour with a 3D eye-model for gaze estimation. In: Proc. of ACCV, pp. 688–697 (2007)

    Google Scholar 

  99. Yang, G., Stewart, C.V., Sofka, M., Tsai, L.C.: Registration of challenging image pairs: initialization, estimation and decision. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1973–1989 (2007)

    Article  Google Scholar 

  100. Zhang, Z.: Le probleme de la mise en correspondance: l’etat de l’art. INRIA, RR N’ 2146 (1993)

    Google Scholar 

  101. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

  102. Zhu, S., Ma, K.: A new diamond search algorithm for fast block-matching motion estimation. IEEE Trans. Image Process. 9, 287–290 (2000)

    Article  Google Scholar 

  103. Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)

    Article  Google Scholar 

  104. Zitova, B., Kautsky, J., Peters, G., Flusser, J.: Robust detection of significant points in multiframe images. Pattern Recognit. Lett. 20, 199–206 (1999)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ISIR (Institut des Systèmes Intelligents et de Robotique), UPMC (Université Paris 6), Paris, France

    Edwige E. Pissaloux

  2. Department of Computer Science and Information Systems, Birkbeck College, University of London, London, UK

    Steve Maybank

  3. Mecatrónica y Control de Sistemas, Universidad Panamericana, Aguascalientes, Mexico

    Ramiro Velázquez

Authors
  1. Edwige E. Pissaloux
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steve Maybank
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramiro Velázquez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edwige E. Pissaloux .

Editor information

Editors and Affiliations

  1. Electrical Engineering Department, Jadavpur University, Kolkata, 700032, West Bengal, India

    Amitava Chatterjee

  2. Dept. of Aerospace Engineering, Sharif University of Technology, Azadi Street, Tehran, Iran

    Hadi Nobahari

  3. Laboratory LiSSi, University of Paris 12, av. du Général de Gaulle 61, Créteil, 94010, France

    Patrick Siarry

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Pissaloux, E.E., Maybank, S., Velázquez, R. (2013). On Image Matching and Feature Tracking for Embedded Systems: A State-of-the-Art. In: Chatterjee, A., Nobahari, H., Siarry, P. (eds) Advances in Heuristic Signal Processing and Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37880-5_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-37880-5_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-37879-9

  • Online ISBN: 978-3-642-37880-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics