Skip to main content
SpringerLink
Log in

Proving the efficiency of template matching-based markerless tracking methods which consider the camera perspective deformations

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Template matching techniques are often used for camera tracking. They provide a good balance between computational cost and robustness to illumination changes. However, they lack robustness to camera orientation and scale changes. Camera movement, and specially rotation, generates perspective deformations that affect the process of patch matching, so the number of inliers (3D–2D correspondences) decreases. This fact affects camera tracking stability. This paper provides the following statistical proof: considering surface normals associated with 3D points substantially increases the number of inliers. So, this paper shows that computing perspective compensation improves the tracking. For instance, in a particular camera path used for experiments in this paper, without compensation, only a \(14\%\) of 3D points projected into the image were found as inliers, while perspective compensation increased that figure up to a \(65\%\). These results must be contextualized in the analysis provided by the paper.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: fast retina keypoint. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 510–517 (2012)

  2. Barrena, N., Sánchez, J.R., García-Alonso, A.: WaPT surface normal estimation for improved template matching in visual tracking. In: Proceedings of the 10th International Conference on Computer Vision Theory and Applications (VISAPP), vol. 3, pp. 496–503. SCITEPRESS (2015)

  3. Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)

    Article  Google Scholar 

  4. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: speeded up robust features. In: Computer Vision–ECCV, pp. 404–417. Springer, New York (2006)

  5. Bellavia, F., Tegolo, D., Valenti, C.: Keypoint descriptor matching with context-based orientation estimation. Image Vis. Comput. 32(9), 559–567 (2014)

    Article  Google Scholar 

  6. Bouguet, J.Y.: Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel Corp. 5(1–10), 4 (2001)

    Google Scholar 

  7. Brunelli, R.: Template Matching Techniques in Computer Vision: Theory and Practice. Wiley, New York (2009)

    Book  Google Scholar 

  8. Charmette, B., Royer, E., Chausse, F.: Vision-based robot localization based on the efficient matching of planar features. Mach. Vis. Appl. 27(4), 415–436 (2016)

    Article  Google Scholar 

  9. Davison, A.J.: Real-time simultaneous localisation and mapping with a single camera. In: Ninth IEEE International Conference on Computer Vision, pp. 1403–1410. IEEE (2003)

  10. Davison, A.J., Reid, I.D., Molton, N.D., Stasse, O.: Monoslam: real-time single camera slam. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1052–1067 (2007)

    Article  Google Scholar 

  11. Furukawa, Y., Ponce, J.: Accurate, dense, and robust multiview stereopsis. IEEE Trans. Pattern Anal. Mach. Intell. 32(8), 1362–1376 (2010)

    Article  Google Scholar 

  12. Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S.M.: Multi-view stereo for community photo collections. In: IEEE 11th International Conference on Computer Vision, ICCV, pp. 1–8. IEEE (2007)

  13. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  14. Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. ISMAR, pp. 225–234. IEEE (2007)

  15. Koenderink, J.J., Van Doorn, A.J.: Affine structure from motion. J. Opt. Soc. Am. 8(2), 377–385 (2007)

    Article  Google Scholar 

  16. Lepetit, V., Fua, P., et al.: Monocular model-based 3d tracking of rigid objects: a survey. Found. Trends Comput. Gr. Vis. 1(1), 1–89 (2005)

    Article  Google Scholar 

  17. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  18. Molton, N., Davison, A.J., Reid, I.: Locally planar patch features for real-time structure from motion. In: BMVC, pp. 1–10 (2004)

  19. Mostofi, N., Moussa, A., Elhabiby, M., El-Sheimy, N.: RGB-D indoor plane-based 3d-modeling using autonomous robot. Int. Arch. Photogram. Remote Sens. Spat. Inf. Sci. 40(1), 301 (2014)

    Article  Google Scholar 

  20. Plackett, R.L.: Karl Pearson and the chi-squared test. Int. Stat. Rev. 51, 59–72 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  21. Rosner, B., Grove, D.: Use of the mann-whitney u-test for clustered data. Stat. Med. 18(11), 1387–1400 (1999)

    Article  Google Scholar 

  22. Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Computer Vision–ECCV 2006, pp. 430–443. Springer, New York (2006)

  23. Schmidt, A.: Prediction-based perspective warping of feature template for improved visual slam accuracy. In: Man–Machine Interactions, vol. 4, pp. 169–177. Springer, New York (2016)

  24. Sivaraman, S., Trivedi, M.M.: Looking at vehicles on the road: a survey of vision-based vehicle detection, tracking, and behavior analysis. IEEE Trans. Intell. Transp. Syst. 14(4), 1773–1795 (2013)

    Article  Google Scholar 

  25. Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, New York (2010)

    MATH  Google Scholar 

  26. Watman, C., Austin, D., Barnes, N., Overett, G., Thompson, S.: Fast sum of absolute differences visual landmark detector. In: Robotics and Automation, 2004. Proceedings. ICRA’04. 2004 IEEE International Conference, vol. 5, pp. 4827–4832. IEEE (2004)

  27. Wu, C., Clipp, B., Li, X., Frahm, J.M., Pollefeys, M.: 3d model matching with viewpoint-invariant patches (VIP). In: Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference, pp. 1–8. IEEE (2008)

Download references

Acknowledgements

The authors wish to thank to David Oyarzun, former head of the Interactive and Computer Graphics department of Vicomtech-Ik4, for his expert advise.

Author information

Authors and Affiliations

  1. Vicomtech-Ik4 Paseo Mikeletegi N57, 20009, Donostia-San Sebastián, Spain

    Nagore Barrena, Jairo R. Sánchez & Ramón J. Ugarte

  2. Facultad de Informática, Universidad del País Vasco, Paseo de Manuel Lardizabal 1, 20018, Donostia-San Sebastián, Spain

    Alejandro García Alonso

Authors
  1. Nagore Barrena
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jairo R. Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramón J. Ugarte
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alejandro García Alonso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nagore Barrena.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barrena, N., Sánchez, J.R., Ugarte, R.J. et al. Proving the efficiency of template matching-based markerless tracking methods which consider the camera perspective deformations. Machine Vision and Applications 29, 573–584 (2018). https://doi.org/10.1007/s00138-018-0914-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-018-0914-2

Keywords

Search

Navigation