Skip to main content
SpringerLink
Log in

Tracking rigid objects using integration of model-based and model-free cues

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Model-based 3-D object tracking has earned significant importance in areas such as augmented reality, surveillance, visual servoing, robotic object manipulation and grasping. Key problems to robust and precise object tracking are the outliers caused by occlusion, self-occlusion, cluttered background, reflections and complex appearance properties of the object. Two of the most common solutions to the above problems have been the use of robust estimators and the integration of visual cues. The tracking system presented in this paper achieves robustness by integrating model-based and model-free cues together with robust estimators. As a model-based cue, a wireframe edge model is used. As model-free cues, automatically generated surface texture features are used. The particular contribution of this work is the integration framework where not only polyhedral objects are considered. In particular, we deal also with spherical, cylindrical and conical objects for which the complete pose cannot be estimated using only wireframe models. Using the integration with the model-free features, we show how a full pose estimate can be obtained. Experimental evaluation demonstrates robust system performance in realistic settings with highly textured objects and natural backgrounds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Vacchetti L., Lepetit V., Fua P.: Stable real-time 3D tracking using online and offline information. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1385–1391 (2004)

    Article  Google Scholar 

  2. Vincze M., Ayromlou M., Ponweiser M., Zillich M.: Edge projected integration of image and model cues for robust model-based object tracking. Int. J. Robot. Res. 20(7), 533–552 (2001)

    Article  Google Scholar 

  3. Taylor, G., Kleeman, L.: Fusion of multimodal visual cues for model-based object tracking. In: Australiasian Conf. on Robotics and Automation, Brisbane, Australia (2003)

  4. Harris C.: Tracking with rigid models. In: Blake, A., Yuille, A. (eds) Active Vision, Ch. 4, pp. 59–73. MIT Press, Cambridge (1992)

    Google Scholar 

  5. Comport, A., Marchand, E., Chaumette, F.: A real-time tracker for markerless augmented reality. In: IEEE Int. Symp. on Mixed and Augmented Reality pp. 36–45 (2003)

  6. Vincze, M., Ayromlou, M., Kubinger, W.: An integrating framework for robust real-time 3D object tracking. In: Int. Conf. on Comp. Vis. Syst., ICVS’99, pp. 135–150 (1999)

  7. Koller D., Daniilidis K., Nagel H.: Model-based object tracking in monocular image sequences of road traffic scenes. Int. J. Comput. Vis. 10(3), 257–281 (1993)

    Article  Google Scholar 

  8. Drummond T., Cipolla R.: Real-time visual tracking of complex structures. IEEE Trans. PAMI 24(7), 932–946 (2002)

    Google Scholar 

  9. Malis E., Chaumette F., Boudet S.: 2-1/2-d visual servoing. IEEE Trans. Robot. Autom. 15(2), 238–250 (1999)

    Article  Google Scholar 

  10. Jurie F., Dhome M.: Real time tracking of 3D objects: an efficient and robust approach. Pattern Recognit. 35, 317–328 (2002)

    Article  MATH  Google Scholar 

  11. Dickmanns E.D., Graefe V.: Dynamic monocular machine vision. Mach. Vis. Appl. 1, 223–240 (1988)

    Article  Google Scholar 

  12. Lowe D.G.: Robust model-based motion tracking through the integration of search and estimation. Int. J. Comput. Vis. 8(2), 113–122 (1992)

    Article  Google Scholar 

  13. Wunsch, P., Hirzinger, G.: Real-time visual tracking of 3-D objects with dynamic handling of occlusion. In: IEEE Int. Conf. on Robotics and Automation, ICRA’97, Albuquerque, New Mexico, USA, pp. 2868–2873 (1997)

  14. Masson, L., Jurie, F., Dhome, M.: Robust real time tracking of 3D objects. In: Int. Conf. Pattern Recognit., vol. 4, pp. 252–255 (2004)

  15. Klein, G., Drummond, T.: Robust visual tracking for non-instrumented augmented reality. In: Proc. 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 113–122 (2003)

  16. Rasmussen C., Hager G.: Probabilistic data association methods for tracking complex visual objects. IEEE Trans. PAMI 23(6), 560–576 (2001)

    Google Scholar 

  17. Triesch, J., der Malsburg, C.V.: Self-organized integration of adaptive visual cues for face tracking. In: Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 102–107 (2000)

  18. Kragic D., Christensen H.I.: Cue integration for visual servoing. IEEE Trans. Robot. Autom. 17(1), 18–27 (2001)

    Article  Google Scholar 

  19. Hayman, E., Eklundh, J.-O.: Statistical background subtraction for a mobile observer. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, pp. 67–74 (2003)

  20. Vacchetti, L., Lepetit, V., Fua, P.: Combining edge and texture information for real-time accurate 3D camera tracking. In: Proceedings of International Symposium on Mixed and Augmented Reality, Arlington, VA, USA (2004)

  21. Rosten, E., Drummond, T.: Fusing points and lines for high performance tracking. In: IEEE International Conference on Computer Vision, vol. 2, pp. 1508–1511. Beijing, China (2005)

  22. Kyrki, V., Kragic, D.: Integration of model-based and model-free cues for visual object tracking in 3D. In: IEEE International Conference on Robotics and Automation, ICRA’05, pp. 1566–1572 (2005)

  23. Kyrki, V., Schmock, K.: Integation methods of model-free features for 3D tracking. In: Scandinavian Conference on Image Analysis, pp. 557–566 (2005)

  24. Brox, T., Rosenhahn, B., Cremers, D., Seidel, H.P.: High accuracy optical flow serves 3-D pose tracking: exploiting contour and flow based constraints. In: European Conference on Computer Vision, ECCV 2006, Graz, Austria, pp. 98–111 (2006)

  25. Pressigout M., Marchand E.: Real-time hybrid tracking using edge and texture information. Int. J. Robot. Res. 26(7), 689–713 (2007)

    Article  Google Scholar 

  26. Hager G., Belhumeur P.: Efficient region tracking with parametric models of geometry and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 20(10), 1025–1039 (1998)

    Article  Google Scholar 

  27. Lu, S., Metaxa, D., Samaras, D., Oliensis, J.: Using multiple cues for hand tracking and model refinement. In: Computer Vision and Pattern Recognition, vol. 2, pp. 443–450. Madison, Wisconsin, USA (2003)

  28. Harris, C.J., Stephens, M.: A combined corner and edge detector. In: Proc. 4th Alvey Vision Conference, Manchester, UK, pp. 147–151 (1988)

  29. Lefebvre T., Bruyninckx H., De Schutter J.: Kalman filters for non-linear systems: a comparison of performance. Int. J. Control 77(7), 639–653 (2004)

    Article  MATH  Google Scholar 

  30. Isard M., Blake A.: CONDENSATION-Conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998)

    Article  Google Scholar 

  31. Bar-Shalom Y., Li X.R., Kirubarajan T.: Estimation with Applications to Tracking and Navigation. Wiley-Interscience, New York (2001)

    Book  Google Scholar 

  32. Welch, G., Bishop, G.: SCAAT: Incremental tracking with incomplete information. In: Proc. Computer graphics and interactive techniques, pp. 333–344. Los Angeles, CA, USA (1997)

  33. Heikkila, J., Silven, O.: A four-step camera calibration procedure with implicit image correction. In: IEEE Computer Vision and Pattern Recognition, pp. 1106–1112 (1997)

  34. Zhang Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

  35. Huber P.J.: Robust estimation of a location parameter. Ann. Math. Stat. 35, 73–101 (1964)

    Article  MATH  MathSciNet  Google Scholar 

  36. Huber P.J.: Robust Statistics. Wiley, New York (1981)

    Book  MATH  Google Scholar 

  37. Press W.H., Teukolsky S.A., Vetterling W.T., Flannery B.P.: Numerical Recipes in C++. Cambridge University Press, Cambridge (2002)

    Google Scholar 

  38. Brent R.P.: Algorithms for Minimization without Derivatives. Prentice-Hall, Englewood Cliffs (1973)

    MATH  Google Scholar 

  39. Tsai R., Lenz R.K.: A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Trans. Robot. Autom. 5(3), 345–358 (1989)

    Article  Google Scholar 

  40. Kyrki, V., Kragic, D., Christensen, H.I.: New shortest-path approaches to visual servoing. In: IEEE/RSJ Int. Conf. Intell. Robots and Systems, Sendai, Japan (2004)

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Lappeenranta University of Technology, P.O. Box 20, 53851, Lappeenranta, Finland

    Ville Kyrki

  2. Centre for Autonomous Systems, School of Computer Science and Communication, Royal Institute of Technology, 10044, Stockholm, Sweden

    Danica Kragic

Authors
  1. Ville Kyrki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Danica Kragic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ville Kyrki.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kyrki, V., Kragic, D. Tracking rigid objects using integration of model-based and model-free cues. Machine Vision and Applications 22, 323–335 (2011). https://doi.org/10.1007/s00138-009-0214-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-009-0214-y

Keywords

Search

Navigation