Skip to main content
Springer Nature Link
Log in

A Generative Approach to Tracking Hands and Their Interaction with Objects

  • Conference paper
  • First Online:
Man–Machine Interactions 4

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 391))

  • 1076 Accesses

Abstract

Markerless 3D tracking of hands in action or in interaction with objects provides rich information that can be used to interpret a number of human activities. In this paper, we review a number of relevant methods we have proposed. All of them focus on hands, objects and their interaction and follow a generative approach. The major strength of such an approach is the straightforward fashion in which arbitrarily complex priors can be easily incorporated towards solving the tracking problem and their capability to generalize to greater and/or different domains. The proposed generative approach is implemented in a single, unified computational framework.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Athitsos, V., Sclaroff, S.: Estimating 3d hand pose from a cluttered image. In: CVPR 2003. vol. 2, pp. 432–439. Madison, USA (2003)

    Google Scholar 

  2. Ballan, L., Taneja, A., Gall, J., Van Gool, L., Pollefeys, M.: Motion Capture of Hands in Action Using Discriminative Salient Points. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 640–653. Springer, Heidelberg (2012)

    Google Scholar 

  3. Bray, M., Koller-Meier, E., Van Gool, L.: Smart particle filtering for high-dimensional tracking. Comput. Vis. Image Underst. 106(1), 116–129 (2007)

    Article  Google Scholar 

  4. de Campos, T., Murray, D.: Regression-based hand pose estimation from multiple cameras. In: CVPR 2006. vol. 1, pp. 782–789. New York, USA (2006)

    Google Scholar 

  5. Douvantzis, P., Oikonomidis, I., Kyriazis, N., Argyros, A.: Dimensionality reduction for efficient single frame hand pose estimation. In: Chen, M., Leibe, B., Neumann, B. (eds.) Computer Vision Systems. LNCS, vol. 7963, pp. 143–152. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  6. Gallese, V., Goldman, A.: Mirror neurons and the simulation theory of mind-reading. Trends Cogn. Sci. 2(12), 493–501 (1998)

    Article  Google Scholar 

  7. Grezes, J., Decety, J.: Functional anatomy of execution, mental simulation, observation, and verb generation of actions: a meta-analysis. Hum. Brain Mapp. 12(1), 1–19 (2001)

    Article  Google Scholar 

  8. Keskin, C., Kirac, F., Kara, Y., Akarun, L.: Real time hand pose estimation using depth sensors. In: ICCV 2011. pp. 1228–1234. Barcelona, Spain (2011)

    Google Scholar 

  9. Keskin, C., Kıraç, F., Kara, Y., Akarun, L.: Hand pose estimation and hand shape classification using multi-layered randomized decision forests. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 852–863. Springer, Heidelberg (2012)

    Google Scholar 

  10. Kim, K., Lepetit, V., Woo, W.: Keyframe-based modeling and tracking of multiple 3D objects. In: ISMAR 2010. pp. 193–198. Seoul, Japan (2010)

    Google Scholar 

  11. Kyriazis, N.: A computational framework for observing and understanding the interaction of humans with objects of their environment. Ph.D. thesis, University of Crete (2014)

    Google Scholar 

  12. Kyriazis, N., Argyros, A.: Physically plausible 3D scene tracking: The single actor hypothesis. In: CVPR 2013. pp. 9–16. Portland, Oregon, USA (2013)

    Google Scholar 

  13. Kyriazis, N., Argyros, A.: Scalable 3D tracking of multiple interacting objects. In: CVPR 2014. pp. 3430–3437. Columbus, Ohio, USA (2014)

    Google Scholar 

  14. MacCormick, J., Isard, M.: Partitioned sampling, articulated objects, and interface-quality hand tracking. In: In: Proceedings of the 6th EuropeanConference on Computer Vision-Part II, ECCV 2000. LNCS, pp. 3–19. Springer, Heidelberg (2000)

    Google Scholar 

  15. Makris, A., Kyriazis, N., Argyros, A.: Hierarchical particle filtering for 3d hand tracking. In: CVPR 2015. pp. 8–17. Boston, USA (2015)

    Google Scholar 

  16. Malassiotis, S., Strintzis, M.G.: Real-time hand posture recognition using range data. Image Vis. Comput. 26(7), 1027–1037 (2008)

    Article  Google Scholar 

  17. Michel, D., Panagiotakis, C., Argyros, A.A.: Tracking the articulated motion of the human body with two RGBD cameras. Mach. Vis. Appl. 26(1), 41–54 (2013)

    Article  Google Scholar 

  18. Oikonomidis, I., Kyriazis, N., Argyros, A.: Full dof tracking of a hand interacting with an object by modeling occlusions and physical constraints. In: ICCV 2011. pp. 2088–2095. Barcelona, Spain (2011)

    Google Scholar 

  19. Oikonomidis, I., Kyriazis, N., Argyros, A., et al.: Full dof tracking of a hand interacting with an object by modeling occlusions and physical constraints. In: ICCV 2011. pp. 2088–2095. Barcelona, Spain (2011)

    Google Scholar 

  20. Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Efficient model-based 3D tracking of hand articulations using kinect. In: BMVC 2011. p. 3. Dundee, UK (2011)

    Google Scholar 

  21. Oikonomidis, I., Kyriazis, N., Tzevanidis, K., Argyros, A., et al.: Tracking hand articulations: relying on 3D visual hulls versus relying on multiple 2D cues. In: ISUVR 2013. pp. 7–10. Daejeon, South Korea (2013)

    Google Scholar 

  22. Oikonomidis, I., Lourakis, M., Argyros, A., et al.: Evolutionary quasi-random search for hand articulations tracking. In: CVPR 2014. pp. 3422–3429. Columbus, Ohio, USA (2014)

    Google Scholar 

  23. Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Tracking the articulated motion of two strongly interacting hands. In: CVPR 2012. pp. 1862–1869. Providence, Rhode Island (2012)

    Google Scholar 

  24. Oikonomidis, I., Kyriazis, N., Argyros, A.: Markerless and efficient 26-DOF hand pose recovery. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) Computer Vision—ACCV 2010. LNCS, vol. 6494, pp. 744–757. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  25. Padeleris, P., Zabulis, X., Argyros, A., et al.: Head pose estimation on depth data based on particle swarm optimization. In: CVPRW 2012. pp. 42–49. Providence, USA (2012)

    Google Scholar 

  26. Patel, M., Ek, C.H., Kyriazis, N., Argyros, A., Miro, J.V., Kragic, D.: Language for learning complex human-object interactions. In: ICRA 2013. pp. 4997–5002. Karlsruhe, Germany (2013)

    Google Scholar 

  27. Pham, T.H., Kheddar, A., Qammaz, A., Argyros, A.A.: Towards force sensing from vision: observing hand-object interactions to infer manipulation forces. In: CVPR 2015. pp. 1893–1902. Boston, USA (2015)

    Google Scholar 

  28. Poli, R., Kennedy, J., Blackwell, T.: Particle swarm optimization. Swarm Intell. 1(1), 33–57 (2007)

    Article  Google Scholar 

  29. Qian, C., Sun, X., Wei, Y., Tang, X., Sun, J.: Realtime and robust hand tracking from depth. In: CVPR 2014. pp. 1106–1113. Columbus, USA (2014)

    Google Scholar 

  30. Qian, C., Sun, X., Wei, Y., Tang, X., Sun, J.: Realtime and robust hand tracking from depth. In: CVPR 2014. pp. 1106–1113. Ohio, USA (2014)

    Google Scholar 

  31. Rehg, J.M., Kanade, T.: Visual tracking of high DOF articulated structures: an application to human hand tracking. In: ECCV 1994. pp. 35–46. Springer, London, UK (1994)

    Google Scholar 

  32. Roditakis, K., Argyros, A.: Quantifying the effect of a colored glove in the 3D tracking of a human hand. In: Nalpantidis, L., Krüger, V., Eklundh, J.O., Gasteratos, A. (eds.) Computer Vision Systems. LNCS, vol. 9163, pp. 404–414. Springer, Heidelberg (2015)

    Chapter  Google Scholar 

  33. Romero, J., Kjellström, H., Ek, C.H., Kragic, D.: Non-parametric hand pose estimation with object context. Image Vis. Comput. 31(8), 555–564 (2013)

    Article  Google Scholar 

  34. Romero, J., Kjellström, H., Kragic, D.: Monocular real-time 3D articulated hand pose estimation. In: IEEE-RAS 2009. pp. 87–92. Paris, France (2009)

    Google Scholar 

  35. Ros, G., del Rincon, J., Mateos, G.: Articulated particle filter for hand tracking. In: ICPR 2012. pp. 3581–3585. Stockholm, Sweden (2012)

    Google Scholar 

  36. Rosales, R., Athitsos, V., Sigal, L., Sclaroff, S.: 3D hand pose reconstruction using specialized mappings. In: ICCV 2001. vol. 1, pp. 378–385. Vancouver, Canada (2001)

    Google Scholar 

  37. Salzmann, M., Urtasun, R.: Physically-based motion models for 3D tracking: a convex formulation. In: ICCV 2011. pp. 2064–2071. Barcelona, Spain (2011)

    Google Scholar 

  38. Schmidt, T., Newcombe, R., Fox, D.: Dart: Dense articulated real-time tracking. In: RSS 2014. vol. 2. Berkeley, USA (2014)

    Google Scholar 

  39. Song, D., Kyriazis, N., Oikonomidis, I., Papazov, C., Argyros, A., Burschka, D., Kragic, D.: Predicting human intention in visual observations of hand/object interactions. In: ICRA 2013. pp. 1608–1615. Karlsruhe, Germany (2013)

    Google Scholar 

  40. Sridhar, S., Oulasvirta, A., Theobalt, C.: Interactive markerless articulated hand motion tracking using RGB and depth data. In: ICCV 2013. pp. 2456–2463. Sydney, Australia (2013)

    Google Scholar 

  41. Sridhar, S., Rhodin, H., Seidel, H.P., Oulasvirta, A., Theobalt, C.: Real-time hand tracking using a sum of anisotropic gaussians model. In: 3DV 2014. Tokyo, Japan (2014)

    Google Scholar 

  42. Stenger, B., Mendonça, P.R., Cipolla, R.: Model-based 3D tracking of an articulated hand. In: CVPR 2001. vol. 2, pp. 310–315. Kauai, HI, USA (2001)

    Google Scholar 

  43. Sudderth, E., Mandel, M., Freeman, W., Willsky, A.: Visual hand tracking using nonparametric belief propagation. In: CVPRW 2004. pp. 189–189. Washington, USA (2004)

    Google Scholar 

  44. Tompson, J., Stein, M., Lecun, Y., Perlin, K.: Real-time continuous pose recovery of human hands using convolutional networks. ACM Trans. Graph. 33(5), 169–169 (2014)

    Article  Google Scholar 

  45. Vo, B., Vo, B., Pham, N., Suter, D.: Joint detection and estimation of multiple objects from image observations. IEEE Trans. Signal Process. 58(10), 5129–5141 (2010)

    Article  MathSciNet  Google Scholar 

  46. Wang, R., Paris, S., Popovic, J.: 6D hands: markerless hand-tracking for computer aided design. In: UIST 2011. pp. 549–558. UIST ’11, ACM, New York, USA (2011)

    Google Scholar 

  47. Wang, Y., Min, J., Zhang, J., Liu, Y., Xu, F., Dai, Q., Chai, J.: Video-based hand manipulation capture through composite motion control. ACM Trans. Graph. (TOG) 32(4), 43:1–43:14 (2013)

    Article  Google Scholar 

  48. Wu, Y., Lin, J.Y., Huang, T.S.: Capturing natural hand articulation. In: ICCV 2001. vol. 2, pp. 426–432. Vancouver, USA (2001)

    Google Scholar 

  49. Xu, C., Cheng, L.: Efficient hand pose estimation from a single depth image. In: ICCV 2013. pp. 3456–3462. Sydney, Australia (2013)

    Google Scholar 

Download references

Acknowledgments

This work was partially supported by the by the EU ISTFP7-IP-288533 project RoboHow.Cog and by the EU FP7-ICT-2011-9-601165 project WEARHAP.

Author information

Authors and Affiliations

  1. Institute of Computer Science, Foundation for Research and Technology, Heraklion, Greece

    Nikolaos Kyriazis, Iason Oikonomidis, Paschalis Panteleris, Damien Michel, Ammar Qammaz, Alexandros Makris, Konstantinos Tzevanidis, Petros Douvantzis, Konstantinos Roditakis & Antonis Argyros

  2. Computer Science Department, University of Crete, Heraklion, Greece

    Nikolaos Kyriazis, Iason Oikonomidis, Konstantinos Tzevanidis, Petros Douvantzis, Konstantinos Roditakis & Antonis Argyros

Authors
  1. Nikolaos Kyriazis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Iason Oikonomidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paschalis Panteleris
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Damien Michel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ammar Qammaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alexandros Makris
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Konstantinos Tzevanidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Petros Douvantzis
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Konstantinos Roditakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Antonis Argyros
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonis Argyros .

Editor information

Editors and Affiliations

  1. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Aleksandra Gruca

  2. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Agnieszka Brachman

  3. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Stanisław Kozielski

  4. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Tadeusz Czachórski

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Kyriazis, N. et al. (2016). A Generative Approach to Tracking Hands and Their Interaction with Objects. In: Gruca, A., Brachman, A., Kozielski, S., Czachórski, T. (eds) Man–Machine Interactions 4. Advances in Intelligent Systems and Computing, vol 391. Springer, Cham. https://doi.org/10.1007/978-3-319-23437-3_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-23437-3_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-23436-6

  • Online ISBN: 978-3-319-23437-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics