skip to main content
10.1145/1178782.1178792acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
Article

Robust real-time upper body limb detection and tracking

Published: 27 October 2006 Publication History

Abstract

We describe an efficient and robust system to detect and track the limbs of a human. Of special consideration in the design of this system are real-time and robustness issues. We thus utilize a detection/tracking scheme in which we detect the face and limbs of a user and then track the forearms of the found limbs. Detection occurs by first finding the face of a user. The location and color information from the face can then be used to find limbs. As skin color is a key visual feature in this system, we continuously search for faces and use them to update skin color information. Along with edge information, this is used in the subsequent forearm tracking. Robustness is implicit in this design, as the system automatically re-detects a limbs when its corresponding forearms is lost. This design is also conducive to real-time processing because while detection of the limbs can take up to seconds, tracking is on the order of milliseconds. Thus reasonable frame rates can be achieved with a short latency. Also, in this system we make use of multiple 2D limb tracking models to enhance tracking of the underlying 3D structure. This includes models for lateral forearm views (waving) as well as for pointing gestures. Experiments on test sequences demonstrate the efficacy of this approach.

References

[1]
G. R. Bradski. The OpenCV Library. Dr. Dobb's Software Tools for the Professional Programmer, November 2000.
[2]
C. Breazeal. Socially intelligent robots. ACM Interactions, 12(2):19--22, 2005.
[3]
C. Bregler and J. Malik. Tracking people with twists and exponential maps. In CVPR, Washington, DC, USA, June 1998.
[4]
I. Cohen and H. Li. Inference of human postures by classification of 3d human body shape. In IEEE Workshop on Analysis and Modeling of Faces and Gestures, pages 74--81, Nice, France, 2003.
[5]
D. Comaniciu, V. Ramesh, and P. Meer. Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(5):564--577, May 2003.
[6]
D. J. Feil-Seifer and M. J. Matarić. Socially assistive robotics. In Proceedings of the International Conference on Rehabilitation Robotics, Chicago, Il, July 2005.
[7]
P. F. Felzenszwalb and D. P. Huttenlocher. Pictorial structures for object recognition. International Journal of Computer Vision, 61(1):55--79, 2005.
[8]
T. Fong, I. Nourbakhsh, and K. Dautenhahn. A survey of socially interactive robots. Robotics and Autonomous Systems, Special issue on Socially Interactive Robots, 42(3-4):143--166, 2003.
[9]
A.R. François. Software architecture for computer vision. In G. Medioni and S. Kang, editors, Emerging Topics in Computer Vision, pages 585--654. Prentice Hall, 2004.
[10]
S. Ioffe and D. A. Forsyth. Probabilistic methods for finding people. International Journal of Computer Vision, 43(1):45--68, June 2001.
[11]
M. W. Lee and I. Cohen. Human upper body pose estimation in static images. In ECCV, pages II: 126--138, 2004.
[12]
D. P. Miller. Assistive robotics: An overview. In Assistive Technology and Artificial Intelligence, Applications in Robotics, User Interfaces and Natural Language Processing, pages 126--136, London, UK, 1998. Springer-Verlag.
[13]
D. D. Morris and J. M. Rehg. Singularity analysis for articulated object tracking. In CVPR, Santa Barbara, CA, June 1998.
[14]
W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, New York, NY, USA, 1992.
[15]
D. Ramanan and D. A. Forsyth. Finding and tracking people from the bottom up. In CVPR, pages II: 467--474, 2003.
[16]
L. Sigal, S. Bhatia, S. Roth, M. J. Black, and M. Isard. Tracking loose-limbed people. In CVPR, pages I: 421--428, 2004.
[17]
C. Sminchisescu and B. Triggs. Covariance scaled sampling for monocular 3d body tracking. In CVPR, Kauai Marriott, Hawaii, December 2001.
[18]
H. Zhang, W. Huang, Z. Huang, and L. Li. Affine object tracking with kernel-based spatial-color representation. In CVPR, pages I: 293--300, 2005.

Cited By

View all
  • (2015)A Stereo Vision Approach for Cooperative Robotic Movement TherapyProceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW)10.1109/ICCVW.2015.74(519-527)Online publication date: 7-Dec-2015
  • (2013)Wireless Body Area Network Node Localization Using Small-Scale Spatial InformationIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2012.223717817:3(715-726)Online publication date: May-2013
  • (2012)A real-time upper limb motion estimation system2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE)10.1109/CSAE.2012.6272749(159-168)Online publication date: May-2012
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
VSSN '06: Proceedings of the 4th ACM international workshop on Video surveillance and sensor networks
October 2006
230 pages
ISBN:1595934960
DOI:10.1145/1178782
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2006

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. body pose estimation
  2. gesture and interaction analysis
  3. limb detection
  4. limb tracking
  5. real-time vision

Qualifiers

  • Article

Conference

MM06
MM06: The 14th ACM International Conference on Multimedia 2006
October 27, 2006
California, Santa Barbara, USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2015)A Stereo Vision Approach for Cooperative Robotic Movement TherapyProceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW)10.1109/ICCVW.2015.74(519-527)Online publication date: 7-Dec-2015
  • (2013)Wireless Body Area Network Node Localization Using Small-Scale Spatial InformationIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2012.223717817:3(715-726)Online publication date: May-2013
  • (2012)A real-time upper limb motion estimation system2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE)10.1109/CSAE.2012.6272749(159-168)Online publication date: May-2012
  • (2012)Human sign recognition for robot manipulationProceedings of the 4th Mexican conference on Pattern Recognition10.1007/978-3-642-31149-9_11(107-116)Online publication date: 27-Jun-2012
  • (2011)Upper body detection in unconstrained still images2011 6th IEEE Conference on Industrial Electronics and Applications10.1109/ICIEA.2011.5975596(287-292)Online publication date: Jun-2011
  • (2011)Using a game controller for relaying deictic gestures in computer-mediated communicationInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2011.01.00269:6(362-374)Online publication date: 1-Jun-2011
  • (2010)Locating human hands for real-time pose estimation from monocular videoProceedings of the 17th ACM Symposium on Virtual Reality Software and Technology10.1145/1889863.1889882(95-96)Online publication date: 22-Nov-2010
  • (2010)Human arm estimation using convex features in depth images2010 IEEE International Conference on Image Processing10.1109/ICIP.2010.5651215(3269-3272)Online publication date: Sep-2010
  • (2010)3D pose estimation and motion analysis of the articulated human hand-forearm limb in an industrial production environment3D Research10.1007/3DRes.03(2010)031:3(1-21)Online publication date: 1-Sep-2010
  • (2009)Real-time robust body part tracking for augmented reality interfaceProceedings of the 8th International Conference on Virtual Reality Continuum and its Applications in Industry10.1145/1670252.1670295(203-207)Online publication date: 14-Dec-2009
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media