Skip to main content

Advertisement

SpringerLink
Log in

Pipeline-Architecture Based Real-Time Active-Vision for Human-Action Recognition

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper presents a generic framework for on-line reconfiguration of a multi-camera active-vision system for time-varying-geometry object/subject action recognition. The proposed methodology utilizes customizable pipeline architecture to select optimal camera poses in real time. Subject visibility is optimized via a depth-limited search algorithm. All stages are developed with real-time operation as the central focus. A human action-sensing implementation example demonstrates viability. Controlled experiments, first with a human analogue and, subsequently, with a real human, illustrate the workings of the proposed framework. A tangible increase in action-recognition success rate over other strategies, particularly those with static cameras, is noteworthy. The proposed framework is also shown to operate in real-time. Further experiments examine the effect of scaling the number of obstacles and cameras, sensing-system mobility, and library actions on real-time performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Capin, T.K., Pandzic, I.S., Thalmann, N.M., Thalmann, D.: A dead-reckoning algorithm for virtual human figures. In: Proceedings of the 1997 Virtual Reality Annual International Symposium (VRAIS ’97), Albuquerque, NM., USA (1997)

  2. Tarabanis, K.A., Allen, P.K., Tsai, R.Y.: A survey of sensor planning in computer vision. IEEE Trans. Robot. Autom. 11(1), 86–104 (1995)

    Article  Google Scholar 

  3. Szeliski, R.: Computer Vision: Algorithms and Applications, 1st edn. Springer-Verlag LLC., New York (2010)

    Google Scholar 

  4. Chellappa, R., Roy-Chowdhury, A.K., Zhou, S.K.: Human Activity Recognition, pp. 53–92. Morgan & Claypool Publishing, San Rafael (2005)

    Google Scholar 

  5. Veres, G.V., Gordon, L., Carter, J.N., Nixon, M.S.: What image information is important in silhouette-based gait recognition? In: IEEE Conference on Computer Vision and Pattern Recognition. Washington, D.C. (2004)

  6. deRuiter, H., Benhabib, B.: Object-of-interest selection for model-based 3d pose tracking with background clutter. In: Proceedings of the International Joint Conferences on Computer, Information, and Systems Sciences, and Engineering (CISSE07) (2008)

  7. Wang, X., Wang, S., Bi, D.: Distributed visual-target-surveillance system in wireless sensor networks. IEEE Trans. Syst. Man Cybern. B Cybern. 39(5), 1134–1146 (2009)

    Article  Google Scholar 

  8. Lee, C.-S., Elgammal, A.: Dynamic shape outlier detection for human locomotion. Comput. Vis. Image Underst. 113(3), 332–344 (2009)

    Article  Google Scholar 

  9. Lu, J., Zhang, E.: Gait recognition for human identification based on ICA and fuzzy SVM through multiple views fusion. Pattern Recogn. Lett. 28(16), 2401–2411 (2007)

    Article  Google Scholar 

  10. Roy, A., Sural, S.: A fuzzy inferencing system for gait recognition. In: Proceedings of the 28th North American Fuzzy Information Processing Society Annual Conference, Cincinnati, OH (2009)

  11. Lam, T.H.W., Lee, R.S.T.: A new representation for human gait recognition: Motion Silhouettes Image (MSI). Lect. Notes Comput. Sci. 3832, 612–618 (2005)

    Article  Google Scholar 

  12. Ioannidis, D., Tzovaras, D., Moustakas, K.: Gait identification using the 3D protrusion transform. In: Proceedings of the IEEE International Conference on Image Processing (ICIP07). San Antonio, TX (2007)

  13. Xu, D., Yan, S., Tao, D., Zhang, L., Li, X., Zhang, H.: Human gait recognition with matrix representation. IEEE Trans. Circuits Syst. Video Technol. 16(7), 896–903 (2006)

    Article  Google Scholar 

  14. Guo, B., Nixon, M.S.: Gait feature subset selection by mutual information. IEEE Trans. Syst. Man Cybern., Part A, Syst. Humans 39(1), 36–46 (2009)

    Article  Google Scholar 

  15. Liu, Z., Sarkar, S.: Effect of silhouette quality on hard problems in gait recognition. IEEE Trans. Syst. Man Cybern. B Cybern. 35(2), 170–183 (2005)

    Article  Google Scholar 

  16. Yu, S., Tan, D., Tan, T.: A framework for evaluating the effect of view angle, clothing, and carrying condition on gait recognition. In: International Conference on Pattern Recognition, Hong Kong (2006)

  17. Mittal, A.: Generalized multi-sensor planning. In: European Conference on Computer Vision. Graz, Austria (2006)

    Google Scholar 

  18. Miura, J., Ikeuchi, K.: Task-oriented generation of visual sensing strategies in assembly tasks. IEEE Trans. Pattern Anal. Mach. Intell. 20(2), 126–138 (1998)

    Article  Google Scholar 

  19. Bakhtari, A.: Multi-Target Surveillance in Dynamic Environments: Sensing-System Reconfiguration. Toronto, ON, Canada (2006)

  20. Gu, X., Marefat, M.M., Ciarallo, F.W.: A robust approach for sensor placement in automated vision dimensional inspection. In: IEEE International Conference on Robotics and Automation, Michigan (1999)

  21. Cowan, C.K., Kovesik, P.D.: Automated sensor placement from vision task requirements. IEEE Trans. Pattern Anal. Mach. Intell. 10(3), 407–416 (1988)

    Article  Google Scholar 

  22. Sakane, S., Ishii, M., Kakikura, M.: Occlusion avoidance of visual sensors based on a hand-eye action simulator system: HEAVEN. Adv. Robot. 2(2), 149–165 (1987)

    Article  Google Scholar 

  23. Sakane, S., Sato, T., Kakikura, M.: Model-based planning of visual sensors using a Hand-eye action simulator: HEAVEN. In: Conference on Advanced Robotics, Versailles, France (1987)

  24. Reed, M.K., Allen, P.K.: Constraint-based sesor planning for scene modeling. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1460–1467 (2000)

    Article  Google Scholar 

  25. Pito, R.: A solution to the next best view problem for automated surface acquisition. IEEE Trans. Pattern Anal. Mach. Intell. 21(10), 1016–1030 (1999)

    Article  Google Scholar 

  26. Hodge, L., Kamel, M.: An agent-based approach to multi-sensor coordination. IEEE Trans. Syst. Man Cybern., Part A, Syst. Humans 33(5), 648–662 (2003)

    Article  Google Scholar 

  27. Murrieta-Cid, R., Tovar, B., Hutchinson, S.: A sampling-based motion planning approach to maintain visibility of unpredictable targets. J. Auto. Rob. 19(3), 285–300 (2005)

    Article  Google Scholar 

  28. Naish, M.D., Croft, E.A., Benhabib, B.: Coordinated dispatching of proximity sensors for the surveillance of maneuvering targets. J. Rob. CIM. 19(3), 283–299 (2003)

    Article  Google Scholar 

  29. Bakhtari, A., Benhabib, B.: An active vision system for multi-target surveillance in dynamic environments. IEEE Trans. Syst. Man Cybern. 37(1), 190–198 (2007)

    Article  Google Scholar 

  30. Bakhtari, A., Mackay, M., Benhabib, B.: Active-vision for the autonomous surveillance of dynamic, multi-object environments. J. Intell. Robot. Syst. 54(4), 567–593 (2009)

    Google Scholar 

  31. Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., Seidel, H.P.: Motion capture using joint skeleton tracking and surface estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09), (2009)

  32. Mackay, M., Fenton, R.G., Benhabib, B.: Time-varying-geometry object surveillance using a multi-camera active-vision system. JSSIS 1(3), 679–704 (2008)

    Google Scholar 

  33. Mackay, M., Benhabib, B.: Active-vision system reconfiguration for form recognition in the presence of dynamic obstacles. In: Lecture Notes on Computer Science, Conference on Articulated Motion and Deformable Objects, Andratx, Mallorca, Spain (2008)

  34. Tomori, Z., Gargalik, R., Hrmo, I.: active segmentation in 3D using kinect sensor. In: Proceedings of the 20th International Conference on Computer Graphics, Visualization, and Computer Vision. Pilsen, Czech Republic (2012)

    Google Scholar 

  35. Williams, S.A.: Programming Models For Parallel Systems. Wiley, New York (1990)

    Google Scholar 

  36. Mackay, M., Benhabib, B.: A multi-camera active-vision system for dynamic form recognition. In: Innovations and Advanced Techniques in Systems, Computing Sciences and Software Engineering, pp. 26–31. Springer Science and Business Media B.V. (2008)

  37. Himmelblau, M.D.: Applied Non-Linear Programming. McGraw Hill (1972)

  38. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: From Proceedings of Imaging Understanding Workshop, pp. 121–130 (1981)

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical and Industrial Engineering, University of Toronto, 5 King’s College Road, Toronto, ON, Canada, M5S 3G8

    Matthew Mackay, Robert G. Fenton & Beno Benhabib

Authors
  1. Matthew Mackay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert G. Fenton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Beno Benhabib
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew Mackay.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mackay, M., Fenton, R.G. & Benhabib, B. Pipeline-Architecture Based Real-Time Active-Vision for Human-Action Recognition. J Intell Robot Syst 72, 385–407 (2013). https://doi.org/10.1007/s10846-012-9810-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-012-9810-6

Keywords

Search

Navigation