Skip to main content
SpringerLink
Log in

Single- and two-person action recognition based on silhouette shape and optical point descriptors

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In this paper, we present action descriptors that are capable of performing single- and two-person simultaneous action recognition. In order to exploit the shape information of action silhouettes, we detect junction points and geometric patterns at the silhouette boundary. The motion information is exploited by using optical flow points. We compute centroid distance signatures to construct the junction points and optical flow-based action descriptors. By taking advantage of the distinct poses, we extract key frames and construct geometric pattern action descriptor, which is based on histograms of the geometric patterns classes obtained by a distance-based classification method. In order to exploit the shape and motion information simultaneously, we follow the information fusion strategy and construct a joint action descriptor by combining geometric patterns and optical flow descriptors. We evaluate the performance of these descriptors on the two widely used action datasets, i.e., Weizmann dataset (single-person actions) and SBU Kinect interaction dataset, clean and noisy versions (two-person actions). The experimental outcomes demonstrate the ability of the individual descriptors to give satisfactory performance on average. It is found that the joint action descriptor shows the best performance among the proposed descriptors due to its high discriminative power and also outperforms state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Mohamed, A.N., Ali, M.M.: Human motion analysis, recognition and understanding in computer vision: a review. J. Eng. Sci. 41(5), 1928–1946 (2013)

    Google Scholar 

  2. Poppe, R.: A survey on vision-based human action recognition. Image Vis. Comput. 28(6), 976–990 (2010)

    Article  Google Scholar 

  3. Chaaraoui, A.A., Climent-Pérez, P., Flórez-Revuelta, F.: Silhouette-based human action recognition using sequences of key poses. Pattern Recognit. Lett. 34(15), 1799–1807 (2013)

    Article  Google Scholar 

  4. Cheema, S., Eweiwi, A., Thurau, C., Bauckhage, C.: Action recognition by learning discriminative key poses. In: ICCV, pp. 1302–1309 (2011)

  5. Vishwakarma, D., Dhiman, A., Maheshwari, R., Kapoor, R.: Human motion analysis by fusion of silhouette orientation and shape features. In: ICRTC, pp. 438–447 (2015)

  6. Yun, K., Honorio, J., Chattopadhyay, D., Berg, T.L., Samaras, D.: Two-person interaction detection using body-pose features and multiple instance learning. In: CVPR, pp. 28–35 (2012)

  7. Gori, I., Aggarwal, J., Matthies, L., Ryoo, M.: Multitype activity recognition in robot-centric scenarios. IEEE Trans. Robot. Autom. 1(1), 593–600 (2016)

    Article  Google Scholar 

  8. Grundmann, M., Meier, F., Essa, I.: 3D shape context and distance transform for action recognition. In: ICPR, pp. 1–4 (2008)

  9. Mokhber, A., Achard, C., Milgram, M.: Recognition of human behavior by space-time silhouette characterization. Pattern Recognit. Lett. 29(1), 81–89 (2008)

    Article  Google Scholar 

  10. Wang, B., Bai, X., Wang, X., Liu, W., Tu, Z.: Object recognition using junctions. In: ECCV, pp. 15–28 (2010)

  11. Bhatti, N.A., Hanbury, A.: Detection and classification of local primitives in line drawings. In: AAPR (2011)

  12. Bergevin, R., Bubel, A.: Detection and characterization of junctions in a 2D image. Comput. Vis. Image Underst. 93(3), 288–309 (2004)

    Article  Google Scholar 

  13. Hasan, H., Haron, H., Hashim, S.Z., Omar, F.S.: Logical heuristic algorithm in extracting 2D structure thinned binary image into freeman chain code (FCC). In: IVIC, pp. 770–778 (2009)

  14. Dedeoğlu, Y., Töreyin, B.U., Güdükbay, U., Çetin, A.E.: Silhouette-based method for object classification and human action recognition in video. In: Comp. Vision in HCI. ECCV, LNCS, pp. 64–77. Springer (2006)

  15. Toreyin, B.U., Dedeoglu, Y., Cetin, A.E.: Contour based smoke detection in video using wavelets. In: EUSIPCO, pp. 1–5 (2006)

  16. Chaaraoui, A., Padilla-Lopez, J., Flórez-Revuelta, F.: Fusion of skeletal and silhouette-based features for human action recognition with rgb-d devices. In: ICCV, pp. 91–97 (2013)

  17. Chen, D.Y., Shih, S.W., Liao, H.Y.M.: Human action recognition using 2-d spatio-temporal templates. In: ICME, pp. 667–670 (2007)

  18. Xia, L., Chen, C.C., Aggarwal, J.: View invariant human action recognition using histograms of 3d joints. In: CVPRW, pp. 20–27 (2012)

  19. Guo, K., Ishwar, P., Konrad, J.: Action recognition in video by covariance matching of silhouette tunnels. In: SIBGRAPI, pp. 299–306 (2009)

  20. Abdelkader, M.F., Abd-Almageed, W., Srivastava, A., Chellappa, R.: Silhouette-based gesture and action recognition via modeling trajectories on riemannian shape manifolds. Comput. Vis. Image Underst. 115(3), 439–455 (2011)

    Article  Google Scholar 

  21. Wu, D., Shao, L.: Silhouette analysis-based action recognition via exploiting human poses. IEEE Trans. Circuits Syst. Video Technol. 23(2), 236–243 (2013)

    Article  MathSciNet  Google Scholar 

  22. Chaaraoui, A.A., Flórez-Revuelta, F.: A low-dimensional radial silhouette-based feature for fast human action recognition fusing multiple views. International Scholarly Research Notices 2014 (2014)

  23. Junejo, I.N., Junejo, K.N., Al Aghbari, Z.: Silhouette-based human action recognition using sax-shapes. Vis. Comput. 30(3), 259–269 (2014)

    Article  Google Scholar 

  24. Venkatesha, S., Turk, M.: Human activity recognition using local shape descriptors. In: ICPR, pp. 3704–3707 (2010)

  25. Abe, T., Fukushi, M., Ueda, D.: Primitive human action recognition based on partitioned silhouette block matching. In: ISVC, pp. 308–317 (2013)

  26. Huang, B., Tian, G., Zhou, F.: Human typical action recognition using gray scale image of silhouette sequence. Comput. Electr. Eng. 38(5), 1177–1185 (2012)

    Article  Google Scholar 

  27. Li, W., Wen, L., Choo Chuah, M., Lyu, S.: Category-blind human action recognition: a practical recognition system. In: ICCV, pp. 4444–4452 (2015)

  28. Benezeth, Y., Jodoin, P.M., Emile, B., Rosenberger, C., Laurent, H.: Comparative study of background subtraction algorithms. J. Electron. Imaging 19(3), 033003 (2010)

  29. Soille, P.: Morphological Image Analysis: Principles and Applications, 2nd edn. Springer, New York (2003)

    MATH  Google Scholar 

  30. Yang, M., Kpalma, K., Ronsin, J.: A survey of shape feature extraction techniques. In: Yin, PY (ed.) Pattern Recognition, pp. 43–90, IN-TECH (2008)

  31. Chen, Y., Wu, Q., He, X., Du, C., Yang, J.: Extracting key postures in a human action video sequence. In: MMSP, pp. 569–573 (2008)

  32. Baumann, F.: Action recognition with hog-of features. In: Pattern Recognit., pp. 243–248. Springer (2013)

  33. Thanh, T.T., Chen, F., Kotani, K., Le, B.: Extraction of discriminative patterns from skeleton sequences for accurate action recognition. Fundamenta Informaticae 130(2), 247–261 (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics, Quaid-i-Azam University, Islamabad, 45320, Pakistan

    Shujah Islam, Tehreem Qasim, Muhammad Yasir, Naeem Bhatti, Hasan Mahmood & Muhammad Zia

Authors
  1. Shujah Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tehreem Qasim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Yasir
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Naeem Bhatti
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hasan Mahmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Muhammad Zia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Naeem Bhatti.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Islam, S., Qasim, T., Yasir, M. et al. Single- and two-person action recognition based on silhouette shape and optical point descriptors. SIViP 12, 853–860 (2018). https://doi.org/10.1007/s11760-017-1228-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-017-1228-y

Keywords

Search

Navigation