Skip to main content
Springer Nature Link
Log in

A Comprehensive Survey on Human Activity Prediction

  • Conference paper
  • First Online:
Computational Science and Its Applications – ICCSA 2017 (ICCSA 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10404))

Included in the following conference series:

  • 2254 Accesses

Abstract

Human activity recognition has been extensively studied and achieves promising results in Computer Vision community. Typical activity recognition methods require observe the whole process, then extract features and build a model to classify the activity. However, in many applications, the ability to early recognition or prediction a human activity before it completes is necessary. This task is challenging because of the lack of information when only a fraction of the activity is observed. To get an accurate prediction, the methods must have high discriminated power with just the beginning part of activity. While activity recognition is very popular and has a lot of surveys, activity prediction is still a new and relatively unexplored problem. To the best of our knowledge, there is no survey specifically focusing on human activity prediction. In this survey, we give a systematic review of current methods for activity prediction and how they overcome the above challenge. Moreover, this paper also compares performances of various techniques on the common dataset to show the current state of research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Koppula, H.S., Saxena, A.: Anticipating human activities using object affordances for reactive robotic response. IEEE Trans. Pattern Anal. Mach. Intell. 38(1), 14–29 (2016)

    Article  Google Scholar 

  2. Ryoo, M.S.: Human activity prediction: early recognition of ongoing activities from streaming videos. In: 2011 International Conference on Computer Vision, pp. 1036–1043, November 2011

    Google Scholar 

  3. Dawn, D.D., Shaikh, S.H.: A comprehensive survey of human action recognition with spatio-temporal interest point (STIP) detector. Vis. Comput. 32(3), 289–306 (2016)

    Article  Google Scholar 

  4. Poppe, R.: A survey on vision-based human action recognition. Image Vis. Comput. 28(6), 976–990 (2010)

    Article  Google Scholar 

  5. Vrigkas, M., Nikou, C., Kakadiaris, I.A.: A review of human activity recognition methods. Front. Robot. AI 2, 28 (2015)

    Article  Google Scholar 

  6. Cao, Y., Barrett, D., Barbu, A., Narayanaswamy, S., Yu, H., Michaux, A., Lin, Y., Dickinson, S., Siskind, J.M., Wang, S.: Recognize human activities from partially observed videos. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013

    Google Scholar 

  7. Li, K., Fu, Y.: Prediction of human activity by discovering temporal sequence patterns. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1644–1657 (2014)

    Article  Google Scholar 

  8. Xu, K., Qin, Z., Wang, G.; Human activities prediction by learning combinatorial sparse representations. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 724–728, September 2016

    Google Scholar 

  9. Kong, Y., Fu, Y.: Max-margin action prediction machine. IEEE Trans. Pattern Anal. Mach. Intell. 38(9), 1844–1858 (2016)

    Article  Google Scholar 

  10. Dollar, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 65–72, October 2005

    Google Scholar 

  11. Raptis, M., Sigal, L.: Poselet key-framing: a model for human activity recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013

    Google Scholar 

  12. Ding, W., Liu, K., Cheng, F., Zhang, J.: Learning hierarchical spatio-temporal pattern for human activity prediction. J. Vis. Comun. Image Represent. 35(C), 103–111 (2016)

    Article  Google Scholar 

  13. Ke, Q., Bennamoun, M., An, S., Boussaid, F., Sohel, F.: Human interaction prediction using deep temporal features. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 403–414. Springer, Cham (2016). doi:10.1007/978-3-319-48881-3_28

    Chapter  Google Scholar 

  14. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893, June 2005

    Google Scholar 

  15. Wang, H., Yang, W., Yuan, C., Ling, H., Hu, W.: Human activity prediction using temporally-weighted generalized time warping. Neurocomputing 225(C), 139–147 (2017)

    Article  Google Scholar 

  16. Wang, H., Klser, A., Schmid, C., Liu, C.L.: Action recognition by dense trajectories. In: CVPR 2011, pp. 3169–3176 (2011)

    Google Scholar 

  17. Ryoo, M.S., Fuchs, T.J., Xia, L., Aggarwal, J.K., Matthies, L.: Robot-centric activity prediction from first-person videos: what will they do to me? In: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2015, pp. 295–302. ACM, New York (2015)

    Google Scholar 

  18. Bourdev, L., Malik, J.: Poselets: body part detectors trained using 3D human pose annotations. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1365–1372, September 2009

    Google Scholar 

  19. Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2014

    Google Scholar 

  20. Ma, S., Sigal, L., Sclaroff, S.: Learning activity progression in LSTMs for activity detection and early detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1942–1950, June 2016

    Google Scholar 

  21. Hu, J.-F., Zheng, W.-S., Ma, L., Wang, G., Lai, J.: Real-time RGB-D activity prediction by soft regression. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 280–296. Springer, Cham (2016). doi:10.1007/978-3-319-46448-0_17

    Chapter  Google Scholar 

  22. Joachims, T., Finley, T., Chun-Nam John, Y.: Cutting-plane training of structural svms. Mach. Learn. 77(1), 27–59 (2009)

    Article  MATH  Google Scholar 

  23. Hoai, M., De la Torre, F.: Max-margin early event detectors. Int. J. Comput. Vis. 107(2), 191–202 (2014)

    Article  MathSciNet  Google Scholar 

  24. Lan, T., Chen, T.-C., Savarese, S.: A hierarchical representation for future action prediction. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 689–704. Springer, Cham (2014). doi:10.1007/978-3-319-10578-9_45

    Google Scholar 

  25. Chakraborty, A., Roy-Chowdhury, A.K.: Context-aware activity forecasting. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9007, pp. 21–36. Springer, Cham (2015). doi:10.1007/978-3-319-16814-2_2

    Google Scholar 

  26. Li, K., Hu, J., Fu, Y.: Modeling complex temporal composition of actionlets for activity prediction. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 286–299. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33718-5_21

    Chapter  Google Scholar 

  27. Vondrick, C., Pirsiavash, H., Torralba, A.: Anticipating visual representations from unlabeled video. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    Google Scholar 

  28. Ryoo, M.S., Aggarwal, J.K.: UT-Interaction Dataset, ICPR contest on Semantic Description of Human Activities (SDHA) (2010). http://cvrc.ece.utexas.edu/SDHA2010/Human_Interaction.html

  29. Koppula, H.S., Gupta, R. Saxena, A.: Learning human activities and object affordances from RGB-D videos. CoRR, abs/1210.1207 (2012)

    Google Scholar 

  30. Kong, Y., Jia, Y., Fu, Y.: Learning human interaction by interactive phrases. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 300–313. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33718-5_22

    Chapter  Google Scholar 

  31. Ryoo, M.S., Matthies, L.: First-person activity recognition: what are they doing to me? In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, June 2013

    Google Scholar 

  32. Ghanem, B., Heilbron, F.C., Escorcia, V., Niebles, J.C.: ActivityNet: a large-scale video benchmark for human activity understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 961–970 (2015)

    Google Scholar 

  33. Kong, Y., Kit, D., Fu, Y.: A discriminative model with multiple temporal scales for action prediction. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 596–611. Springer, Cham (2014). doi:10.1007/978-3-319-10602-1_39

    Google Scholar 

  34. Simonyan,K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556 (2014)

    Google Scholar 

Download references

Acknowledgments

This research is funded by Vietnam National Foundation for Science and Technology Development (NAFOSTED).

Author information

Authors and Affiliations

  1. Japan Advanced Institute of Science and Technology, Nomi, Ishikawa, Japan

    Nghia Pham Trong, Hung Nguyen & Kotani Kazunori

  2. University of Science, Vietnam National University Ho Chi Minh City, Ho Chi Minh City, Vietnam

    Bac Le Hoai

Authors
  1. Nghia Pham Trong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hung Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kotani Kazunori
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bac Le Hoai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nghia Pham Trong .

Editor information

Editors and Affiliations

  1. University of Perugia, Perugia, Italy

    Osvaldo Gervasi

  2. University of Basilicata, Potenza, Italy

    Beniamino Murgante

  3. Covenant University, Ota, Nigeria

    Sanjay Misra

  4. University of Trieste, Trieste, Italy

    Giuseppe Borruso

  5. Polytechnic University of Bari, Bari, Italy

    Carmelo M. Torre

  6. University of Minho, Braga, Portugal

    Ana Maria A.C. Rocha

  7. Monash University, Clayton, Victoria, Australia

    David Taniar

  8. Kyushu Sangyo University, Fukuoka, Japan

    Bernady O. Apduhan

  9. Saint Petersburg State University, Saint Petersburg, Russia

    Elena Stankova

  10. University of Trieste, Trieste, Italy

    Alfredo Cuzzocrea

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Trong, N.P., Nguyen, H., Kazunori, K., Le Hoai, B. (2017). A Comprehensive Survey on Human Activity Prediction. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2017. ICCSA 2017. Lecture Notes in Computer Science(), vol 10404. Springer, Cham. https://doi.org/10.1007/978-3-319-62392-4_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-62392-4_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-62391-7

  • Online ISBN: 978-3-319-62392-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics