Skip to main content
SpringerLink
Log in

Video based exercise recognition and correct pose detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Human pose estimation has gained significant attention from researchers of the present era. Personal exercise sessions can be monitored and supervised with the help of pose recognition. Existing work on exercise classification primarily relies on external or wearable sensors for recognizing poses. However, such sensors often fail to differentiate amongst similar exercises. Some essential extensions of human pose estimation are activity detection and activity prediction. In this paper, we first classify an individual’s exercises and then predict whether the pose corresponding to an exercise is correct or not. The tasks mentioned above are performed with the help of 2-dimensional pose coordinates. We have used an RGB camera to capture the poses during exercises performed by individuals. We formulate our model with 2D coordinates obtained from the 2D pose. We consider 2D coordinates of 18 joints of a human body as the primary features to classify different exercises and predict correctness about the poses. We have developed a benchmark dataset consisting of human subjects of various age groups with varying heights. An accuracy of 97.01% has been obtained, and it is better than existing work when tested on our dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. https://github.com/CMU-Perceptual-Computing-Lab/OpenPose

  2. http://tensorflow.org

References

  1. Andriluka M, Pishchulin L, Gehler P, Schiele B (2014) 2d human pose estimation: New benchmark and state of the art analysis. In: Proceedings of the IEEE Conference on computer Vision and Pattern Recognition, pp 3686–3693

  2. Cao Z, Hidalgo G, Simon T, Wei S-E, Sheikh Y (2019) Openpose: realtime multi-person 2d pose estimation using part affinity fields. IEEE Trans Pattern Anal Mach Intell 43(1):172–186

    Article  Google Scholar 

  3. Cao Z, Simon T, Wei S-E, Sheikh Y (2017) Realtime multi-person 2d pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7291–7299

  4. Dhar P, Alvarez J M, Roy P P (2017) Efficient framework for action recognition using reduced fisher vector encoding. In: Proceedings of International Conference on Computer Vision and Image Processing, pp 343–354

  5. Feichtenhofer C, Pinz A, Wildes R P (2017) Spatiotemporal multiplier networks for video action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4768–4777

  6. Hidalgo G, Raaj Y, Idrees H, Xiang D, Joo H, Simon T, Sheikh Y (2019) Single-network whole-body pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp 6982–6991

  7. Hu J-F, Zheng W-S, Lai J, Gong S, Xiang T (2015) Exemplar-based recognition of human–object interactions. IEEE Trans Circ Syst Video Technol 26(4):647–660

    Article  Google Scholar 

  8. Huang G, Liu Z, Van Der Maaten L, Weinberger K Q (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  9. Hwang D-H, Kim S, Monet N, Koike H, Bae S (2020) Lightweight 3d human pose estimation network training using teacher-student learning. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pp 479–488

  10. Ke S-R, Zhu L, Hwang J-N, Pai H-I, Lan K-M, Liao C-P (2010) Real-time 3d human pose estimation from monocular view with applications to event detection and video gaming. In: Proceedings of the IEEE International Conference on Advanced Video and Signal Based Surveillance, pp 489–496

  11. Koskimäki H, Siirtola P (2014) Recognizing gym exercises using acceleration data from wearable sensors. In: Proceedings of the IEEE Symposium on Computational Intelligence and Data Mining (CIDM), pp 321–328

  12. Kumar P, Mukherjee S, Saini R, Kaushik P, Roy P P, Dogra D P (2018) Multimodal gait recognition with inertial sensor data and video using evolutionary algorithm. IEEE Trans Fuzzy Syst 27(5):956–965

    Article  Google Scholar 

  13. Kumar P, Saini R, Roy P P, Dogra D P (2018) A position and rotation invariant framework for sign language recognition using kinect. Multimed Tools Appl 77(7):8823–8846

    Article  Google Scholar 

  14. Kumar P, Saini R, Yadava M, Roy P P, Dogra D P, Balasubramanian R (2017) Virtual trainer with real-time feedback using kinect sensor. In: Proceedings of the IEEE Region 10 Symposium (TENSYMP), pp 1–5

  15. Kunze K, Lukowicz P (2008) Dealing with sensor displacement in motion-based onbody activity recognition systems. In: Proceedings of the 10th international conference on Ubiquitous computing, pp 20–29

  16. Lester J, Choudhury T, Kern N, Borriello G, Hannaford B (2005) A hybrid discriminative/generative approach for modeling human activities. In: Proceedings of the 19th international joint conference on Artificial intelligence, pp 766–772

  17. Linna M, Kannala J, Rahtu E (2018) Real-time human pose estimation with convolutional neural networks. In: Proceedings of the VISIGRAPP, pp 335–342

  18. Lukowicz P, Ward J A, Junker H, Stäger M, Tröster G, Atrash A, Starner T (2004) Recognizing workshop activity using body worn microphones and accelerometers. In: International conference on pervasive computing. Springer, pp 18–32

  19. Maurer U, Smailagic A, Siewiorek D P, Deisher M (2006) Activity recognition and monitoring using multiple sensors on different body positions. In: International Workshop on Wearable and Implantable Body Sensor Networks. IEEE, pp 1–4

  20. O’Reilly M, Caulfield B, Ward T, Johnston W, Doherty C (2018) Wearable inertial sensor systems for lower limb exercise detection and evaluation: a systematic review. Sports Med 48(5):1221–1246

    Article  Google Scholar 

  21. Parkka J, Ermes M, Korpipaa P, Mantyjarvi J, Peltola J, Korhonen I (2006) Activity classification using realistic data from wearable sensors. IEEE Trans Inf Technol Biomed 10(1):119–128

    Article  Google Scholar 

  22. Saini R, Kumar P, Kaur B, Roy P P, Dogra D P, Santosh KC (2019) Kinect sensor-based interaction monitoring system using the blstm neural network in healthcare. Int J Mach Learn Cybern 10(9):2529–2540

    Article  Google Scholar 

  23. Saini R, Kumar P, Roy P P, Dogra D P (2018) A novel framework of continuous human-activity recognition using kinect. Neurocomputing 311:99–111

    Article  Google Scholar 

  24. Seeger C, Buchmann A, Van Laerhoven K (2011) myhealthassistant: a phone-based body sensor network that captures the wearer’s exercises throughout the day. In: Proceedings of the 6th International Conference on Body Area Networks. Institute for Computer Sciences and Social-Informatics, pp 1–7

  25. Shotton J, Girshick R, Fitzgibbon A, Sharp T, Cook M, Finocchio M, Moore R, Kohli P, Criminisi A, Kipman A et al (2012) Efficient human pose estimation from single depth images. IEEE Trans Pattern Anal Mach Intell 35(12):2821–2840

    Article  Google Scholar 

  26. Simon T, Joo H, Matthews I, Sheikh Y (2017) Hand keypoint detection in single images using multiview bootstrapping. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp 1145–1153

  27. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  28. Toshev A, Szegedy C (2014) Deeppose: Human pose estimation via deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1653–1660

  29. Ukita N, Uematsu Y (2018) Semi-and weakly-supervised human pose estimation. Comput Vis Image Underst 170:67–78

    Article  Google Scholar 

  30. Wang H, Ullah M M, Klaser A, Laptev I, Schmid C (2009) Evaluation of local spatio-temporal features for action recognition. In: Bmvc 2009-british machine vision conference. BMVA Press, pp 124–1

  31. Wang L, Xiong Y, Wang Z, Qiao Y, Lin D, Tang X, Van Gool L (2016) Temporal segment networks: Towards good practices for deep action recognition. In: Proceedings of the European conference on computer vision. Springer, pp 20–36

  32. Wei S-E, Ramakrishna V, Kanade T, Sheikh Y (2016) Convolutional pose machines. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp 4724–4732

  33. Xiaohan Nie B, Xiong C, Zhu S-C (2015) Joint action recognition and pose estimation from video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1293–1301

  34. Zhang Z, Wang N, Cui L (2018) Fine-fit: A fine-grained gym exercises recognition system. In: Proceedings of the 24th Asia-Pacific Conference on Communications (APCC), pp 492–497

  35. Zhu A, Zhang S, Huang Y, Hu F, Cui R, Hua G (2019) Exploring hard joints mining via hourglass-based generative adversarial network for human pose estimation. AIP Adv 9(035321-3):1–9

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, IIT Roorkee, Roorkee, 247667, India

    Tushar Rangari, Sudhanshu Kumar & Partha Pratim Roy

  2. School of Electrical Sciences, IIT Bhubaneswar, Odisha, 752050, India

    Debi Prosad Dogra

  3. Department of IT Engineering, Sookmyung Women’s University, Seoul, 04310, South Korea

    Byung-Gyu Kim

Authors
  1. Tushar Rangari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sudhanshu Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Partha Pratim Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Debi Prosad Dogra
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Byung-Gyu Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sudhanshu Kumar.

Ethics declarations

Conflicts of Interest

The authors declared that they have no conflicts of interest to this work.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rangari, T., Kumar, S., Roy, P.P. et al. Video based exercise recognition and correct pose detection. Multimed Tools Appl 81, 30267–30282 (2022). https://doi.org/10.1007/s11042-022-12299-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12299-z

Keywords

Search

Navigation