Skip to main content

Advertisement

Springer Nature Link
Log in

A View Independent Classification Framework for Yoga Postures

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Yoga is a globally acclaimed and widely recommended practice for a healthy living. Maintaining correct posture while performing a Yogasana is of utmost importance. In this work, we employ transfer learning from human pose estimation models for extracting 136 key-points spread all over the body to train a random forest classifier which is used for estimation of the Yogasanas. The results are evaluated on an in-house collected extensive yoga video database of 51 subjects recorded from four different camera angles. We use a three step scheme for evaluating the generalizability of a Yoga classifier by testing it on (1) unseen frames, (2) unseen subjects, and (3) unseen camera angles. We argue that for most of the applications, validation accuracies on unseen subjects and unseen camera angles would be most important. We empirically analyze over three public datasets, the advantage of transfer learning and the possibilities of target leakage. We further demonstrate that the classification accuracies critically depend on the cross validation method employed and can often be misleading. To promote further research, we have made key-points dataset and code publicly available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. The dataset and code can be found here: https://github.com/mustafa1728/Yogasana-Classifier.

  2. Images in [43] dataset are very similar to those in Yoga-82 [52], but [43] is relatively smaller in size. Thus, we experiment on Yoga-82 [52] and do not explore [43].

  3. mean Average Precision and Multiple Object Tracking Accuracy.

  4. Percentage of Correct Keypoints based on the head.

  5. Joint Angular Displacement Maps.

  6. Bi-lateral asanas were labelled as asana_left and asana_right separately. Unilateral asanas were labelled normally as asana.

  7. 21 left hand key-points, 21 right hand key-points and 68 face key-points are the three key-point sets considered.

  8. Here, we refer to the mathematical continuity of the function describing frame dynamics.

  9. 136 keypoints subsequently reduced to 35, as descriibed in Methodology.

References

  1. Andriluka Mykhaylo, Pishchulin Leonid, Gehler Peter, Schiele Bernt. 2d human pose estimation: New benchmark and state of the art analysis. In Proceedings of the IEEE Conference on computer Vision and Pattern Recognition. 2014;3686–3693.

  2. Li Jiefenf, Wang Can, Zhu Hao, Mao Yihuan, Fang Hao-Shu, Lu Cewu. CrowdPose: Efficient Crowded Scenes Pose Estimation and a New Benchmark. In Proceedings of the IEEE Conference on computer Vision and Pattern Recognition 2019;10855–10864.

  3. Lin Weiyao, Liu Huabin, Liu Shuzhan, Li Yuxi, Qian Rui, Wang Tao, Xu Ning, Xiong Hongkai, Qi Guo-Jun, Sebe Nicu. Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events. arXiv 2020.

  4. Andriluka Mykhaylo, Iqbal Umar, Insafutdinov Eldar, Pishchulin Leonid, Milan Anton, Gall Juergen, Schiele Bernt. PoseTrack: A Benchmark for Human Pose Estimation and Tracking. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018;5167–5176

  5. Ma Ningning, Zhang Xiangyu, Zheng Hai-Tao, Sun Jian. Shufflenet v2: Practical guidelines for efficient cnn architecture design. European conference on computer vision (ECCV). 2018;116–131

  6. Beddiar Djamila Romaissa, Nini Brahim, Sabokrou Mohammad, Hadid Abdenour. Vision-based human activity recognition: a survey. Multimedia Tools and Applications 79, 41 (2020), 2020;30509–30555.

  7. Breiman L. Random forests. Mach Learn. 2001;45(1):5–32.

    Article  MATH  Google Scholar 

  8. Büssing Arndt, Michalsen Andreas, Khalsa Sat Bir S, Telles Shirley, Sherman Karen J. 2012. Effects of yoga on mental and physical health: a short summary of reviews. Evidence-based complementary and alternative medicine 2012 (2012).

  9. Chen Hua-Tsung, He Yu-Zhen, Hsu Chun-Chieh. Computer-assisted yoga training system. Multimedia Tools and Applications 77, 18 (2018), 2018;23969–23991.

  10. Chen Hua-Tsung, He Yu-Zhen, Hsu Chun-Chieh, Chou Chien-Li, Lee Suh-Yin, Lin Bao-Shuh P. Yoga posture recognition for self-training. In International Conference on Multimedia Modeling. Springer, 2014;496–505.

  11. Fang Hao-Shu, Xie Shuqin, Tai Yu-Wing, Lu Cewu. RMPE: Regional Multi-person Pose Estimation. In Proceedings of the IEEE International Conference on Computer Vision. 2017;2334–2343.

  12. Xiao Bin, Wu Haiping, Wei Yichen. Simple Baselines for Human Pose Estimation and Tracking. In Proceedings of the European Conference on Computer Vision (ECCV). 2018;466–481

  13. Sun Ke, Xiao Bin, Liu Dong, Wang Jingdong. Deep high-resolution representation learning for human pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019;5693–5703

  14. Yu Changqian, Xiao Bin, Gao Changxin, Yuan Lu, Zhang Lei, Sang Nong, Wang Jingdong. Lite-HRNet: A Lightweight High-Resolution Network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021;10435–10445

  15. Geng Zigang, Sun Ke, Xiao Bin, Zhang Zhaoxiang, Wang Jingdong. Bottom-Up Human Pose Estimation via Disentangled Keypoint Regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021;14676–14686

  16. Li Jiefeng, Bian Siyuan, Zeng Ailing, Wang Can, Pang Bo, Liu Wentao, Lu Cewu. Human pose regression with residual log-likelihood estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021;11025–11034

  17. Li Shichao, Ke Lei, Pratama Kevin, Tai Yu-Wing, Tang Chi-Keung, Cheng Kwang-Ting. Cascaded Deep Monocular 3D Human Pose Estimation with Evolutionary Training Data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020;6173–6183

  18. McNally William, Vats Kanav, Wong Alexander, McPhee John. Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation. In arXiv preprint arXiv:2111.08557 (Accepted to ECCV 2022).2021.

  19. Zhang Feng, Zhu Xiatian, Dai Hanbin, Ye Mao, Zhu Ce. Distribution-Aware Coordinate Representation for Human Pose Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020;7093–7102

  20. Liu Zhenguang, Chen Haoming, Feng Runyang, Wu Shuang, Ji Shouling, Yang Bailin, Wang Xun. Deep Dual Consecutive Network for Human Pose Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021;525–534

  21. Kocabas Muhammed, Karagoz Salih, Akbas Emre. MultiPoseNet: Fast Multi-Person Pose Estimation Using Pose Residual Network. In European Conference on Computer Vision (ECCV). 2018;437–453

  22. Cao Zhe, Hidalgo Gines, Simon Tomas, Wei Shih-En, Sheikh Yaser. OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. IEEE transactions on pattern analysis and machine intelligence 43, 1 (2019), 2019;172–186.

  23. Cheng Bowen, Xiao Bin, Wang Jingdong, Shi Honghui, Huang Thomas S., Zhang Lei. HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020;5386–5395.

  24. Kreiss Sven, Bertoni Lorenzo, Alahi Alexandre. PifPaf: Composite Fields for Human Pose Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019;11969–11978.

  25. Zhang Jiabin, Zhu Zheng, Lu Jiwen, Huang Junjie, Huang Guan, Zhou Jie. SIMPLE: SIngle-network with Mimicking and Point Learning for Bottom-up Human Pose Estimation. In Proceedings of the AAAI Conference on Artificial Intelligence. 2021;3342–3350.

  26. He Kaiming, Zhang Xiangyu, Ren Shaoqing, Sun Jian. Deep Residual Learning for Image Recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016;770-778.

  27. Xie Saining, Girshick Ross. Aggregated residual transformations for deep neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017.

  28. Friedman JH. Stochastic gradient boosting. Computational statistics & data analysis. 2002;38(2002):367–78.

    Article  MathSciNet  MATH  Google Scholar 

  29. Gothe Neha P, Khan Imadh, Hayes Jessica, Erlenbach Emily, Damoiseaux Jessica S. Yoga effects on brain health: a systematic review of the current literature. Brain Plasticity 5, 1 (2019), 2019;105–122.

  30. Groessl Erik J, Chopra Deepak, Mills Paul J. An overview of yoga research for health and well-being. Journal of Yoga & Physical Therapy 5, 4 (2015), 2015;1.

  31. Gupta Ashish, Gupta Hari Prabhat. YogaHelp: Leveraging Motion Sensors for Learning Correct Execution of Yoga with Feedback. IEEE Transactions on Artificial Intelligence (2021). 2021.

  32. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9(8):1735–80.

    Article  Google Scholar 

  33. Huang Gao, Liu Zhuang, Maaten Laurens Van Der, Weinberger Kilian Q. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2017;4700–4708.

  34. Islam Muhammad Usama, Mahmud Hasan, Ashraf Faisal Bin, Hossain Iqbal, Hasan Md Kamrul. Yoga posture recognition by detecting human joint points in real time using microsoft kinect. In 2017 IEEE Region 10 humanitarian technology conference (R10-HTC). IEEE, 2017;668–673.

  35. Jain Shrajal, Rustagi Aditya, Saurav Sumeet, Saini Ravi, Singh Sanjay. Three-dimensional CNN-inspired deep learning architecture for Yoga pose recognition in the real-world environment. Neural Computing and Applications 33, 12 (2021), 2021;6427–6441.

  36. Ji Shuiwang, Xu Wei, Yang Ming, Yu Kai. 3D convolutional neural networks for human action recognition. IEEE transactions on pattern analysis and machine intelligence 35, 1 (2012), 2012;221–231.

  37. Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu T-Y. Lightgbm: A highly efficient gradient boosting decision tree. Adv Neural Inf Process Syst. 2017;30(2017):3146–54.

    Google Scholar 

  38. LeCun, Yann, Bottou, Léon, Bengio, Yoshua, Haffner, Patrick. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 1998;2278–2324.

  39. Li Yong-Lu, Xu Liang, Liu Xinpeng, Huang Xijie, Xu Yue, Wang Shiyi, Fang Hao-Shu, Ma Ze, Chen Mingyang, Lu Cewu. Pastanet: Toward human activity knowledge engine. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020;382–391.

  40. Lin Tsung-Yi, Maire Michael, Belongie Serge, Hays James, Perona Pietro, Ramanan Deva, Dollár Piotr, Zitnick C Lawrence. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 2014;740–755.

  41. Maddala Teja Kiran Kumar, Kishore PVV, Eepuri Kiran Kumar, Dande Anil Kumar. YogaNet: 3-D Yoga Asana Recognition Using Joint Angular Displacement Maps With ConvNets. IEEE Transactions on Multimedia 21, 10 (2019), 2019;2492–2503.

  42. Newell Alejandro, Yang Kaiyu, Deng Jia. Stacked hourglass networks for human pose estimation. In European conference on computer vision. Springer, 2016;483–499.

  43. Pandit Niharika. New Yoga Pose Dataset. 2020. https://www.kaggle.com/general/192938

  44. Ranasinghe Suneth, Al Machot Fadi, Mayr Heinrich C. A review on applications of activity recognition systems with regard to performance and evaluation. International Journal of Distributed Sensor Networks 12, 8 (2016), 2016;1550147716665520.

  45. Ross Alyson, Thomas Sue. The health benefits of yoga and exercise: a review of comparison studies. The journal of alternative and complementary medicine 16, 1 (2010), 2010;3–12.

  46. Sahni Pooja Swami, Singh Kamlesh, Sharma Nitesh, Garg Rahul. Yoga an effective strategy for self-management of stress-related problems and wellbeing during COVID19 lockdown: A cross-sectional study. PloS one 16, 2 (2021), 2021;e0245214.

  47. Sandler Mark, Howard Andrew, Zhu Menglong, Zhmoginov Andrey, Chen Liang-Chieh. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2018;4510–4520.

  48. Schapire Robert E. Explaining adaboost. In Empirical inference. Springer, 2013;37–52.

  49. Skurichina M, Duin RPW. Bagging for linear classifiers. Pattern Recogn. 1998;31(7):909–30.

    Article  Google Scholar 

  50. Toshev Alexander, Szegedy Christian. Deeppose: Human pose estimation via deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2014;1653–1660.

  51. Trejo Edwin W, Yuan Peijiang. Recognition of Yoga poses through an interactive system with Kinect device. In 2018 2nd International Conference on Robotics and Automation Sciences (ICRAS). IEEE, 2018;1–5.

  52. Verma Manisha, Kumawat Sudhakar, Nakashima Yuta, Raman Shanmuganathan. Yoga-82: a new dataset for fine-grained classification of human poses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020;1038–1039.

  53. Wang Donna. The use of yoga for physical and mental health among older adults: a review of the literature. International Journal of Yoga Therapy 19, 1 (2009), 2009;91–96.

  54. Woodyard Catherine. Exploring the therapeutic effects of yoga and its ability to increase quality of life. International journal of yoga 4, 2 (2011), 2011;49.

  55. Yadav Santosh Kumar, Singh Amitojdeep, Gupta Abhishek, Raheja Jagdish Lal. Real-time Yoga recognition using deep learning. Neural Computing and Applications 31, 2019;12(2019),9349–9361.

  56. Zhang Zhengyou. Microsoft kinect sensor and its effect. IEEE multimedia 19, 2012;2 (2012), 4–10.

  57. Zhu Rui, Galoogahi Hamed Kiani, Wang Chaoyang, Lucey Simon. Rethinking reprojection: Closing the loop for pose-aware shape reconstruction from a single image. In Proceedings of the IEEE International Conference on Computer Vision. 2017;57–65.

Download references

Acknowledgements

We would like to thank Prof. Varsha Singh and Prof. Sanjeev Jain from IIT Delhi for conceptualization of the Yoga dataset collection and would like to thank Kunal Singh for his contribution in dataset collection. We would like to acknowledge D&L 1-2-3-4 projects (MI01722G D&L-2017) at IIT Delhi for motivating the analysis presented in this work. We would also like to acknowledge Siddhaant Priyam, Misha Mishra, and Ankita Mandal from IIT Delhi for their efforts in setting up multiple pose estimation models and generating AlphaPose inferences on the Yoga dataset.

Author information

Authors and Affiliations

  1. Computer Science and Engineering, Indian Institute of Technology Delhi, Delhi, India

    Mustafa Chasmai & Rahul Garg

  2. Electrical Engineering, Indian Institute of Technology Delhi, Delhi, India

    Nirjhar Das

  3. School of Information Technology, Indian Institute of Technology Delhi, Delhi, India

    Aman Bhardwaj

Authors
  1. Mustafa Chasmai
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Nirjhar Das
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Aman Bhardwaj
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Rahul Garg
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Mustafa Chasmai.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Funding

The authors declare that they had no grant related to the funding of this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 4661 KB)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chasmai, M., Das, N., Bhardwaj, A. et al. A View Independent Classification Framework for Yoga Postures. SN COMPUT. SCI. 3, 476 (2022). https://doi.org/10.1007/s42979-022-01376-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-022-01376-7

Keywords