Skip to main content
SpringerLink
Log in

Nrityamanthan: Unravelling the Intent of the Dancer Using Deep Learning

  • Chapter
  • First Online:
Heritage Preservation

Abstract

This work focuses on understanding small video sequences of Bharatanatyam, which as a whole comprise of hand gestures, facial expressions and dynamic body postures called Adavus. Initially, we attempt to recognize static as well as dynamic hand gestures with 2D and 3D Convolutional Neural Networks (CNNs). Subsequently, we also attempt to interpret the basic dance steps in Bharatanatyam, i.e Adavu. Since no public datasets are available for Indian Classical Dance (ICD), we propose datasets for static and dynamic hand gestures captured under controlled laboratory settings as well as from real-world videos. We also create a separate dataset for Adavus. For detecting static hand gestures in video frames, an adaptive boosting (AdaBoost) (Schapire and Singer, Mach Learn 37(3):297–336, 1999) [4] approach and region-based CNN (R-CNN) (Girshick et al., Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587, 2014) [5] are used. Furthermore, a skin detector is used for refining the localization of hands in the images. Finally, we demonstrate the interesting possibility of using the proposed algorithms for understanding a video of a Shloka enacted in Bharatanatyam.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Meduri, A.: Bharatha natyam-what are you? Asian Theatre J. 5(1), 1–22 (1988)

    Article  Google Scholar 

  2. Mohanty, A., Duttagupta, A., Sahay, R.R.: Hand gesture recognition to understand indian classical dance videos. In: Accepted in Proceedings of Indian Conference on Computer Vision, Graphics and Image Processing (2016)

    Google Scholar 

  3. Mohanty, A., Vaishnavi, P., Jana, P., Majumdar, A., Ahmed, A., Goswami, T., Sahay, R.R.: Nrityabodha: towards understanding Indian classical dance using a deep learning approach. Signal Process.: Image Commun. 47, 529–548 (2016)

    Google Scholar 

  4. Schapire, R.E., Singer, Y.: Improved boosting algorithms using confidence-rated predictions. Mach. Learn. 37(3), 297–336 (1999)

    Article  Google Scholar 

  5. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

    Google Scholar 

  6. Gupt, B.: Dramatic Concepts Greek and Indian: A Study of the Poetics and the Natyasastra. DK Printworld (1994)

    Google Scholar 

  7. Raut, M.: Odissi, What, Why and How-: Evolution. Revival and Technique, BR Rhythms (2007)

    Google Scholar 

  8. Mohanty, A., Rambhatla, S.S., Sahay, R.R.: Deep gesture: static hand gesture recognition using CNN. In: Proceedings of International Conference on Computer Vision and Image Processing, pp. 449–461 (2017)

    Google Scholar 

  9. Mittal, A., Zisserman, A., Torr, P.H.: Hand detection using multiple proposals. In: Proceedings of British Machine Vision Conference, pp. 1–11 (2011)

    Google Scholar 

  10. Pisharady, P.K., Vadakkepat, P., Loh, A.P.: Attention based detection and recognition of hand postures against complex backgrounds. Int. J. Comput. Vis. 101(3), 403–419 (2013)

    Article  Google Scholar 

  11. Kurakin, A., Zhang, Z., Liu, Z.: A real time system for dynamic hand gesture recognition with a depth sensor. In: Proceedings of the 20th IEEE European Signal Processing Conference, pp. 1975–1979 (2012)

    Google Scholar 

  12. Shin, S., Sung, W.: Dynamic hand gesture recognition for wearable devices with low complexity recurrent neural networks. In: IEEE International Symposium on Circuits and Systems, pp. 2274–2277 (2016)

    Google Scholar 

  13. Oikonomidis, I., Kyriazis, N., Argyros, A.A.: Efficient model-based 3D tracking of hand articulations using kinect. In: Proceedings of British Machine Vision Conference, vol. 1, p. 3 (2011)

    Google Scholar 

  14. Tompson, J., Stein, M., Lecun, Y., Perlin, K.: Real-time continuous pose recovery of human hands using convolutional networks. ACM Trans. Graph. 33(5), 169 (2014)

    Article  Google Scholar 

  15. Sridhar, S., Mueller, F., Oulasvirta, A., Theobalt, C.: Fast and robust hand tracking using detection-guided optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213–3221 (2015)

    Google Scholar 

  16. Yamato, J., Ohya, J., Ishii, K.: Recognizing human action in time-sequential images using hidden Markov model. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 379–385 (1992)

    Google Scholar 

  17. Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  18. Yang, M.H., Ahuja, N.: Recognizing hand gestures using motion trajectories. In: Proceedings of Face Detection and Gesture Recognition for Human-Computer Interaction, pp. 53–81 (2001)

    Chapter  Google Scholar 

  19. Hong, P., Turk, M., Huang, T.S.: Gesture modeling and recognition using finite state machines. In: Proceedings of Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 410–415 (2000)

    Google Scholar 

  20. Henia, O.B., Bouakaz, S.: A new depth-based function for 3D hand motion tracking. In: International Conference on Computer Vision Theory and Applications, pp. 653–658 (2011)

    Google Scholar 

  21. Van den Bergh, M., Van Gool, L.: Combining RGB and ToF cameras for real-time 3D hand gesture interaction. In: IEEE Workshop on Applications of Computer Vision, pp. 66–72 (2011)

    Google Scholar 

  22. Suryanarayan, P., Subramanian, A., Mandalapu, D.: Dynamic hand pose recognition using depth data. In: Proceedings of 20th IEEE International Conference on Pattern Recognition, pp. 3105–3108 (2010)

    Google Scholar 

  23. Kollorz, E., Penne, J., Hornegger, J., Barke, A.: Gesture recognition with a time-of-flight camera. Int. J. Intell. Syst. Technol. Appl. 5(3–4), 334–343 (2008)

    Google Scholar 

  24. Erdem, C., Ulukaya, S., Karaali, A., Erdem, A.T.: Combining Haar feature and skin color based classifiers for face detection. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1497–1500 (2011)

    Google Scholar 

  25. Nalepa, J., Grzejszczak, T., Kawulok, M.: Wrist localization in color images for hand gesture recognition. In: Man-Machine Interactions 3, pp. 79–86. Springer International Publishing (2014)

    Google Scholar 

  26. Stöttinger, J., Hanbury, A., Liensberger, C., Khan, R.: Skin paths for contextual flagging adult videos. In: Proceedings Part II of 5th International Symposium on Advances in Visual Computing, pp. 303–314. Springer (2009)

    Chapter  Google Scholar 

  27. Chan, C.S., Liu, H., Brown, D.J.: Recognition of human motion from qualitative normalised templates. J. Intell. Robot. Syst. 48(1), 79–95 (2007)

    Article  Google Scholar 

  28. Zhang, Z., Gunes, H., Piccardi, M.: Head detection for video surveillance based on categorical hair and skin colour models. In: 16th IEEE International Conference on Image Processing, pp. 1137–1140 (2009)

    Google Scholar 

  29. Vezhnevets, V., Sazonov, V., Andreeva, A.: A survey on pixel-based skin color detection techniques. Proceeding Graphicon 3, 85–92 (2003)

    Google Scholar 

  30. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

    Google Scholar 

  31. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html

  32. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  33. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  34. LeCun, Y., Huang, F.J., Bottou, L.: Learning methods for generic object recognition with invariance to pose and lighting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2, 97–104 (2004)

    Google Scholar 

  35. Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  36. Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)

    Google Scholar 

  37. Vedaldi, A., Lenc, K.: MatConvNet: convolutional neural networks for MATLAB. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 689–692 (2015)

    Google Scholar 

  38. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computational Vision Laboratory, Department of Electrical Engineering, Indian Institute of Technology, Kharagpur, India

    Aparna Mohanty, Kankana Roy & Rajiv R. Sahay

Authors
  1. Aparna Mohanty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kankana Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajiv R. Sahay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aparna Mohanty .

Editor information

Editors and Affiliations

  1. Electronics and Communication Sciences Unit, Indian Statistical Institute, Kolkata, West Bengal, India

    Bhabatosh Chanda

  2. Department of Electrical Engineering, Indian Institute of Technology Bombay, Mumbai, Maharashtra, India

    Subhasis Chaudhuri

  3. CSIR-CEERI, Pilani, Rajasthan, India

    Santanu Chaudhury

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Mohanty, A., Roy, K., Sahay, R.R. (2018). Nrityamanthan: Unravelling the Intent of the Dancer Using Deep Learning. In: Chanda, B., Chaudhuri, S., Chaudhury, S. (eds) Heritage Preservation. Springer, Singapore. https://doi.org/10.1007/978-981-10-7221-5_11

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-7221-5_11

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-7220-8

  • Online ISBN: 978-981-10-7221-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation