Skip to main content
SpringerLink
Log in
Book cover

International Conference on Analysis of Images, Social Networks and Texts

AIST 2019: Analysis of Images, Social Networks and Texts pp 263–269Cite as

Hand Gestures Detection, Tracking and Classification Using Convolutional Neural Network

  • Conference paper
  • First Online:

  • 633 Accesses

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1086))

Abstract

The article describes a software pipeline for detecting, tracking and classification of static hand gestures of the Russian Sign Language in a video stream using computer vision and deep learning techniques. The dataset used for this task is original, includes 10 classes and consists of more than 2000 unique images. The solution includes a hand detection module that uses a color mask, a gesture tracking module, a static gestures classification module in the detected region of the image based on convolutional neural network, as well as an auxiliary image preprocessing module and dataset augmentation module.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Source code and dataset (gestureset) are available on the GitHub: https://github.com/olpotkin/DNN-Gesture-Classifier.

References

  1. Krüger, J., Lien, T., Verl, A.: Cooperation of human and machines in assembly lines. CIRP Ann. Manuf. Technol. 58, 628–646 (2009)

    Article  Google Scholar 

  2. Bauer, A., Wollherr, D., Buss, M.: Human-robot collaboration: a survey. Int. J. Humanoid Rob. 5, 47–66 (2008)

    Article  Google Scholar 

  3. DailyTechInfo: DICE - gesture-based control HMI from Mersedes-Benz (2012). [https://dailytechinfo.org/auto/3291-dice-sistema-zhestovogo-upravleniya-avtomobilem-ot-mersedes-benz.html]

  4. Urmson, C., Dolgov, D., Nemec, P.: Driving pattern recognition and safety control (2011). [https://patents.google.com/patent/US8634980]

  5. Shumilov, A., Philippovich, A.: Gesture-based animated CAPTCHA. Inf. Comput. Secur. 24(3), 242–254 (2015)

    Article  Google Scholar 

  6. Letessier, J., Bérard, F.: Visual tracking of bare fingers for interactive surfaces. In: Proceedings of the 17th Annual ACM symposium on User interface software and technology, pp. 119–122 (1970)

    Google Scholar 

  7. Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. Pattern Anal. Mach. Intell. 24, 509–522 (2002)

    Article  Google Scholar 

  8. Doliotis, P., Stefan, A., McMurrough, C., Eckhard, D., Athitsos, V.: Comparing gesture recognition accuracy using color and depth information. In: Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments - PETRA 2011 (2011)

    Google Scholar 

  9. Nalepa, J., Grzejszczak, T., Kawulok, M.: Wrist Localization in Color Images for Hand Gesture Recognition. In: Gruca, D.A., Czachórski, T., Kozielski, S. (eds.) Man-Machine Interactions 3. AISC, vol. 242, pp. 79–86. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-02309-0_8

    Chapter  Google Scholar 

  10. Habili, N., Lim, C., Moini, A.: Segmentation of the face and hands in sign language video sequences using color and motion cues. IEEE Trans. Circuits Syst. Video Technol. 14(8), 1086–1097 (2004)

    Article  Google Scholar 

  11. Shaik, K., Ganesan, P., Kalist, V., Sathish, B.: Comparative study of skin color detection and segmentation in HSV and YCbCr color space. Procedia Comput. Sci. 57, 41–48 (2016)

    Article  Google Scholar 

  12. Förstner, W.: Image preprocessing for feature extraction in digital intensity, color and range images. In: Dermanis, A., Grün, A., Sansó, F. (eds.) Geomatic Method for the Analysis of Data in the Earth Sciences. LNEARTH, vol. 95, pp. 165–189. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45597-3_4

    Chapter  Google Scholar 

  13. Wang, J., Perez, L.: The effectiveness of data augmentation in image classification using deep learning. Stanford University (2017)

    Google Scholar 

  14. LeCun, Y., Jackel, L., Bottou, L.: Learning algorithms for classification: a comparison on handwritten digit recognition. AT&T Bell Laboratories (1995)

    Google Scholar 

  15. Potkin, O., Philippovich, A.: Static gestures classification using convolutional neural networks on the example of the Russian sign language. In: Supplementary Proceedings of the Seventh International Conference on Analysis of Images, Social Networks and Texts (AIST 2018), pp. 229–234 (2018)

    Google Scholar 

  16. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2015)

    Google Scholar 

  17. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B.: Generative adversarial nets. Advances in Neural Information Processing Systems 27 (NIPS 2014) (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Moscow Polytechnic University, Moscow, 107023, Russian Federation

    Oleg Potkin & Andrey Philippovich

Authors
  1. Oleg Potkin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrey Philippovich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oleg Potkin .

Editor information

Editors and Affiliations

  1. RWTH Aachen University, Aachen, Germany

    Wil M. P. van der Aalst

  2. University of Ljubljana, Ljubljana, Slovenia

    Vladimir Batagelj

  3. National Research University Higher School of Economics, Moscow, Russia

    Dmitry I. Ignatov

  4. Institute of Mathematics and Mechanics Yekaterinburg, Yekaterinburg, Russia

    Michael Khachay

  5. National Research University Higher School of Economics, Moscow, Russia

    Valentina Kuskova

  6. University of Oslo, Oslo, Norway

    Andrey Kutuzov

  7. National Research University Higher School of Economics, Moscow, Russia

    Sergei O. Kuznetsov

  8. National Research University Higher School of Economics, Moscow, Russia

    Irina A. Lomazova

  9. Moscow State University, Moscow, Russia

    Natalia Loukachevitch

  10. Loria, Vandoeuvre lès Nancy, France

    Amedeo Napoli

  11. University of Florida, Gainesville, USA

    Panos M. Pardalos

  12. Ca' Foscari University of Venice, Venezia Mestre, Italy

    Marcello Pelillo

  13. National Research University Higher School of Economics, Nizhny Novgorod, Russia

    Andrey V. Savchenko

  14. Kazan Federal University, Kazan, Russia

    Elena Tutubalina

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Potkin, O., Philippovich, A. (2020). Hand Gestures Detection, Tracking and Classification Using Convolutional Neural Network. In: van der Aalst, W., et al. Analysis of Images, Social Networks and Texts. AIST 2019. Communications in Computer and Information Science, vol 1086. Springer, Cham. https://doi.org/10.1007/978-3-030-39575-9_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-39575-9_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-39574-2

  • Online ISBN: 978-3-030-39575-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation