Skip to main content
Springer Nature Link
Log in

Gesture Recognition-Based Human–Computer Interaction Interface for Multimedia Applications

  • Chapter
  • First Online:
Digitisation of Culture: Namibian and International Perspectives

  • 847 Accesses

Abstract

We present an approach that uses visual hand detection, tracking and gesture recognition to provide an interactive interface between the user and a multimedia application. We use the Viola–Jones cascade detector to locate the hand and then utilise the unscented Kalman filter and a pre-constructed hand model for efficient object tracking. We then use a semantic-probabilistic method to recognise hand gestures that provide a simple and user-friendly way for the user to interact with the application. We perform experimental testing of our tracking system, validate the gesture recognition approach and evaluate the proposed approach in an augmented reality application. Finally, we discuss several possible applications that can use our gesture recognition approach, including a virtual reality application, a smart home control system and an interface to interact with museum exhibits.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Itkarkar, R.R., Nandi, A.V.: A survey of 2D and 3D imaging used in hand gesture recognition for human-computer interaction (HCI). In: 2016 IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE), Pune, pp. 188–193 (2016)

    Google Scholar 

  2. De Jesus Oliveira, V.A., Nedel, L., Maciel, A.: Proactive haptic articulation for intercommunication in collaborative virtual environments. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI), Greenville, SC, pp. 91–94 (2016)

    Google Scholar 

  3. Sreejith, M., Rakesh, S., Gupta, S., Biswas, S., Das, P.P.: Real-time hands-free immersive image navigation system using Microsoft Kinect 2.0 and Leap Motion Controller. In: 2015 Fifth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG), Patna, pp. 1–4 (2015)

    Google Scholar 

  4. Invitto, S., Faggiano, C., Sammarco, S., De Luca, V., De Paolis, L.T.: Interactive entertainment, virtual motion training and brain ergonomy. In: 7th International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN), 2015, Turin, pp. 88–94 (2015)

    Google Scholar 

  5. Chang, C.-C., Chen, I.-Y., Huang, Y.-S.: Hand pose recognition using curvature scale space. In: IEEE International Conference on Pattern Recognition (2002)

    Google Scholar 

  6. Utsumi, A., Miyasato, T., Kishino, F.: Multi-camera hand pose recognition system using skeleton image. In: IEEE International Workshop on Robot and Human Communication, pp. 219–224 (1995)

    Google Scholar 

  7. Aoki, Y., Tanahashi, S., Xu, J.: Sign language image processing for intelligent communication by communication satellite. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (1994)

    Google Scholar 

  8. Yang, J., Zhu, C., Yuan, J.: Real time hand gesture recognition via finger-emphasized multi-scale description. In: 2017 IEEE International Conference on Multimedia and Expo (ICME), Hong Kong, Hong Kong, pp. 631–636 (2017)

    Google Scholar 

  9. Zhao, S., Tan, W., Wu, C., Wen, L.: A Novel Interactive Method of Virtual Reality System Based on Hand Gesture Recognition, pp. 5879–5882. IEEE (2009). ISBN: 978-1-4244-2723-9/09

    Google Scholar 

  10. Sepehri, A., Yacoob, Y., Davis, L.: Employing the hand as an interface device. J. Multimed. 1(7), 18–29 (2006)

    Article  Google Scholar 

  11. Guan, Y., Zheng, M.: Real-time 3D pointing gesture recognition for natural HCI. In: Proceedings of the World Congress on Intelligent Control and Automation, China, pp. 2433–2436 (2008)

    Google Scholar 

  12. Freeman, W., Weissman, C.: Television control by hand gesture. In: IEEE International Workshop on Automatic Face and Gesture Recognition, Zurich (1995)

    Google Scholar 

  13. Kumar, P., Saini, R., Behera, S.K., Dogra, D.P., Roy, P.P.: Real-time recognition of sign language gestures and air-writing using leap motion. In: 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), Nagoya, pp. 157–160 (2017)

    Google Scholar 

  14. Erol, A., Bebis, G., Nicolescu, M., Boyle, R., Twombly, X.: Vision-based hand pose estimation: a review. Comput. Vis. Image Underst. 108, 52–73 (2007)

    Article  Google Scholar 

  15. Bao, P., Binh, N., Khoa, T.: A new approach to hand tracking and gesture recognition by a new feature type and HMM. In: International Conference on Fuzzy Systems and Knowledge Discovery. IEEE Computer Society (2009)

    Google Scholar 

  16. Kovalenko, M., Antoshchuk, S., Sieck, J.: Real-time hand tracking and gesture recognition using a semantic-probabilistic network. In: Proceedings of 16th International Conference on Computer Modelling and Simulation, Cambridge, pp. 269–274 (2014)

    Google Scholar 

  17. Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  18. Open Computer Vision Library. Retrieved from 26 June 2016. http://sourceforge.net/projects/opencvlibrary/

  19. Tayal, Y., Lamba, R., Padhee, S.: Automatic face detection using colour based segmentation. Int. J. Sci. Res. Publ. 2(6), 1–7 (2012)

    Google Scholar 

  20. Wang, X., Wang, J.: Simulation analysis of EKF and UKF implementations in PHD filter. In: 2016 IEEE 13th International Conference on Networking, Sensing, and Control (ICNSC), Mexico City, pp. 1–6 (2016)

    Google Scholar 

  21. Cooper, G., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)

    MATH  Google Scholar 

  22. Murphy, K.: Dynamic bayesian networks: representation, inference and learning. Ph.D. thesis, University of California at Berkley (2002)

    Google Scholar 

  23. Wobbrock, J.O., Wilson, A.D., Li, Y.: Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the ACM Symposium on User Interface Software and Technology, Newport, Rhode Island, pp. 159–168, 7–10 Oct 2007

    Google Scholar 

  24. Apolinário, A.L., Giraldi, G.A., Macedo, M.C., Souza, A.C.: A markerless augmented reality approach based on real-time 3D reconstruction using Kinect. In: Workshop of Works in Progress (WIP) in SIBGRAPI 2013, XXVI Conference on Graphics, Patterns and Images, Salvador, Brazil (2014)

    Google Scholar 

  25. Khandelwal, P., Swarnalatha, P., Bisht, N., Prabu, S.: Detection of features to track objects and segmentation using GrabCut for application in marker-less augmented reality. Procedia Comput. Sci. 58, 698–705 (2015)

    Article  Google Scholar 

  26. Sridhar, S., Oulasvirta, A., Theobalt, C.: Interactive markerless articulated hand motion tracking using RGB and depth data. In: Proceedings of the IEEE International Conference on Computer Vision (2013)

    Google Scholar 

  27. Baumann, A., Boltz, M., Ebling, J.: A review and comparison of measures for automatic video surveillance systems. EURASIP J. Image Video Process. 3, 280–312 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Odessa National Polytechnic University, Ave. Shevtchenko 1F, Odessa, 65044, Ukraine

    Svitlana Antoshchuk & Mykyta Kovalenko

  2. University of Applied Science Berlin, Wilhelminenhofstr. 75A, 12459, Berlin, Germany

    Jürgen Sieck

  3. Namibia University of Science and Technology (NUST), 13 Storch Street, Windhoek, Namibia

    Jürgen Sieck

Authors
  1. Svitlana Antoshchuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mykyta Kovalenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jürgen Sieck
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mykyta Kovalenko .

Editor information

Editors and Affiliations

  1. Computer Science, Namibia University of Science and Technology, Windhoek, Namibia

    Dharm Singh Jat

  2. Computer Science, HTW Berlin, Berlin, Germany

    Jürgen Sieck

  3. Computer Science, Namibia University of Science and Technology, Windhoek, Namibia

    Hippolyte N'Sung-Nza Muyingi

  4. Computer Science, Namibia University of Science and Technology, Windhoek, Namibia

    Heike Winschiers-Theophilus

  5. Computing and Informatics, Namibia University of Science and Technology, Windhoek, Namibia

    Anicia Peters

  6. Computer Science, Namibia University of Science and Technology, Windhoek, Namibia

    Shawulu Nggada

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Antoshchuk, S., Kovalenko, M., Sieck, J. (2018). Gesture Recognition-Based Human–Computer Interaction Interface for Multimedia Applications. In: Jat, D., Sieck, J., Muyingi, HN., Winschiers-Theophilus, H., Peters, A., Nggada, S. (eds) Digitisation of Culture: Namibian and International Perspectives. Springer, Singapore. https://doi.org/10.1007/978-981-10-7697-8_16

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-7697-8_16

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-7696-1

  • Online ISBN: 978-981-10-7697-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics