Skip to main content
SpringerLink
Log in

Easy Hand Gesture Control of a ROS-Car Using Google MediaPipe for Surveillance Use

  • Conference paper
  • First Online:
HCI in Business, Government and Organizations (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13327))

Included in the following conference series:

Abstract

Hand gestures are a relatively new way for humans to communicate with computers. The goal of gesture recognition is to bridge the physical and digital worlds. Hand gestures make it much easier to communicate our intentions and ideas to the computer. There are numerous methods for a computer to recognize a hand gesture, one of which is image recognition. The use of a Convolutional Neural Network (CNN) allows for the detection of human gestures. However, training a CNN necessitates a massive dataset of human gesture images. In this paper, we employ Google MediaPipe, a Machine Learning (ML) pipeline that combines Palm Detection and Hand Landmark Models, to develop a simple hand tracking method to control a Robot Operating System (ROS) based surveillance car with socket programming. The study demonstrates control of a ROS car’s steering direction and speed. Hand-gesture-controlled surveillance vehicles could aid in the improvement of security systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sun, J.H., Ji, T.T., Bin Zhang, S., Yang, J.K., Ji, G.R.: Research on the hand gesture recognition based on deep learning. In: 2018 12th International Symposium on Antennas, Propagation and EM Theory, ISAPE 2018 - Proceedings, pp. 10–13 (2019)

    Google Scholar 

  2. Song, S., Yan, D., Xie, Y.: Design of control system based on hand gesture recognition. In: ICNSC 2018 - 15th IEEE International Conference on Networking, Sensing and Control, no. 16, pp. 1–4 (2018)

    Google Scholar 

  3. Kabir, R., Ahmed, N., Roy, N., Islam, M.R.: A novel dynamic hand gesture and movement trajectory recognition model for non-touch HRI interface. In: 2019 IEEE Eurasia Conference on IOT, Communication and Engineering, ECICE 2019, pp. 505–508 (2019)

    Google Scholar 

  4. Meng, Z.Y., Pan, J.S., Tseng, K.K., Zheng, W.M.: Dominant points based hand finger counting for recognition under skin color extraction in hand gesture control system. In: Proceedings - 2012 6th International Conference on Genetic and Evolutionary Computing, ICGEC 2012, pp. 364–367 (2012)

    Google Scholar 

  5. Lamb, K., Madhe, S.: Automatic bed position control based on hand gesture recognition for disabled patients. In: International Conference on Automatic Control and Dynamic Optimization Techniques, ICACDOT 2016, pp. 148–153 (2017)

    Google Scholar 

  6. Koo, Y., et al.: An intelligent motion control of two wheel driving robot based voice recognition. In: ICCAS, pp. 13–15 (2014)

    Google Scholar 

  7. Chew, M.T., Penver, K.: Low-cost eye gesture communication system for people with motor disabilities. In: I2MTC 2019 - 2019 International Instrumentation and Measurement Technology Conference Proceedings, vol. 2019-May, pp. 8–12 (2019)

    Google Scholar 

  8. Mankar, S.M., Chhabria, S.A.: Review on hand gesture based mobile control application. In: 2015 International Conference on Pervasive Computing, ICPC 2015, vol. 00, no. c, pp. 58–59 (2015)

    Google Scholar 

  9. Wang, K., Zhao, R., Ji, Q.: Human computer interaction with head pose, eye gaze and body gestures. In: Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018, p. 789 (2018)

    Google Scholar 

  10. Wardhany, V.A., Kurnia, M.H., Sukaridhoto, S., Sudarsono, A., Pramadihanto, D.: Smart presentation system using hand gestures and Indonesian speech command. In: Proceedings - 2015 International Electronics Symposium, IES 2015, pp. 68–72 (2016)

    Google Scholar 

  11. Salunke, T.P.: Recognition based on hog feature extraction and K-NN classification. In: ICCMC, pp. 1151–1155 (2017)

    Google Scholar 

  12. Tsai, C.C., Kuo, C.C., Chen, Y.L.: 3D hand gesture recognition for drone control in unity. In: International Conference on Automation Science and Engineering, vol. 2020-August, pp. 985–988 (2020)

    Google Scholar 

  13. Yu, Y., Wang, X., Zhong, Z., Zhang, Y.: ROS-based UAV control using hand gesture recognition. In: Proceedings of the 29th Chinese Control And Decision Conference, CCDC 2017, vol. 410072, pp. 6795–6799 (2017)

    Google Scholar 

  14. Natarajan, K., Nguyen, T.H.D., Mete, M.: Hand gesture controlled drones: an open source library. In: Proceedings - 2018 1st International Conference on Data Intelligence and Security, ICDIS 2018, pp. 168–175 (2018)

    Google Scholar 

  15. Ghasemi, H., Mirfakhar, A., Masouleh, M.T., Kalhor, A.: Control a drone using hand movement in ROS based on single shot detector approach. In: 2020 28th Iranian Conference on Electrical Engineering, ICEE 2020 (2020)

    Google Scholar 

  16. Jain, M., et al.: Object detection and gesture control of four-wheel mobile robot. In: Proceedings of the 4th International Conference on Communication and Electronics Systems, ICCES 2019, pp. 303–308 (2019)

    Google Scholar 

  17. Wang, Y., Song, G., Qiao, G., Zhang, Y., Zhang, J., Wang, W.: Wheeled robot control based on gesture recognition using the Kinect sensor. In: 2013 International Conference on Robotics and Biomimetics, ROBIO 2013, no. December, pp. 378–383 (2013)

    Google Scholar 

  18. Sriram, K.N.V., Palaniswamy, S.: Mobile robot assistance for disabled and senior citizens using hand gestures. In: 1st International Conference on Power Electronics Applications and Technology in Present Energy Scenario, PETPES (2019)

    Google Scholar 

  19. Raheja, J.L., Shyam, R., Kumar, U., Prasad, P.B.: Real-time robotic hand control using hand gestures. In: ICMLC 2010 - 2nd International Conference on Machine Learning and Computing, pp. 12–16 (2010)

    Google Scholar 

  20. Bularka, S., Szabo, R., Otesteanu, M., Babaita, M.: Robotic arm control with hand movement gestures, pp. 543–546 (2018)

    Google Scholar 

  21. Choudhary, G.B., Chethan, R.B.V.: Real time robotic arm control using hand gestures. In: 2014 International Conference on High Performance Computing and Applications, ICHPCA 2014, pp. 5–7 (2015)

    Google Scholar 

  22. Islam, M.R., Mitu, U.K., Bhuiyan, R.A., Shin, J.: Hand gesture feature extraction using deep convolutional neural network for recognizing American sign language. In: 2018 4th International Conference on Frontiers of Signal Processing, ICFSP 2018, pp. 115–119 (2018)

    Google Scholar 

  23. Dhall, I., Vashisth, S., Aggarwal, G.: Automated hand gesture recognition using a deep convolutional neural network model. In: 2020 10th International Conference on Cloud Computing, Data Science & Engineering (Confluence), pp. 811–816 (2020). https://doi.org/10.1109/Confluence47617.2020.9057853

  24. Zhan, F.: Hand gesture recognition with convolution neural networks. In: Proceedings - 2019 IEEE 20th International Conference on Information Reuse and Integration for Data Science, IRI 2019, pp. 295–298 (2019)

    Google Scholar 

  25. Oluwatosin, H.S.: Client-server model. IOSR J. Comput. Eng. 16(1), 57–71 (2014)

    Article  Google Scholar 

  26. Kratky, S., Reichenberger, C.: Client/server development based on the apple event object model, Atlanta (2013

    Google Scholar 

  27. Yuqing, D.L.Z., Wu, W.: Efficient client assignment for ClientServer Systems. IEEE Trans. Netw. Serv. Manag. 13(4), 835–847 (2016)

    Article  Google Scholar 

  28. Xue, M., Zhu, C.: The socket programming and software design for communication based on client/server. In: Proceedings of the 2009 Pacific-Asia Conference on Circuits, Communications and Systems, PACCS 2009, pp. 775–777 (2009)

    Google Scholar 

  29. Zhang, F., et al.: MediaPipe hands: on-device real-time hand tracking. arXiv (2020)

    Google Scholar 

  30. About ROS. https://www.ros.org/about-ros/

  31. Meier, L.: Understanding ROS Nodes (2019). http://wiki.ros.org/ROS/Tutorials/UnderstandingNodes

  32. Kurzaj, D.: Messages (2016). http://wiki.ros.org/Messages

  33. Topics (2019). http://wiki.ros.org/Topics

  34. Kutluca, H.: Robot Operating System 2 (ROS 2) Architecture. https://medium.com/software-architecture-foundations/robot-operating-system-2-ros-2-architecture-731ef1867776

  35. Jazba, M.: Introduction to rosserial_arduino (2018). https://atadiat.com/en/e-rosserial-arduino-introduction/

Download references

Author information

Authors and Affiliations

  1. Minghsin University of Science and Technology, 30401, Xinfeng, Hsinchu County, Taiwan

    Christian Diego Allena, Ryan Collin De Leon & Yung-Hao Wong

  2. Adamson University, 900 Ermita, Manila, Philippines

    Ryan Collin De Leon

Authors
  1. Christian Diego Allena
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ryan Collin De Leon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yung-Hao Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yung-Hao Wong .

Editor information

Editors and Affiliations

  1. City University of Hong Kong, Hong Kong, Hong Kong

    Fiona Fui-Hoon Nah

  2. City University of Hong Kong, Hong Kong, Hong Kong

    Keng Siau

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Allena, C.D., De Leon, R.C., Wong, YH. (2022). Easy Hand Gesture Control of a ROS-Car Using Google MediaPipe for Surveillance Use. In: Fui-Hoon Nah, F., Siau, K. (eds) HCI in Business, Government and Organizations. HCII 2022. Lecture Notes in Computer Science, vol 13327. Springer, Cham. https://doi.org/10.1007/978-3-031-05544-7_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05544-7_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05543-0

  • Online ISBN: 978-3-031-05544-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation