Skip to main content
SpringerLink
Log in

3D-CNNs-Based Touchless Human-Machine Interface

  • Conference paper
  • First Online:
Interactive Collaborative Robotics (ICR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14214))

Included in the following conference series:

  • 324 Accesses

Abstract

Interacting with machines via hand gestures is a common way for people to communicate with robots. Human utilize gestures in a regular talk to convey meaning and emotions to one another. Gesture-based interactions are utilized in a wide range of applied to a wide range of fields, as telephones, TVs, monitors, video games, and other electronic devices. By technological improvements, gesture recognition is now a more realistic and appealing approach in the context of human interaction. In this research, the relevant experiments are conducted using numerous types of convolutional neural networks, including the proposed customized model, to see which ones performs the best. Because of the introduction of such Microsoft Kinect sensor, increased depth and vision sensing has been widely important for several purposes. Given its ability to measure ranges to objects at a fast frame rate, these types of sensors are widely being employed for 3D acquisitions, as well as for other purposes in robotics and machine learning. This research made use of the Kinect sensor and the use of an RGB-D camera and a 3D convolution neural network, which offer a novel approach for fingertips identification and hand gesture classification in real time that is both accurate and fast (3DCNN).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hasan, H., Abdul-Kareem, S.: Human–computer interaction using vision-based hand gesture recognition systems: a survey. Neural Comput. Appl. 25, 251–261 (2014)

    Article  Google Scholar 

  2. Klompmaker, F., Nebe, K., Fast, A.: dSensingNI: a framework for advanced tangible interaction using a depth camera. In: Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, pp. 217–224 (2012)

    Google Scholar 

  3. Gallo, L., Placitelli, A.P., Ciampi, M.: Controller-free exploration of medical image data: experiencing the Kinect. In: Proceedings of the 2011 24th International Symposium on Computer-Based Medical Systems (CBMS), pp. 1–6 (2011)

    Google Scholar 

  4. Krejov, P., Bowden, R.: Multi-touchless: real-time fingertip detection and tracking using geodesic maxima. In: Proceeding of 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–7 (2013)

    Google Scholar 

  5. Freeman, W.T., Roth, M.: Orientation histograms for hand gesture recognition. In: International Workshop on Automatic Face and Gesture Recognition, vol. 12, pp. 296–301 (1995)

    Google Scholar 

  6. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)

  7. Fossati, A., Gall, J., Grabner, H., Ren, X., Konolige, K.: Consumer Depth Cameras for Computer Vision: Research Topics and Applications. Springer, Heidelberg (2012)

    Google Scholar 

  8. Dao, T.T., Tannous, H., Pouletaut, P., Gamet, D., Istrate, D., Tho, M.H.B.: Interactive and connected rehabilitation systems for e-Health. Irbm 37(5–6), 289–296 (2016)

    Article  Google Scholar 

  9. Reza, M.N., Hossain, M.S., Ahmad, M.: Real time mouse cursor control based on bare finger movement using webcam to improve HCI. In: 2015 International Conference on Electrical Engineering and Information Communication Technology (ICEEICT), pp. 1–5. IEEE (2015)

    Google Scholar 

  10. Pisharady, P.K., Saerbeck, M.: Recent methods and databases in vision-based hand gesture recognition: a review. Comput. Vis. Image Underst. 141, 152–165 (2015)

    Article  Google Scholar 

  11. Amin, M.A., Yan, H.: Sign language finger alphabet recognition from Gabor-PCA representation of hand gestures. In: 2007 International Conference on Machine Learning and Cybernetics, vol. 4, pp. 2218–2223. IEEE (2007)

    Google Scholar 

  12. Sharma, A., Mittal, A., Singh, S., Awatramani, V.: Hand gesture recognition using image processing and feature extraction techniques. Procedia Comput. Sci. 173, 181–190 (2020)

    Article  Google Scholar 

  13. Lian, S., Hu, W., Wang, K.: Automatic user state recognition for hand gesture based low-cost television control system. IEEE Trans. Consum. Electron. 60(1), 107–115 (2014)

    Article  Google Scholar 

  14. Sharp, T., et al.: Accurate, robust, and flexible real-time hand tracking. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 3633–3642 (2015)

    Google Scholar 

  15. Fossati, A., Gall, J., Grabner, H., Ren, X., Konolige, K.: Consumer Depth Cameras for Computer Vision: Research Topics and Applications. Springer, Heıdelberg (2012)

    Google Scholar 

  16. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  17. Bagdanov, A.D., Del Bimbo, A., Seidenari, L., Usai, L.: Real-time hand status recognition from RGB-D imagery. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 2456–2459. IEEE (2012)

    Google Scholar 

  18. Tsai, T.H., Huang, C.C., Zhang, K.L.: Embedded virtual mouse system by using hand gesture recognition. In: 2015 IEEE International Conference on Consumer Electronics-Taiwan, pp. 352–353. IEEE (2015)

    Google Scholar 

  19. Wang, R.Y., Popović, J.: Real-time hand-tracking with a color glove. ACM Trans. Graph. (TOG) 28(3), 1–8 (2009)

    Google Scholar 

  20. Wang, P., Li, W., Ogunbona, P., Wan, J., Escalera, S.: RGB-D-based human motion recognition with deep learning: a survey. Comput. Vis. Image Underst. 171, 118–139 (2018)

    Article  Google Scholar 

  21. Grif, H.-S., Farcas, C.C.: Mouse cursor control system based on hand gesture. Procedia Technol. 22, 657–661 (2016)

    Article  Google Scholar 

  22. Andersen, M.R., et al.: Kinect depth sensor evaluation for computer vision applications. Aarhus University, pp. 1–37 (2012)

    Google Scholar 

  23. Shou, Z., Wang, D., Chang, S.-F.: Temporal action localization in untrimmed videos via multi-stage CNNs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1049–1058 (2016)

    Google Scholar 

  24. Molchanov, P., Yang, X., Gupta, S., Kim, K., Tyree, S., Kautz, J.: Online detection and classification of dynamic hand gestures with recurrent 3D convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4207–4215 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Process Automation Engineering Department, Baku Higher Oil School, 30, Khojaly Avenue, Baku, 1025, Azerbaijan

    Ali Asgarov & Ali Parsayan

Authors
  1. Ali Asgarov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Parsayan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ali Parsayan .

Editor information

Editors and Affiliations

  1. St. Petersburg Federal Research Center of the Russian Academy of Sciences, St. Petersburg, Russia

    Andrey Ronzhin

  2. Institute of Control Systems of Ministry of Science and Education of the Republic of Azerbaijan, Baku, Azerbaijan

    Aminagha Sadigov

  3. V.A. Trapeznikov Institute of Control Sciences of the Russian Academy of Sciences, Moscow, Russia

    Roman Meshcheryakov

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Asgarov, A., Parsayan, A. (2023). 3D-CNNs-Based Touchless Human-Machine Interface. In: Ronzhin, A., Sadigov, A., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2023. Lecture Notes in Computer Science(), vol 14214. Springer, Cham. https://doi.org/10.1007/978-3-031-43111-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43111-1_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43110-4

  • Online ISBN: 978-3-031-43111-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation