skip to main content
10.1145/3557738.3557854acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiconetsiConference Proceedingsconference-collections
research-article

Hand Detection and Hand Recognition Application Design for Human Computer Interaction Using SSD and Hand Landmark

Published: 21 November 2022 Publication History

Abstract

This research is intended to support computer interaction technology, especially the operation of computers to minimize the adverse effects of the dangers of direct physical contact between hands and technological objects. This research was conducted with the aim of utilizing the movement of the limbs of the hand as an object of interaction that bridges between humans and computers. With this research, it is hoped that the design results can minimize the risk of spreading bacteria and viruses that cause health problems. The objects used in the design will focus on the pattern/shape and movement of the hands, and in training the pattern/shape of the hand as a pointing tool and keyboard shortcut input. The design was built by using a webcam as a sensor to capture images. This design used the help of the field of artificial intelligence, hand detection using the Single-Shot Detector method with hand recognition using Hand Landmark. The value of success in functionality obtained from the application was 94.73%. The results of the model evaluation obtained that the final average recall value was 96%, the precision value was 100%, and the accuracy value was 96%. From the research that has been done, it is possible to replace the mouse function and several keyboard shortcuts with right- and left-hand movements.

References

[1]
Z. Nan and Y. Li, "Transmission of influenza A in a student office based on realistic person-to-person contact and surface touch behaviour," International journal of environmental research and public health, vol. 15, p. 1699, 2018.
[2]
J. D. Asmoro, Virtual Mouse Menggunakan Hand Gesture Recognition Berbasis Hand Landmark Model Untuk Pointing Device, Surabaya: Universitas Islam Negeri Sunan Ampel, 2022.
[3]
A. Lora and P. D. Arnesia, Vimo Applications as Virtual Mouse with Real Time, Depok: Seminar Nasional Teknologi Informasi dan Komunikasi STI&K (SeNTIK), 2020.
[4]
T.-H. Tsai, C.-C. Huang and K.-L. Zhang, Design of hand gesture recognition system for human-computer interaction, Jung-Li: National Central University, 2019.
[5]
H. T. Thang, D. V. Hoa and D. N. Tuan, "Computer control in human-machine interaction systems by hand movements," Computer science and Control engineering, pp. 42-48, 2021.
[6]
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu and A. C. Berg, "SSD: Single Shot MultiBox Detector," in Computer Vision - ECCV 2016, Springer International Publishing, 2016, pp. 21-37.
[7]
M. Everingham and J. Winn, "The PASCAL Visual Object Classes Challenge," 7 June 2007. [Online]. Available: http://host.robots.ox.ac.uk/pascal/VOC/voc2007/.
[8]
M. Everingham and J. Winn, "The PASCAL Visual Object Classes Challenge," 18 May 2012. [Online]. Available: http://host.robots.ox.ac.uk/pascal/VOC/voc2012/.
[9]
T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, Z. C. Lawrence and P. Dollár, "Microsoft COCO: Common Objects in Context," ArXiv.org, 2014.

Index Terms

  1. Hand Detection and Hand Recognition Application Design for Human Computer Interaction Using SSD and Hand Landmark

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICONETSI '22: Proceedings of the 2022 International Conference on Engineering and Information Technology for Sustainable Industry
    September 2022
    450 pages
    ISBN:9781450397186
    DOI:10.1145/3557738
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 November 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Artificial Intelligence
    2. Hand Landmark
    3. Hand Recognition
    4. Human and Computer Interaction
    5. Single-Shot Detector

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICONETSI '22

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 51
      Total Downloads
    • Downloads (Last 12 months)19
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media