skip to main content
10.1145/3379336.3381507acmconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
extended-abstract

Developing a Hand Gesture Recognition System for Mapping Symbolic Hand Gestures to Analogous Emoji in Computer-Mediated Communications

Published: 17 March 2020 Publication History

Abstract

Recent trends in computer-mediated communications (CMC) have seen messaging with richer media not only in images and videos, but in visual communication markers (VCM) such as emoticons, emojis, and stickers. VCMs could prevent a potential loss of subtle emotional conversation in CMC, which is delivered by nonverbal cues that convey affective and emotional information. However, as the number of VCMs grows in the selection set, the problem of VCM entry needs to be addressed. Furthermore, conventional means of accessing VCMs continue to rely on input entry methods that are not directly and intimately tied to expressive nonverbal cues. In this work, we aim to address this issue, by facilitating the use of an alternative form of VCM entry: hand gestures. To that end, we propose a user-defined hand gesture set that is highly representative of a number of VCMs and a two-stage hand gesture recognition system (trajectory-based, shape-based) that can identify these user-defined hand gestures with an accuracy of 82%. By developing such a system, we aim to allow people using low-bandwidth forms of CMCs to still enjoy their convenient and discreet properties, while also allowing them to experience more of the intimacy and expressiveness of higher-bandwidth online communication.

References

[1]
Susan Goldin-Meadow. 2005. Hearing gesture: How our hands help us think. Harvard University Press, Cambridge, MA.
[2]
Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The WEKA Data Mining Software: An Update. SIGKDD Explor. Newsl. 11, 1 (Nov. 2009), 10--18. https://doi.org/10.1145/1656274.1656278
[3]
Mika Hatano, Shinji Sako, and Tadashi Kitamura. 2015. Contour-based Hand Pose Recognition for Sign Language Recognition. In 6th Workshop on Speech and Language Processing for Assistive Technologies (SLPAT) (Interspeech '15). SIG-SLPAT, Dresden, Germany, 17.
[4]
Jung In Koh, Josh Cherian, Paul Taele, and Tracy Hammond. 2019. Developing a Hand Gesture Recognition System for Mapping Symbolic Hand Gestures to Analogous Emojis in Computer-Mediated Communication. ACM Trans. Interact. Intell. Syst. 9, 1, Article 6 (March 2019), 35 pages. https://doi.org/10.1145/3297277
[5]
Chiew Seng Sean Tan, Johannes Schöning, Kris Luyten, and Karin Coninx. 2013. Informing intelligent user interfaces by inferring affective states from body postures in ubiquitous computing environments. In Proceedings of the 2013 international conference on Intelligent user interfaces (IUI '13). ACM, Santa Monica, CA, USA, 235--246.
[6]
Cuong Tran and Mohan Manubhai Trivedi. 2012. 3-D posture and gesture recognition for interactivity in smart spaces. IEEE Transactions on Industrial Informatics 8, 1 (2012), 178--187.
[7]
Joseph B Walther and Kyle P D'Addario. 2001. The impacts of emoticons on message interpretation in computer-mediated communication. Social science computer review 19, 3 (August 2001), 324--347.
[8]
Jacob O. Wobbrock, Htet Htet Aung, Brandon Rothrock, and Brad A. Myers. 2005. Maximizing the Guessability of Symbolic Input. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, New York, NY, USA, 1869--1872. https://doi.org/10.1145/1056808.1057043

Cited By

View all
  • (2022)Recognition of Complex Gestures for Real-Time Emoji AssignmentImage Analysis and Processing. ICIAP 2022 Workshops10.1007/978-3-031-13324-4_19(215-227)Online publication date: 23-May-2022

Index Terms

  1. Developing a Hand Gesture Recognition System for Mapping Symbolic Hand Gestures to Analogous Emoji in Computer-Mediated Communications

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        IUI '20 Companion: Companion Proceedings of the 25th International Conference on Intelligent User Interfaces
        March 2020
        153 pages
        ISBN:9781450375139
        DOI:10.1145/3379336
        Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 17 March 2020

        Check for updates

        Author Tags

        1. CMC
        2. emoji
        3. hand gesture recognition

        Qualifiers

        • Extended-abstract
        • Research
        • Refereed limited

        Conference

        IUI '20
        Sponsor:

        Acceptance Rates

        Overall Acceptance Rate 746 of 2,811 submissions, 27%

        Upcoming Conference

        IUI '25

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)9
        • Downloads (Last 6 weeks)0
        Reflects downloads up to 17 Jan 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2022)Recognition of Complex Gestures for Real-Time Emoji AssignmentImage Analysis and Processing. ICIAP 2022 Workshops10.1007/978-3-031-13324-4_19(215-227)Online publication date: 23-May-2022

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media