Yuki Yamato
Buntarou Shizuki
Shin Takahashi
ABSTRACT
We propose a hand gesture interaction method using a low-resolution infrared image sensor on an inner wrist. We attach the sensor to the strap of a wrist-worn device, on the palmar side, and apply machine-learning techniques to recognize the gestures made by the opposite hand. As the sensor is placed on the inner wrist, the user can naturally control its direction to reduce privacy invasion. Our method can recognize four types of hand gestures: static hand poses, dynamic hand gestures, finger motion, and the relative hand position. We developed a prototype that does not invade surrounding people's privacy using an 8 x 8 low-resolution infrared image sensor. Then we conducted experiments to validate our prototype, and our results imply that the low-resolution sensor has sufficient capabilities for recognizing a rich array of hand gestures. In this paper, we introduce an implementation of a mapping application that can be controlled by our specified hand gestures, including gestures that use both hands.
- Feiyu Chen, Jia Deng, Zhibo Pang, Majid Baghaei Nejad, Huayong Yang, and Geng Yang. 2018. Finger Angle-Based Hand Gesture Recognition for Smart Infrastructure Using Wearable Wrist-Worn Camera. Applied Sciences 8, 3 (2018). https://doi.org/10.3390/app8030369Google Scholar
- Pascal Chiu, Kazuki Takashima, Kazuyuki Fujita, and Yoshifumi Kitamura. 2019. Pursuit Sensing: Extending Hand Tracking Space in Mobile VR Applications. In Symposium on Spatial User Interaction (New Orleans, LA, USA) (SUI '19). Association for Computing Machinery, New York, NY, USA, Article 1, 5 pages. https://doi.org/10.1145/3357251.3357578Google ScholarDigital Library
- Andrew Clark and Deshendran Moodley. 2016. A System for a Hand Gesture-Manipulated Virtual Reality Environment. In Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists (Johannesburg, South Africa) (SAICSIT '16). Association for Computing Machinery, New York, NY, USA, Article 10, 10 pages. https://doi.org/10.1145/2987491.2987511Google ScholarDigital Library
- Jun Gong, Yang Zhang, Xia Zhou, and Xing-Dong Yang. 2017. Pyro: Thumb-Tip Gesture Recognition Using Pyroelectric Infrared Sensing. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST '17). Association for Computing Machinery, New York, NY, USA, 553--563. https://doi.org/10.1145/3126594.3126615Google ScholarDigital Library
- David Kim, Otmar Hilliges, Shahram Izadi, Alex D. Butler, Jiawen Chen, Iason Oikonomidis, and Patrick Olivier. 2012. Digits: Freehand 3D Interactions Anywhere Using a Wrist-Worn Gloveless Sensor. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (Cambridge, Massachusetts, USA) (UIST 12). Association for Computing Machinery, New York, NY, USA, 167--176. https://doi.org/10.1145/2380116.2380139Google ScholarDigital Library
- Jungsoo Kim, Jiasheng He, Kent Lyons, and Thad Starner. 2007. The Gesture Watch: A Wireless Contact-free Gesture based Wrist Interface. In 2007 11th IEEE International Symposium on Wearable Computers. IEEE Computer Society, USA, 15--22. https://doi.org/10.1109/ISWC.2007.4373770Google ScholarDigital Library
- Yichen Li, Tianxing Li, Ruchir A. Patel, Xing-Dong Yang, and Xia Zhou. 2018. Self-Powered Gesture Recognition with Ambient Light. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST '18). Association for Computing Machinery, New York, NY, USA, 595--608. https://doi.org/10.1145/3242587.3242635Google ScholarDigital Library
- Hui Liang, Junsong Yuan, Daniel Thalmann, and Nadia Magnenat Thalmann. 2015. AR in Hand: Egocentric Palm Pose Tracking and Gesture Recognition for Augmented Reality Applications. In Proceedings of the 23rd ACM International Conference on Multimedia (Brisbane, Australia) (MM 15). Association for Computing Machinery, New York, NY, USA, 743--744. https://doi.org/10.1145/2733373.2807972Google ScholarDigital Library
- Manuel Prätorius, Dimitar Valkov, Ulrich Burgbacher, and Klaus Hinrichs. 2014. DigiTap: An Eyes-Free VR/AR Symbolic Input Device. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology (Edinburgh, Scotland) (VRST '14). Association for Computing Machinery, New York, NY, USA, 9--18. https://doi.org/10.1145/2671015.2671029Google ScholarDigital Library
- Katie A. Siek, Yvonne Rogers, and Kay H. Connelly. 2005. Fat Finger Worries: How Older and Younger Users Physically Interact with PDAs. In Proceedings of the 2005 IFIP TC13 International Conference on Human-Computer Interaction (Rome, Italy) (INTERACT '05). Springer-Verlag, Berlin, Heidelberg, 267--280. https://doi.org/10.1007/11555261_24Google ScholarDigital Library
- Srinath Sridhar, Anders Markussen, Antti Oulasvirta, Christian Theobalt, and Sebastian Boring. 2017. WatchSense: On- and Above-Skin Input Sensing through a Wearable Depth Sensor. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 3891--3902. https://doi.org/10.1145/3025453.3026005Google ScholarDigital Library
- Emi Tamaki, Takashi Miyaki, and Jun Rekimoto. 2009. Brainy Hand: An Ear-Worn Hand Gesture Interaction Device. In CHI '09 Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA) (CHI EA '09). Association for Computing Machinery, NewYork, NY, USA, 4255--4260. https://doi.org/10.1145/1520340.1520649Google ScholarDigital Library
- Theophilus Teo, Gun A. Lee, Mark Billinghurst, and Matt Adcock. 2018. Hand Gestures and Visual Annotation in Live 360 Panorama-Based Mixed Reality Remote Collaboration. In Proceedings of the 30th Australian Conference on Computer-Human Interaction (Melbourne, Australia) (OzCHI '18). Association for Computing Machinery, New York, NY, USA, 406--410. https://doi.org/10.1145/3292147.3292200Google ScholarDigital Library
- Andrew Vardy, John Robinson, and Li-Te Cheng. 1999. The WristCam as Input Device. In Proceedings of the 3rd IEEE International Symposium on Wearable Computers (ISWC '99). IEEE Computer Society, USA, 199.Google ScholarCross Ref
- Hongyi Wen, Julian Ramos Rojas, and Anind K. Dey. 2016. Serendipity: Finger Gesture Recognition Using an Off-the-Shelf Smartwatch. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 3847--3851. https://doi.org/10.1145/2858036.2858466Google ScholarDigital Library
- Piotr Wojtczuk, Alistair Armitage, David Binnie, and Timothy Chamberlain. 2011. Recognition of simple gestures using a PIR sensor array. Sensors & Transducers 14 (2011), 83--94. http://researchrepository.napier.ac.uk/id/eprint/5228Google Scholar
- Robert Xiao, Teng Cao, Ning Guo, Jun Zhuo, Yang Zhang, and Chris Harrison. 2018. LumiWatch: On-Arm Projected Graphics and Touch Input. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, Article 95, 11 pages. https://doi.org/10.1145/3173574.3173669Google ScholarDigital Library
- Chao Xu, Parth H. Pathak, and Prasant Mohapatra. 2015. Finger-Writing with Smartwatch: A Case for Finger and Hand Gesture Recognition Using Smart-watch. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications (Santa Fe, New Mexico, USA) (HotMobile '15). Association for Computing Machinery, New York, NY, USA, 9--14. https://doi.org/10.1145/2699343.2699350Google ScholarDigital Library
- Hui-Shyong Yeo, Erwin Wu, Juyoung Lee, Aaron Quigley, and Hideki Koike. 2019. Opisthenar: Hand Poses and Finger Tapping Recognition by Observing Back of Hand Using Embedded Wrist Camera. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST '19). Association for Computing Machinery, New York, NY, USA, 963--971. https://doi.org/10.1145/3332165.3347867Google ScholarDigital Library
Index Terms
- Hand Gesture Interaction with a Low-Resolution Infrared Image Sensor on an Inner Wrist
Recommendations
BeamBand: Hand Gesture Sensing with Ultrasonic Beamforming
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing SystemsBeamBand is a wrist-worn system that uses ultrasonic beamforming for hand gesture sensing. Using an array of small transducers, arranged on the wrist, we can ensem-ble acoustic wavefronts to project acoustic energy at spec-ified angles and focal ...
Human-Machine Interaction based on Hand Gesture Recognition using Skeleton Information of Kinect Sensor
ICAIT'2018: Proceedings of the 3rd International Conference on Applications in Information TechnologyThe hand gesture provides a natural and intuitive communication medium for the human and machine interaction. Because, it can use in virtual reality, language detection, computer games, and other human-computer or human-machine instruction applications. ...
Designing and Evaluating Hand-to-Hand Gestures with Dual Commodity Wrist-Worn Devices
Hand gestures provide a natural and easy-to-use way to input commands. However, few works have studied the design space of bimanual hand gestures or attempted to infer gestures that involve devices on both hands. We explore the design space of hand-to-...
Comments