skip to main content
10.1145/2789168.2790129acmconferencesArticle/Chapter ViewAbstractPublication PagesmobicomConference Proceedingsconference-collections
research-article

WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices

Published: 07 September 2015 Publication History

Abstract

This paper demonstrates that it is possible to leverage WiFi signals from commodity mobile devices to enable hands-free drawing in the air. While prior solutions require the user to hold a wireless transmitter, or require custom wireless hardware, or can only determine a pre-defined set of hand gestures, this paper introduces WiDraw, the first hand motion tracking system using commodity WiFi cards, and without any user wearables. WiDraw harnesses the Angle-of-Arrival values of incoming wireless signals at the mobile device to track the user's hand trajectory. We utilize the intuition that whenever the user's hand occludes a signal coming from a certain direction, the signal strength of the angle representing the same direction will experience a drop. Our software prototype using commodity wireless cards can track the user's hand with a median error lower than 5 cm. We use WiDraw to implement an in-air handwriting application that allows the user to draw letters, words, and sentences, and achieves a mean word recognition accuracy of 91%.

References

[1]
Microsoft kinect. http://www.microsoft.com/en-us/kinectforwindows/.
[2]
Leap motion, inc. leap motion: Mac & pc gesture controller for game, design and more. https://www.leapmotion.com/.
[3]
Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. Whole-home gesture recognition using wireless signals. In Proceedings of ACM Mobicom, 2013.
[4]
Fadel Adib and Dina Katabi. See through walls with WiFi! In Proceedings of ACM SIGCOMM, 2013.
[5]
Bryce Kellogg, Vamsi Talla, and Shyam Gollakota. Bringing gesture recognition to all devices. In Proceedings of USENIX NSDI, 2014.
[6]
Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C Miller. 3D tracking via body radio reflections. In Proceedings of USENIX NSDI, 2014.
[7]
Jue Wang, Deepak Vasisht, and Dina Katabi. RF-IDraw: Virtual Touch Screen in the Air Using RF Signals. In Proceedings of ACM SIGCOMM, 2014.
[8]
Pedro Melgarejo, Xinyu Zhang, Parameswaran Ramanathan, and David Chu. Leveraging directional antenna capabilities for fine-grained gesture recognition. In Proceedings of ACM Ubicomp, 2014.
[9]
Ralph O Schmidt. Multiple emitter location and signal parameter estimation. IEEE Transactions on Antennas and Propagation, AP-34(3):276--280, 1986.
[10]
Jon Gjengset, Jie Xiong, Graeme McPhillips, and Kyle Jamieson. Phaser: enabling phased array signal processing on commodity wifi access points. In Proceedings of ACM Mobicom, 2014.
[11]
Souvik Sen, Jeongkeun Lee, Kyu-Han Kim, and Paul Congdon. Avoiding multipath to revive inbuilding wifi localization. In Proceeding of ACM MobiSys, 2013.
[12]
Kainam Thomas Wong and Michael D Zoltowski. Root-music-based azimuth-elevation angle-of-arrival estimation with uniformly spaced but arbitrarily oriented velocity hydrophones. Signal Processing, IEEE Transactions on, 47(12):3250--3260, 1999.
[13]
Li Sun, Souvik Sen, and Dimitrios Koutsonikolas. Bringing mobility-awareness to WLANs using PHY layer information. In Proceedings of the ACM CoNEXT, 2014.
[14]
Myscript. Myscript stylus app. http://myscript.com/.
[15]
Wenwei Wang, Anja Brakensiek, and Gerhard Rigoll. Combination of multiple classifiers for handwritten word recognition. In Frontiers in Handwriting Recognition, 2002. Proceedings. Eighth International Workshop on, pages 117--122. IEEE, 2002.
[16]
Réjean Plamondon and Sargur N Srihari. On-line and off-line handwriting recognition: A comprehensive survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1):63--84, 2000.
[17]
James M Rehg and Takeo Kanade. Visual tracking of high DOF articulated structures: an application to human hand tracking. In Proceedings of the European Conference on Computer Vision, 1994.
[18]
Thad Starner and Alex Pentland. Real-time american sign language recognition from video using hidden markov models. In Proceedings of the International Symposium on Computer Vision, 1995.
[19]
Nintendo wii. http://www.nintendo.com/wiiu.
[20]
Sandip Agrawal, Ionut Constandache, Shravan Gaonkar, Romit Roy Choudhury, Kevin Caves, and Frank DeRuyter. Using mobile phones to write in air. In Proceedings of ACM MobiSys, 2011.
[21]
Taiwoo Park, Jinwon Lee, Inseok Hwang, Chungkuk Yoo, Lama Nachman, and Junehwa Song. E-Gesture: A Collaborative Architecture for Energy-efficient Gesture Recognition with Hand-worn Sensor and Mobile Devices. In Proceedings of ACM SenSys, 2011.
[22]
Chao Xu, Parth H. Pathak, and Prasant Mohapatra. Finger-writing with smartwatch: A case for finger and hand gesture recognition using smartwatch. In Proceedings of ACM HotMobile, 2015.
[23]
Myo: Gesture control armband. https://www.thalmic.com/en/myo/.
[24]
Abhinav Parate, Meng-Chieh Chiu, Chaniel Chadowitz, Deepak Ganesan, and Evangelos Kalogerakis. RisQ: recognizing smoking gestures with inertial sensors on a wristband. In Proceedings of ACM MobiSys, 2014.
[25]
Jeremy Gummeson, Bodhi Priyantha, and Jie Liu. An energy harvesting wearable ring platform for gestureinput on surfaces. In Proceedings of ACM MobiSys, 2014.
[26]
Shahriar Nirjon, Jeremy Gummeson, Dan Gelb, and Kyu-Han Kim. TypingRing: A Wearable Ring Platform for Text Input. In Proceedings of ACM MobiSys, 2015.
[27]
Fadel Adib, Zach Kabelac, and Dina Katabi. Multi-Person Localization via RF Body Reflections. In Proceedings of USENIX NSDI, 2015.
[28]
Rajalakshmi Nandakumar, Bryce Kellogg, and Shyamnath Gollakota. Wi-fi gesture recognition on existing devices. http://arxiv.org/ftp/arxiv/papers/1411/1411.5394.pdf.
[29]
Heba Abdelnasser, Moustafa Youssef, and Khaled Harras. Wigest: A ubiquitous wifi-based gesture recognition system. In Proceedings of IEEE INFOCOM, 2015.
[30]
IEEE 802.11ac -- Amendment 4: Enhancements for Very High Throughput for Operation in Bands below 6 GHz. IEEE P802.11ac/D5.0, 2013.
[31]
Ioannis Pefkianakis, Henrik Lundgren, Augustin Soule, Jaideep Chandrashekar, Pascal Le Guyadec, Christophe Diot, Martin May, Karel Van Doorselaer, and Koen Van Oost. Characterizing Home Wireless Performance: The Gateway View. In Proceedings of IEEE INFOCOM, 2015.

Cited By

View all
  • (2024)WiFi-CSI Difference ParadigmProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596088:2(1-29)Online publication date: 15-May-2024
  • (2024)Room-scale Location Trace Tracking via Continuous Acoustic WavesACM Transactions on Sensor Networks10.1145/364913620:3(1-23)Online publication date: 13-Apr-2024
  • (2024)KeyStubProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314427:4(1-23)Online publication date: 12-Jan-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MobiCom '15: Proceedings of the 21st Annual International Conference on Mobile Computing and Networking
September 2015
638 pages
ISBN:9781450336192
DOI:10.1145/2789168
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 September 2015

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. angle-of-arrival
  2. channel state information
  3. gesture recognition
  4. motion tracking
  5. wireless

Qualifiers

  • Research-article

Conference

MobiCom'15
Sponsor:

Acceptance Rates

MobiCom '15 Paper Acceptance Rate 38 of 207 submissions, 18%;
Overall Acceptance Rate 440 of 2,972 submissions, 15%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)74
  • Downloads (Last 6 weeks)6
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)WiFi-CSI Difference ParadigmProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596088:2(1-29)Online publication date: 15-May-2024
  • (2024)Room-scale Location Trace Tracking via Continuous Acoustic WavesACM Transactions on Sensor Networks10.1145/364913620:3(1-23)Online publication date: 13-Apr-2024
  • (2024)KeyStubProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314427:4(1-23)Online publication date: 12-Jan-2024
  • (2024)UniFiProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314297:4(1-29)Online publication date: 12-Jan-2024
  • (2024)Gesture Recognition Using Visible Light on Mobile DevicesIEEE/ACM Transactions on Networking10.1109/TNET.2024.336999632:4(2920-2935)Online publication date: Aug-2024
  • (2024)AMT+: Acoustic Multi-Target Tracking With Smartphone MIMO SystemIEEE Transactions on Mobile Computing10.1109/TMC.2024.341747423:12(12650-12665)Online publication date: Dec-2024
  • (2024)A Handwriting Recognition System With WiFiIEEE Transactions on Mobile Computing10.1109/TMC.2023.327960823:4(3391-3409)Online publication date: Apr-2024
  • (2024)WiRITE: General and Practical Wi-Fi Based Hand-Writing RecognitionIEEE Transactions on Mobile Computing10.1109/TMC.2023.326598823:4(2943-2957)Online publication date: Apr-2024
  • (2024)Passive Handwriting Tracking via Weak mmWave Communication SignalsIEEE Wireless Communications Letters10.1109/LWC.2023.334887513:3(874-878)Online publication date: Mar-2024
  • (2024)EchoGest: A Highly Scalable Unseen Gesture Recognition System Based on Feature-Wise TransformationIEEE Internet of Things Journal10.1109/JIOT.2024.340463111:18(29709-29727)Online publication date: 15-Sep-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

EPUB

View this article in ePub.

ePub

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media