research-article

SideSwipe: detecting in-air gestures around mobile devices using actual GSM signal

Authors:

Md Tanvir Islam Aumi,

Matthew S. ReynoldsAuthors Info & Claims

UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technology

Pages 527 - 534

https://doi.org/10.1145/2642918.2647380

Published: 05 October 2014 Publication History

Abstract

Current smartphone inputs are limited to physical buttons, touchscreens, cameras or built-in sensors. These approaches either require a dedicated surface or line-of-sight for interaction. We introduce SideSwipe, a novel system that enables in-air gestures both above and around a mobile device. Our system leverages the actual GSM signal to detect hand gestures around the device. We developed an algorithm to convert the discrete and bursty GSM pulses to a continuous wave that can be used for gesture recognition. Specifically, when a user waves their hand near the phone, the hand movement disturbs the signal propagation between the phone's transmitter and added receiving antennas. Our system captures this variation and uses it for gesture recognition. To evaluate our system, we conduct a study with 10 participants and present robust gesture recognition with an average accuracy of 87.2% across 14 hand gestures.

Supplementary Material

suppl.mov (uistf2558-file3.mp4)

Supplemental video

Download
1.60 KB

References

[1]

Adib, F. and Katabi, D., See Through Walls with WiFi!. In Proc SIGCOMM 2013

Digital Library

[2]

Butler, A., Izadi, S., and Hodges, S. SideSight: multitouch interaction around small devices., In Proc UIST 2013, pp. 201--204.

Digital Library

[3]

Chen, K., Lyons, K., White, S., and Patel, S. uTrack: 3D input using two magnetic sensors. In Proc UIST 2013, pp. 237--244.

Digital Library

[4]

Chetty, K., Smith, G., Guo, H., and Woodbridge, K. Target detection in high clutter using passive bistatic WiFi radar. 2009 IEEE Radar Conference, pp. 1--5.

[5]

Chintalapudi, K., Padmanabha Iyer, A., and Padmanabhan, V.N. Indoor localization without the pain. In Proc MobiCom 2010, 173.

Digital Library

[6]

Cohn, G., Morris, D., Patel, S., and Tan, D. Your noise is my command: sensing gestures using the body as an antenna. In Proc CHI 2011, pp. 791--800.

Digital Library

[7]

Cohn, G., Morris, D., Patel, S., and Tan, D. Humantenna: using the body as an antenna for realtime whole-body interaction, In Proc CHI 2012, pp. 1901--1910

Digital Library

[8]

Gupta, S., Morris, D., Patel, S., and Tan, D. Soundwave: using the doppler effect to sense gestures. In Proc CHI 2012, pp. 1911--1914.

Digital Library

[9]

Harrison, C. and Hudson, S. Scratch input: creating large, inexpensive, unpowered and mobile finger input surfaces. In Proc UIST 2008, pp. 205--208

Digital Library

[10]

Harrison, C., Tan, D., and Morris, D. Skinput: appropriating the body as an input surface, In Proc CHI 2010, pp. 453--462

Digital Library

[11]

Hudson, S.E. Abracadabra : Wireless, High-Precision, and Unpowered Finger Input for Very Small Mobile Devices, In Proc UIST 2009, pp. 121--124.

Digital Library

[12]

Kellogg, B., Talla, V., and Gollakota, S. Bringing Gesture Recognition To All Devices. In Proc NSDI 2014,

Digital Library

[13]

Kim, D., Hilliges, O., Izadi, S., and Butler, A. Digits: freehand 3D interactions anywhere using a wristworn gloveless sensor, In Proc UIST 2012, pp. 167--

Digital Library

[14]

Marvasti, F. Nonuniform sampling: theory and practice. Springer, 2001.

[15]

Minka, T. and Choudhury, R.R. Spot, Precise Localization using PHY Layer Information, In Proc HotNets-X 2012

[16]

Pu, Q., Gupta, S., Gollakota, S., and Patel, S. WholeHome Gesture Recognition Using Wireless Signals, In Proc MobiCom 2013, pp. 27--38.

Digital Library

[17]

Ring, M.F., Ashbrook, D., Baudisch, P., and White, S. Nenya : Subtle and Eyes-Free Mobile Input with a magnetically-tracked finger ring, In Proc CHI 2011, pp. 2043--2046.

Digital Library

[18]

Shotton, J., Fitzgibbon, A., Cook, M., et al. Real-time human pose recognition in parts from single depth images. In Proc CVPR 2011, pp. 297--1304.

Digital Library

[19]

Starner, T., Auxier, J., Ashbrook, D., and Gandy, M. The gesture pendant: a self-illuminating, wearable, infrared computer vision system for home automation control and medical monitoring. In Proc ISWC 2000, pp. 87--94.

Digital Library

[20]

Wigdor, D., Forlines, C., Baudisch, P., Barnwell, J., and Shen, C. LucidTouch : A See-Through Mobile Device, In Proc UIST 2007, pp. 269--278.

Digital Library

[21]

Xiong, J. and Jamieson, K. Towards Fine-Grained Radio-Based Indoor Location Categories and Subject Descriptors, In Proc HotMobile 2012.

Digital Library

Cited By

Matulic FKashima TBeker DSuzuo DFujiwara HVogel D(2024)Above-Screen Fingertip Tracking and Hand Representation for Precise Touch Input with a Phone in Virtual RealityProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670961(1-15)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670961
Liu LCao ZLi T(2023)FaceTouch: Practical Face Touch Detection with a Multimodal Wearable System for Epidemiological SurveillanceProceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation10.1145/3576842.3582368(13-26)Online publication date: 9-May-2023
https://dl.acm.org/doi/10.1145/3576842.3582368
Matulic FKashima TBeker DSuzuo DFujiwara HVogel D(2023)Above-Screen Fingertip Tracking with a Phone in Virtual RealityExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585728(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585728
Show More Cited By

Index Terms

SideSwipe: detecting in-air gestures around mobile devices using actual GSM signal
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
    2. Interaction paradigms
      1. Graphical user interfaces

Recommendations

AtaTouch: Robust Finger Pinch Detection for a VR Controller Using RF Return Loss
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Handheld controllers are an essential part of VR systems. Modern sensing techniques enable them to track users’ finger movements to support natural interaction using hands. The sensing techniques, however, often fail to precisely determine whether two ...
Dual-Band Circularly-Polarized EBG-Based Antenna for Wi-MAX/WLAN/ISM Band Applications
Abstract
A co-planar waveguide (CPW)-fed dual-band circularly-polarized (CP) disc-shaped monopole antenna with an electromagnetic band gap (EBG) array is presented for Wi-MAX, WLAN, and ISM band applications. The proposed monopole antenna is comprised of ...
Design and experiment of a conical double-ridged horn antenna for 6–18 GHz

A broadband conical double-ridged horn (DRH) antenna with symmetrical radiation patterns, low side lobe level, and low cross polarization is presented. Experimental investigations and detailed simulations are conducted to understand its behavior and to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technology

October 2014

722 pages

ISBN:9781450330695

DOI:10.1145/2642918

General Chair:
Hrvoje Benko
Microsoft Research, USA
,
Program Chairs:
Mira Dontcheva
Adobe, USA
,
Daniel Wigdor
University of Toronto, Canada

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 October 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '14

Sponsor:

UIST '14: The 27th Annual ACM Symposium on User Interface Software and Technology

October 5 - 8, 2014

Hawaii, Honolulu, USA

Acceptance Rates

UIST '14 Paper Acceptance Rate 74 of 333 submissions, 22%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

53
Total Citations
View Citations
911
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Matulic FKashima TBeker DSuzuo DFujiwara HVogel D(2024)Above-Screen Fingertip Tracking and Hand Representation for Precise Touch Input with a Phone in Virtual RealityProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670961(1-15)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670961
Liu LCao ZLi T(2023)FaceTouch: Practical Face Touch Detection with a Multimodal Wearable System for Epidemiological SurveillanceProceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation10.1145/3576842.3582368(13-26)Online publication date: 9-May-2023
https://dl.acm.org/doi/10.1145/3576842.3582368
Matulic FKashima TBeker DSuzuo DFujiwara HVogel D(2023)Above-Screen Fingertip Tracking with a Phone in Virtual RealityExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585728(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585728
Grubert J(2023)Mixed Reality Interaction TechniquesSpringer Handbook of Augmented Reality10.1007/978-3-030-67822-7_5(109-129)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-3-030-67822-7_5
Costagliola GDe Rosa MFuccella V(2023)Gesture‐Based ComputingHandbook of Human‐Machine Systems10.1002/9781119863663.ch32(397-408)Online publication date: 7-Jul-2023
https://doi.org/10.1002/9781119863663.ch32
Al-rawashdeh MKeikhosrokiani PBelaton BAlawida MZwiri A(2022)IoT Adoption and Application for Smart Healthcare: A Systematic ReviewSensors10.3390/s2214537722:14(5377)Online publication date: 19-Jul-2022
https://doi.org/10.3390/s22145377
Wei HLi ZGalvan ASu ZZhang XPahlavan KSolovey E(2022)IndexPenProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346016:2(1-39)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534601
Čopič Pucihar KAttygalle NKljun MSandor CLeiva L(2022)Solids on SoliProceedings of the ACM on Human-Computer Interaction10.1145/35322126:EICS(1-19)Online publication date: 17-Jun-2022
https://dl.acm.org/doi/10.1145/3532212
Sun ZWang SLiu CMa X(2022)Metaphoraction: Support Gesture-based Interaction Design with Metaphorical MeaningsACM Transactions on Computer-Human Interaction10.1145/351189229:5(1-33)Online publication date: 20-Oct-2022
https://dl.acm.org/doi/10.1145/3511892
Song WSinghose WFrakes D(2022)Soli Radar Image-Based Target Localization2022 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP46576.2022.9897226(746-750)Online publication date: 16-Oct-2022
https://doi.org/10.1109/ICIP46576.2022.9897226
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten