Ke Sun
Ting Zhao
ABSTRACT
Enabling touch gesture sensing on all surfaces of the mobile device, not limited to the touchscreen area, leads to new user interaction experiences. In this paper, we propose VSkin, a system that supports fine-grained gesture-sensing on the back of mobile devices based on acoustic signals. VSkin utilizes both the structure-borne sounds, i.e., sounds propagating through the structure of the device, and the air-borne sounds, i.e., sounds propagating through the air, to sense finger tapping and movements. By measuring both the amplitude and the phase of each path of sound signals, VSkin detects tapping events with an accuracy of 99.65% and captures finger movements with an accuracy of 3.59mm.
- Md Tanvir Islam Aumi, Sidhant Gupta, Mayank Goel, Eric Larson, and Shwetak Patel. 2013. Doplink: Using the doppler effect for multi-device interaction. In Proceedings of ACM UbiComp. Google Scholar
Digital Library
- Ian H Chan. 2010. Swept Sine Chirps for Measuring Impulse Response. Stanford Research Systems.Google Scholar
- Ke-Yu Chen, Daniel Ashbrook, Mayank Goel, Sung-Hyuck Lee, and Shwetak Patel. 2014. AirLink: sharing files between multiple devices using in-air gestures. In Proceedings of ACM UbiComp. Google ScholarDigital Library
- Mayank Goel, JacobWobbrock, and Shwetak Patel. 2012. GripSense: using built-in sensors to detect hand posture and pressure on commodity mobile phones. In Proceedings of ACM UIST. Google ScholarDigital Library
- Emilio Granell and Luis A Leiva. 2017. "Tap: back-of-device tap input with built-in sensors. In Proceedings of of MobileHCI. Google ScholarDigital Library
- Sidhant Gupta, Daniel Morris, Shwetak Patel, and Desney Tan. 2012. Soundwave: using the doppler effect to sense gestures. In Proceedings of ACM CHI. Google ScholarDigital Library
- Chris Harrison, Julia Schwarz, and Scott E Hudson. 2011. TapSense: enhancing finger interaction on touch surfaces. In Proceedings of ACM UIST. Google ScholarDigital Library
- Jouni Hartikainen, Arno Solin, and Simo Särkkä. 2011. Optimal filtering with Kalman filters and smoothers. Aalto University School of Science.Google Scholar
- Seongkook Heo and Geehyuk Lee. 2011. Forcetap: extending the input vocabulary of mobile touch screens by adding tap gestures. In Proceedings of ACM MobileHCI. Google ScholarDigital Library
- Sungjae Hwang, Andrea Bianchi, and Kwang-yun Wohn. 2013. Vib- Press: estimating pressure input using vibration absorption on mobile devices. In Proceedings of ACM MobileHCI. Google ScholarDigital Library
- Huy Viet Le, Sven Mayer, Patrick Bader, and Niels Henze. 2017. A smartphone prototype for touch interaction on the whole device surface. In Proceedings of of MobileHCI. Google ScholarDigital Library
- Jian Liu, Yingying Chen, Marco Gruteser, and Yan Wang. 2017. Vib- Sense: Sensing Touches on Ubiquitous Surfaces through Vibration. In Proceedings of IEEE SECON.Google Scholar
- Jian Liu, Chen Wang, Yingying Chen, and Nitesh Saxena. 2017. Vib- Write: Towards Finger-input Authentication on Ubiquitous Surfaces via Physical Vibration. In Proceedings of ACM CCS. Google ScholarDigital Library
- Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. FingerIO: Using Active Sonar for Fine-Grained Finger Tracking. In Proceedings of ACM CHI. Google ScholarDigital Library
- Makoto Ono, Buntarou Shizuki, and Jiro Tanaka. 2015. Sensing touch force using active acoustic sensing. In Proceedings of ACM TEI. Google ScholarDigital Library
- Chunyi Peng, Guobin Shen, Yongguang Zhang, Yanlin Li, and Kun Tan. 2007. Beepbeep: a high accuracy acoustic ranging system using COTS mobile devices. In Proceedings of ACM SenSys. Google ScholarDigital Library
- Corey R Pittman and Joseph J LaViola Jr. 2017. Multiwave: Complex Hand Gesture Recognition Using the Doppler Effect. In Proceedings of ACM GI. Google ScholarDigital Library
- Branislav M Popovic. 1992. Generalized chirp-like polyphase sequences with optimum correlation properties. IEEE Transactions on Information Theory 38, 4 (1992), 1406--1409. Google ScholarDigital Library
- Swadhin Pradhan, Eugene Chai, Karthikeyan Sundaresan, Lili Qiu, Mohammad A Khojastepour, and Sampath Rangarajan. 2017. RIO: A Pervasive RFID-based Touch Gesture Interface. In Proceedings of ACM MobiCom. Google ScholarDigital Library
- A Rodrìguez Valiente, A Trinidad, JR García Berrocal, C Górriz, and R Ramírez Camacho. 2014. Extended high-frequency (9--20 kHz) audiometry reference thresholds in 645 healthy subjects. International journal of audiology 53, 8 (2014), 531--545.Google Scholar
- Wenjie Ruan, Quan Z Sheng, Lei Yang, Tao Gu, Peipei Xu, and Longfei Shangguan. 2016. AudioGest: enabling fine-grained hand gesture detection by decoding echo signal. In Proceedings of ACM Ubicomp. Google ScholarDigital Library
- Shaikh Shawon Arefin Shimon, Sarah Morrison-Smith, Noah John, Ghazal Fahimi, and Jaime Ruiz. 2015. Exploring user-defined back-ofdevice gestures for mobile devices. In Proceedings of ACM MobileHCI. Google ScholarDigital Library
- Ke Sun, Wei Wang, Alex X. Liu, and Haipeng Dai. 2018. Depth aware finger tapping on virutal displays. In Proceedings of ACM MobiSys. Google ScholarDigital Library
- Thomas L Szabo and Junru Wu. 2000. A model for longitudinal and shear wave propagation in viscoelastic media. The Journal of the Acoustical Society of America 107, 5 (2000), 2437--2446.Google ScholarCross Ref
- Yu-Chih Tung and Kang G Shin. 2016. Expansion of human-phone interface by sensing structure-borne sound propagation. In Proceedings of ACM MobiSys. Google ScholarDigital Library
- Michael Vorländer. 2007. Auralization: fundamentals of acoustics, modelling, simulation, algorithms and acoustic virtual reality. Springer Science & Business Media. Google ScholarDigital Library
- Hao Wang, Daqing Zhang, Junyi Ma, Yasha Wang, Yuxiang Wang, Dan Wu, Tao Gu, and Bing Xie. 2016. Human respiration detection with commodity WiFi devices: do user location and body orientation matter?. In Proceedings of ACM Ubicomp. Google ScholarDigital Library
- WeiWang, Alex X. Liu, and Ke Sun. 2016. Device-free gesture tracking using acoustic signals. In Proceedings of ACM MobiCom. Google ScholarDigital Library
- Teng Wei and Xinyu Zhang. 2015. mTrack: High-Precision Passive Tracking Using Millimeter Wave Radios. In Proceedings of ACM Mobi- Com. Google ScholarDigital Library
- Daniel Wigdor, Clifton Forlines, Patrick Baudisch, John Barnwell, and Chia Shen. 2007. Lucid touch: a see-through mobile device. In Proceedings of ACM UIST. Google ScholarDigital Library
- Pui Chung Wong, Hongbo Fu, and Kening Zhu. 2016. Back-Mirror: back-of-device one-handed interaction on smartphones. In Proceedings of Symposium on Mobile Graphics and Interactive Applications, SIGGRAPH ASIA. Google ScholarDigital Library
- Xiang Xiao, Teng Han, and Jingtao Wang. 2013. LensGesture: augmenting mobile interactions with back-of-device finger gestures. In Proceedings of ACM ICMI. Google ScholarDigital Library
- Sangki Yun, Yi-Chao Chen, and Lili Qiu. 2015. Turning a Mobile Device into a Mouse in the Air. In Proceedings of ACM MobiSys. Google ScholarDigital Library
- Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. 2017. Strata: Fine-Grained Acoustic-based Device-Free Tracking. In Proceedings of ACM MobiSys. Google ScholarDigital Library
- Cheng Zhang, Aman Parnami, Caleb Southern, Edison Thomaz, Gabriel Reyes, Rosa Arriaga, and Gregory D Abowd. 2013. BackTap: robust four-point tapping on the back of an off-the-shelf smartphone. In Proceedings of ACM UIST. Google ScholarDigital Library
- Lide Zhang, Birjodh Tiwana, Zhiyun Qian, ZhaoguangWang, Robert P. Dick, Zhuoqing Morley Mao, and Lei Yang. 2010. Accurate online power estimation and automatic battery behavior based power model generation for smartphones. In Proceedings of IEEE CODES+ISSS. Google ScholarDigital Library
- Zengbin Zhang, David Chu, Xiaomeng Chen, and Thomas Moscibroda. 2012. Swordfight: Enabling a new class of phone-to-phone action games on commodity phones. In Proceedings of ACM MobiSys. Google ScholarDigital Library
Index Terms
- VSkin: Sensing Touch Gestures on Surfaces of Mobile Devices Using Acoustic Signals
Recommendations
Device-free gesture tracking using acoustic signals
MobiCom '16: Proceedings of the 22nd Annual International Conference on Mobile Computing and NetworkingDevice-free gesture tracking is an enabling HCI mechanism for small wearable devices because fingers are too big to control the GUI elements on such small screens, and it is also an important HCI mechanism for medium-to-large size mobile devices because ...
Gesturemote: interacting with remote displays through touch gestures
AVI '14: Proceedings of the 2014 International Working Conference on Advanced Visual InterfacesWe present Gesturemote, a technique for interacting with remote displays through touch gestures on a handheld touch surface. By combining a variety of different touch gestures and connecting them smoothly, Gesturemote supports a wide range of ...
Towards estimating web search result relevance from touch interactions on mobile devices
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing SystemsFine-grained search interactions such as mouse cursor movements and scrolling have been shown to be valuable for modeling user attention and preferences of Web search results, in the desktop setting. However, users increasingly search the Web on touch-...
Comments