Li Liu
Zhichao Cao
ABSTRACT
In this paper, we propose FaceTouch, a low-power and versatile method that enables accurate face touch detection with a multimodal wearable system. FaceTouch consists of two sensing components, an inertial sensor on the wrist and a novel vibration sensor on the finger. We leverage the wrist inertial sensor to detect the face-touch gesture that the hand moves towards the face area. To achieve this goal in a computation-efficient manner, we develop a cascading classification model including three classifiers to filter out irrelevant gestures to significantly extend the battery life while keeping a high recall. Once a face-touch gesture is triggered, we activate the vibration sensor to detect touch events. We implement FaceTouch using commercial off-the-shelf hardware components and evaluate its performance with various user activities and false-positive behaviors. FaceTouch achieves 93.5% F-1 score of face touch detection. The entire system only consumes 60.89 μ W power on average in normal daily usage and 209.15 μ W in extremely heavy usage, which is several magnitudes lower than the state-of-the-art systems, and FaceTouch can continuously detect face-touch events for 79 – 273 days using a small 400 mWh battery depending on usage.
- 2010. TDK PiezoListen™ Piezo Transducers. https://product.tdk.com/en/search/sw_piezo/speaker/piezolisten/info?part_no=PHUA2010-049B-00-000&utm_source=piezolisten_commercial_phu_en.pdf&utm_medium=catalog. Accessed: 2010-09-30.Google Scholar
- 2021. Face Touch Aware : the Apple Watch App. https://facetouch.app. Accessed: 2021-11-03.Google Scholar
- 2021. Immutouch: Stay healthy and hygienic with Immutouch. https://immutouch.com/. Accessed: 2021-11-03.Google Scholar
- 2021. Monsoon High Voltage Power Monitor. https://www.msoon.com/high-voltage-power-monitor. Accessed: 2021-11-03.Google Scholar
- 2021. Nudge. https://www.nudgeband.co.uk/. Accessed: 2021-11-03.Google Scholar
- 2022. Adafruit Bluefruit LE UART Friend - Bluetooth Low Energy (BLE). https://www.adafruit.com/product/2479. (2022). Accessed: 2022-01-30.Google Scholar
- Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C Miller. 2014. 3d tracking via body radio reflections. In 11th { USENIX} Symposium on Networked Systems Design and Implementation ({ NSDI} 14). 317–329.Google Scholar
- Marco Altini and Hannu Kinnunen. 2021. The promise of sleep: A multi-sensor approach for accurate sleep stage detection using the oura ring. Sensors 21, 13 (2021), 4302.Google ScholarCross Ref
- Xiang Anthony’Chen. 2020. FaceOff: Detecting Face Touching with a Wrist-Worn Accelerometer. arXiv e-prints (2020), arXiv–2008.Google Scholar
- Dongyao Chen, Mingke Wang, Chenxi He, Qing Luo, Yasha Iravantchi, Alanson Sample, Kang G Shin, and Xinbing Wang. 2021. MagX: wearable, untethered hands tracking with passive magnets. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. 269–282.Google ScholarDigital Library
- Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).Google Scholar
- Massimiliano De Zambotti, Nicola Cellini, Aimee Goldstone, Ian M Colrain, and Fiona C Baker. 2019. Wearable sleep technology in clinical and research settings. Medicine and science in sports and exercise 51, 7 (2019), 1538.Google Scholar
- James Diebel. 2006. Representing attitude: Euler angles, unit quaternions, and rotation vectors. Matrix 58, 15-16 (2006), 1–35.Google Scholar
- Stuart Dimond and Rashida Harries. 1984. Face touching in monkeys, apes and man: Evolutionary origins and cerebral asymmetry. Neuropsychologia 22, 2 (1984), 227–233.Google ScholarCross Ref
- Nicole D’Aurizio, Tommaso Lisini Baldi, Sara Marullo, Gianluca Paolocci, and Domenico Prattichizzo. 2021. Reducing Face-Touches to Limit COVID-19 Outbreak: an Overview of Solutions. In 2021 29th Mediterranean Conference on Control and Automation (MED). IEEE, 645–650.Google ScholarCross Ref
- Sebastiano Foti, Carlo G Lai, Glenn J Rix, and Claudio Strobbia. 2014. Surface wave methods for near-surface site characterization. CRC press.Google Scholar
- Jerome H Friedman. 2001. Greedy function approximation: a gradient boosting machine. Annals of statistics (2001), 1189–1232.Google Scholar
- Jun Gong, Aakar Gupta, and Hrvoje Benko. 2020. Acustico: Surface tap detection and localization using wrist-based acoustic tdoa sensing. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 406–419.Google ScholarDigital Library
- Aziz Guellich, Emilie Tella, Molka Ariane, Camille Grodner, Hoai-Nam Nguyen-Chi, and Emmanuel Mahé. 2021. The face mask-touching behavior during the COVID-19 pandemic: Observational study of public transportation users in the greater Paris region: The French-mask-touch study. Journal of transport & health 21 (2021), 101078.Google ScholarCross Ref
- Sidhant Gupta, Daniel Morris, Shwetak Patel, and Desney Tan. 2012. Soundwave: using the doppler effect to sense gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1911–1914.Google ScholarDigital Library
- Roya Haratian. 2019. Assistive Wearable Technology for Mental Wellbeing: Sensors and Signal Processing Approaches. In 2019 5th International Conference on Frontiers of Signal Processing (ICFSP). IEEE, 7–11.Google Scholar
- Takeshi Hatta and Stuart J Dimond. 1984. Differences in face touching by Japanese and British people. Neuropsychologia 22, 4 (1984), 531–534.Google ScholarCross Ref
- FSA William Jones. 2019. Finger-ring Lore: Historical, Legendary, Anecdotal. Good Press.Google Scholar
- Vimal Kakaraparthi, Qijia Shao, Charles J Carver, Tien Pham, Nam Bui, Phuc Nguyen, Xia Zhou, and Tam Vu. 2021. FaceSense: Sensing Face Touch with an Ear-worn System. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1–27.Google ScholarDigital Library
- Bryce Kellogg, Vamsi Talla, and Shyamnath Gollakota. 2014. Bringing gesture recognition to all devices. In 11th { USENIX} Symposium on Networked Systems Design and Implementation ({ NSDI} 14). 303–316.Google Scholar
- Wolf Kienzle, Eric Whitmire, Chris Rittaler, and Hrvoje Benko. 2021. ElectroRing: Subtle Pinch and Touch Detection with a Ring. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12.Google ScholarDigital Library
- David G Kleinbaum, K Dietz, M Gail, Mitchel Klein, and Mitchell Klein. 2002. Logistic regression. Springer.Google Scholar
- Yen Lee Angela Kwok, Jan Gralton, and Mary-Louise McLaws. 2015. Face touching: a frequent habit that has implications for hand hygiene. American journal of infection control 43, 2 (2015), 112–114.Google Scholar
- Xingyu Li, Hongjun Dai, Lizhen Cui, and Ya Wang. 2017. SonicOperator: Ultrasonic gesture recognition with deep neural network on mobiles. In 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI). IEEE, 1–7.Google Scholar
- Mingbin Liu, Jianming Ou, Lijie Zhang, Xiaona Shen, Rongtao Hong, Huilai Ma, Bao-Ping Zhu, and Robert E Fontaine. 2016. Protective effect of hand-washing and good hygienic habits against seasonal influenza: a case-control study. Medicine 95, 11 (2016).Google ScholarCross Ref
- Anna Maijala, Hannu Kinnunen, Heli Koskimäki, Timo Jämsä, and Maarit Kangas. 2019. Nocturnal finger skin temperature in menstrual cycle tracking: ambulatory pilot study using a wearable Oura ring. BMC Women’s Health 19, 1 (2019), 1–10.Google Scholar
- Neil J Mansfield. 2004. Human response to vibration. CRC press.Google Scholar
- Wenguang Mao, Jian He, and Lili Qiu. 2016. Cat: high-precision acoustic motion tracking. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. 69–81.Google ScholarDigital Library
- S Marnilo, T Lisini Baldi, G Paolocci, N D’Aurizio, and D Prattichizzo. 2021. No face-touch: Exploiting wearable devices and machine learning for gesture detection. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 4187–4193.Google Scholar
- William C McGrew and Linda F Marchant. 1997. On the other hand: current issues in and meta-analysis of the behavioral laterality of hand function in nonhuman primates. American Journal of Physical Anthropology: The Official Publication of the American Association of Physical Anthropologists 104, S25 (1997), 201–232.Google Scholar
- Mary-Louise McLaws, Abrar A Chughtai, Sharon Salmon, and C Raina MacIntyre. 2016. A highly precautionary doffing sequence for health care workers after caring for wet Ebola patients to further reduce occupational acquisition of Ebola. American journal of infection control 44, 7 (2016), 740–744.Google Scholar
- Scott Menard. 2002. Applied logistic regression analysis. Vol. 106. Sage.Google Scholar
- Rahul Mishra, Hari Prabhat Gupta, and Tanima Dutta. 2020. A survey on deep neural network compression: Challenges, overview, and solutions. arXiv preprint arXiv:2010.03954 (2020).Google Scholar
- Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. 2018. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence 41, 8 (2018), 1979–1993.Google Scholar
- Kentaro Morita, Kaho Hashimoto, Masayuki Ogata, Hitomi Tsutsumi, Shin-ichi Tanabe, and Satoshi Hori. 2019. Measurement of face-touching frequency in a simulated train. In E3S Web of Conferences, Vol. 111. EDP Sciences, 02027.Google Scholar
- Stephanie Margarete Mueller, Sven Martin, and Martin Grunwald. 2019. Self-touch: contact durations and point of touch of spontaneous facial self-touches differ depending on cognitive and emotional load. PloS one 14, 3 (2019), e0213677.Google ScholarCross Ref
- Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. Fingerio: Using active sonar for fine-grained finger tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 1515–1525.Google ScholarDigital Library
- Rosemery O Nelson, Ronald A Boykin, and Steven C Hayes. 1982. Long-term effects of self-monitoring on reactivity and on accuracy. Behaviour research and therapy 20, 4 (1982), 357–363.Google Scholar
- Abhinav Parate, Meng-Chieh Chiu, Chaniel Chadowitz, Deepak Ganesan, and Evangelos Kalogerakis. 2014. Risq: Recognizing smoking gestures with inertial sensors on a wristband. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. 149–161.Google ScholarDigital Library
- Dung Phan, Lee Yee Siong, Pubudu N Pathirana, and Aruna Seneviratne. 2015. Smartwatch: Performance evaluation for long-term heart rate monitoring. In 2015 International symposium on bioelectronics and bioinformatics (ISBB). IEEE, 144–147.Google ScholarCross Ref
- Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. 2013. Whole-home gesture recognition using wireless signals. In Proceedings of the 19th annual international conference on Mobile computing & networking. 27–38.Google ScholarDigital Library
- Yang Qifan, Tang Hao, Zhao Xuebing, Li Yin, and Zhang Sanfeng. 2014. Dolphin: Ultrasonic-based gesture recognition on smartphone platform. In 2014 IEEE 17th International Conference on Computational Science and Engineering. IEEE, 1461–1468.Google ScholarDigital Library
- Dhanush Rachaveti, Niranjan Chakrabhavi, Vaisakh Shankar, and SKM Varadhan. 2018. Thumbs up: movements made by the thumb are smoother and larger than fingers in finger-thumb opposition tasks. PeerJ 6 (2018), e5763.Google ScholarCross Ref
- Lesley J Rogers, Giorgio Vallortigara, and Richard J Andrew. 2013. Divided brains: the biology and behaviour of brain asymmetries. Cambridge University Press.Google Scholar
- Camilo Rojas, Niels Poulsen, Mileva Van Tuyl, Daniel Vargas, Zipporah Cohen, Joe Paradiso, Pattie Maes, Kevin Esvelt, and Fadel Adib. 2021. A scalable solution for signaling face touches to reduce the spread of surface-based pathogens. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1 (2021).Google ScholarDigital Library
- Nirupam Roy, Mahanth Gowda, and Romit Roy Choudhury. 2015. Ripple: Communicating through physical vibration. In 12th { USENIX} Symposium on Networked Systems Design and Implementation ({ NSDI} 15). 265–278.Google Scholar
- Wenjie Ruan, Quan Z Sheng, Lei Yang, Tao Gu, Peipei Xu, and Longfei Shangguan. 2016. AudioGest: enabling fine-grained hand gesture detection by decoding echo signal. In Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing. 474–485.Google ScholarDigital Library
- Raymond A Serway and Chris Vuille. 2014. College physics. Cengage Learning.Google Scholar
- Tanu Singhal. 2020. A review of coronavirus disease-2019 (COVID-19). The Indian Journal of Pediatrics (2020), 1–6.Google Scholar
- Zhipeng Song, Zhichao Cao, Zhenjiang Li, and Jiliang Wang. 2020. Magic Wand: Towards Plug-and-Play Gesture Recognition on Smartwatch. In 2020 16th International Conference on Mobility, Sensing and Networking (MSN). IEEE, 275–282.Google Scholar
- Zhipeng Song, Zhichao Cao, Zhenjiang Li, Jiliang Wang, and Yunhao Liu. 2021. Inertial motion tracking on mobile and wearable devices: Recent advancements and challenges. Tsinghua Science and Technology 26, 5 (2021), 692–705.Google ScholarCross Ref
- Wei Wang, Lin Yang, and Qian Zhang. 2016. Touch-and-guard: secure pairing through hand resonance. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 670–681.Google ScholarDigital Library
- Sarah L Warnes, Zoë R Little, and C William Keevil. 2015. Human coronavirus 229E remains infectious on common touch surface materials. MBio 6, 6 (2015), e01697–15.Google ScholarCross Ref
- Zachary Witkower and Jessica L Tracy. 2019. Bodily communication of emotion: Evidence for extrafacial behavioral expressions and available coding systems. Emotion Review 11, 2 (2019), 184–193.Google ScholarCross Ref
- Raymond E Wright. 1995. Logistic regression. (1995).Google Scholar
- Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. 2017. Strata: Fine-grained acoustic-based device-free tracking. In Proceedings of the 15th annual international conference on mobile systems, applications, and services. 15–28.Google ScholarDigital Library
- Cheng Zhang, Qiuyue Xue, Anandghan Waghmare, Ruichen Meng, Sumeet Jain, Yizeng Han, Xinyu Li, Kenneth Cunefare, Thomas Ploetz, Thad Starner, 2018. FingerPing: Recognizing fine-grained hand poses using active acoustic on-body sensing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–10.Google ScholarDigital Library
- Nan Zhang, Wei Jia, Peihua Wang, Marco-Felipe King, Pak-To Chan, and Yuguo Li. 2020. Most self-touches are with the nondominant hand. Scientific reports 10, 1 (2020), 1–13.Google Scholar
- Nan Zhang, Peihua Wang, Te Miao, Pak-To Chan, Wei Jia, Pengcheng Zhao, Boni Su, Xuguang Chen, and Yuguo Li. 2021. Real human surface touch behavior based quantitative analysis on infection spread via fomite route in an office. Building and Environment 191 (2021), 107578.Google ScholarCross Ref
- Yang Zhang, Wolf Kienzle, Yanjun Ma, Shiu S Ng, Hrvoje Benko, and Chris Harrison. 2019. ActiTouch: Robust touch detection for on-skin AR/VR interfaces. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 1151–1159.Google ScholarDigital Library
- Yang Zhang, Junhan Zhou, Gierad Laput, and Chris Harrison. 2016. Skintrack: Using the body as an electrical waveguide for continuous finger tracking on the skin. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 1491–1503.Google ScholarDigital Library
- Chen Zhao, Ke-Yu Chen, Md Tanvir Islam Aumi, Shwetak Patel, and Matthew S Reynolds. 2014. SideSwipe: detecting in-air gestures around mobile devices using actual GSM signal. In Proceedings of the 27th annual ACM symposium on User interface software and technology. 527–534.Google ScholarDigital Library
Recommendations
FaceTouch: Enabling Touch Interaction in Display Fixed UIs for Mobile Virtual Reality
UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and TechnologyWe present FaceTouch, a novel interaction concept for mobile Virtual Reality (VR) head-mounted displays (HMDs) that leverages the backside as a touch-sensitive surface. With FaceTouch, the user can point at and select virtual content inside their field-...
FaceTouch: Touch Interaction for Mobile Virtual Reality
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsWe present FaceTouch, a mobile Virtual Reality (VR) headmounted display (HMD) that leverages the backside as a touch-sensitive surface. FaceTouch allows the user to point at and select virtual content inside their field-of-view by touching the ...
Using 3D hand gestures and touch input for wearable AR interaction
CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing SystemsWhile wearable devices have been developed that incorporate computing, sensing and display technology into a head-worn package, they often have limited input methods that might not be appropriate for natural 3D interaction which is necessary for ...
Comments