Ashkan Arabi
ABSTRACT
As years pass, smartphones are becoming a larger part of daily lives, causing users to interact with them more than ever. There are moments, however, when it becomes difficult for the user to operate their device directly. Currently, a user can either touch their devices for direct interaction, or use voice commands for simpler tasks. Although these two methods are very capable means of interacting with the devices, they have their limitations. Touching a physical device is not always practical, while voice commands become ineffective in loud environments. A good example would be if the user is washing dishes in a noisy environment, where neither physical control nor voice commands are convenient. Existing systems of smartphone CSI gesture recognition rely on manual feature extraction which could be hard to implement as gestures grow in number and complexity. We study the feasibility of using lightweight image classification models with minimal preprocessing by implementing and testing the performance of such an architecture. We collect data for five gestures from three setups and two phones, on which our system is able to obtain 90.0% accuracy. Additionally, we investigate the impact of different people, distances, and phones on the system's performance.
- Abdelnasser, H., Youssef, M., and Harras, K. A. Wigest: A ubiquitous wifi-based gesture recognition system. In 2015 IEEE Conference on Computer Communications (INFOCOM) (2015), pp. 1472--1480.Google Scholar
Cross Ref
- Ahmed, H. F. T., Ahmad, H., and Aravind, C. Device free human gesture recognition using wi-fi csi: A survey. Engineering Applications of Artificial Intelligence 87 (2020), 103281.Google ScholarDigital Library
- Al-qaness, M. A. A., and Li, F. Wiger: Wifi-based gesture recognition system. ISPRS International Journal of Geo-Information 5, 6 (2016).Google Scholar
- Al-Shamayleh, A. S., Ahmad, R., Abushariah, M. A., Alam, K. A., and Jomhari, N. A systematic literature review on vision based gesture recognition techniques. Multimedia Tools and Applications 77 (2018), 28121--28184.Google ScholarDigital Library
- Alsheakhali, M., Skaik, A., Aldahdouh, M., and Alhelou, M. Hand gesture recognition system. Information & Communication Systems 132 (2011).Google Scholar
- Bu, Q., Yang, G., Ming, X., Zhang, T., Feng, J., and Zhang, J. Deep transfer learning for gesture recognition with wifi signals. Personal and Ubiquitous Computing (2020), 1--12.Google Scholar
- Chin-Shyurng, F., Lee, S.-E., and Wu, M.-L. Real-time musical conducting gesture recognition based on a dynamic time warping classifier using a single-depth camera. Applied Sciences 9, 3 (2019).Google ScholarCross Ref
- Gringoli, F., Schulz, M., Link, J., and Hollick, M. Free your csi: A channel state information extraction platform for modern wi-fi chipsets. In Proceedings of the 13th International Workshop on Wireless Network Testbeds, Experimental Evaluation & Characterization (New York, NY, USA, 2019), WiNTECH '19, Association for Computing Machinery, p. 21--28.Google ScholarDigital Library
- Kim, K., Kim, J., Choi, J., Kim, J., and Lee, S. Depth camera-based 3d hand gesture controls with immersive tactile feedback for natural mid-air gesture interactions. Sensors 15, 1 (2015), 1022--1046.Google Scholar
- Kim, Y., and Toomajian, B. Application of doppler radar for the recognition of hand gestures using optimized deep convolutional neural networks. In 2017 11th European Conference on Antennas and Propagation (EUCAP) (2017).Google ScholarCross Ref
- Kresge, K., Martino, S., Zhao, T., and Wang, Y. Wifi-based contactless gesture recognition using lightweight cnn. In 2021 IEEE 18th International Conference on Mobile Ad Hoc and Smart Systems (MASS) (2021), IEEE, pp. 645--650.Google ScholarCross Ref
- Li, T., Shi, C., Li, P., and Chen, P. A novel gesture recognition system based on csi extracted from a smartphone with nexmon firmware. Sensors 21, 1 (2021).Google Scholar
- Lien, J., Gillian, N., Karagozler, M. E., Amihood, P., Schwesig, C., Olson, E., Raja, H., and Poupyrev, I. Soli: Ubiquitous gesture sensing with millimeter wave radar. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1--19.Google ScholarDigital Library
- McIntosh, J., Marzo, A., Fraser, M., and Phillips, C. Echoflex: Hand gesture recognition using ultrasound imaging. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2017), CHI '17, Association for Computing Machinery, p. 1923--1934.Google Scholar
- Shukor, A. Z., Miskon, M. F., Jamaluddin, M. H., bin Ali@Ibrahim, F., Asyraf, M. F., and bin Bahar, M. B. A new data glove approach for malaysian sign language detection. Procedia Computer Science 76 (2015), 60--67. 2015 IEEE International Symposium on Robotics and Intelligent Sensors (IEEE IRIS2015).Google ScholarCross Ref
- Simonyan, K., and Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- Zhang, H., Zhang, D., Guan, J., Wang, D., Tang, M., Ma, Y., and Xia, H. A flexible wearable strain sensor for human-motion detection and a human-machine interface. Journal of Materials Chemistry C 10, 41 (2022), 15554--15564.Google ScholarCross Ref
Index Terms
- Phone-based CSI Hand Gesture Recognition with Lightweight Image-Classification Model
Recommendations
Smart Hand Device Gesture Recognition with Dynamic Time-Warping Method
BDIOT '17: Proceedings of the International Conference on Big Data and Internet of ThingIn this paper, we present a smart wearable hand-gesture recognition system based on the movement of the hand and fingers. The proposed smart wearable system is built using the fewest sensors necessary for gesture recognition. Thus, motion sensors are ...
Finger identification and hand gesture recognition techniques for natural user interface
APCHI '13: Proceedings of the 11th Asia Pacific Conference on Computer Human InteractionThe natural user interface using hand gesture have been popular field in Human-Computer-Interaction(HCI). Many research papers have been proposed in this field. They proposed vision-based, glove-based and depth-based approach for hand gesture ...
Enabling fast and effortless customisation in accelerometer based gesture interaction
MUM '04: Proceedings of the 3rd international conference on Mobile and ubiquitous multimediaAccelerometer based gesture control is proposed as a complementary interaction modality for handheld devices. Predetermined gesture commands or freely trainable by the user can be used for controlling functions also in other devices. To support ...
Comments