Abstract
In the current critical situation of novel coronavirus, the use of contactless gesture recognition method can reduce human contact and decrease the probability of virus transmission. In this context, ultrasound-based sensing has been widely concerned for its slow propagation speed, low sampling rate, and easy access to devices. However, limited by the complexity of gestural movements and insufficient training data, the accuracy and robustness of gesture recognition are low. To solve this problem, we propose UltrasonicG, a system for highly robust gesture recognition on ultrasonic devices. The system first converts a single audio signal into a Doppler shift and subsequently extracts the feature values using the Residual Neural Network (ResNet34) and uses Bi-directional Long Short-Term Memory (Bi-LSTM) for gesture recognition. The method effectively improves the accuracy of gesture recognition by combining the information of feature dimension with time dimension. To overcome the challenge of insufficient dataset, we use data extension to expand the dataset. We have conducted extensive experiments and evaluations on UltrasonicG in a variety of real scenarios. The experimental results show that UltrasonicG can recognize 15 kinds of gestures with a recognition distance of 0.5 m. And it has a high accuracy and robustness with a comprehensive recognition rate of 98.8% under different environments and influencing factors.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Fan, C.: Prediction of epidemic spread of the 2019 novel coronavirus driven by spring festival transportation in China: a population-based study. Int. J. Environ. Res. Public Health 17(5), 1679 (2020)
Gao, Y.: EchoWhisper: exploring an Acoustic-based Silent Speech Interface for Smartphone Users. Proc. ACM Interact. Mobile Wear. Ubiq. Technol. 4(3), 1–27 (2020)
Wang, W., Liu, A.X.: Device-free gesture tracking using acoustic signals. In: Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, pp. 82–94 (2016)
Yun, S., Chen, Y.C.: Strata: fine-grained acoustic-based device-free tracking. In: Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services, pp. 15–28 (2017)
Cihan Camgoz, N., Hadfield, S.: SubuNets: end-to-end hand shape and continuous sign language recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3056–3065 (2017)
Namdeo, A.: Performance measures for a three-unit compact circuit. Int. J. Adv. Trends Comput. Sci. Eng. 4, 15107–15115 (2020)
Chen, Z.: WiFi CSI based passive human activity recognition using attention based BLSTM. IEEE Trans. Mob. Comput. 18(11), 2714–2724 (2018)
Tian, Z.: WiCatch: A Wi-Fi based hand gesture recognition system. IEEE Access 6, 16911–16923 (2018)
Zheng, Y.-Zhang, Y.: Zero-effort cross-domain gesture recognition with Wi-Fi. In: Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services, pp. 313–325 (2019)
Nandakumar, R., Iyer, V.: FingeriO: using active sonar for fine-grained finger tracking. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 1515–1525 (2016)
Ling, K.: Ultragesture: fine-grained gesture sensing and recognition. IEEE Trans. Mobile Comput. (2020)
Zou, Y., Yang, Q.: EchoWrite: an acoustic-based finger input system without training. In: 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pp. 778–787 (2019)
Mao, W., He, J.: Cat: high-precision acoustic motion tracking. In: Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, pp. 69–81 (2016)
Wang, Y.: Push the limit of acoustic gesture recognition. In: IEEE INFOCOM 2020 - IEEE Conference on Computer Communications (2020)
He, K.-Zhang, X.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Cai, C.: Acoustic software defined platform: a versatile sensing and general benchmarking platform. IEEE Trans. Mobile Comput. (Early access), 1–15 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hao, Z., Wang, Y., Zhang, D., Dang, X. (2022). UltrasonicG: Highly Robust Gesture Recognition on Ultrasonic Devices. In: Wang, L., Segal, M., Chen, J., Qiu, T. (eds) Wireless Algorithms, Systems, and Applications. WASA 2022. Lecture Notes in Computer Science, vol 13472. Springer, Cham. https://doi.org/10.1007/978-3-031-19214-2_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-19214-2_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19213-5
Online ISBN: 978-3-031-19214-2
eBook Packages: Computer ScienceComputer Science (R0)