Abstract
Contents typed via keyboards prove to be vulnerable to attacks based on acoustic emanations analysis. However, previous works achieve the attacks under controlled environment, e.g., neglecting the noises or requiring the keyboard to be located in fixed locations. In this study, we present a localization-free online keystroke tracking system (LOL), which enables people to use prior knowledge obtained from the keyboard in one location to recognize real-time keystrokes of the same type of keyboard in any other places, despite various background noises. Combined with support vector machine, we design an detection model to separate keystroke signals from noises. By analyzing the properties of acoustics transmission, we propose an angle-based sampling method with a single microphone to decrease the dependence on certain locations, and it also increases the diversity of signals in the meantime. Our real-world experiments demonstrate a 99.47% keystroke detection rate, a 97.27% recognition accuracy under ideal condition, and an 84.55% content recovery accuracy despite changing locations of the keyboard. Most commercial off-the-shelf sound recording devices, e.g., smartphones, can be used in our system to record acoustic emanations from keystrokes. LOL could attract more community to study security of keyboard devices and promote users to enhance privacy protection awareness.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ali K, Liu AX, Wang W, Shahzad M (2015) Keystroke recognition using wifi signals. In: ACM MobiCom
Ali K, Liu AX, Wang W, Shahzad M (2017) Recognizing keystrokes using WiFi devices. IEEE J Sel Areas Commun PP(99): 1–1
Asonov D, Agrawal R (2004) Keyboard acoustic emanations. In: IEEE symposium on security and privacy
Baynath P, Soyjaudah KMS, Khan HM (2017) Keystroke recognition using neural network. In: International symposium on computational and business intelligence, pp 86–90
Berger Y, Wool A, Yeredor A (2006) Dictionary attacks using keyboard acoustic emanations. In: ACM CCS
Chen B, Yenamandra V, Srinivasan K (2015) Tracking keystrokes using wireless signals. In: ACM MobiSys
Chen H, Li F, Wang Y (2017) Echotrack: acoustic device-free hand tracking on smart phones. In: IEEE INFOCOM
Li F, Wang X, Chen H, Sharif K, Wang Y (2017) Clickleak: keystroke leaks through multimodal sensors in cyber-physical social networks. IEEE Access 5:27311–27321
Li M, Meng Y, Liu J, Zhu H, Liang X, Liu Y, Ruan N (2016) When CSI meets public WiFi: inferring your mobile phone password via WiFi signals. In: ACM CCS
Liu J, Wang Y, Kar G, Chen Y, Yang J, Gruteser M (2015) Snooping keystrokes with mm-level audio ranging on a single phone. In: ACM MobiCom
Liu X, Zhou Z, Diao W, Li Z, Zhang K (2015) When good becomes evil: keystroke inference with smartwatch. In: ACM CCS
Maiti A, Armbruster O, Jadliwala M, He J (2016) Smartwatch-based keystroke inference attacks and context-aware protection mechanisms. In: ACM CCS
Mao W, He J, Qiu L (2016) CAT: high-precision acoustic motion tracking. In: ACM MobiCom
Marquardt P, Verma A, Carter H, Traynor P (2011) (sp) iphone: decoding vibrations from nearby keyboards using mobile phone accelerometers. In: ACM CCS
Miluzzo E, Varshavsky A, Balakrishnan S, Choudhury RR (2012) Tapprints: your finger taps have fingerprints. In: ACM MobiSys
Nirjon S, Gummeson J, Gelb D, Kim KH (2015) Typingring: a wearable ring platform for text input. In: ACM MobiSys
Raguram R, White AM, Goswami D, Monrose F, Frahm JM (2011) ispy: automatic reconstruction of typed input from compromising reflections. In: ACM CCS
Shukla D, Kumar R, Serwadda A, Phoha VV (2014) Beware, your hands reveal your secrets! In: ACM CCS
Wang H, Lai TTT, Roy Choudhury R (2015) Mole: motion leaks through smartwatch sensors. In: ACM MobiCom
Wang J, Ruby R, Wang L, Wu K (2016) Accurate combined keystrokes detection using acoustic signals. In: IEEE MSN
Wang J, Zhao K, Zhang X, Peng C (2014) Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization. In: ACM MobiSys
Wang W, Liu AX, Sun K (2016) Device-free gesture tracking using acoustic signals. In: ACM MobiCom
Xu Y, Heinly J, White AM, Monrose F, Frahm JM (2013) Seeing double: reconstructing obscured typed input from repeated compromising reflections. In: ACM CCS
Yin Y, Li Q, Xie L, Yi S, Novak E, Lu S (2016) Camk: a camera-based keyboard for small mobile devices. In: IEEE INFOCOM
Yue Q, Ling Z, Fu X, Liu B, Ren K, Zhao W (2014) Blind recognition of touched keys on mobile devices. In: ACM CCS
Zhu T, Ma Q, Zhang S, Liu Y (2014) Context-free attacks using keyboard acoustic emanations. In: ACM CCS
Zhuang L, Zhou F, Tygar JD (2009) Keyboard acoustic emanations revisited. ACM Transactions on Information and System Security (TISSEC)
Acknowledgements
The work is supported by the Fundamental Research Funds for the Central Universities, No. DUT17RC(3)094, the Fundamental Research Funds for the Central University with No. DUT17 LAB16 and the Program for Liaoning Excellent Talents in University, No. LR2017009.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Qin, Z., Du, J., Han, G. et al. LOL: localization-free online keystroke tracking using acoustic signals. Soft Comput 23, 11063–11075 (2019). https://doi.org/10.1007/s00500-018-3659-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-018-3659-y