Dan Ping
ABSTRACT
Today's smartphones are equipped with precise motion sensors like accelerometer and gyroscope, which can measure tiny motion and rotation of devices. While they make mobile applications more functional, they also bring risks of leaking users' privacy. Researchers have found that tap locations on screen can be roughly inferred from motion data of the device. They mostly utilized this side-channel for inferring short input like PIN numbers and passwords, with repeated attempts to boost accuracy. In this work, we study further for longer input inference, such as chat record and e-mail content, anything a user ever typed on a soft keyboard. Since people increasingly rely on smartphones for daily activities, their inputs directly or indirectly expose privacy about them. Thus, it is a serious threat if their input text is leaked.
To make our attack practical, we utilize the shared memory side-channel for detecting window events and tap events of a soft keyboard. The up or down state of the keyboard helps triggering our Trojan service for collecting accelerometer and gyroscope data. Machine learning algorithms are used to roughly predict the input text from the raw data and language models are used to further correct the wrong predictions. We performed experiments on two real-life scenarios, which were writing emails and posting Twitter messages, both through mobile clients. Based on the experiments, we show the feasibility of inferring long user inputs to readable sentences from motion sensor data. By applying text mining technology on the inferred text, more sensitive information about the device owners can be exposed.
- System permissions. http://developer.android.com/guide/topics/security/permissions.html.Google Scholar
- ios security. https://www.apple.com/br/privacy/docs/iOS_Security_Guide_Oct_2014.pdf, 2014.Google Scholar
- Motion sensors. http://developer.android.com/guide/topics/sensors/sensors_motion.html.Google Scholar
- Sensorevent. http://developer.android.com/reference/android/hardware/SensorEvent.html.Google Scholar
- Core motion framework reference. https://developer.apple.com/library/ios/documentation/CoreMotion/Reference/CoreMotion_Reference/index.html#//apple_ref/doc/uid/TP40009686.Google Scholar
- Liang Cai and Hao Chen. Touchlogger: Inferring keystrokes on touch screen from smartphone motion. In HotSec, 2011. Google ScholarDigital Library
- Zhi Xu, Kun Bai, and Sencun Zhu. Taplogger: Inferring user inputs on smartphone touchscreens using on-board motion sensors. In Proceedings of the fifth ACM conference on Security and Privacy in Wireless and Mobile Networks, pages 113--124. ACM, 2012. Google ScholarDigital Library
- Emmanuel Owusu, Jun Han, Sauvik Das, Adrian Perrig, and Joy Zhang. Accessory: password inference using accelerometers on smartphones. In Proceedings of the Twelfth Workshop on Mobile Computing Systems & Applications, page 9. ACM, 2012. Google ScholarDigital Library
- Emiliano Miluzzo, Alexander Varshavsky, Suhrid Balakrishnan, and Romit Roy Choudhury. Tapprints: your finger taps have fingerprints. In Proceedings of the 10th international conference on Mobile systems, applications, and services, pages 323--336. ACM, 2012. Google ScholarDigital Library
- Liang Cai and Hao Chen. On the practicality of motion based keystroke inference attack. Springer, 2012.Google ScholarDigital Library
- Tayfun Kucukyilmaz, B Barla Cambazoglu, Cevdet Aykanat, and Fazli Can. Chat mining: Predicting user and message attributes in computer-mediated communication. Information Processing & Management, 44(4):1448--1466, 2008. Google ScholarDigital Library
- Guanting Tang, Jian Pei, and Wo-Shun Luk. Email mining: tasks, common techniques, and tools. Knowledge and Information Systems, 41(1):1--31, 2014. Google ScholarDigital Library
- Wikipedia. Identity theft. http://en.wikipedia.org/wiki/Identity_theft.Google Scholar
- Xiaoyong Zhou, Soteris Demetriou, Dongjing He, Muhammad Naveed, Xiaorui Pan, XiaoFeng Wang, Carl A Gunter, and Klara Nahrstedt. Identity, location, disease and more: Inferring your secrets from android public resources. In Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security, pages 1017--1028. ACM, 2013. Google ScholarDigital Library
- Qi Alfred Chen, Zhiyun Qian, and Z Morley Mao. Peeking into your app without actually seeing it: Ui state inference and novel android attacks. In Proc. 23rd USENIX Security Symposium (SEC'14), USENIX Association, 2014. Google ScholarDigital Library
- Popstar. https://play.google.com/store/search?q=popstar.Google Scholar
- Manifest.permission. http://developer.android.com/reference/android/Manifest.permission.html.Google Scholar
- Sashank Narain, Amirali Sanatinia, and Guevara Noubir. Single-stroke language-agnostic keylogging using stereo-microphones and domain specific machine learning. In Proceedings of the 2014 ACM conference on Security and privacy in wireless & mobile networks, pages 201--212. ACM, 2014. Google ScholarDigital Library
- Managing your app's memory. https://developer.android.com/training/articles/memory.html.Google Scholar
- Settings.secure. http://developer.android.com/reference/android/provider/Settings.Secure.html.Google Scholar
- Compression algorithm. http://www.gzip.org/algorithm.txt.Google Scholar
- Sky McKinley and Megan Levine. Cubic spline interpolation. College of the Redwoods, 45:1049--1060, 1998.Google Scholar
- Popupwindow. http://developer.android.com/reference/android/widget/PopupWindow.html.Google Scholar
- Textview. http://developer.android.com/reference/android/widget/TextView.html.Google Scholar
- Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten. The weka data mining software: an update. ACM SIGKDD explorations newsletter, 11(1):10--18, 2009. Google ScholarDigital Library
- Thomas G Dietterich. Ensemble methods in machine learning. In Multiple classifier systems, pages 1--15. Springer, 2000. Google ScholarCross Ref
- Niels Landwehr, Mark Hall, and Eibe Frank. Logistic model trees. Machine Learning, 59(1-2):161--205, 2005. Google ScholarDigital Library
- John Platt et al. Fast training of support vector machines using sequential minimal optimization. Advances in kernel methods, a lsupport vector learning, 3, 1999. Google ScholarDigital Library
- Leo Breiman. Random forests. Machine learning, 45(1):5--32, 2001. Google ScholarDigital Library
- David W Aha, Dennis Kibler, and Marc K Albert. Instance-based learning algorithms. Machine learning, 6(1):37--66, 1991. Google ScholarDigital Library
- Lawrence R Rabiner and Biing-Hwang Juang. Fundamentals of speech recognition, volume 14. PTR Prentice Hall Englewood Cliffs, 1993. Google ScholarDigital Library
- Mehryar Mohri, Fernando Pereira, and Michael Riley. Weighted finite-state transducers in speech recognition. Computer Speech & Language, 16(1):69--88, 2002.Google ScholarDigital Library
- Cyril Allauzen, Michael Riley, Johan Schalkwyk, Wojciech Skut, and Mehryar Mohri. Openfst: A general and efficient weighted finite-state transducer library. In Implementation and Application of Automata, pages 11--23. Springer, 2007. Google ScholarCross Ref
- Wikipedia. Letter frequency. http://en.wikipedia.org/wiki/Letter_frequency.Google Scholar
- Brian Roark, Richard Sproat, Cyril Allauzen, Michael Riley, Jeffrey Sorensen, and Terry Tai. The opengrm open-source finite-state grammar software libraries. In Proceedings of the ACL 2012 System Demonstrations, pages 61--66. Association for Computational Linguistics, 2012. Google ScholarDigital Library
- Justine Jordan. 53% of emails opened on mobile; outlook opens decrease 33%. https://litmus.com/blog/53-of-emails-opened-on-mobile-outlook-opens-decrease-33Google Scholar
- Bryan Klimt and Yiming Yang. Introducing the enron corpus. In CEAS, 2004.Google Scholar
- Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, Clifford Stein, et al. Introduction to algorithms, volume 2. MIT press Cambridge, 2001. Google ScholarDigital Library
- Martin Vuagnoux and Sylvain Pasini. Compromising electromagnetic emanations of wired and wireless keyboards. In USENIX security symposium, pages 1--16, 2009. Google ScholarDigital Library
- Li Zhuang, Feng Zhou, and J Doug Tygar. Keyboard acoustic emanations revisited. ACM Transactions on Information and System Security (TISSEC), 13(1):3, 2009. Google ScholarDigital Library
- Denis Foo Kune and Yongdae Kim. Timing attacks on pin input devices. In Proceedings of the 17th ACM conference on Computer and communications security, pages 678--680. ACM, 2010. Google ScholarDigital Library
- Rahul Raguram, Andrew M White, Dibyendusekhar Goswami, Fabian Monrose, and Jan-Michael Frahm. ispy: automatic reconstruction of typed input from compromising reflections. In Proceedings of the 18th ACM conference on Computer and communications security, pages 527--536. ACM, 2011. Google ScholarDigital Library
- Yi Xu, Jared Heinly, Andrew M White, Fabian Monrose, and Jan-Michael Frahm. Seeing double: Reconstructing obscured typed input from repeated compromising reflections. In Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security, pages 1063--1074. ACM, 2013. Google ScholarDigital Library
- W Nelson Francis and Henry Kucera. Brown corpus manual. Brown University, 1979.Google Scholar
- Philip Marquardt, Arunabh Verma, Henry Carter, and Patrick Traynor. (sp) iphone: decoding vibrations from nearby keyboards using mobile phone accelerometers. In Proceedings of the 18th ACM conference on Computer and communications security, pages 551--562. ACM, 2011. Google ScholarDigital Library
- Yan Michalevsky, Dan Boneh, and Gabi Nakibly. Gyrophone: Recognizing speech from gyroscope signals. In Proc. 23rd USENIX Security Symposium (SEC '14), USENIX Association, 2014. Google ScholarDigital Library
Index Terms
- TextLogger: inferring longer inputs on touch screen using motion sensors
Recommendations
Single-stroke language-agnostic keylogging using stereo-microphones and domain specific machine learning
WiSec '14: Proceedings of the 2014 ACM conference on Security and privacy in wireless & mobile networksMobile phones are equipped with an increasingly large number of precise and sophisticated sensors. This raises the risk of direct and indirect privacy breaches. In this paper, we investigate the feasibility of keystroke inference when user taps on a ...
IntentFuzzer: detecting capability leaks of android applications
ASIA CCS '14: Proceedings of the 9th ACM symposium on Information, computer and communications securityCapability leak is a vulnerability in Android applications, which violates the enforcement of permission model and threatens the secure usage of Android phone users. Malicious applications can launch permission escalation attacks with this ...
Unified security enhancement framework for the Android operating system
In these days there are many malicious applications that collect sensitive information owned by third-party applications by escalating their privileges to the higher level on the Android operating system. An attack of obtaining the root-level privilege ...
Comments