Connor Bolton
Jun Han
Josiah Hester
ABSTRACT
Smartphone motion sensors provide cybersecurity attackers with a stealthy way to eavesdrop on nearby acoustic information. Eavesdropping on touchtones emitted by smartphone speakers when users input numbers into their phones exposes sensitive information such as credit card information, banking PINs, and social security card numbers to malicious applications with access to only motion sensor data. This work characterizes this new security threat of touchtone eavesdropping by providing an analysis based on physics and signal processing theory. We show that advanced adversaries who selectively integrate data from multiple motion sensors and multiple sensor axes can achieve over 99% accuracy on recognizing 12 unique touchtones. We further design, analyze, and evaluate several mitigations which could be implemented in a smartphone update. We found that some apparent mitigations such as low-pass filters can undesirably reduce the motion sensor data to benign applications by 83% but only reduce an advanced adversary’s accuracy by less than one percent. Other more informed designs such as anti-aliasing filters can fully preserve the motion sensor data to support benign application functionality while reducing attack accuracy by 50.1%.
- S Abhishek Anand, Chen Wang, Jian Liu, Nitesh Saxena, and Yingying Chen. 2019. Spearphone: A Speech Privacy Exploit via Accelerometer-Sensed Reverberations from Smartphone Loudspeakers. arXiv preprint arXiv:1907.05972 (2019).Google Scholar
- Zhongjie Ba, Tianhang Zheng, Xinyu Zhang, Zhan Qin, Baochun Li, Xue Liu, and Kui Ren. 2020. Learning-based Practical Smartphone Eavesdropping with Built-in Accelerometer.. In NDSS.Google Scholar
- Stephen Beeby, Graham Ensel, Neil M White, and Michael Kraft. 2004. MEMS mechanical sensors. Artech House.Google Scholar
- Hristo Bojinov, Yan Michalevsky, Gabi Nakibly, and Dan Boneh. 2014. Mobile device identification via sensor fingerprinting. arXiv preprint arXiv:1408.1416 (2014).Google Scholar
- Connor Bolton, Sara Rampazzi, Chaohao Li, Andrew Kwong, Wenyuan Xu, and Kevin Fu. 2018. Blue Note: How intentional acoustic interference damages availability and integrity in hard disk drives and operating systems. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 1048–1062.Google ScholarCross Ref
- Raj Bridgelall. 2015. Inertial sensor sample rate selection for ride quality measures. Journal of Infrastructure Systems 21, 2 (2015), 04014039.Google ScholarCross Ref
- Tianqi Chen and Carlos Guestrin. 2016. XGBoost. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Aug 2016). https://doi.org/10.1145/2939672.2939785Google ScholarDigital Library
- Yun Chan Cho and Jae Wook Jeon. 2008. Remote robot control system based on DTMF of mobile phone. In 2008 6th IEEE International Conference on Industrial Informatics. IEEE, 1441–1446.Google Scholar
- Android Developers. 2023. Android Debug Bridge. https://developer.android.com/studio/command-line/adb.Google Scholar
- Denis Foo Kune, John Backes, Shane S Clark, Daniel Kramer, Matthew Reynolds, Kevin Fu, Yongdae Kim, and Wenyuan Xu. 2013. Ghost talk: Mitigating EMI signal injection attacks against analog sensors. In Proceedings of the 34th IEEE Symposium on Security and Privacy (SP). IEEE, 145–159.Google Scholar
- Raffaele Gravina, Parastoo Alinia, Hassan Ghasemzadeh, and Giancarlo Fortino. 2017. Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges. Information Fusion 35 (2017), 68–80.Google ScholarCross Ref
- Jun Han, Albert Jin Chung, and Patrick Tague. 2017. PitchIn: Eavesdropping via Intelligible Speech Reconstruction using Non-Acoustic Sensor Fusion. In 16th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).Google ScholarDigital Library
- Jun Han, Emmanuel Owusu, Le T. Nguyen, Adrian Perrig, and Joy Zhang. 2012. ACComplice: Location inference using accelerometers on smartphones. In Communication Systems and Networks (COMSNETS). https://doi.org/10.1109/COMSNETS.2012.6151305Google ScholarCross Ref
- Pengfei Hu, Hui Zhuang, Panneer Selvam Santhalingam, Riccardo Spolaor, Parth Pathak, Guoming Zhang, and Xiuzhen Cheng. 2022. Accear: Accelerometer acoustic eavesdropping with unconstrained vocabulary. In 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 1757–1773.Google ScholarCross Ref
- Zhe Hu, Lu Yuan, Stephen Lin, and Ming-Hsuan Yang. 2016. Image deblurring using smartphone inertial sensors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1855–1864.Google ScholarCross Ref
- Intel. 2023. Intel NUC. https://www.intel.com/content/www/us/en/products/boards-kits/nuc.html.Google Scholar
- Alexandre Karpenko, David Jacobs, Jongmin Baek, and Marc Levoy. 2011. Digital video stabilization and rolling shutter correction using gyroscopes. CSTR 1, 2 (2011), 13.Google Scholar
- Adil Mehmood Khan, Muhammad Hameed Siddiqi, and Seok-Won Lee. 2013. Exploratory data analysis of acceleration signals to select light-weight and accurate features for real-time activity recognition on smartphones. Sensors 13, 10 (2013), 13099–13122.Google ScholarCross Ref
- Tuljappa M Ladwa, Sanjay M Ladwa, R Sudharshan Kaarthik, Alok Ranjan Dhara, and Nayan Dalei. 2009. Control of remote domestic system using DTMF. In International Conference on Instrumentation, Communication, Information Technology, and Biomedical Engineering 2009. IEEE, 1–6.Google ScholarCross Ref
- Seoungjun Lee, Dongsoo Har, and Dongsuk Kum. 2016. Drone-assisted disaster management: Finding victims via infrared camera and lidar sensor fusion. In 2016 3rd Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE). IEEE, 84–89.Google ScholarCross Ref
- R. Gary Leonard and George R. Doddington. 1993. TIDIGITS. https://catalog.ldc.upenn.edu/LDC93S10.Google Scholar
- Mark W Maciejewski, Harry Z Qui, Iulian Rujan, Mehdi Mobli, and Jeffrey C Hoch. 2009. Nonuniform sampling and spectral aliasing. Journal of Magnetic Resonance 199, 1 (2009), 88–93.Google ScholarCross Ref
- Ahmed Tanvir Mahdad, Cong Shi, Zhengkun Ye, Tianming Zhao, Yan Wang, Yingying Chen, and Nitesh Saxena. 2022. EarSpy: Spying Caller Speech and Identity through Tiny Vibrations of Smartphone Ear Speakers. arXiv preprint arXiv:2212.12151 (2022).Google Scholar
- Robert J. Marks, II. 1991. Introduction to Shannon Sampling and Interpolation Theory. Springer-Verlag, Berlin, Heidelberg.Google Scholar
- Philip Marquardt, Arunabh Verma, Henry Carter, and Patrick Traynor. 2011. (Sp)iPhone: Decoding Vibrations from Nearby Keyboards Using Mobile Phone Accelerometers. In Conference on Computer and Communications Security (CCS). ACM, New York, NY, USA. https://doi.org/10.1145/2046707.2046771Google ScholarDigital Library
- Václav Matyáš and Zdeněk Říha. 2002. Biometric authentication—security and usability. In Advanced communications and multimedia security. Springer, 227–239.Google Scholar
- Yan Michalevsky, Dan Boneh, and Gabi Nakibly. 2014. Gyrophone: Recognizing Speech from Gyroscope Signals. In 23rd USENIX Security Symposium (USENIX Security 14). USENIX Association, San Diego, CA, 1053–1067. https://www.usenix.org/conference/usenixsecurity14/technical-sessions/presentation/michalevskyGoogle ScholarDigital Library
- Emiliano Miluzzo, Alexander Varshavsky, Suhrid Balakrishnan, and Romit Roy Choudhury. 2012. Tapprints: Your Finger Taps Have Fingerprints. In Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services (Low Wood Bay, Lake District, UK) (MobiSys ’12). ACM, New York, NY, USA, 323–336. https://doi.org/10.1145/2307636.2307666Google ScholarDigital Library
- S. Narain, T. D. Vo-Huu, K. Block, and G. Noubir. 2016. Inferring User Routes and Locations Using Zero-Permission Mobile Sensors. In 2016 IEEE Symposium on Security and Privacy (SP). 397–413. https://doi.org/10.1109/SP.2016.31Google ScholarCross Ref
- Emmanuel Owusu, Jun Han, Sauvik Das, Adrian Perrig, and Joy Zhang. 2012. ACCessory: Password Inference Using Accelerometers on Smartphones(HotMobile). ACM, New York, NY, USA. https://doi.org/10.1145/2162081.2162095Google ScholarDigital Library
- Thilo Pfau and Patrick Reilly. 2021. How low can we go? Influence of sample rate on equine pelvic displacement calculated from inertial sensor data. Equine Veterinary Journal 53, 5 (2021), 1075–1081.Google ScholarCross Ref
- Rahul Potharaju, Andrew Newell, Cristina Nita-Rotaru, and Xiangyu Zhang. 2012. Plagiarizing smartphone applications: attack strategies and defense techniques. In International symposium on engineering secure software and systems. Springer, 106–120.Google ScholarDigital Library
- C.E. Shannon. 1949. Communication in the Presence of Noise. Proceedings of the IRE 37, 1 (Jan 1949), 10–21. https://doi.org/10.1109/JRPROC.1949.232969Google ScholarCross Ref
- Yunmok Son, Hocheol Shin, Dongkwan Kim, Youngseok Park, Juhwan Noh, Kibum Choi, Jungwoo Choi, and Yongdae Kim. 2015. Rocking drones with intentional sound noise on gyroscopic sensors. In 24th USENIX Security Symposium. 881–896.Google ScholarDigital Library
- Weigao Su, Daibo Liu, Taiyuan Zhang, and Hongbo Jiang. 2021. Towards device independent eavesdropping on telephone conversations with built-in accelerometer. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 4 (2021), 1–29.Google ScholarDigital Library
- General Tools. 2023. DSM403SD. https://generaltools.com/class-1-sound-level-meter-with-excel-formatted-data-logging-sd-card.Google Scholar
- Timothy Trippel, Ofir Weisse, Wenyuan Xu, Peter Honeyman, and Kevin Fu. 2017. WALNUT: Waging doubt on the integrity of MEMS accelerometers with acoustic injection attacks. In 2017 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 3–18.Google ScholarCross Ref
- Yannis Tsividis. 2004. Digital signal processing in continuous time: a possibility for avoiding aliasing and reducing quantization error. In 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 2. IEEE, ii–589.Google ScholarCross Ref
- Yazhou Tu, Zhiqiang Lin, Insup Lee, and Xiali Hei. 2018. Injected and delivered: fabricating implicit control over actuation systems by spoofing inertial sensors. In 27th USENIX Security Symposium. 1545–1562.Google Scholar
- International Telecommunication Union. 1988. Technical Features of Push-Button Telephone Sets. General Recommendations on Telephone Switching and Signalling (25 11 1988). https://www.itu.int/rec/T-REC-Q.23-198811-I/en.Google Scholar
- Matt Vasilogambros. 2019. Voting by phone is easy. But is it secure?https://gcn.com/articles/2019/07/18/vote-by-phone.aspx.Google Scholar
- Alma Whitten and J Doug Tygar. 1999. Why Johnny Can’t Encrypt: A Usability Evaluation of PGP 5.0.. In USENIX security symposium, Vol. 348. 169–184.Google Scholar
- Steve Winder. 2002. Analog and digital filter design. Elsevier.Google Scholar
- Chen Yan, Yan Long, Xiaoyu Ji, and Wenyuan Xu. 2019. The catcher in the field: A fieldprint based spoofing detection for text-independent speaker verification. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 1215–1229.Google ScholarDigital Library
- Li Zhang, Parth H. Pathak, Muchen Wu, Yixin Zhao, and Prasant Mohapatra. 2015. AccelWord: Energy Efficient Hotword Detection Through Accelerometer. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (Florence, Italy) (MobiSys ’15). ACM, New York, NY, USA, 301–315. https://doi.org/10.1145/2742647.2742658Google ScholarDigital Library
- Yang Zhang, Peng Xia, Junzhou Luo, Zhen Ling, Benyuan Liu, and Xinwen Fu. 2012. Fingerprint attack against touch-enabled devices. In Proceedings of the second ACM workshop on Security and privacy in smartphones and mobile devices. 57–68.Google ScholarDigital Library
Index Terms
- Characterizing and Mitigating Touchtone Eavesdropping in Smartphone Motion Sensors
Recommendations
Quantifying eavesdropping vulnerability in sensor networks
DMSN '05: Proceedings of the 2nd international workshop on Data management for sensor networksWith respect to security, sensor networks have a number of considerations that separate them from traditional distributed systems. First, sensor devices are typically vulnerable to physical compromise. Second, they have significant power and processing ...
Taxonomy of Distributed Denial of Service DDoS Attacks and Defense Mechanisms in Present Era of Smartphone Devices
This article describes how in the summer of 1999, the Computer Incident Advisory Capability first reported about Distributed Denial of Service DDoS attack incidents and the nature of Denial of Service DoS attacks in a distributed environment that ...
mmEve: eavesdropping on smartphone's earpiece via COTS mmWave device
MobiCom '22: Proceedings of the 28th Annual International Conference on Mobile Computing And NetworkingEarpiece mode of smartphones is often used for confidential communication. In this paper, we proposed a remote(>2m) and motion-resilient attack on smartphone earpiece. We developed an end-to-end eavesdropping system mmEve based on a commercial mmWave ...
Comments