Chao Wang
ABSTRACT
Earpiece mode of smartphones is often used for confidential communication. In this paper, we proposed a remote(>2m) and motion-resilient attack on smartphone earpiece. We developed an end-to-end eavesdropping system mmEve based on a commercial mmWave sensor to recover speech emitted from smartphone earpiece. The rationale of the attack is based on our observation that, soundwaves emitted from the smartphone's earpiece have a strong correlation with reflected mmWaves from the smartphone's rear. However, we find the recovered speech suffers from the sensor's self-noise and smartphone user's motion which limit attack distance to less than 2m, causing limited threats in real world. We modeled the motion interference under mmWave sensing and proposed a motion-resilient solution by optimizing the fitting function on I/Q plane. To achieve a practical attack with reasonable attack distance, we developed a GAN-based denoising scheme to eliminate the noise pattern of the sensor, which boosted the attack range to 6--8m. We evaluated mmEve with extensive experiments and find 23 different models of smartphones manufactured by Samsung, Huawei, etc. can be compromised by the proposed attack.
- Alankrita Aggarwal, Mamta Mittal, and Gopi Battineni. 2021. Generative adversarial network: An overview of theory and applications. International Journal of Information Management Data Insights 1, 1 (2021), 100004.Google Scholar
Cross Ref
- Amazon. 2022. Amazon Transcribe. https://aws.amazon.com/transcribe/Google Scholar
- Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al. 2016. Deep speech 2: End-to-end speech recognition in english and mandarin. In International conference on machine learning. PMLR, 173--182.Google Scholar
- S Abhishek Anand and Nitesh Saxena. 2018. Speechless: Analyzing the threat to speech privacy from smartphone motion sensors. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 1000--1017.Google ScholarCross Ref
- S Abhishek Anand, Chen Wang, Jian Liu, Nitesh Saxena, and Yingying Chen. 2021. Spearphone: a lightweight speech privacy exploit via accelerometer-sensed reverberations from smartphone loudspeakers. In Proceedings of the 14th ACM Conference on Security and Privacy in Wireless and Mobile Networks. 288--299.Google ScholarDigital Library
- Zhongjie Ba, Tianhang Zheng, Xinyu Zhang, Zhan Qin, Baochun Li, Xue Liu, and Kui Ren. 2020. Learning-based Practical Smartphone Eavesdropping with Built-in Accelerometer. In 27th Annual Network and Distributed System Security Symposium, NDSS 2020, San Diego, California, USA, February 23--26, 2020. The Internet Society.Google Scholar
- S. Basak and M. Gowda. 2022. mmSpy: Spying Phone Calls using mmWave Radars. In 2022 2022 IEEE Symposium on Security and Privacy (SP) (SP). IEEE Computer Society, Los Alamitos, CA, USA, 995--1012. Google ScholarCross Ref
- Alexander Bunkowski, Bertram Bödeker, Stefan Bader, Michael Westhoff, Patric Litterst, and Jörg Ingo Baumbach. 2009. MCC/IMS signals in human breath related to sarcoidosis---results of a feasibility study using an automated peak finding procedure. Journal of Breath Research 3, 4 (2009), 046001.Google ScholarCross Ref
- John Cunnison Catford et al. 1988. A practical introduction to phonetics. Clarendon Press Oxford. 161 pages.Google Scholar
- Chi-Lung Cheng and Hans Schneeweiss. 1998. Polynomial regression with errors in the variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 60, 1 (1998), 189--199.Google ScholarCross Ref
- Jieun Choi, Hae-Yong Yang, and Dong-Ho Cho. 2020. TEMPEST Comeback: A Realistic Audio Eavesdropping Threat on Mixed-signal SoCs. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security. 1085--1101.Google ScholarDigital Library
- Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham J Mysore, Fredo Durand, and William T Freeman. 2014. The visual microphone: Passive recovery of sound from video. (2014).Google Scholar
- Philippa Demonte. 2019. HARVARD Speech Corpus---Audio Recording 2019. University of Salford Collection (2019).Google Scholar
- Szu-Wei Fu, Yu Tsao, Hsin-Te Hwang, and Hsin-Min Wang. 2018. Quality-Net: An end-to-end non-intrusive speech quality assessment model based on BLSTM. arXiv preprint arXiv:1808.05344 (2018).Google Scholar
- Szu-Wei Fu, Cheng Yu, Tsun-An Hsieh, Peter Plantinga, Mirco Ravanelli, Xugang Lu, and Yu Tsao. 2021. MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement. arXiv preprint arXiv:2104.03538 (2021).Google Scholar
- Munther Gdeisat and Francis Lilley. 2011. One-dimensional phase unwrapping problem. signal 4 (2011), 6.Google Scholar
- Mordechai Guri, Yosef Solewicz, Andrey Daidakulov, and Yuval Elovici. 2017. SPEAKE (a) R: Turn speakers to microphones for fun and profit. In 11th {USENIX} Workshop on Offensive Technologies ({WOOT} 17).Google Scholar
- HStatista. 2021. Smartphone users worldwide 2016-2021. Technical Report. New York, NY, USA.Google Scholar
- TeamViewer Inc. 2021. TeamViewer. https://www.teamviewer.com/Google Scholar
- Texas Instruments. 2020. AWR1843Boost. https://www.ti.com/lit/ug/spruim4b/spruim4b.pdf?ts=1638342429747Google Scholar
- Md Asif Iqbal, Mohammad AZ Al-Khateeb, Lukasz Krzczanowicz, Ian D Phillips, Paul Harper, and Wladek Forysiak. 2019. Linear and nonlinear noise characterisation of dual stage broadband discrete Raman amplifiers. Journal of Lightwave Technology 37, 14 (2019), 3679--3688.Google ScholarCross Ref
- Chengkun Jiang, Junchen Guo, Yuan He, Meng Jin, Shuai Li, and Yunhao Liu. 2020. mmVib: micrometer-level vibration measurement with mmwave radar. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 1--13.Google ScholarDigital Library
- S Kumudini and R Sinha. 2015. Normalized least mean square (Nlms) adaptive filter for noise cancellation. International Journal of Progresses in Engineering, Management, Science and Humanities 1, 1 (2015), 49.Google Scholar
- Andrew Kwong, Wenyuan Xu, and Kevin Fu. 2019. Hard Drive of Hearing: Disks that Eavesdrop with a Synthesized Microphone. In 2019 IEEE Symposium on Security and Privacy (SP). 905--919. Google ScholarCross Ref
- Christophe Leys, Christophe Ley, Olivier Klein, Philippe Bernard, and Laurent Licata. 2013. Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of experimental social psychology 49, 4 (2013), 764--766.Google ScholarCross Ref
- Dong Li, Jialin Liu, Sunghoon Ivan Lee, and Jie Xiong. 2022. LASense: Pushing the Limits of Fine-grained Activity Sensing Using Acoustic Signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1--27.Google ScholarDigital Library
- William McGrath. 2005. Technique and device for through-the-wall audio surveillance. US Patent App. 11/095,122.Google Scholar
- Ephrem Tibebe Mekonnen, Alessio Brutti, and Daniele Falavigna. 2022. End-to-End Low Resource Keyword Spotting Through Character Recognition and Beam-Search Re-Scoring. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 8182--8186.Google Scholar
- Yan Michalevsky, Dan Boneh, and Gabi Nakibly. 2014. Gyrophone: Recognizing speech from gyroscope signals. In 23rd {USENIX} Security Symposium ({USENIX} Security 14). 1053--1067.Google Scholar
- Felix Modes. 2021. BMW Automotive sensors: assistance systems' sense organs. https://www.bmw.com/en/innovation/automotive-sensors.htmlGoogle Scholar
- Masanori Morise, Fumiya Yokomori, and Kenji Ozawa. 2016. WORLD: a vocoder-based high-quality speech synthesis system for real-time applications. IEICE TRANSACTIONS on Information and Systems 99, 7 (2016), 1877--1884.Google ScholarCross Ref
- Ralph P Muscatell. 1984. Laser microphone. The Journal of the Acoustical Society of America 76, 4 (1984), 1284--1284.Google ScholarCross Ref
- Ben Nassi, Yaron Pirutin, Tomer Galor, Yuval Elovici, and Boris Zadov. 2021. Glowworm Attack: Optical TEMPEST Sound Recovery via a Device's Power Indicator LED. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 1900--1914.Google ScholarDigital Library
- Ben Nassi, Yaron Pirutin, Adi Shamir, Yuval Elovici, and Boris Zadov. 2020. Lam-phone: Real-time passive sound recovery from light bulb vibrations. Cryptology ePrint Archive (2020).Google Scholar
- John Nolan, Kun Qian, and Xinyu Zhang. 2021. RoS: Passive Smart Surface for Roadside-to-Vehicle Communication. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference (Virtual Event, USA) (SIGCOMM '21). Association for Computing Machinery, New York, NY, USA, 165--178. Google ScholarDigital Library
- Tomofumi Oyama, Hisao Nakashima, Shoichiro Oda, Tomohiro Yamauchi, Zhenning Tao, Takeshi Hoshida, and Jens C Rasmussen. 2014. Robust and efficient receiver-side compensation method for intra-channel nonlinear effects. In OFC 2014. IEEE, 1--3.Google ScholarCross Ref
- Piya Pal and PP Vaidyanathan. 2009. Frequency invariant MVDR beamforming without filters and implementation using MIMO radar. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2081--2084.Google ScholarDigital Library
- Pery Pearson. 1993. Sound Sampling. http://www.hitl.washington.edu/projects/knowledge_base/virtual-worlds/EVE/I.B.3.a.SoundSampling.htmlGoogle Scholar
- Sandeep Rao. 2017. Introduction to mmWave sensing: FMCW radars. Texas Instruments (TI) mmWave Training Series (2017).Google Scholar
- Wit Rigs. 2019. Samsung Galaxy Note10 Teardown. https://www.youtube.com/watch?v=RfHLb5TagS8Google Scholar
- Wit Rigs. 2020. Samsung Galaxy S20 Teardown. https://www.youtube.com/watch?v=zaZsd4Sz83QGoogle Scholar
- Vittorio Rizzoli, Diego Masotti, and Franco Mastri. 1994. Full nonlinear noise analysis of microwave mixers. In 1994 IEEE MTT-S International Microwave Symposium Digest (Cat. No. 94CH3389-4). IEEE, 961--964.Google ScholarCross Ref
- Nirupam Roy and Romit Roy Choudhury. 2016. Listening through a vibration motor. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services. 57--69.Google ScholarDigital Library
- Sriram Sami, Yimin Dai, Sean Rui Xiang Tan, Nirupam Roy, and Jun Han. 2020. Spying with your robot vacuum cleaner: eavesdropping via lidar sensors. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems. 354--367.Google ScholarDigital Library
- Elahe Soltanaghaei, Avinash Kalyanaraman, and Kamin Whitehouse. 2017. Peripheral wifi vision: Exploiting multipath reflections for more sensitive human sensing. In Proceedings of the 4th International on Workshop on Physical Analytics. 13--18.Google ScholarDigital Library
- Elahe Soltanaghaei, Akarsh Prabhakara, Artur Balanuta, Matthew Anderson, Jan M. Rabaey, Swarun Kumar, and Anthony Rowe. 2021. Millimetro: MmWave Retro-Reflective Tags for Accurate, Long Range Localization. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking (MobiCom '21). Association for Computing Machinery, New York, NY, USA, 69--82. Google ScholarDigital Library
- Andrew Sowter. 2003. The derivation of phase integer ambiguity from single InSAR pairs: implications for differential interferometry. In Proceedings of the 11th FIG Symposium on Deformation Measurements, Santorini, Greece. 149--156.Google Scholar
- Cees H Taal, Richard C Hendriks, Richard Heusdens, and Jesper Jensen. 2011. An algorithm for intelligibility prediction of time-frequency weighted noisy speech. IEEE Transactions on Audio, Speech, and Language Processing 19, 7 (2011), 2125--2136.Google ScholarDigital Library
- Cassia Valentini-Botinhao et al. 2017. Noisy speech database for training speech enhancement algorithms and tts models. (2017).Google Scholar
- Chao Wang, Feng Lin, Tiantian Liu, Ziwei Liu, Yijie Shen, Wenyao Xu, and Kui Ren. 2022. mmPhone: Acoustic Eavesdropping on Loudspeakers via mmWave-characterized Piezoelectric Effect. In IEEE INFOCOM 2022 - IEEE Conference on Computer Communications.Google ScholarDigital Library
- Chen-Chia Wang, Sudhir Trivedi, and etc. 2009. High sensitivity pulsed laser vibrometer and its application as a laser microphone. Applied Physics Letters 94, 5 (2009), 051112.Google ScholarCross Ref
- Ziqi Wang, Zhe Chen, Akash Deep Singh, Luis Garcia, Jun Luo, and Mani B Srivastava. 2020. UWHear: through-wall extraction and separation of audio vibrations using wireless signals. In SenSys'20. 1--14.Google Scholar
- Teng Wei, Shu Wang, Anfu Zhou, and Xinyu Zhang. 2015. Acoustic eavesdropping through wireless vibrometry. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, New York, NY, USA, 130--141.Google ScholarDigital Library
- Wikipedia contributors. 2020. Word error rate --- Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=Word_error_rate&oldid=939575741 [Online; accessed 6-August-2022].Google Scholar
- Chenhan Xu, Zhengxiong Li, Hanbin Zhang, Aditya Singh Rathore, Huining Li, Chen Song, Kun Wang, and Wenyao Xu. 2019. Waveear: Exploring a mmwave-based noise-resistant speech sensing for voice-user interface. In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services. 14--26.Google ScholarDigital Library
- Bin Zhang, Min Kong, and Cong Bing WU. 2009. Research of spectrum leakage with window function. Informationization 11 (2009), 10--12.Google Scholar
- Li Zhang, Parth H Pathak, Muchen Wu, Yixin Zhao, and Prasant Mohapatra. 2015. Accelword: Energy efficient hotword detection through accelerometer. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. 301--315.Google ScholarDigital Library
Index Terms
- mmEve: eavesdropping on smartphone's earpiece via COTS mmWave device
Recommendations
Characterizing and Mitigating Touchtone Eavesdropping in Smartphone Motion Sensors
RAID '23: Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and DefensesSmartphone motion sensors provide cybersecurity attackers with a stealthy way to eavesdrop on nearby acoustic information. Eavesdropping on touchtones emitted by smartphone speakers when users input numbers into their phones exposes sensitive ...
Reverberation canceling wireless aid for hearing impaired
The stringent requirements on size and power consumption constrain the conventional hearing aid devices. Besides providing an economic and user friendly aid, reverberation/echo cancellation is an important requirement. With the technological ...
A jamming approach to enhance enterprise Wi-Fi secrecy through spatial access control
Prevalent Wi-Fi networks have adopted various protections to prevent eavesdropping caused by the intrinsic shared nature of wireless medium. However, many of them are based on pre-shared secret incurring key management costs, and are still vulnerable ...
Comments