skip to main content
research-article

CovertBand: Activity Information Leakage using Music

Published: 11 September 2017 Publication History

Abstract

This paper contributes a novel method for low-cost, covert physical sensing and, by doing so, surfaces new privacy threats. We demonstrate how a smartphone and portable speaker playing music with embedded, inaudible signals can track multiple individuals’ locations and activities both within a room and through barriers in 2D space. We achieve this by transforming a smartphone into an active sonar system that emits a combination of a sonar pulse and music and listens to the reflections off of humans in the environment. Our implementation, CovertBand, monitors minute changes to these reflections to track multiple people concurrently and to recognize different types of motion, leaking information about where people are in addition to what they may be doing. We evaluated CovertBand by running experiments in five homes in the Seattle area, showing that we can localize both single and multiple individuals through barriers. These tests show CovertBand can track walking subjects with a mean tracking error of 18 cm and subjects moving at a fixed position with an accuracy of 8 cm at up to 6 m in line-of-sight and 3 m through barriers. We test a variety of rhythmic motions such as pumping arms, jumping, and supine pelvic tilts in through-wall scenarios and show that they produce discernibly different spectrograms from walking in the acoustic reflections. In tests with 33 subjects, we also show that even in ideal scenarios, listeners were unlikely to detect a CovertBand attack.

References

[1]
2016. VideoRay ROV Sonar, VideoRay Imaging Sonar. (2016). http://www.videoray.com/homepage/options/sonar/blueview-imaging-sonars.html.
[2]
Acoustic Localization 2017. Acoustic Localization through wall/ceiling/floor. (2017). https://redecomposition.wordpress.com/acousticvideo/.
[3]
Fadel Adib, Chen-Yu Hsu, Hongzi Mao, Dina Katabi, and Frédo Durand. 2015. Capturing the human figure through a wall. ACM Trans. Graph. 34 (2015), 219:1--219:13.
[4]
HO Ahmed, JH Dennis, O Badran, M Ismail, SG Ballal, A Ashoor, and D Jerwood. 2001. High-frequency (10--18 kHz) hearing thresholds: reliability, and effects of age and occupational noise exposure. Occupational Medicine 51, 4 (2001), 245--258.
[5]
Amprobe 2016. Amprobe sound power meter. (2016). http://www.amprobe.com/amprobe/usen/environmental-test/sound/amp-sm-10.htm?PID=73334.
[6]
Md Tanvir Islam Aumi, Sidhant Gupta, Mayank Goel, Eric Larson, and Shwetak Patel. 2013. DopLink: Using the Doppler Effect for Multi-device Interaction. In UbiComp.
[7]
Auvio 2016. Auvio Portable Bluetooth Speaker. (2016). http://www.amazon.com/Portable-Bluetooth-Speaker-PBT1000-Gadgets/dp/B00D2D46S4.
[8]
Beats 2016. Beats Pill. (2016). http://www.beatsbydre.com/beatspill.html.
[9]
Bose 2016. Bose Sound Link 2. (2016). https://www.bose.com/products/speakers/wireless_speakers/soundlink_mini_ii.html.
[10]
Bo Chen, Vivek Yenamandra, and Kannan Srinivasan. 2015. Tracking Keystrokes Using Wireless Signals. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services.
[11]
Ke-Yu Chen, Daniel Ashbrook, Mayank Goel, Sung-Hyuck Lee, and Shwetak Patel. 2014. AirLink: Sharing Files Between Multiple Devices Using In-air Gestures. In UbiComp.
[12]
Ivan Cowie. 2008. Through-Wall Surveillance for Locating Individuals Within Buildings. Technical Report. Time Domain Corporation and United States of America. Final Scientific and Technical Report.
[13]
Fadel Ddib, Zachary Kabelac, and Dina Katabi. 2015. Multi-Person Localization via RF Body Reflections. In NSDI.
[14]
Fadel Ddib, Zach Kabelac, Dina Katabi, and Robert C Miller. 2014. 3D Tracking via Body Radio Reflections. In NSDI.
[15]
Fadel Ddib and Dina Katabi. 2013. Seeing Through Walls Using WiFi!. In SIGCOMM.
[16]
Ivan DokmaniÄĞa, Reza Parhizkara, Andreas Walthera, Yue M. Lub, and Martin Vetterlia. 2013. Acoustic echoes reveal room shape. In Proceedings of the National Academy of Sciences, Vol. 110. 12186--12191.
[17]
Echo 2016. Amazon Echo. (2016). http://www.amazon.com/Amazon-SK705DI-Echo/dp/B00X4WHP5E.
[18]
FLIR 2016. FLIR One Thermal Imaging. (2016). http://www.flir.com/flirone/explore.cfm.
[19]
Sidhant Gupta, Daniel Morris, Shwetak Patel, and Desney Tan. 2012. SoundWave: Using the Doppler Effect to Sense Gestures. In CHI.
[20]
Ruonan Han, Yaming Zhang, Youngwan Kim, Dae Yeon Kim, H. Shichijo, E. Afshari, and O. Kenneth. 2012. 280GHz and 860GHz image sensors using Schottky-barrier diodes in 0.13 um digital CMOS. In ISSCC.
[21]
Home 2017. Google Home. (2017). https://madeby.google.com/home/.
[22]
Home Theater 2016. Costco: Bose Acoustimass 10 Series V Home Theater Onkyo Bundle. (2016). http://www.costco.com/Bose%C2%AE-Acoustimass-10-Series-V-Home-Theater-Onkyo-Bundle.product.100147047.html.
[23]
Wenchao Huang, Yan Xiong, Xiang-Yang Li, Hao Lin, XuFei Mao, Panlong Yang, and Yunhao Liu. 2014. Shake and walk: Acoustic direction finding and fine-grained indoor localization using smartphones. In INFOCOM.
[24]
Humming 2016. Humming Bird Side Imaging. (2016). http://www.humminbird.com/Category/Technology/Side-Imaging/.
[25]
Hunted 2016. Hunted: The War Against Gays in Russia. (2016). http://www.hbo.com/documentaries/hunted-the-war-against-gays-in-russia.
[26]
IEEE. {n. d.}. IEEE 802.11g-2003: Further Higher Data Rate Extension in the 2.4 GHz Band. In Standard, 2003.
[27]
ifixit 2017. Amazon Echo Teardown. (2017). https://www.ifixit.com/Teardown/Amazon+Echo+Teardown/33953.
[28]
Incest 2016. Man killed on suspicion of incest. http://timesofindia.indiatimes.com/city/ahmedabad/Man-killed-on-suspicion-of-incest/articleshow/35271666.cms. (2016).
[29]
Interior Door Dimensions 2017. Standard Inside Door Sizes. (2017). http://homeguides.sfgate.com/standard-inside-door-sizes-84805.html.
[30]
Interior Wall Dimensions 2017. Building Requirements for Partition Walls. (2017). http://homeguides.sfgate.com/building-requirements-partition-walls-62677.html
[31]
JBL 2016. JBL Flip 2. (2016). http://www.jbl.com/bluetooth-speakers/JBL+FLIP+II.html.
[32]
Kiran Raj Joshi, Dinesh Bharadia, Manikanta Kotaru, and Sachin Katti. 2015. WiDeo: Fine-grained Device-free Motion Tracing using RF Backscatter. In NSDI.
[33]
Bryce Kellogg, Vamsi Talla, and Shyamnath Gollakota. 2014. Bringing Gesture Recognition To All Devices. In NSDI.
[34]
Manikanta Kotaru, Kiran Raj Joshi, Dinesh Bharadia, and Sachin Katti. 2015. SpotFi: Decimeter Level Localization Using WiFi. Computer Communication Review 45 (2015), 269--282.
[35]
Qiang Lin and Yuan Yue. 2015. Device-Free Passive Human Detection Using Wi-Fi Technology: Current State and Future Trend. 2015 IEEE 12th Intl Conf on Ubiquitous Intelligence and Computing and 2015 IEEE 12th Intl Conf on Autonomic and Trusted Computing and 2015 IEEE 15th Intl Conf on Scalable Computing and Communications and Its Associated Workshops (UIC-ATC-ScalCom) (2015), 1717--1723.
[36]
LTE 2016. An Introduction to LTE, 3GPP LTE Encyclopedia. (2016). https://sites.google.com/site/lteencyclopedia/home.
[37]
mobisante 2016. Smartphone Ultrasound: The MobiUS SP1 System. (2016). http://www.mobisante.com/products/product-overview/.
[38]
Rajalakshmi Nandakumar, Shyamnath Gollakota, and Nathaneil Watson. 2015. Contactless Sleep Apnea Detection on Smartphones. In MobiSys.
[39]
Rajalakshmi Nandakumar, Vikram Iyer, Shyamnath Gollakota, and Desney Tan. 2016. FingerIO: Fine-Grained Finger Tracking Using Active Sonar. In CHI.
[40]
Rajalakshmi Nandakumar, Bryce Kellogg, and Shyamnath Gollakota. 2014. Wi-Fi Gesture Recognition on Existing Devices. CoRR abs/1411.5394 (2014).
[41]
Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. 2013. Whole-Home Gesture Recognition Using Wireless Signals. In MOBICOM.
[42]
Radar Through Wall 2012. Through-the-Wall Sensors for Law Enforcement. (October 2012). https://www.justnet.org/pdf/00-WallSensorReport-508.pdf.
[43]
T.S. Ralston, G.L. Charvat, and J.E. Peabody. 2010. Real-time through-wall imaging using an ultrawideband multiple-input multiple-output (MIMO) phased array radar system. In ARRAY.
[44]
Franziska Roesner, Tadayoshi Kohno, Alexander Moshchuk, Bryan Parno, Helen J. Wang, and Crispin Cowan. 2012. User-Driven Access Control: Rethinking Permission Granting in Modern Operating Systems. 2012 IEEE Symposium on Security and Privacy (2012), 224--238.
[45]
Ahmed Saeed, Ahmed E. Kosba, and Moustafa Youssef. 2014. Ichnaea: A Low-Overhead Robust WLAN Device-Free Passive Localization System. IEEE Journal of Selected Topics in Signal Processing 8 (2014), 5--15.
[46]
Seek 2016. Seek Thermal Camera Review: Smartphone thermal vision in a tiny package. (2016). http://www.cnet.com/products/seek-thermal-camera/.
[47]
Moustafa Seifeldin, Ahmed Saeed, Ahmed E. Kosba, Amr El-Keyi, and Moustafa Youssef. 2013. Nuzzer: A Large-Scale Device-Free Passive Localization System for Wireless Environments. IEEE Transactions on Mobile Computing 12 (2013), 1321--1334.
[48]
Sharp TV 2017. Sharp LC-42SB45UT 42” 1080p LCD TV. (2017). https://www.amazon.com/Sharp-LC-42SB45UT-42-1080p-LCD/dp/B001F0QS9G.
[49]
Adam Smith, Hari Balakrishnan, Michel Goraczko, and Nissanka Priyantha. 2014. Tracking Moving Devices with the Cricket Location System. In Mobisys.
[50]
sonardyne 2016. Sonardyne: Sound in Depth. (2016). http://www.sonardyne.com/.
[51]
Stoned 2015. Afghan woman stoned to death for ’adultery’. (2015). http://www.cnn.com/2015/11/04/asia/afghanistan-taliban-woman-stoning/.
[52]
Li Sun, Souvik Sen, Dimitrios Koutsonikolas, and Kyu-Han Kim. {n. d.}. WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking (MobiCom ’15).
[53]
Zheng Sun, Aveek Purohit, Raja Bose, and Pei Zhang. 2013. Spartacus: Spatially-aware Interaction for Mobile Devices Through Energy-efficient Audio Sensing. In Mobisys.
[54]
H. Tanoto, J. H. Teng, Q. Y. Wu, Z. N. Chen, S. A. Maier, B. Wang, C. C. Chum, G. Y. Si, and A. J. Danner. 2013. Retraction: Greatly enhanced continuous-wave terahertz emission by nano-electrodes in a photoconductive photomixer. In Nat Photon.
[55]
Thermal Fact 2016. Thermal Imaging: Facts versus Fiction. (2016). https://pr-infrared.com/about-thermal-imaging/thermal-imaging-facts-vs-fiction/.
[56]
Ju Wang, Hongbo Jiang, Jie Xiong, Kyle Jamieson, Xiaojiang Chen, Dingyi Fang, and Binbin Xie. 2016. LiFS: low human-effort, device-free localization with fine-grained subcarrier information. In MobiCom.
[57]
Junjue Wang, Kaichen Zhao, Xinyu Zhang, and Chunyi Peng. 2014. Ubiquitous Keyboard for Small Mobile Devices: Harnessing Multipath Fading for Fine-grained Keystroke Localization. In MobiSys.
[58]
Wei Wang, Alex X. Liu, and Muhammad Shahzad. 2016. Gait recognition using wifi signals. In UbiComp.
[59]
Wei Wang, Alex X. Liu, and Ke Sun. 2016. Device-free gesture tracking using acoustic signals. In MobiCom.
[60]
Yan Wang, Jian Liu, Yingying Chen, Marco Gruteser, Jie Yang, and Hongbo Liu. 2014. E-eyes: Device-free Location-oriented Activity Identification Using Fine-grained WiFi Signatures. In MOBICOM.
[61]
Teng Wei, Shu Wang, Anfu Zhou, and Xinyu Zhang. 2015. Acoustic Eavesdropping Through Wireless Vibrometry. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking.
[62]
Norbert Wild. 2001. Ultrasonic through-the-wall surveillance system. Technical Report. International Society for Optics and Photonics. 167--176 pages.
[63]
J. Wilson and N. Patwari. 2011. See-Through Walls: Motion Tracking Using Variance-Based Radio Tomography Networks. IEEE Transactions on Mobile Computing (2011).
[64]
Dan Wu, Zhang Daqing, Chenren Xu, Yasha Wang, and Hao Wang. 2016. WiDir: walking direction estimation using wireless signals. In UBICOMP.
[65]
Jiang Xiao, Kaishun Wu, Youwen Yi, Lu Wang, and Lionel M. Ni. 2013. Pilot: Passive Device-Free Indoor Localization Using Channel State Information. 2013 IEEE 33rd International Conference on Distributed Computing Systems (2013), 236--245.
[66]
Robert Xiao, Greg Lew, James Marsanico, Divya Hariharan, Scott Hudson, and Chris Harrison. 2014. Toffee: Enabling Ad Hoc, Around-device Interaction with Acoustic Time-of-arrival Correlation. In MobileHCI.
[67]
Jie Xiong and Kyle Jamieson. 2012. Towards fine-grained radio-based indoor location. In HotMobile.
[68]
Qinyi Xu, Yan Chen, BeiBei Wang, and K. J. Ray Liu. 2017. Radio Biometrics: Human Recognition Through a Wall. IEEE Transactions on Information Forensics and Security 12 (2017), 1141--1155.
[69]
Lei Yang, Qiongzheng Lin, Xiangyang Li, Tianci Liu, and Yunhao Liu. 2015. See Through Walls with COTS RFID System!. In MOBICOM.
[70]
Sangki Yun, Yi-Chao Chen, and Lili Qiu. 2015. Turning a Mobile Device into a Mouse in the Air. In MobiSys.
[71]
Y. Zhao and N. Patwari. 2014. Robust Estimators for Variance-Based Device-Free Localization and Tracking. IEEE Transactions on Mobile Computing (2014).
[72]
Yanzi Zhu, Yibo Zhu, Ben Y. Zhao, and Haitao Zheng. 2015. Reusing 60GHz Radios for Mobile Radar Imaging. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking (MobiCom ’15).

Cited By

View all
  • (2025)VitalHide: Enabling Privacy-Aware Wireless Sensing of Vital SignsProceedings of the 26th International Workshop on Mobile Computing Systems and Applications10.1145/3708468.3711880(37-42)Online publication date: 26-Feb-2025
  • (2024)VBCNet: A Hybird Network for Human Activity RecognitionSensors10.3390/s2423779324:23(7793)Online publication date: 5-Dec-2024
  • (2024)Room-scale Location Trace Tracking via Continuous Acoustic WavesACM Transactions on Sensor Networks10.1145/364913620:3(1-23)Online publication date: 13-Apr-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 1, Issue 3
September 2017
2023 pages
EISSN:2474-9567
DOI:10.1145/3139486
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 September 2017
Accepted: 01 July 2017
Revised: 01 May 2017
Received: 01 February 2017
Published in IMWUT Volume 1, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Active Sonar
  2. Information leakage
  3. Motion tracking
  4. Music processing

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)60
  • Downloads (Last 6 weeks)11
Reflects downloads up to 03 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)VitalHide: Enabling Privacy-Aware Wireless Sensing of Vital SignsProceedings of the 26th International Workshop on Mobile Computing Systems and Applications10.1145/3708468.3711880(37-42)Online publication date: 26-Feb-2025
  • (2024)VBCNet: A Hybird Network for Human Activity RecognitionSensors10.3390/s2423779324:23(7793)Online publication date: 5-Dec-2024
  • (2024)Room-scale Location Trace Tracking via Continuous Acoustic WavesACM Transactions on Sensor Networks10.1145/364913620:3(1-23)Online publication date: 13-Apr-2024
  • (2024)Robust Indoor Location Identification for Smartphones Using Echoes From Dominant ReflectorsIEEE Transactions on Mobile Computing10.1109/TMC.2023.3307695(1-17)Online publication date: 2024
  • (2024)CSI-GLSTN: A Location-Independent CSI Human Activity Recognition Method Based on Spatio-Temporal and Channel Feature FusionIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.337431273(1-12)Online publication date: 2024
  • (2024)Time to Think the Security of WiFi-Based Behavior Recognition SystemsIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.326132821:1(449-462)Online publication date: Jan-2024
  • (2024)Station: Gesture-Based Authentication for Voice InterfacesIEEE Internet of Things Journal10.1109/JIOT.2024.338272111:12(22668-22683)Online publication date: 15-Jun-2024
  • (2024)EarHear: Enabling the Deaf to Hear the World via Smartphone Speakers and MicrophonesIEEE Internet of Things Journal10.1109/JIOT.2023.332363111:6(9708-9724)Online publication date: 15-Mar-2024
  • (2024)Beyond-Voice: Towards Continuous 3D Hand Pose Tracking on Commercial Home Assistant Devices2024 23rd ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)10.1109/IPSN61024.2024.00017(151-162)Online publication date: 13-May-2024
  • (2024)One is Enough: Enabling One-shot Device-free Gesture Recognition with COTS WiFiIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621091(1231-1240)Online publication date: 20-May-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media