Skip to main content
Springer Nature Link
Log in

CondioSense: high-quality context-aware service for audio sensing system via active sonar

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Audio sensing has been applied in various mobile applications for sensing personal and environmental information to improve user’s life quality. However, the quality of audio sensing is distorted seriously, while the sensing service is working in incorrect context or the ability of the acoustic sensing is limited (i.e., aging effect of the microphone or interference due to hand covering). To address this challenge, we present CondioSense, a CONtext-aware service for auDIO SENSing system, which identifies the current phone context (i.e., pocket, bag, car, indoor and outdoor) and detects the microphone sensing states. The main idea behind context detection is to extract multipath features from actively generated acoustic signal to identify various contexts since the space size and material among various contexts is different. The sound of physical vibration is explored on microphone sensing state detection, by leveraging that the frequency response of recorded vibration sound changes when the signal propagation in the air is blocked with the microphone covered. We prototype CondioSense on smartphones as an application and perform extensive evaluations. It offers the possibility to recognize various phone contexts with an accuracy exceeding \(92\%\) and the accuracy of microphone sensing states detection exceeding \(90\%\).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Qiu J, Chu D, Meng X, Moscibroda T (2011) On the feasibility of real-time phone-to-phone 3D localization. In: Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems (Sensys). ACM, pp 190–203

  2. Peng C, Shen G, Zhang Y, Li Y, Tan K (2007) Beepbeep: a high accuracy acoustic ranging system using COTS mobile devices. In: Proceedings of the 5th international conference on Embedded networked sensor systems (Sensys). ACM, pp 1–14

  3. Bo C, Jian X, Jung T, Han J, Li X-Y, Mao X, Wang Y (2016) Detecting driver’s smartphone usage via non-intrusively sensing driving dynamics. IEEE Internet of Things Journal

  4. Bo C, Jung T, Mao X, Li X-Y, Wang Y, (2016) Smartloc: Sensing landmarks silently for smartphone based metropolitan localization. EURASIP Journal on Wireless Communications and Networking, 2016

  5. Ren Y, Wang C, Yang J, Chen Y (2015) Fine-grained sleep monitoring: Hearing your breathing with smartphones. In: 2015 IEEE Conference on Computer Communications (INFOCOM). IEEE, pp 1194–1202

  6. Yang J, Sidhom S, Chandrasekaran G, Vu T, Liu H, Cecan N, Chen Y, Gruteser M, Martin RP (2011) Detecting driver phone use leveraging car speakers. In: Proceedings of the 17th annual international conference on Mobile computing and networking (Mobicom). ACM, pp 97–108

  7. Rana RK, Chou CT, Kanhere SS, Bulusu N, Hu W (2010) earphone: an end-to-end participatory urban noise mapping system. In: Proceedings of the 9th ACM/IEEE International Conference on Information Processing in Sensor Networks. ACM, pp 105–116

  8. Li H, Li T, Li F, Wang W, Wang Y (2016) Enhancing participant selection through caching in mobile crowd sensing. Proc. of ACM/IEEE IWQoS

  9. Li J, Cai Z, Yan M, Li Y (2016) Using crowdsourced data in location-based social networks to explore influence maximization. In: Proc. of 35th Annual IEEE International Conference on Computer Communications (INFOCOM 2016)

  10. Tung Y-C, Shin KG (2015) EchoTag: accurate infrastructure-free indoor location tagging with smartphones. In: Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, pp 525–536

  11. Lazik P, Rowe A (2012) Indoor pseudo-ranging of mobile devices using ultrasonic chirps. In: Sensys. ACM, pp 99–112

  12. Chen H, Li F, Wang Y (2016) EchoLoc: Accurate device-free hand localization using COTS devices. In: Proceedings of the 45th International Conference on Parallel Processing (ICPP). IEEE, pp 334–339

  13. Wang K, Yang Z, Zhou Z, Liu Y, Ni L (2015) Ambient rendezvous: Energy-efficient neighbor discovery via acoustic sensing. In: 2015 IEEE Conference on Computer Communications (INFOCOM). IEEE, pp 2704–2712

  14. Sun Z, Purohit A, Bose R, Zhang P (2013) Spartacus: spatially-aware interaction for mobile devices through energy-efficient audio sensing. In: Proceeding of the 11th annual international conference on Mobile systems, applications, and services (MobiSys). ACM, pp 263–276

  15. Zhang Z, Chu D, Chen X, Moscibroda T (2012) Swordfight: Enabling a new class of phone-to-phone action games on commodity phones. In: Proceedings of the 10th international conference on Mobile systems, applications, and services (MobiSys). ACM, pp 1–14

  16. Xu C, Li S, Liu G, Zhang Y, Miluzzo E, Chen Y-F, Li J, Firner B (2013) Crowd++: unsupervised speaker count with smartphones. In: Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. ACM, pp 43–52

  17. Xu C, Li S, Zhang Y, Miluzzo E, Chen Y-F (2014) Crowdsensing the speaker count in the wild: Implications and applications. IEEE Communications Magazine 52(10):92–99

    Article  Google Scholar 

  18. Roy N, Gowda M, Choudhury RR (2015) Ripple: Communicating through physical vibration. In: 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15), pp 265–278

  19. Roy N, Choudhury RR (2016) Ripple ii: faster communication through physical vibration. In: 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16), pp 671–684

  20. Weka, http://www.cs.waikato.ac.nz/ml/weka/. Accessed 10 Aug 2016

  21. Wang J, Zhao K, Zhang X, Peng C (2014) Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization. In: Proceedings of the 12th annual international conference on Mobile systems, applications, and services. ACM, pp 14–27

  22. Li M, Zhou P, Zheng Y, Li Z, Shen G (2015) Iodetector: A generic service for indoor/outdoor detection. ACM Transactions on Sensor Networks (TOSN) 11(2):28

    Google Scholar 

  23. Radu V, Katsikouli P, Sarkar R, Marina MK (2014) A semi-supervised learning approach for robust indoor-outdoor detection with smartphones. In: Proceedings of the 12th ACM Conference on Embedded Network Sensor Systems. ACM, pp 280–294

  24. Park H, Ahn D, Won M, Son SH, Park T (2014) Poster: Are you driving? non-intrusive driver detection using built-in smartphone sensors. In: Proceedings of the 20th annual international conference on Mobile computing and networking. ACM, pp 397–400

  25. Rossi M, Feese S, Amft O, Braune N, Martis S, Tröster G (2013) Ambientsense: a real-time ambient sound recognition system for smartphones. In: Pervasive Computing and Communications Workshops (PERCOM Workshops), 2013 IEEE International Conference on. IEEE, pp 230–235

  26. Antos SA, Albert MV, Kording KP (2014) Hand, belt, pocket or bag: Practical activity tracking with mobile phones. Journal of neuroscience methods 231:22–30

    Article  Google Scholar 

  27. Yang J, Munguia-Tapia E, Gibbs S (2013) Efficient in-pocket detection with mobile phones. In: Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication. ACM, pp 31–34

  28. Bisio I, Delfino A, Lavagetto F (2015) A simple ultrasonic indoor/outdoor detector for mobile devices. In: 2015 International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, pp 137–141

  29. Diaconita I, Reinhardt A, Englert F, Christin D, Steinmetz R (2014) Do you hear what i hear? using acoustic probing to detect smartphone locations. In: Pervasive Computing and Communications Workshops (PERCOM Workshops), 2014 IEEE International Conference on. IEEE, pp 1–9

  30. Tung Y-C, Shin KG (2016) Expansion of human-phone interface by sensing structure-borne sound propagation. In: Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services. ACM, pp 277–289

  31. Findling RD, Mayrhofer R (2015) Towards device-to-user authentication: protecting against phishing hardware by ensuring mobile device authenticity using vibration patterns. In: Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia. ACM, pp 131–135

  32. Anand SA, Saxena N (2016) Vibreaker: Securing vibrational pairing with deliberate acoustic noise. In: Proceedings of the 9th ACM Conference on Security & Privacy in Wireless and Mobile Networks. ACM, pp 103–108

  33. Wang W, Yang L, Zhang Q (2016) Touch-and-guard: secure pairing through hand resonance. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, pp 670–681

  34. Darbar R, Samanta D (2015) Surfacesense: Smartphone can recognize where it is kept. In: Proceedings of the 7th International Conference on HCI, IndiaHCI 2015. ACM, pp 39–46

  35. Nandakumar R, Iyer V, Tan D, Gollakota S (2016) Fingerio: Using active sonar for fine-grained finger tracking. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, pp 1515–1525

  36. Zhao K, Fang D, Chen X, Zhang Y, Nie W, Xing T (2014) Poster: doppler effect based device-free moving object localization. In: Proceedings of the 20th annual international conference on Mobile computing and networking (Mobicom). ACM, pp 441–444

  37. Gupta S, Morris D, Patel S, Tan D (2012) Soundwave: using the doppler effect to sense gestures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp 1911–1914

  38. Yun S, Chen Y-C, Qiu L (2015) Turning a mobile device into a mouse in the air. In: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (Mobisys). ACM, pp 15–29

  39. Liu J, Wang Y, Kar G, Chen Y, Yang J, Gruteser M (2015) Snooping keystrokes with mm-level audio ranging on a single phone. In: Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, pp 142–154

  40. Han H, Yi S, Li Q, Shen G, Liu Y, Novak E (2016) AMIL: localizing neighboring mobile devices through a simple gesture. In: Proc. of IEEE Infocom. IEEE

Download references

Acknowledgements

The work is partially supported by the National Natural Science Foundation of China under Grant No. 61370192, 61432015, 61428203 and 61572347, and the US National Science Foundation under Grant Nos. CNS-1319915 and CNS-1343355.

Author information

Authors and Affiliations

  1. School of Computer Science, Beijing Institute of Technology, Beijing, 100081, China

    Fan Li, Huijie Chen, Xiaoyu Song, Qian Zhang & Youqi Li

  2. Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC, 28223, USA

    Yu Wang

Authors
  1. Fan Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Huijie Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xiaoyu Song
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Qian Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Youqi Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Yu Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Fan Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, F., Chen, H., Song, X. et al. CondioSense: high-quality context-aware service for audio sensing system via active sonar. Pers Ubiquit Comput 21, 17–29 (2017). https://doi.org/10.1007/s00779-016-0981-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-016-0981-1

Keywords