skip to main content
research-article

Smartphone-based Acoustic Indoor Space Mapping

Published: 05 July 2018 Publication History

Abstract

Constructing a map of indoor space has many important applications, such as indoor navigation, VR/AR, construction, safety, facility management, and network condition prediction. Existing indoor space mapping requires special hardware (e.g., indoor LiDAR equipment) and well-trained operators. In this paper, we develop a smartphone-based indoor space mapping system that lets a regular user quickly map an indoor space by simply walking around while holding a phone in his/her hand. Our system accurately measures the distance to nearby reflectors, estimates the user's trajectory, and pairs different reflectors the user encounters during the walk to automatically construct the contour. Using extensive evaluation, we show our contour construction is accurate: the median errors are 1.5 cm for a single wall and 6 cm for multiple walls (due to longer trajectory and the higher number of walls). We show that our system provides a median error of 30 cm and a 90-percentile error of 1 m, which is significantly better than the state-of-the-art smartphone acoustic mapping system BatMapper [64], whose corresponding errors are 60 cm and 2.5 m respectively, even after multiple walks. We further show that the constructed indoor contour can be used to predict wireless received signal strength (RSS).

References

[1]
2017. Affordable Scanning LiDAR for everyone. http://scanse.io. {Online} Last Accessed: 11/30/2017.
[2]
2017. AirLoc: Pedestrian dead reckoning for passenger localization. http://www.es.ewi.tudelft.nl/msc-theses/2017-deMoes.pdf. {Online} Last Accessed: 11/30/2017.
[3]
2017. Android ARCore. https://developers.google.com/ar/. {Online} Last Accessed: 11/30/2017.
[4]
2017. Apple ARKit. https://developer.apple.com/arkit/. {Online} Last Accessed: 11/30/2017.
[5]
2017. AR MeasureKit. https://itunes.apple.com/app/id1258270451. {Online} Last Accessed: 11/30/2017.
[6]
2017. Background noise reduction: one of your smartphone's greatest tools. http://www.techradar.com/news/phone-and-communications/mobile-phones/background-noise-reduction-one-of-your-smartphone-s-greatest-tools-1229667. {Online} Last Accessed: 11/30/2017.
[7]
2017. Bumblebee Camera. https://www.ptgrey.com/bumblebee2-firewire-stereo-vision-camera-systems. {Online} Last Accessed: 30/11/2017.
[8]
2017. Cook's Distance. http://bit.ly/2n2iQtv. {Online} Last Accessed: 11/30/2017.
[9]
2017. Housecraft App. https://itunes.apple.com/us/app/housecraft/id1261483849. {Online} Last Accessed: 11/30/2017.
[10]
2017. IKEA Place App. https://itunes.apple.com/us/app/ikea-place/id1279244498. {Online} Last Accessed: 11/30/2017.
[11]
2017. IPhone X. www.apple.com/iphone-x/. {Online} Last Accessed: 11/30/2017.
[12]
2017. Leica DISTO E7100i 200ft Laser Distance Measure with Bluetooth. http://amzn.to/2m2WSr4. {Online} Last Accessed: 11/30/2017.
[13]
2017. Lenovo Phab Series Smartphones. https://www3.lenovo.com/us/en/smart-devices/-lenovo-smartphones/phab-series/c/phab-series. {Online} Last Accessed: 11/30/2017.
[14]
2017. Microsoft Hololens. https://www.microsoft.com/microsoft-hololens/en-us. {Online} Last Accessed: 11/30/2017.
[15]
2017. Oculus VR Rift headset. https://www.oculus.com/rift/. {Online} Last Accessed: 11/30/2017.
[16]
2017. PMD{Vision} Camcube Depth Camera. https://www.youtube.com/watch?v=f0bYnxy74fE. {Online} Last Accessed: 30/11/2017.
[17]
2017. Portrait mode on the Pixel 2 and Pixel 2 XL smartphones. https://research.googleblog.com/2017/10/portrait-mode-on-pixel-2-and-pixel-2-xl.html. {Online} Last Accessed: 30/11/2017.
[18]
2017. Project Tango. https://get.google.com/tango/. {Online} Last Accessed: 11/30/2017.
[19]
2017. Sample compass code. https://github.com/iutinvg/compass. {Online} Last Accessed: 11/30/2017.
[20]
2017. THe average walking stride length. http://www.livestrong.com/article/206902-how-to-reset-a-new-balance-sport-pedometer/. {Online} Last Accessed: 11/30/2017.
[21]
Moustafa Alzantot and Moustafa Youssef. 2012. CrowdInside: Automatic Construction of Indoor Floorplans. In SIGSPATIAL '12. ACM, New York, NY, USA, 99--108.
[22]
Moustafa Alzantot and Moustafa Youssef. 2012. CrowdInside: Automatic Construction of Indoor Floorplans. In SIGSPATIAL '12. ACM, New York, NY, USA, 99--108.
[23]
S. Ayub, B. M. Heravi, A. Bahraminasab, and B. Honary. 2012. Pedestrian Direction of Movement Determination Using Smartphone. In 2012 Sixth International Conference on Next Generation Mobile Applications, Services and Technologies. 64--69.
[24]
Martin Azizyan, Ionut Constandache, and Romit Roy Choudhury. 2009. SurroundSense: Mobile Phone Localization via Ambience Fingerprinting. In MobiCom '09. 261--272.
[25]
Richard H. Bartels, John C. Beatty, and Brian A. Barsky. 1987. An Introduction to Splines for Use in Computer Graphics 8 Geometric Modeling. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
[26]
Si Chen, Muyuan Li, Kui Ren, and Chunming Qiao. 2015. Crowd Map: Accurate Reconstruction of Indoor Floor Plans from Crowdsourced Sensor-Rich Videos. In ICDCS 2015, Columbus, OH, USA. 1--10.
[27]
S. Chen, M. Li, K. Ren, and C. Qiao. 2015. Crowd Map: Accurate Reconstruction of Indoor Floor Plans from Crowdsourced Sensor-Rich Videos. In 2015 IEEE 35th International Conference on Distributed Computing Systems. 1--10.
[28]
R. Dennis Cook. 1979. Influential observations in linear regression. J. Amer. Statist. Assoc. 74, 365 (1979), 169--174.
[29]
Graham Daniel, Simmons George, T.Nguyen David, and Zhou Gang. 2015. A Software Based Sonar Ranging Sensor for Smart Phones. IoT (2015).
[30]
Ivan Dokmanića, Reza Parhizkara, Andreas Walthera, Yue M. Lub, and Martin Vetterlia. 2013. Acoustic echoes reveal room shape. Proceedings of National Academy of Sciences (PNAS) (2013).
[31]
Jiang Dong, Yu Xiao, Marius Noreikis, Zhonghong Ou, and Antti Ylä-Jääski. 2015. iMoon: Using Smartphones for Image-based Indoor Navigation. In SenSys. ACM, 85--97.
[32]
Jorge Fuentes-Pacheco, José Ruiz-Ascencio, and Juan Manuel Rendón-Mancha. 2015. Visual Simultaneous Localization and Mapping: A Survey. Artif. Intell. Rev. 43, 1 (2015), 55--81.
[33]
J. H. Gao and Li-Shiuan Peh. 2016. A smartphone-based laser distance sensor for outdoor environments. In 2016 IEEE International Conference on Robotics and Automation (ICRA). 2922--2929.
[34]
Ruipeng Gao, Mingmin Zhao, Tao Ye, Fan Ye, Yizhou Wang, Kaigui Bian, Tao Wang, and Xiaoming Li. 2014. Jigsaw: Indoor Floor Plan Reconstruction via Mobile Crowdsensing. In MobiCom '14. ACM, 249--260.
[35]
Ravi Garg, Vijay Kumar B. G, and Ian D. Reid. 2016. Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue. CoRR abs/1603.04992 (2016). arXiv:1603.04992 http://arxiv.org/abs/1603.04992
[36]
Daniel Graham, Gang Zhou, Ed Novak, and Jeffrey Buffkin. 2016. A Smartphone Compatible SONAR Ranging Attachment for 2-D Mapping. IoT (2016).
[37]
Daniel Halperin, Wenjun Hu, Anmol Sheth, and David Wetherall. 2011. Tool Release: Gathering 802.11n Traces with Channel State Information. ACM SIGCOMM CCR 41, 1 (Jan. 2011), 53.
[38]
Yifei Jiang, Yun Xiang, Xin Pan, Kun Li, Qin Lv, Robert P. Dick, Li Shang, and Michael Hannigan. 2013. Hallway Based Automatic Indoor Floorplan Construction Using Room Fingerprints. In UbiComp '13. ACM, 315--324.
[39]
Yifei Jiang, Yun Xiang, Xin Pan, Kun Li, Qin Lv, Robert P. Dick, Li Shang, and Michael Hannigan. 2013. Hallway Based Automatic Indoor Floorplan Construction Using Room Fingerprints. In UbiComp '13. ACM, New York, NY, USA, 315--324.
[40]
Kourosh Khoshelham and Er Oude Elberink. 2012. Accuracy and resolution of kinect depth data for indoor mapping applications. In Sensors, 12, 1437--1454. 2013. 8238.
[41]
Kourosh Khoshelham and Er Oude Elberink. 2013. Accuracy and resolution of kinect depth data for indoor mapping applications. In Sensors 2012, 12, 1437--1454. 8238.
[42]
Miranda Krekovic, Ivan Dokmanic, and Martin Vetterli. 2016. EchoSLAM: Simultaneous localization and mapping with acoustic echoes. In ICASSP 2016, Shanghai, China, March 20-25, 2016. IEEE, 11--15.
[43]
Lawrence Berkeley National Laboratory, United States. Department of Energy. Office of Scientific, and Technical Information. 2001. The National Human Activity Pattern Survey (NHAPS): A Resource for Assessing Exposure to Environmental Pollutants. Lawrence Berkeley National Laboratory. https://books.google.com/books?id=njh3AQAACAAJ
[44]
Kaikai Liu, Xinxin Liu, and Xiaolin Li. 2013. Guoguo: Enabling Fine-grained Indoor Localization via Smartphone. In MobiSys '13. 235--248.
[45]
Wenguang Mao, Jian He, and Lili Qiu. 2016. High-precision Acoustic Motion Tracking. In MobiCom '16. 491--492.
[46]
Yuya Murata, Katsuhiko Kaji, Kei Hiroi, and Nobuo Kawaguchi. 2014. Pedestrian Dead Reckoning Based on Human Activity Sensing Knowledge. In UbiComp '14 Adjunct. ACM, New York, NY, USA, 797--806.
[47]
Rajalakshmi Nandakumar, Shyamnath Gollakota, and Nathaniel Watson. 2015. Contactless Sleep Apnea Detection on Smartphones. In MobiSys '15. ACM, New York, NY, USA, 45--57.
[48]
Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. FingerIO: Using Active Sonar for Fine-Grained Finger Tracking. In CHI '16. 1515--1525.
[49]
Yasuharu Onishi, Jun Kuroda, Yukio Murata, Motoyoshi Komoda, Kazuyuki Tsunoda, Yukio Yokoyama, Yasuhiro Oikawa, and Yoshio Yamasaki. 2010. The acoustical design of mobile phones. Vol. 2. 1250--1257.
[50]
Chunyi Peng, Guobin Shen, Yongguang Zhang, Yanlin Li, and Kun Tan. 2007. BeepBeep: A High Accuracy Acoustic Ranging System Using COTS Mobile Devices. In SenSys '07. ACM, New York, NY, USA, 1--14.
[51]
Nirupam Roy, He Wang, and Romit Roy Choudhury. 2014. I am a smartphone and I can tell my users walking direction. In In Proceedings of ACM International Conference onMobile Systems, Applications, and Services (MobiSys).
[52]
R. Scheibler, I. Dokmanić, and M. Vetterli. 2015. Raking echoes in the time domain. In ICASSP '15. 554--558.
[53]
Wen-Yuah Shih, Liang-Yu Chen, and Kun-Chan Lan. 2012. Estimating Walking Distance with a Smart Phone. In PAAP '12. IEEE Computer Society, Washington, DC, USA, 166--171.
[54]
Yuanchao Shu, Kang G. Shin, Tian He, and Jiming Chen. 2015. Last-Mile Navigation Using Smartphones. In MobiCom '15. ACM, New York, NY, USA, 512--524.
[55]
Keith N. Snavely. 2009. Scene Reconstruction and Visualization from Internet Photo Collections. Ph.D. Dissertation. Seattle, WA, USA. Advisor(s) Seitz, Steve and Szeliski, Rick.
[56]
Arno Solin, Santiago Cortes, Esa Rahtu, and Juho Kannala. 2017. Inertial Odometry on Handheld Smartphones. CoRR abs/1703.00154 (2017).
[57]
Hartmut Surmann, Andreas Nüchter, and Joachim Hertzberg. 2003. An autonomous mobile robot with a 3D laser range finder for 3D exploration and digitalization of indoor environments. Robotics and Autonomous Systems 3--4 (2003), 181--198.
[58]
Juan D. Tardós, José Neira, Paul M. Newman, and John J. Leonard. 2002. Robust Mapping and Localization in Indoor Environments Using Sonar Data. I. J. Robotic Res. 21, 4 (2002), 311--330.
[59]
He Wang, Souvik Sen, Ahmed Elgohary, Moustafa Farid, Moustafa Youssef, and Romit Roy Choudhury. 2012. No Need to War-drive: Unsupervised Indoor Localization. In MobiSys '12. ACM, New York, NY, USA, 197--210.
[60]
Tianben Wang, Daqing Zhang, Yuanqing Zheng, Tao Gu, Xingshe Zhou, and Bernadette Dorizzi. 2017. C-FMCW Based Contactless Respiration Detection Using Acoustic Signal. IMWUT 1, 4 (2017), 170:1--170:20.
[61]
Wei Wang, Alex X. Liu, and Ke Sun. 2016. Device-free Gesture Tracking Using Acoustic Signals. In MobiCom '16. ACM, New York, NY, USA, 82--94.
[62]
Sangki Yun, Yi-Chao Chen, and Lili Qiu. 2015. Turning a Mobile Device into a Mouse in the Air. In MobiSys '15. 15--29.
[63]
Huanle Zhang, Wan Du, Pengfei Zhou, Mo Li, and Prasant Mohapatra. 2016. DopEnc: Acoustic-based Encounter Profiling Using Smartphones. In MobiCom '16. 294--307.
[64]
Bing Zhou, Mohammed Elbadry, Ruipeng Gao, and Fan Ye. 2017. BatMapper: Acoustic Sensing Based Indoor Floor Plan Construction Using Smartphones. In MobiSys '17. ACM, New York, NY, USA, 42--55.
[65]
Pengfei Zhou, Mo Li, and Guobin Shen. 2014. Use It Free: Instantly Knowing Your Phone Attitude. In MobiCom '14. ACM, 605--616.

Cited By

View all
  • (2025)ALCDNet: Loop Closure Detection Based on Acoustic EchoesIEEE Robotics and Automation Letters10.1109/LRA.2024.351990610:2(1473-1480)Online publication date: Feb-2025
  • (2025)An echolocation method for mapping using parametric acoustic arrayMeasurement10.1016/j.measurement.2025.116734(116734)Online publication date: Jan-2025
  • (2024)Human–AI Collaboration for Remote Sighted Assistance: Perspectives from the LLM EraFuture Internet10.3390/fi1607025416:7(254)Online publication date: 18-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 2, Issue 2
June 2018
741 pages
EISSN:2474-9567
DOI:10.1145/3236498
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2018
Accepted: 01 April 2018
Revised: 01 February 2018
Received: 01 November 2017
Published in IMWUT Volume 2, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Acoustic Sensing
  2. FMCW
  3. Mapping
  4. Smartphone

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)105
  • Downloads (Last 6 weeks)8
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)ALCDNet: Loop Closure Detection Based on Acoustic EchoesIEEE Robotics and Automation Letters10.1109/LRA.2024.351990610:2(1473-1480)Online publication date: Feb-2025
  • (2025)An echolocation method for mapping using parametric acoustic arrayMeasurement10.1016/j.measurement.2025.116734(116734)Online publication date: Jan-2025
  • (2024)Human–AI Collaboration for Remote Sighted Assistance: Perspectives from the LLM EraFuture Internet10.3390/fi1607025416:7(254)Online publication date: 18-Jul-2024
  • (2024)SyncEcho: Echo-Based Single Speaker Time Offset Estimation for Time-of-Flight LocalizationProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699369(718-729)Online publication date: 4-Nov-2024
  • (2024)Ubiquitous Indoor Mapping Using Mobile Radio TomographyIEEE Transactions on Mobile Computing10.1109/TMC.2024.344243923:12(14031-14043)Online publication date: Dec-2024
  • (2024)Indoor Location Identification for Smart Speakers Leveraging 3-D Acoustic ImagesIEEE Transactions on Mobile Computing10.1109/TMC.2024.338016223:12(10720-10733)Online publication date: Dec-2024
  • (2024)Indoor Smartphone SLAM With Acoustic EchoesIEEE Transactions on Mobile Computing10.1109/TMC.2023.332339323:6(6634-6649)Online publication date: Jun-2024
  • (2024)Robust Indoor Location Identification for Smartphones Using Echoes From Dominant ReflectorsIEEE Transactions on Mobile Computing10.1109/TMC.2023.3307695(1-17)Online publication date: 2024
  • (2024)Orientation Estimation Piloted by Deep Reinforcement Learning2024 IEEE/ACM Ninth International Conference on Internet-of-Things Design and Implementation (IoTDI)10.1109/IoTDI61053.2024.00016(134-145)Online publication date: 13-May-2024
  • (2024)Composite material performance testing and indoor decoration visualization simulation based on sensor image recognitionThe International Journal of Advanced Manufacturing Technology10.1007/s00170-024-14964-zOnline publication date: 26-Dec-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media