Abstract
Smartphone localization is essential to a wide spectrum of applications in the era of mobile computing. The ubiquity of smartphone mobile cameras and surveillance ambient cameras holds promise for offering sub-meter accuracy localization services thanks to the maturity of computer vision techniques. In general, ambient-camera-based solutions are able to localize pedestrians in video frames at fine-grained, but the tracking performance under dynamic environments remains unreliable. On the contrary, mobile-camera-based solutions are capable of continuously tracking pedestrians; however, they usually involve constructing a large volume of image database, a labor-intensive overhead for practical deployment. We observe an opportunity of integrating these two most promising approaches to overcome above limitations and revisit the problem of smartphone localization with a fresh perspective. However, fusing mobile-camera-based and ambient-camera-based systems is non-trivial due to disparity of camera in terms of perspectives, parameters and incorrespondence of localization results. In this article, we propose iMAC, an integrated mobile cameras and ambient cameras based localization system that achieves sub-meter accuracy and enhanced robustness with zero-human start-up effort. The key innovation of iMAC is a well-designed fusing frame to eliminate disparity of cameras including a construction of projection map function to automatically calibrate ambient cameras, an instant crowd fingerprints model to describe user motion patterns, and a confidence-aware matching algorithm to associate results from two sub-systems. We fully implement iMAC on commodity smartphones and validate its performance in five different scenarios. The results show that iMAC achieves a remarkable localization accuracy of 0.68 m, outperforming the state-of-the-art systems by >75%.
- Frank C. Anderson and Marcus G. Pandy. 2001. Dynamic optimization of human walking. J. Biomech. Eng. 123, 5 (2001), 381–390.Google ScholarCross Ref
- Apurva Bedagkar-Gala and Shishir K. Shah. 2014. A survey of approaches and trends in person re-identification. Image Vis. Comput. 32, 4 (2014), 270–286. Google ScholarDigital Library
- Siyuan Cao and He Wang. 2018. Enabling public cameras to talk to the public. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 2 (2018), 1–20. Google ScholarDigital Library
- Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2018. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 1 (2019), 172–186.Google ScholarDigital Library
- Si Chen, Muyuan Li, Kui Ren, Xinwen Fu, and Chunming Qiao. 2015. Rise of the indoor crowd: Reconstruction of building interior view via mobile crowdsourcing. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. 59–71. Google ScholarDigital Library
- Mark Cummins and Paul Newman. 2008. FAB-MAP: Probabilistic localization and mapping in the space of appearance. Int. J. Robot. Res. 27, 6 (2008), 647–665. Google ScholarDigital Library
- Erqun Dong, Jingao Xu, Chenshu Wu, Yunhao Liu, and Zheng Yang. 2019. Pair-Navi: Peer-to-peer indoor navigation with mobile visual SLAM. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM’19). IEEE, 1189–1197.Google ScholarCross Ref
- Liang Dong, Jingao Xu, Guoxuan Chi, Danyang Li, Xinglin Zhang, Jianbo Li, Qiang Ma, and Zheng Yang. 2020. Enabling surveillance cameras to navigate. In Proceedings of the 2020 29th International Conference on Computer Communications and Networks (ICCCN’20). IEEE, 1–10.Google ScholarCross Ref
- Ruipeng Gao, Mingmin Zhao, Tao Ye, Fan Ye, Yizhou Wang, Kaigui Bian, Tao Wang, and Xiaoming Li. 2014. Jigsaw: Indoor floor plan reconstruction via mobile crowdsensing. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking. 249–260. Google ScholarDigital Library
- Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861. Retrieved from https://arxiv.org/abs/1704.04861.Google Scholar
- Wenchao Jiang and Zhaozheng Yin. 2017. Combining passive visual cameras and active IMU sensors for persistent pedestrian tracking. J. Vis. Commun. Image Represent. 48 (2017), 419–431. Google ScholarDigital Library
- David C. Lee, Martial Hebert, and Takeo Kanade. 2009. Geometric reasoning for single image structure recovery. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2136–2143.Google ScholarCross Ref
- Vincent Lepetit, Francesc Moreno-Noguer, and Pascal Fua. 2009. Epnp: An accurate o (n) solution to the pnp problem. Int. J. Comput. Vis. 81, 2 (2009), 155. Google ScholarDigital Library
- Fan Li, Chunshui Zhao, Guanzhong Ding, Jian Gong, Chenxing Liu, and Feng Zhao. 2012. A reliable and accurate indoor localization method using phone inertial sensors. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. 421–430. Google ScholarDigital Library
- Shiqi Li, Chi Xu, and Ming Xie. 2012. A robust O (n) solution to the perspective-n-point problem. IEEE Trans. Pattern Anal. Mach. Intelligence 34, 7 (2012), 1444–1450. Google ScholarDigital Library
- Xiaochen Liu, Yurong Jiang, Puneet Jain, and Kyu-Han Kim. 2018. TAR: Enabling fine-grained targeted advertising in retail stores. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. 323–336. Google ScholarDigital Library
- Justin Gregory Manweiler, Puneet Jain, and Romit Roy Choudhury. 2012. Satellites in our pockets: An object positioning system using smartphones. In Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services. 211–224. Google ScholarDigital Library
- Eitan Marder-Eppstein. 2016. Project tango. In Proceedings of the ACM SIGGRAPH 2016 Real-Time Live! 25–25. Google ScholarDigital Library
- Savvas Papaioannou, Hongkai Wen, Andrew Markham, and Niki Trigoni. 2014. Fusion of radio and camera sensor data for accurate indoor positioning. In Proceedings of the 2014 IEEE 11th International Conference on Mobile Ad Hoc and Sensor Systems. IEEE, 109–117. Google ScholarDigital Library
- Jerry Ratcliffe. 2006. Video Surveillance of Public Places. Citeseer.Google Scholar
- Noah Snavely, Steven M. Seitz, and Richard Szeliski. 2006. Photo tourism: Exploring photo collections in 3D. ACM Trans. Graphics (2006), 835–846. Google ScholarDigital Library
- Takafumi Taketomi, Hideaki Uchiyama, and Sei Ikeda. 2017. Visual SLAM algorithms: A survey from 2010 to 2016. IPSJ Trans. Comput. Vis. Appl. 9, 1 (2017), 1–11.Google ScholarCross Ref
- Jin Teng, Boying Zhang, Junda Zhu, Xinfeng Li, Dong Xuan, and Yuan F. Zheng. 2014. EV-Loc: Integrating electronic and visual signals for accurate localization. IEEE/ACM Trans. Netw. (2014). Google ScholarDigital Library
- Yang Tian, Ruipeng Gao, Kaigui Bian, Fan Ye, Tao Wang, Yizhou Wang, and Xiaoming Li. 2014. Towards ubiquitous indoor localization service leveraging environmental physical features. In Proceedings of the IEEE Annual Conference on Computer Communications (IEEE’14). IEEE, 55–63.Google ScholarCross Ref
- Changchang Wu. 2013. Towards linear-time incremental structure from motion. In Proceedings of the 2013 International Conference on 3D Vision-3DV 2013. IEEE, 127–134. Google ScholarDigital Library
- Chenshu Wu, Jingao Xu, Zheng Yang, Nicholas D. Lane, and Zuwei Yin. 2017. Gain without pain: Accurate wifi-based localization with fingerprint spatial gradient. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 1, 2 (2017), 1–19. Google ScholarDigital Library
- Chenshu Wu, Zheng Yang, and Chaowei Xiao. 2017. Automatic radio map adaptation for indoor localization using smartphones. IEEE Trans. Mobile Comput. 17, 3 (2017), 517–528.Google ScholarCross Ref
- Han Xu, Zheng Yang, Zimu Zhou, Longfei Shangguan, Ke Yi, and Yunhao Liu. 2015. Enhancing wifi-based localization with visual clues. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 963–974. Google ScholarDigital Library
- Han Xu, Zheng Yang, Zimu Zhou, Longfei Shangguan, Ke Yi, and Yunhao Liu. 2016. Indoor localization via multi-modal sensing on smartphones. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 208–219. Google ScholarDigital Library
- Jingao Xu, Hengjie Chen, Kun Qian, Erqun Dong, Min Sun, Chenshu Wu, Li Zhang, and Zheng Yang. 2019. iVR: Integrated vision and radio localization with zero human effort. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1–22. Google ScholarDigital Library
- Jingao Xu, Zheng Yang, Hengjie Chen, Yunhao Liu, Xianchun Zhou, Jinbo Li, and Nicholas Lane. 2018. Embracing spatial awareness for reliable wifi-based indoor location systems. In Proceedings of the IEEE International Conference on Mobile Ad-Hoc and Smart Systems (MASS’18). IEEE, 281–289.Google ScholarCross Ref
- Zheng Yang, Chenshu Wu, and Yunhao Liu. 2012. Locating in fingerprint space: Wireless indoor localization with little human intervention. In Proceedings of the 18th Annual International Conference on Mobile Computing and Networking. 269–280. Google ScholarDigital Library
- Zheng Yang, Chenshu Wu, Zimu Zhou, Xinglin Zhang, Xu Wang, and Yunhao Liu. 2015. Mobility increases localizability: A survey on wireless indoor localization using inertial sensors. Comput. Surv. 47, 3 (2015), 1–34. Google ScholarDigital Library
- Zheng Yang, Zimu Zhou, and Yunhao Liu. 2013. From RSSI to CSI: Indoor localization via channel response. ACM Comput. Surv. 46, 2 (2013), 1–32. Google ScholarDigital Library
- Zuwei Yin, Chenshu Wu, Zheng Yang, and Yunhao Liu. 2017. Peer-to-peer indoor navigation using smartphones. IEEE J. Select. Areas Commun. 35, 5 (2017), 1141–1153.Google ScholarDigital Library
- Zhengyou Zhang. 2000. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 11 (2000), 1330–1334. Google ScholarDigital Library
- Liang Zheng, Yi Yang, and Alexander G. Hauptmann. 2016. Person re-identification: Past, present and future. arXiv:1610.02984. Retrieved from https://arxiv.org/abs/arXiv:1610.02984.Google Scholar
- Yuanqing Zheng, Guobin Shen, Liqun Li, Chunshui Zhao, Mo Li, and Feng Zhao. 2017. Travi-navi: Self-deployable indoor navigation system. IEEE/ACM Trans. Network. 25, 5 (2017), 2655–2669. Google ScholarDigital Library
- Yue Zheng, Yi Zhang, Kun Qian, Guidong Zhang, Yunhao Liu, Chenshu Wu, and Zheng Yang. 2019. Zero-effort cross-domain gesture recognition with Wi-Fi. In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services. 313–325. Google ScholarDigital Library
- Wei Zhong, Huchuan Lu, and Ming-Hsuan Yang. 2012. Robust object tracking via sparsity-based collaborative model. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. 1838–1845. Google ScholarDigital Library
Index Terms
- Enabling Surveillance Cameras to Navigate
Recommendations
SpotFi: Decimeter Level Localization Using WiFi
SIGCOMM'15This paper presents the design and implementation of SpotFi, an accurate indoor localization system that can be deployed on commodity WiFi infrastructure. SpotFi only uses information that is already exposed by WiFi chips and does not require any ...
SpotFi: Decimeter Level Localization Using WiFi
SIGCOMM '15: Proceedings of the 2015 ACM Conference on Special Interest Group on Data CommunicationThis paper presents the design and implementation of SpotFi, an accurate indoor localization system that can be deployed on commodity WiFi infrastructure. SpotFi only uses information that is already exposed by WiFi chips and does not require any ...
Indoor Localization Using FM Signals
The major challenge for accurate fingerprint-based indoor localization is the design of robust and discriminative wireless signatures. Even though WiFi received signal strength indicator (RSSI) signatures are widely available indoors, they vary ...
Comments