Skip to main content
SpringerLink
Log in

ISR: indoor shop recognition via user-friendly and efficient fingerprinting on smartphones

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Indoor shop recognition can not only help mobile users to quickly recognize a shop to know about the information of interest without needing to enter the shop, but also assist in achieving more accurate user localization in a shopping mall. However, the existing Wi-Fi fingerprint-based approaches or image-based approaches cannot accomplish this goal well due to a huge cost of constructing large-scale fingerprint database and poor accuracy. In order to address these issues, we proposed a user-friendly and efficient fingerprinting method to collect various valuable sensory data with smartphones, which can not only reduce the randomness of fingerprints and the negative impact of pedestrians in image matching, but also be used to derive the user-to-shop distance based on the perspective projection model for assisting in determining an accurate fingerprint searching scope. We also proposed an efficient fingerprint searching and matching method to improve the recognition accuracy. We implemented a prototype system and collected fingerprint datasets in a shopping mall. Extensive experiments demonstrate that our solution achieves promising results in realistic scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Azizyan, M., Constandache, I., Roy Choudhury, R.: Surroundsense: mobile phone localization via ambience fingerprinting. In: Proceedings of ACM MobiCom, pp. 261–272 (2009)

  2. Bahl, P., Padmanabhan, V.N.: Radar: an in-building RF-based user location and tracking system. In: Proceedings of IEEE INFOCOM, pp. 775–784 (2000)

  3. Chen, D.M., Baatz, G., Koser, K., Tsai, S.S., Vedantham, R., Pylvanainen, T., Roimela, K., Chen, X., Bach, J., Pollefeys, M., et al.: City-scale landmark identification on mobile devices. In: Proceedings of IEEE CVPR, pp. 737–744 (2011)

  4. Chon, Y., Lane, N.D., Li, F., Cha, H., Zhao, F.: Automatically characterizing places with opportunistic crowdsensing using smartphones. In: Proceedings of ACM UbiComp, pp. 481–490 (2012)

  5. Debnath, H., Borcea, C.: Tagpix: Automatic real-time landscape photo tagging for smartphones. In: Proceedings of IEEE MobilWare, pp. 176–184 (2013)

  6. Fornaciari, M., Prati, A., Grana, C., Cucchiara, R.: Lightweight sign recognition for mobile devices. In: Proceedings of IEEE ICDSC, pp. 1–6 (2013)

  7. Gao, G., Zhang, H., Chen, H.: A robust video text extraction and recognition approach using OCR feedback information. In: Proceedings of PCM, pp. 507–517 (2015)

  8. Guo, B., Chen, H., Han, Q., Yu, Z., Zhang, D., Wang, Y.: Worker-contributed data utility measurement for visual crowdsensing systems. IEEE Trans. Mob. Comput. (2016). doi:10.1109/MC.2016.2620980

  9. Guo, B., Wang, Z., Yu, Z., Wang, Y., Yen, N.Y., Huang, R., Zhou, X.: Mobile crowd sensing and computing: the review of an emerging human-powered sensing paradigm. ACM Comput. Surv. (CSUR) 48(1), 7 (2015)

    Article  Google Scholar 

  10. Hilsenbeck, S., Bobkov, D., Schroth, G., Huitl, R., Steinbach, E.: Graph-based data fusion of pedometer and WiFi measurements for mobile indoor positioning. In: Proceedings of ACM UbiComp, pp. 147–158 (2014)

  11. Hossain, A.M., Jin, Y., Soh, W.S., Van, H.N.: SSD: a robust RF location fingerprint addressing mobile devices’ heterogeneity. IEEE Trans. Mob. Comput. 12(1), 65–77 (2013)

    Article  Google Scholar 

  12. Kjærgaard, M.B.: Indoor location fingerprinting with heterogeneous clients. Pervasive Mob. Comput. 7(1), 31–43 (2011)

    Article  Google Scholar 

  13. Li, F., Zhao, C., Ding, G., Gong, J., Liu, C., Zhao, F.: A reliable and accurate indoor localization method using phone inertial sensors. In: Proceedings of ACM UbiComp, pp. 421–430 (2012)

  14. Liu, W., Ma, H., Qi, H., Zhao, D., Chen, Z.: Deep learning hashing for mobile visual search. EURASIP J. Image Video Process. (2017). doi:10.1186/s13640-017-0167-4

  15. Liu, W., Mei, T., Zhang, Y.: Instant mobile video search with layered audio–video indexing and progressive transmission. IEEE Trans. Multimed. 16(8), 2242–2255 (2014)

    Article  Google Scholar 

  16. Ma, H., Zhao, D., Yuan, P.: Opportunities in mobile crowd sensing. IEEE Commun. Mag. 52(8), 29–35 (2014)

  17. Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: Proceedings of IEEE CVPR, pp. 2161–2168 (2006)

  18. Park, J.g., Curtis, D., Teller, S., Ledlie, J.: Implications of device diversity for organic localization. In: Proceedings of IEEE INFOCOM, pp. 3182–3190 (2011)

  19. Rai, A., Chintalapudi, K.K., Padmanabhan, V.N., Sen, R.: Zee: zero-effort crowdsourcing for indoor localization. In: Proceedings of ACM MobiCom, pp. 293–304 (2012)

  20. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: an efficient alternative to sift or surf. In: Proceedings of IEEE ICCV, pp. 2564–2571 (2011)

  21. Schindler, G., Brown, M., Szeliski, R.: City-scale location recognition. In: Proceedings of IEEE CVPR, pp. 1–7 (2007)

  22. Schroth, G., Huitl, R., Chen, D., Abu-Alqumsan, M., Al-Nuaimi, A., Steinbach, E.: Mobile visual location recognition. IEEE Signal Process. Mag. 28(4), 77–89 (2011)

    Article  Google Scholar 

  23. Shen, G., Chen, Z., Zhang, P., Moscibroda, T., Zhang, Y.: Walkie-markie: indoor pathway mapping made easy. In: Proceedings of USENIX NSDI, pp. 85–98 (2013)

  24. Sun, W., Liu, J., Wu, C., Yang, Z., Zhang, X., Liu, Y.: Moloc: on distinguishing fingerprint twins. In: Proceedings of IEEE ICDCS, pp. 226–235 (2013)

  25. Wang, H., Zhao, D., Ma, H., Xu, H., Hou, X.: Crowdsourcing based mobile location recognition with richer fingerprints from smartphone sensors. In: Proceedings of IEEE ICPADS, pp. 156–163 (2015)

  26. Wang, J., Fang, D., Yang, Z., Jiang, H., Chen, X., Cai, L., et al.: E-hipa: an energy-efficient framework for high-precision multi-target-adaptive device-free localization. IEEE Trans. Mob. Comput. 16(3), 716–729 (2017)

  27. Xu, H., Yang, Z., Zhou, Z., Shangguan, L., Yi, K., Liu, Y.: Enhancing wifi-based localization with visual clues. In: Proceedings of ACM UbiComp, pp. 963–974 (2015)

  28. Xu, H., Zhao, D., An, J., Liu, L.: Indoor shop recognition via simple but efficient fingerprinting on smartphones. In: Proceedings of IEEE CCIS (2016)

  29. Yang, Z., Wu, C., Liu, Y.: Locating in fingerprint space: wireless indoor localization with little human intervention. In: Proceedings of ACM MobiCom, pp. 269–280 (2012)

  30. Yang, Z., Wu, C., Zhou, Z., Zhang, X., Wang, X., Liu, Y.: Mobility increases localizability: a survey on wireless indoor localization using inertial sensors. ACM Comput. Surv. (CSUR) 47(3), 54 (2015)

    Article  Google Scholar 

  31. Youssef, M., Agrawala, A.: The Horus WLAN location determination system. In: Proceedings of ACM MobiSys, pp. 205–218 (2005)

  32. Yu, Z., Xu, H., Yang, Z., Guo, B.: Personalized travel package with multi-point-of-interest recommendation based on crowdsourced user footprints. IEEE Trans. Hum. Mach. Syst. 46(1), 151–158 (2016)

    Article  Google Scholar 

  33. Zhang, X., Yang, Z., Sun, W., Liu, Y., Tang, S., Xing, K., Mao, X.: Incentives for mobile crowd sensing: a survey. IEEE Commun. Surv. Tutor. 18(1), 54–67 (2016)

    Article  Google Scholar 

  34. Zhao, D., Li, X.Y., Ma, H.: Budget-feasible online incentive mechanisms for crowdsourcing tasks truthfully. IEEE/ACM Trans. Netw. 24(2), 647–661 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant Nos. 61502051 and 61332005, the Funds for Creative Research Groups of China under Grant No. 61421061, the Cosponsored Project of Beijing Committee of Education, and the Beijing Training Project for the Leading Talents in S&T (ljrc201502).

Author information

Authors and Affiliations

  1. Beijing Key Lab of Intelligent Telecommunications Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Dong Zhao, Huaiyu Xu, Jiaqi An, Liang Liu & Huadong Ma

Authors
  1. Dong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huaiyu Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiaqi An
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Huadong Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dong Zhao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, D., Xu, H., An, J. et al. ISR: indoor shop recognition via user-friendly and efficient fingerprinting on smartphones. Machine Vision and Applications 28, 781–791 (2017). https://doi.org/10.1007/s00138-017-0838-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-017-0838-2

Keywords

Search

Navigation