Skip to main content
SpringerLink
Log in

A survey on context-aware mobile visual recognition

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

The phenomenal growth of the usage of mobile devices (e.g., mobile phones and tablet PCs) opens up a new service, namely mobile visual recognition, which has been widely used in many areas, such as mobile shopping and augmented reality. The rich contextual information (e.g., location, time and direction information), easily acquired by the mobile devices, provides useful clues to facilitate mobile visual recognition, including speeding up the recognition time and improving the recognition performance. This survey focuses on recent advances in Context-Aware Mobile Visual Recognition (CAMVR) and reviews related work regarding to different contextual information, recognition methods, recognition types, and various application scenarios. Finally, we discuss future research directions in this field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://www.google.com/mobile/goggles.

  2. http://www.snaptell.com.

  3. http://www.kooaba.com.

  4. http://mclab.citi.sinica.edu.tw/dataset/ops62/ops62.html.

  5. https://purl.stanford.edu/rb470rw0983.

  6. https://purl.stanford.edu/vn158kj2087.

  7. https://en.wikipedia.org/wiki/Mobile_search

  8. http://flow.a9.com.

  9. http://pointandfind.nokia.com.

  10. http://www.omoby.com.

  11. http://marketingland.com/mobile-visual-search-begins-bridge-gap-real-digital-world-101673.

References

  1. Ahern, S., Davis, M., Eckles, D., King, S., Naaman, M., Nair, R., Spasojevic, M., Yang, J.: Zonetag: Designing context-aware mobile media capture to increase participation. In: Proceedings of the Pervasive Image Capture and Sharing, 8th Int. Conf. on Ubiquitous Computing, California (2006)

  2. Amlacher, K., Paletta, L.: Geo-indexed object recognition for mobile vision tasks. In: Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services, pp. 371–374. ACM (2008)

  3. Arandjelović, R., Zisserman, A.: Name that sculpture. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, p. 3. ACM (2012)

  4. Arandjelović, R., Zisserman, A.: Three things everyone should know to improve object retrieval. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 2911–2918. IEEE (2012)

  5. Bacha, S., Benblidia, N.: Combining context and content for automatic image annotation on mobile phones. In: IT Convergence and Security (ICITCS), 2013 International Conference on, pp. 1–4. IEEE (2013)

  6. Baeza-Yates, R., Ribeiro-Neto, B., et al.: Modern information retrieval, vol. 463 (1999)

  7. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: Speeded up robust features. In: Computer vision—ECCV 2006, pp. 404–417. Springer, Berlin (2006)

  8. Chandrasekhar, V., Takacs, G., Chen, D., Tsai, S., Grzeszczuk, R., Girod, B.: Chog: Compressed histogram of gradients a low bit-rate feature descriptor. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp. 2504–2511. IEEE (2009)

  9. Chandrasekhar, V.R., Chen, D.M., Tsai, S.S., Cheung, N.M., Chen, H., Takacs, G., Reznik, Y., Vedantham, R., Grzeszczuk, R., Bach, J., et al.: The stanford mobile visual search data set. In: Proceedings of the Second Annual ACM Conference on Multimedia Systems, pp. 117–122. ACM (2011)

  10. Chatzilari, E., Liaros, G., Nikolopoulos, S., Kompatsiaris, Y.: A comparative study on mobile visual recognition. In: Machine Learning and Data Mining in Pattern Recognition, pp. 442–457. Springer, Berlin (2013)

  11. Chen, D.M., Baatz, G., Köser, K., Tsai, S.S., Vedantham, R., Pylvä, T., Roimela, K., Chen, X., Bach, J., Pollefeys, M., et al.: City-scale landmark identification on mobile devices. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 737–744. IEEE (2011)

  12. Chen, D.M., Makar, M., Araujo, A.F., Girod, B.: Interframe coding of global image signatures for mobile augmented reality. In: Data Compression Conference (DCC), 2014, pp. 33–42. IEEE (2014)

  13. Chen, D.M., Tsai, S.S., Chandrasekhar, V., Takacs, G., Singh, J., Girod, B.: Tree histogram coding for mobile image matching. In: Data Compression Conference, 2009. DCC’09., pp. 143–152. IEEE (2009)

  14. Chen, D.M., Tsai, S.S., Vedantham, R., Grzeszczuk, R., Girod, B.: Streaming mobile augmented reality on mobile phones. In: Mixed and Augmented Reality, 2009. ISMAR 2009. 8th IEEE International Symposium on, pp. 181–182. IEEE (2009)

  15. Chen, T., Fan, J., Lu, S.: Context-aware codebook learning for mobile landmark recognition. In: Image Processing (ICIP), 2014 IEEE International Conference on, pp. 3963–3967. IEEE (2014)

  16. Chen, T., Li, Z., Yap, K.H., Wu, K., Chau, L.P.: A multi-scale learning approach for landmark recognition using mobile devices. In: Information, Communications and Signal Processing, 2009. ICICS 2009. 7th International Conference on, pp. 1–4. IEEE (2009)

  17. Chen, T., Lu, S., Fan, J.: Context-aware vocabulary tree for mobile landmark recognition. J. Vis. Commun. Image Represent. 30, 289–298 (2015)

    Article  Google Scholar 

  18. Chen, T., Yap, K.H.: Context-aware discriminative vocabulary learning for mobile landmark recognition. Circuits Syst. Video Technol. IEEE Trans. 23(9), 1611–1621 (2013)

    Article  Google Scholar 

  19. Chen, T., Yap, K.H.: Discriminative bow framework for mobile landmark recognition. Cybern. IEEE Trans. 44(5), 695–706 (2014)

    Article  Google Scholar 

  20. Chen, T., Yap, K.H., Chau, L.P.: Content and context information fusion for mobile landmark recognition. In: Information, Communications and Signal Processing (ICICS) 2011 8th International Conference on, pp. 1–4. IEEE (2011)

  21. Chen, T., Yap, K.H., Chau, L.P.: Integrated content and context analysis for mobile landmark recognition. Circuits Syst. Video Technol. IEEE Trans. 21(10), 1476–1486 (2011)

    Article  Google Scholar 

  22. Chen, T., Yap, K.H., Zhang, D.: Discriminative soft bag-of-visual phrase for mobile landmark recognition. Multimed. IEEE Trans. 16(3), 612–622 (2014)

    Article  Google Scholar 

  23. Chen, W.C., Xiong, Y., Gao, J., Gelfand, N., Grzeszczuk, R.: Efficient extraction of robust image features on mobile devices. In: Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 1–2. IEEE Computer Society (2007)

  24. Cheng, Z., Ren, J., Shen, J., Miao, H.: Building a large scale test collection for effective benchmarking of mobile landmark search. In: Advances in Multimedia Modeling, pp. 36–46. Springer, Berlin (2013)

  25. Chi, H.Y., Chen, C.C., Cheng, W.H., Chen, M.S.: Ubishop: commercial item recommendation using visual part-based object representation. Multimed. Tools Appl. pp. 1–23 (2015)

  26. Chi, H.Y., Cheng, W.H., Chen, M.S., Tsui, A.W.: Mosro: Enabling mobile sensing for realscene objects with grid based structured output learning. In: International Conference on Multimedia Modeling, pp. 207–218. Springer (2014)

  27. Chum, O., Mikulik, A., Perdoch, M., Matas, J.: Total recall II: query expansion revisited. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 889–896. IEEE (2011)

  28. Cushen, G., Nixon, M.S., et al.: Mobile visual clothing search. In: Multimedia and Expo Workshops (ICMEW), 2013 IEEE International Conference on, pp. 1–6. IEEE (2013)

  29. Di, W., Wah, C., Bhardwaj, A., Piramuthu, R., Sundaresan, N.: Style finder: Fine-grained clothing style detection and retrieval. In: Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, pp. 8–13. IEEE (2013)

  30. Duan, L.Y., Ji, R., Chen, J., Yao, H., Huang, T., Gao, W.: Learning from mobile contexts to minimize the mobile location search latency. Signal Process. Image Commun. 28(4), 368–385 (2013)

    Article  Google Scholar 

  31. Duan, L.Y., Ji, R., Chen, Z., Huang, T., Gao, W.: Towards mobile document image retrieval for digital library. Multimed. IEEE Trans. 16(2), 346–359 (2014)

    Article  Google Scholar 

  32. Fritz, G., Seifert, C., Paletta, L.: A mobile vision system for urban detection with informative local descriptors. In: Computer Vision Systems, 2006 ICVS’06. IEEE International Conference on, pp. 30–30. IEEE (2006)

  33. Girod, B., Chandrasekhar, V., Chen, D.M., Cheung, N.M., Grzeszczuk, R., Reznik, Y., Takacs, G., Tsai, S.S., Vedantham, R.: Mobile visual search. Signal Process. Mag. IEEE 28(4), 61–76 (2011)

    Article  Google Scholar 

  34. Guan, T., He, Y., Duan, L., Yang, J., Gao, J., Yu, J.: Efficient bof generation and compression for on-device mobile visual location recognition. MultiMed. IEEE 21(2), 32–41 (2014)

    Article  Google Scholar 

  35. Guan, T., He, Y., Gao, J., Yang, J., Yu, J.: On-device mobile visual location recognition by integrating vision and inertial sensors. Multimed. IEEE Trans. 15(7), 1688–1699 (2013)

    Article  Google Scholar 

  36. Gui, Z., Wang, Y., Liu, Y., Chen, J.: Mobile visual recognition on smartphones. J. Sens. 2013, 1–9 (2013)

    Article  Google Scholar 

  37. Guillaumin, M., Mensink, T., Verbeek, J., Schmid, C.: Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In: Computer Vision, 2009 IEEE 12th International Conference on, pp. 309–316. IEEE (2009)

  38. Hao, J., Wang, G., Seo, B., Zimmermann, R.: Point of interest detection and visual distance estimation for sensor-rich video. Multimed. IEEE Trans. 16(7), 1929–1941 (2014)

    Article  Google Scholar 

  39. Hauptmann, A.G., Christel, M.G.: Successful approaches in the trec video retrieval evaluations. In: Proceedings of the 12th Annual ACM International Conference on Multimedia, pp. 668–675. ACM (2004)

  40. He, J., Feng, J., Liu, X., Cheng, T., Lin, T.H., Chung, H., Chang, S.F.: Mobile product search with bag of hash bits and boundary reranking. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 3005–3012. IEEE (2012)

  41. He, J., Lin, T.H., Feng, J., Chang, S.F.: Mobile product search with bag of hash bits. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 839–840. ACM (2011)

  42. Herranz, L., Xu, R., Jiang, S.: A probabilistic model for food image recognition in restaurants. In: Proceedings of the IEEE ICME (2015)

  43. Houle, M.E., Oria, V., Satoh, S., Sun, J.: Annotation propagation in image databases using similarity graphs. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 10(1), 7 (2013)

    Google Scholar 

  44. Huang, K., Ding, X., Chen, G., Saenko, K.: Automatic mobile photo tagging using context. In: TENCON 2013-2013 IEEE Region 10 Conference (31194), pp. 1–5. IEEE (2013)

  45. Ivanov, I., Vajda, P., Goldmann, L., Lee, J.S., Ebrahimi, T.: Object-based tag propagation for semi-automatic annotation of images. In: Proceedings of the International Conference on Multimedia Information Retrieval, pp. 497–506. ACM (2010)

  46. Järvelin, K., Kekäläinen, J.: Ir evaluation methods for retrieving highly relevant documents. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 41–48. ACM (2000)

  47. Je, S.k., Lee, S., Oh, W.G.: Mobile visual search applications. In: Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV), p. 1. The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp) (2014)

  48. Ji, R., Duan, L.Y., Chen, J., Yao, H., Gao, W.: When codeword frequency meets geographical location. In: Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, pp. 2400–2403. IEEE (2011)

  49. Ji, R., Duan, L.Y., Chen, J., Yao, H., Huang, T., Gao, W.: Learning compact visual descriptor for low bit rate mobile landmark search. In: IJCAI Proceedings-International Joint Conference on Artificial Intelligence, vol. 22, p. 2456 (2011)

  50. Ji, R., Duan, L.Y., Chen, J., Yao, H., Rui, Y., Chang, S.F., Gao, W.: Towards low bit rate mobile visual search with multiple-channel coding. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 573–582. ACM (2011)

  51. Ji, R., Duan, L.Y., Chen, J., Yao, H., Yuan, J., Rui, Y., Gao, W.: Location discriminative vocabulary coding for mobile landmark search. Int. J. Comput. Vis. 96(3), 290–314 (2012)

    Article  MATH  Google Scholar 

  52. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the ACM International Conference on Multimedia, pp. 675–678. ACM (2014)

  53. Kawano, Y., Yanai, K.: Real-time mobile food recognition system. In: Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, pp. 1–7. IEEE (2013)

  54. Kawano, Y., Yanai, K.: Foodcam-256: a large-scale real-time mobile food recognitionsystem employing high-dimensional features and compression of classifier weights. In: Proceedings of the ACM International Conference on Multimedia, pp. 761–762. ACM (2014)

  55. Kawano, Y., Yanai, K.: Foodcam: a real-time mobile food recognition system employing fisher vector. In: MultiMedia Modeling, pp. 369–373. Springer, Berlin (2014)

  56. Kim, D., Hwang, E., Rho, S.: Location-based large-scale landmark image recognition scheme for mobile devices. In: Mobile, Ubiquitous, and Intelligent Computing (MUSIC), 2012 Third FTRA International Conference on, pp. 47–52 (2012)

  57. Kuo, Y.H., Lee, W.Y., Hsu, W.H., Cheng, W.H.: Augmenting mobile city-view image retrieval with context-rich user-contributed photos. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 687–690. ACM (2011)

  58. Lee, Y.H., Kim, B., Kim, H.J.: Photograph indexing and retrieval using combined geo-information and visual features. In: Complex, Intelligent and Software Intensive Systems (CISIS), 2010 International Conference on, pp. 790–793. IEEE (2010)

  59. Li, Y., Lim, J.H.: Outdoor place recognition using compact local descriptors and multiple queries with user verification. In: Proceedings of the 15th International Conference on Multimedia, pp. 549–552. ACM (2007)

  60. Li, Z., Yap, K.H.: Content and context boosting for mobile landmark recognition. Signal Process. Lett. IEEE 19(8), 459–462 (2012)

    Article  Google Scholar 

  61. Li, Z., Yap, K.H.: Context-aware discriminative vocabulary tree learning for mobile landmark recognition. Digital Signal Process. 24, 124–134 (2014)

    Article  Google Scholar 

  62. Li, Z., Yap, K.H., Tan, K.W.: Context-aware mobile image annotation for media search and sharing. Signal Process. Image Commun. 28(6), 624–641 (2013)

    Article  Google Scholar 

  63. Lim, J.H., Li, Y., You, Y., Chevallet, J.P.: Scene recognition with camera phones for tourist information access. In: Multimedia and Expo, 2007 IEEE International Conference on, pp. 100–103. IEEE (2007)

  64. Lin, J., Wu, V.: Tagging content with metadata pre-filtered by context (2013). https://www.google.com/patents/US8370358. US Patent 8,370,358

  65. Liu, H., Li, H., Mei, T., Luo, J.: Accurate sensing of scene geo-context via mobile visual localization. Multimed. Syst. 21(3), 255–265 (2015)

    Article  Google Scholar 

  66. Liu, H., Mei, T., Li, H., Luo, J., Li, S.: Robust and accurate mobile visual localization and its applications. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 9(1s), 51 (2013)

    Google Scholar 

  67. Liu, H., Mei, T., Luo, J., Li, H., Li, S.: Finding perfect rendezvous on the go: accurate mobile visual localization and its applications to routing. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 9–18. ACM (2012)

  68. Liu, S., Song, Z., Liu, G., Xu, C., Lu, H., Yan, S.: Street-to-shop: cross-scenario clothing retrieval via parts alignment and auxiliary set. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 3330–3337. IEEE (2012)

  69. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  70. Mai, W., Dodds, G., Tweed, C. (eds.): A pda-based system for recognizing buildings from user-supplied images. In: Mobile and Ubiquitous Information Access, pp. 143–157. Springer, Berlin (2004)

  71. Maruyama, T., Kawano, Y., Yanai, K.: Real-time mobile recipe recommendation system using food ingredient recognition. In: Proceedings of the 2nd ACM International Workshop on Interactive Multimedia on Mobile and Portable Devices, pp. 27–34. ACM (2012)

  72. Mei, T., Rui, Y., Li, S., Tian, Q.: Multimedia search reranking: a literature survey. ACM Comput. Surv. (CSUR) 46(3), 38 (2014)

    Article  Google Scholar 

  73. Min, W., Xu, C., Xu, M., Xiao, X., Bao, B.K.: Mobile landmark search with 3d models. Multimed. IEEE Trans. 16(3), 623–636 (2014)

    Article  Google Scholar 

  74. Mouine, S., Yahiaoui, I., Verroust-Blondet, A., Joyeux, L., Selmi, S., Goëau, H.: An android application for leaf-based plant identification. In: Proceedings of the 3rd ACM Conference on International Conference on Multimedia Retrieval, pp. 309–310. ACM (2013)

  75. Naaman, M., Nair, R.: Zonetag’s collaborative tag suggestions: What is this person doing in my phone? MultiMed. IEEE 15(3), 34–40 (2008)

    Article  Google Scholar 

  76. Naaman, M., Paepcke, A., Garcia-Molina, H.: From where to what: Metadata sharing for digital photographs with geographic coordinates. In: On the Move to Meaningful Internet Systems 2003: CoopIS, DOA, and ODBASE, pp. 196–217. Springer, Berlin (2003)

  77. Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, vol. 2, pp. 2161–2168. IEEE (2006)

  78. O’Hare N., Gurrin C., Jones G.J., Smeaton A.F. Combination of content analysis and context features for digital photograph retrieval. In: Proceedings of 2nd IEE European Workshop on the Integration of Knowledge, Semantic and Digital Media Technologies, pp. 323–328, IEEE Computer Society, London, UK, Washington, DC, USA, November 29–December 1, 2005

  79. Oliveira, L., Costa, V., Neves, G., Oliveira, T., Jorge, E., Lizarraga, M.: A mobile, lightweight, poll-based food identification system. Pattern Recognit 47(5), 1941–1952 (2014)

    Article  Google Scholar 

  80. Panda, J., Sharma, S., Jawahar, C.: Heritage app: annotating images on mobile phones. In: Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing, p. 3. ACM (2012)

  81. Pei, D., Ji, R., Sun, F., Liu, H.: Estimating viewing angles in mobile street view search. In: Image Processing (ICIP), 2012 19th IEEE International Conference on, pp. 441–444. IEEE (2012)

  82. Proß, B., Schöning, J., Krüger, A.: ipiccer: automatically retrieving and inferring tagged location information from web repositories. In: Proceedings of the 11th International Conference on Human–Computer Interaction with Mobile Devices and Services, p. 69. ACM (2009)

  83. Qin, C., Bao, X., Choudhury, R.R., Nelakuditi, S.: Tagsense: leveraging smartphones for automatic image tagging. Mob. Comput. IEEE Trans. 13(1), 61–74 (2014)

    Article  Google Scholar 

  84. Quack, T., Bay, H., Van Gool, L.: Object recognition for the internet of things. In: Floerkemeier, C., Langheinrich, M., Fleisch, E., Mattern, F., Sarma, S.E. (eds.) The Internet of Things, pp. 230–246. Springer, Berlin (2008)

  85. Ruf, B., Kokiopoulou, E., Detyniecki, M.: Mobile museum guide based on fast sift recognition. In: Detyniecki, M., Leiner, U., Nürnberger, A. (eds.) Adaptive Multimedia Retrieval. Identifying, Summarizing, and Recommending Image and Music, pp. 170–183. Springer, Berlin (2010)

  86. Runge, N., Wenig, D., Malaka, R.: Keep an eye on your photos: automatic image tagging on mobile devices. In: Proceedings of the 16th International Conference on Human–Computer Interaction with Mobile Devices and Services, pp. 513–518. ACM (2014)

  87. Sang, J., Mei, T., Xu, Y.Q., Zhao, C., Xu, C., Li, S.: Interaction design for mobile visual search. Multimed. IEEE Trans. 15(7), 1665–1676 (2013)

    Article  Google Scholar 

  88. Schroth, G., Huitl, R., Abu-Alqumsan, M., Schweiger, F., Steinbach, E.: Exploiting prior knowledge in mobile visual location recognition. In: Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, pp. 2357–2360. IEEE (2012)

  89. Schroth, G., Huitl, R., Chen, D., Abu-Alqumsan, M., Al-Nuaimi, A., Steinbach, E.: Mobile visual location recognition. Signal Process. Mag. IEEE 28(4), 77–89 (2011)

    Article  Google Scholar 

  90. Seifert, C., Paletta, L., Jeitler, A., Hödl, E., Andreu, J.P., Luley, P., Almer, A.: Visual object detection for mobile road sign inventory. In: Mobile Human–Computer Interaction-MobileHCI 2004, pp. 491–495. Springer, Berlin (2004)

  91. Shen, X., Lin, Z., Brandt, J., Wu, Y.: Mobile product image search by automatic query object extraction. In: Computer Vision–ECCV 2012, pp. 114–127. Springer, Berlin (2012)

  92. Sinha, P., Jain, R.: Classification and annotation of digital photos using optical context data. In: Proceedings of the 2008 International Conference on Content-Based Image and Video Retrieval, pp. 309–318. ACM (2008)

  93. Song, X., Jiang, S., Xu, R., Herranz, L.: Semantic features for food image recognition with geo-constraints. In: Data Mining Workshop (ICDMW), 2014 IEEE International Conference on, pp. 1020–1025. IEEE (2014)

  94. Takacs, G., Chandrasekhar, V., Gelfand, N., Xiong, Y., Chen, W.C., Bismpigiannis, T., Grzeszczuk, R., Pulli, K., Girod, B.: Outdoors augmented reality on mobile phone using loxel-based visual feature organization. In: Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval, pp. 427–434. ACM (2008)

  95. Tsai, C.M., Qamra, A., Chang, E.Y., Wang, Y.F.: Extent: inferring image metadata from context and content. In: Multimedia and Expo, 2005. ICME 2005. IEEE International Conference on, pp. 1270–1273. IEEE (2005)

  96. Tsai, S.S., Chen, D., Chandrasekhar, V., Takacs, G., Cheung, N.M., Vedantham, R., Grzeszczuk, R., Girod, B.: Mobile product recognition. In: Proceedings of the International Conference on Multimedia, pp. 1587–1590. ACM (2010)

  97. Tsai, S.S., Chen, D., Takacs, G., Chandrasekhar, V., Singh, J.P., Girod, B.: Location coding for mobile image retrieval. In: Proceedings of the 5th International ICST Mobile Multimedia Communications Conference, p. 8. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering) (2009)

  98. Tsai, T.H., Cheng, W.H., You, C.W., Hu, M.C., Tsui, A.W., Chi, H.Y.: Learning and recognition of on-premise signs from weakly labeled street view images. Image Process. IEEE Trans. 23(3), 1047–1059 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  99. Viana, W., Braga, R., Lemos, F.D., de Souza, J.M., Carmo, R., Andrade, R., Martin, H., et al.: Mobile photo recommendation and logbook generation using context-tagged images. MultiMed. IEEE 21(1), 24–34 (2014)

    Article  Google Scholar 

  100. Xia, J., Gao, K., Zhang, D., Mao, Z.: Geometric context-preserving progressive transmission in mobile visual search. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 953–956. ACM (2012)

  101. Xie, X., Lu, L., Jia, M., Li, H., Seide, F., Ma, W.Y.: Mobile search with multimodal queries. Proc. IEEE 96(4), 589–601 (2008)

    Article  Google Scholar 

  102. Xu, R., Herranz, L., Jiang, S., Wang, S., Song, X., Jain, R.: Geolocalized modeling for dish recognition. Multimed. IEEE Trans. 17(8), 1187–1199 (2015)

    Article  Google Scholar 

  103. Yang, D.S., Lee, Y.H.: Mobile image retrieval using integration of geo-sensing and visual descriptor. In: Network-Based Information Systems (NBiS), 2012 15th International Conference on, pp. 743–748. IEEE (2012)

  104. Yap, K.H., Chen, T., Li, Z., Wu, K.: A comparative study of mobile-based landmark recognition techniques. Intell. Syst. IEEE 25(1), 48–57 (2010)

    Article  Google Scholar 

  105. You, C.W., Cheng, W.H., Tsui, A.W., Tsai, T.H., Campbell, A.: Mobilequeue: an image-based queue card management system through augmented reality phones. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pp. 651–652. ACM (2012)

  106. You, Q., Yuan, J., Wang, J., Guo, P., Luo, J.: Snap n’shop: visual search-based mobile shopping made a breeze by machine and crowd intelligence. In: Semantic Computing (ICSC), 2015 IEEE International Conference on, pp. 173–180. IEEE (2015)

  107. Yu, F.X.: Intelligent query formulation for mobile visual search. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 861–862. ACM (2011)

  108. Yu, F.X., Ji, R., Chang, S.F.: Active query sensing for mobile location search. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 3–12. ACM (2011)

  109. Zamir, A.R., Dehghan, A., Shah, M.: Visual business recognition: a multimodal approach. In: ACM Multimedia, pp. 665–668. Citeseer (2013)

  110. Zhang, C., Zhang, Y., Zhu, X., Xue, Z., Qin, L., Huang, Q., Tian, Q.: Socio-mobile landmark recognition using local features with adaptive region selection. Neurocomputing (2015). doi:10.1016/j.neucom.2014.10.105

  111. Zhang, N., Mei, T., Hua, X.S., Guan, L., Li, S.: Interactive mobile visual search for social activities completion using query image contextual model. In: Multimedia Signal Processing (MMSP), 2012 IEEE 14th International Workshop on, pp. 238–243. IEEE (2012)

  112. Zhu, C., Li, K., Lv, Q., Shang, L., Dick, R.P.: iscope: personalized multi-modality image search for mobile devices. In: Proceedings of the 7th International Conference on Mobile Systems, Applications, and Services, pp. 277–290. ACM (2009)

Download references

Acknowledgments

This work was supported in part by the National Basic Research 973 Program of China under Grant No. 2012CB316400, the National Natural Science Foundation of China under Grant Nos. 61532018, 61322212, 61303160, 61572488 and 61550110505, China Postdoctoral Science Foundation under Grant No. 2016M590135, the National High Technology Research and Development 863 Program of China under Grant No. 2014AA015202. This work is also funded by Lenovo Outstanding Young Scientists Program (LOYS).

Author information

Authors and Affiliations

  1. Key Lab of Intelligent Information Processing, Institute of Computing Technology, CAS, Beijing, 100190, China

    Weiqing Min, Shuqiang Jiang, Shuhui Wang, Ruihan Xu & Luis Herranz

  2. Higher Education Institution Teacher Online Training Center, Beijing, China

    Yushan Cao

  3. Lenovo Corporate Research, Beijing, 100085, China

    Zhiqiang He

Authors
  1. Weiqing Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuqiang Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuhui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruihan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yushan Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Luis Herranz
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zhiqiang He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuqiang Jiang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Min, W., Jiang, S., Wang, S. et al. A survey on context-aware mobile visual recognition. Multimedia Systems 23, 647–665 (2017). https://doi.org/10.1007/s00530-016-0523-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-016-0523-8

Keywords

Search

Navigation