Skip to main content
SpringerLink
Log in

Organizing photographs with geospatial and image semantics

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

The popularity of GPS-equipped gadgets and mapping mashup applications has motivated the growth of geotagged Web resources as well as georeferenced multimedia applications. More and more research attention have been put on mining collaborative knowledge from mass user-contributed geotagged contents. However, little attention has been paid to generating high-quality geographical clusters, which is an important preliminary data-cleaning process for most geographical mining works. Previous works mainly use geotags to derive geographical clusters. Simply using one channel information is not sufficient for generating distinguishable clusters, especially when the location ambiguity problem occurs. In this paper, we propose a two-level clustering framework to utilize both the spatial and the semantic features of photographs for clustering. For the first-level geoclustering phase, we cluster geotagged photographs according to their spatial ties to roughly partition the dataset in an efficient way. Then we leverage the textual semantics in photographs' annotation to further refine the grouping results in the second-level semantic clustering phase. To effectively measure the semantic correlation between photographs, a semantic enhancement method as well as a new term weighting function have been proposed. We also propose a method for automatic parameter determination for the second-level spectral clustering process. Evaluation of our implementation on real georeferenced photograph dataset shows that our algorithm performs well, producing distinguishable geographical cluster with high accuracy and mutual information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The definition of geographical distance in Wikipedia, http://en.wikipedia.org/wiki/Geographical_distance.

References

  1. Kennedy, L.S., Naaman, M.: Generating diverse and representative image search results for landmarks. In: WWW, pp. 297–306 (2008)

  2. Quack, T., Leibe, B., Van Gool, L.J.: World-scale mining of objects and events from community photo collections. In: CIVR, pp. 47–56 (2008)

  3. Crandall, D.J., Backstrom, L., Huttenlocher, D.P., Kleinberg, M.: Mapping the world’s photos. In: WWW, pp. 761–770 (2009)

  4. Serdyukov, P., Murdock, V., van Zwol, R.: Placing flickr photos on a map. In: SIGIR, pp. 484–491 (2009)

  5. Kleban, J., Moxley, E., Xu, J., Manjunath, B.S.: Global annotation on georeferenced photographs. In: CIVR (2009)

  6. Joshi, D., Gallagher, A.C., Yu, J., Luo, J.: Inferring photographic location using geotagged web images. Multimed. Tools Appl. 56, 131–153 (2012)

    Article  Google Scholar 

  7. Joshi, D., Luo, J., Yu, J., Lei, P., Gallagher, A.: Using geotags to derive rich tag-clouds for image annotation. In: Hoi, S.C.H, Luo, J., Boll, S., Xu, D., Jin, R., King, I. (eds.) Social Media Modeling and Computing, pp. 239–256. Springer, London (2011)

  8. Cao, L., Yu, J., Luo, J., Huang, T.S.: Enhancing semantic and geographic annotation of web images via logistic canonical correlation regression. In: Proceedings of the 17th ACM international conference on Multimedia, New York, pp. 125–134 (2009)

  9. Zhang, L., Han, Y., Yang, Y., Song, M., Yan, S., Tian, Q.: Discovering discriminative graphlets for aerial image categories recognition. IEEE Trans. Image Process. 22, 5071–5084 (2013)

    Article  MathSciNet  Google Scholar 

  10. Zhang, L., Gao, Y., Zimmermann, R., Tian, Q., Li, X.: Fusion of multichannel local and global structural cues for photo aesthetics evaluation. IEEE Trans. Image Process. 23, 1419–1429 (2014)

    Article  MathSciNet  Google Scholar 

  11. Zhang, L., Song, M., Zhao, Q., Liu, X., Chen, C.: Probabilistic graphlet transfer for photo cropping. IEEE Trans. Image Process. 22, 802–815 (2013)

    Article  MathSciNet  Google Scholar 

  12. Zhang, L., Gao, Y., Ji, R., Xia, Y., Dai, Q., Li, X.: Actively learning human gaze shifting paths for semantics-aware photo cropping. IEEE Trans. Image Process. 23, 2235–2245 (2014)

    Article  MathSciNet  Google Scholar 

  13. Zhang, L., Song, M., Liu, Z., Liu, X., Bu, J., Chen, C.: Probabilistic graphlet cut: exploiting spatial structure cue for weakly supervised image segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1908–1915 (2013)

  14. Zhang, L., Song, M., Yang, Y., Zhao, Q., Zhao, C., Sebe, N.: Weakly supervised photo cropping. IEEE Trans. Multimed. 16, 94–107 (2014)

    Article  Google Scholar 

  15. Avrithis, Y.S., Kalantidis, Y., Tolias, G., Spyrou, E.: Retrieving landmark and non-landmark images from community photo collections. In: Proceedings of the international conference on Multimedia, New York, pp. 153–162 (2010)

  16. The Panoramio Website. http://www.panoramio.com, Google Panoramio (2007)

  17. Ankerst, M., Breunig, M.M., Kriegel, H.-P., Sander, J.: OPTICS: ordering points to identify the clustering structure. In: SIGMOD, pp. 49–60 (1999)

  18. Toyama, K., Logan, R., Roseway, A.: Geographic location tags on digital images. In: Proceedings of the eleventh ACM international conference on Multimedia, New York, pp. 156–166 (2003)

  19. Jaffe, A., Naaman, M., Tassa, T., Davis, M.: Generating summaries and visualization for large collections of geo-referenced photographs. In: Proceedings of the 8th ACM international workshop on Multimedia information retrieval, New York, pp. 89–98 (2006)

  20. Ahern, S., Naaman, M., Nair, R., Yang, J.H.-I.: World explorer: visualizing aggregate data from unstructured text in geo-referenced collections. In: JCDL, pp. 1–10 (2007)

  21. Popescu, A., Mollic, P.A., Kanellos, L.: ThemExplorer: finding and browsing geo-referenced images. In: CBMI, pp. 576–583 (2008)

  22. Zhu, Z., Shou, L., Mao, K., Chen, G.: Location disambiguation for geo-tagged images. In: SIGIR, pp. 1165–1166 (2011)

  23. Naaman, M., Paepcke, A., Garcia-Molina, H.: From where to what: metadata sharing for digital photographs with geographic coordinates. In: CoopIS/DOA/ODBASE, pp. 196–217 (2003)

  24. Davis, M., King, S., Good, N., Sarvas, R.: From context to content: leveraging context to infer media metadata. In: Proceedings of the 12th annual ACM international conference on Multimedia, New York, pp. 188–195 (2004)

  25. Rattenbury, T., Good, N., Naaman, M.: Towards automatic extraction of event and place semantics from flickr tags. In: SIGIR, pp. 103–110 (2007)

  26. Hoashi, K., Uemukai, T., Matsumoto, K., Takishima, Y.: Constructing a landmark identification system for Geo-tagged photographs based on Web data analysis. In: ICME, pp. 606–609 (2009)

  27. Cao, X., Cong, G., Jensen, C.S.: Mining significant semantic locations from GPS data. PVLDB 3, 1009–1020 (2010)

    Google Scholar 

Download references

Acknowledgments

This work is supported in part by the China 863 Project No. 2013AA040601.

Author information

Authors and Affiliations

  1. Zhejiang University, Hangzhou, China

    Zhu Zhu & Chang Xu

Authors
  1. Zhu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhu Zhu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, Z., Xu, C. Organizing photographs with geospatial and image semantics. Multimedia Systems 23, 53–61 (2017). https://doi.org/10.1007/s00530-014-0426-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-014-0426-5

Keywords

Search

Navigation