Skip to main content

Towards hierarchical context: unfolding visual community potential for interactive video retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Community structure as an interesting property of network has attracted wide attention from many research fields. In this paper, we exploit the visual community structure in visual-temporal correlation network and utilize it to improve interactive video retrieval. Firstly, we propose a hierarchical community-based feedback algorithm. By re-ranking the video shots through diffusion processes respectively on the inter-community and intra-community level, the feedback algorithm can make full use of the limited user feedback. Furthermore, since it avoids entire graph computation, the feedback algorithm can make quick responses to user feedback, which is particularly important for the large video collections. Secondly, we propose a community-based visualization interface called VideoMap. By organizing the video shots following the community structure, the VideoMap presents a comprehensive and informative view of the whole dataset to facilitate users’ access. Moreover, the VideoMap can help users to quickly locate the potential relevant regions and make active annotation according to the distribution of labeled samples on the VideoMap. Experiments on TRECVID 2009 search dataset demonstrate the efficiency of the feedback algorithm and the effectiveness of the visualization interface.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Ahn Y-Y, Bagrow JP, Lehmann S (2009) Communities and hierarchical organization of links in complex networks. arXiv:0903.3178

  2. Argyriou A (2004) Efficient approximation methods for harmonic semisupervised learning. Master’s thesis, University College London.

  3. Blei D, Ng A, Jordan M (2003) Latent Dirichlet allocation. J Mach Learn Res 3:993–1022

    Article  MATH  Google Scholar 

  4. Blondel VD, Guillaume J-L, Lambiotte R, Lefebvre E (2008) Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment 2008(10):10 008+

    Article  Google Scholar 

  5. Cao J, Zhang YD, Feng BL, Bao L, Pang L, Li JT (2009) MCG-ICT-CAS TRECVID2009 search task report. In: Proceedings of TRECVID Workshop

  6. Cao J, Zhang YD, Guo JB, Bao L, Li JT (2009) VideoMap: an interactive video retrieval system of MCG-ICT-CAS. In: ACM International Conference on Image and Video Retrieval

  7. Cao J, Jing HF, Ngo CW, Zhang YD (2009) Distribution-based concept selection for concept-based video retrieval. In: ACM International Conference on Multimedia (MM), pp 19–24

  8. Chen Y, Zhou X, and Huang T (2001) One-class SVM for learning in image retrieval. In :Proceedings of the International Conference on Image Processing, Vol. 1,page 34–37.

  9. Cheng X, Dale C, and Liu J (2008) Statistics and Social Networking of YouTube Videos. In: Proc. of IEEE IWQoS, page 229-238

  10. Clauset A, Newman MEJ, Moore C (2004) Finding community structure in very large networks. Physical Review E 70(6):066111

    Article  Google Scholar 

  11. Clauset A, Moore C, Newman MEJ (2008) Hierarchical structure and the prediction of missing links in networks. Nature 453:98–101

    Article  Google Scholar 

  12. Eckmann J-P, Moses E (2002) Curvature of co-links uncovers hidden thematic layers in the World Wide Web. In: Proc National Acad Sciences, 99, 5825

  13. Flake G, Lawrence S, Giles C (2000) Efficient identification of web communities. In: Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 150-160

  14. Flake GW, Lawrence S, Giles CL, Coetzee FM (2002) Self-organization and identification of web communities. IEEE Computer.

  15. Fortunato S (2009) Community Detection in Graphs. arXiv: 0906.0612

  16. Ghosh R, Lerman K (2008) Community detection using a measure of global influence. In: Proceedings of SNAKDD 2008: KDD Workshop on Social Network Mining and Analysis

  17. Girvan M and Newman MEJ (2002) Community Structure in Social and Biological Networks, Proceedings of the National Academy of Sciences, Vol. 99, No. 12

  18. Gonzalez MC, Hidalgo CA et al (2008) Understanding individual human mobility patterns. Nature 453:779–782

    Article  Google Scholar 

  19. Guo G, Jain A, MaWY, and Zhang HJ. (2001) Learning similarity measure for natural image retrieval with relevance feedback. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, Vol 13, No. 4, page 811-820

  20. Hauptmann AG., Lin WH, Yan R, Yang J, Chen MY (2006) Extreme video retrieval: joint maximization of human and computer performance. In: Proceedings of the 14th annual ACM international conference on Multimedia, pp 385–389

  21. He JR, Li MJ, Zhang HJ, Tong HH, and Zhang CS (2004) Manifold-ranking based image retrieval.In: Proceedings of the 12th annual ACM international conference on Multimedia, pp 9–16

  22. Heesch D, Howarth P, Magalhaes J, May A et al. (2004)Video retrieval using search and browsing. In: Proc. TRECVID

  23. Hsu WH, Kennedy LS and Chang SF(2007) Video Search Reranking through Random Walk over Document-Level Context Graph. In: Proceeding of 14th ACM International Conference on Multimedia, page 971–980

  24. Jiang YG, Ngo CW, Yang J (2007) Towards optimal bag-of-features for object categorization and semantic video retrieval. In: CIVR, pp 494–501

  25. Jing YS, Baluja S (2008) VisualRank: applying pagerank to large-scale image search. Transactions on Pattern Analysis and Machine Intelligence

  26. Kamvar SD, Haveliwala TH, Manning CD, Golub GH (2003) Extrapolation methods for accelerating pagerank computations. In International WWW Conference

  27. Kernighan BW, Lin S (1970) An effcient heuristic procedure for partitioning graphs. Bell System Technical Journal

  28. Leskovec J, Lang KJ, Dasgupta A, Mahoney MW (2008) Community structure in large networks: natural cluster sizes and the absence of large well-defined clusters. arxiv;0810.1355

  29. Manevitz L, Yousef M (2004) One-class SVMs for document classification. J Mach Learn Res 2:139–154

    Article  Google Scholar 

  30. Newman M (2006) Modularity and community structure in networks. Proceedings of the National Academy of Sciences of the United States of America 103(23):8577–8582

    Article  Google Scholar 

  31. Newman MEJ (2006) Finding community structure in networks using the eigenvectors of matrices. Physical Review E 74:036104

    Article  MathSciNet  Google Scholar 

  32. Newman MEJ, and Girvan M (2004) Finding and evaluating community structure in networks. Physical Review E, 69(2)

  33. Onnela J-P, Saram¨aki J, Hyv¨onen J et al. (2007) Structure and tie strengths in mobile communication networks.Proceedings of the National Academy of Sciences, Vol 104, 7332–7336

    Google Scholar 

  34. Palla G, Deŕenyi I, Farkas I, Vicsek T (2005) Uncovering the overlapping community structure of complex networks in nature and society. Nature 435:814–818

    Article  Google Scholar 

  35. Palla G, Barabási A-L, Vicsek T (2007) Quantifying social group evolution. Nature 446:664–667

    Article  Google Scholar 

  36. Pan J-Y, et al. (2004) Gcap: graph-based automatic image captioning. In International Workshop on Multimedia Data and Document Engineering

  37. Papadopoulos S, Kompatsiaris Y, Vakali A (2010) A graph-based clustering scheme for identifying related tags in folksonomies. In DaWaK 2010

  38. Papadopoulos S, Zigkolis C, et al (2010) Image clustering through community detection on hybrid image similarity graphs. In ICIP 2010

  39. Pons P, Latapy M (2006) Computing communities in large networks using random walks. Journal of Graph Algorithms and Applications 10(2):191–218

    MathSciNet  MATH  Google Scholar 

  40. Porter MA, Onnela JP, Mucha PJ (2009) Communities in Networks. Notices of the American Mathematical Society, Vol. 56, No. 9

  41. Rautiainen M, Ojala T, Seppänen T (2003) Cluster-temporal video browsing with semantic filtering. In :Proc. Advanced Concepts for Intelligent Vision Systems, pp 116–123

  42. Rautianen M, Ojala T, and Seppänen T (2004) Cluster-temporal browsing of large news video databases. In: IEEE ICME, pp 751–754

  43. Snoek CGM, Worring M, Gemert JV, Geusebroek JM et al. (2005) Mediamill: exploring news video archives based on learned semantics. In: ACM Multimedia, pp 225–226

  44. Tong S. and Chang E. (2001) Support vector machine active learning for image retrieval. In: Proc. 9th ACM Int. Conf. on Multimedia, pp 107–118

  45. TREC Video Retrieval (TRECVID). http://www-nlpir.nist.gov/projects/trecvid/.

  46. Wang M, Mei T, Yuan X, Song Y, Dai LR(2007)Video annotation by graph-based learning with neighborhood similarity. In: Proc. of ACM Multimedia, page 325–328

  47. White S and Smyth P. (2005) A spectral clustering approach to finding communities in graphs. In: Proceedings of the 5th SIAM International Conference on Data Mining, pp 76–84.

  48. Worring M, Snoek CGM, Rooij O, Nguyen GP, and Smeulders A(2007)The MediaMill semantic video search engine. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing

  49. Zhou H and Lipowsky R.(2004) Network Brownian Motion: A New Method to Measure Vertex-Vertex Proximity and to Identify Communities and Subcommunities. LNCS, 3038, 1062–1069.

  50. Zhu X (2005) Semi-supervised learning literature survey. Technical report, University of Wisconsin-Madison

  51. Zhu X, and Ghahramani Z (2002). Learning from labeled and unlabeled data with label propagation (Technical Report CMU-CALD-02-107). Carnegie Mellon University.

Download references

Acknowledgments

This work was supported by the National Basic Research Program of China (973 Program, 2007CB311100), National Nature Science Foundation of China (60902090), Beijing New Star Project on Science & Technology (2007B071), Co-building Program of Beijing Municipal Education Commission.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lin Pang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pang, L., Cao, J., Bao, L. et al. Towards hierarchical context: unfolding visual community potential for interactive video retrieval. Multimed Tools Appl 55, 151–178 (2011). https://doi.org/10.1007/s11042-010-0605-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-010-0605-0

Keywords