Abstract
Community structure as an interesting property of network has attracted wide attention from many research fields. In this paper, we exploit the visual community structure in visual-temporal correlation network and utilize it to improve interactive video retrieval. Firstly, we propose a hierarchical community-based feedback algorithm. By re-ranking the video shots through diffusion processes respectively on the inter-community and intra-community level, the feedback algorithm can make full use of the limited user feedback. Furthermore, since it avoids entire graph computation, the feedback algorithm can make quick responses to user feedback, which is particularly important for the large video collections. Secondly, we propose a community-based visualization interface called VideoMap. By organizing the video shots following the community structure, the VideoMap presents a comprehensive and informative view of the whole dataset to facilitate users’ access. Moreover, the VideoMap can help users to quickly locate the potential relevant regions and make active annotation according to the distribution of labeled samples on the VideoMap. Experiments on TRECVID 2009 search dataset demonstrate the efficiency of the feedback algorithm and the effectiveness of the visualization interface.
Similar content being viewed by others
References
Ahn Y-Y, Bagrow JP, Lehmann S (2009) Communities and hierarchical organization of links in complex networks. arXiv:0903.3178
Argyriou A (2004) Efficient approximation methods for harmonic semisupervised learning. Master’s thesis, University College London.
Blei D, Ng A, Jordan M (2003) Latent Dirichlet allocation. J Mach Learn Res 3:993–1022
Blondel VD, Guillaume J-L, Lambiotte R, Lefebvre E (2008) Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment 2008(10):10 008+
Cao J, Zhang YD, Feng BL, Bao L, Pang L, Li JT (2009) MCG-ICT-CAS TRECVID2009 search task report. In: Proceedings of TRECVID Workshop
Cao J, Zhang YD, Guo JB, Bao L, Li JT (2009) VideoMap: an interactive video retrieval system of MCG-ICT-CAS. In: ACM International Conference on Image and Video Retrieval
Cao J, Jing HF, Ngo CW, Zhang YD (2009) Distribution-based concept selection for concept-based video retrieval. In: ACM International Conference on Multimedia (MM), pp 19–24
Chen Y, Zhou X, and Huang T (2001) One-class SVM for learning in image retrieval. In :Proceedings of the International Conference on Image Processing, Vol. 1,page 34–37.
Cheng X, Dale C, and Liu J (2008) Statistics and Social Networking of YouTube Videos. In: Proc. of IEEE IWQoS, page 229-238
Clauset A, Newman MEJ, Moore C (2004) Finding community structure in very large networks. Physical Review E 70(6):066111
Clauset A, Moore C, Newman MEJ (2008) Hierarchical structure and the prediction of missing links in networks. Nature 453:98–101
Eckmann J-P, Moses E (2002) Curvature of co-links uncovers hidden thematic layers in the World Wide Web. In: Proc National Acad Sciences, 99, 5825
Flake G, Lawrence S, Giles C (2000) Efficient identification of web communities. In: Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 150-160
Flake GW, Lawrence S, Giles CL, Coetzee FM (2002) Self-organization and identification of web communities. IEEE Computer.
Fortunato S (2009) Community Detection in Graphs. arXiv: 0906.0612
Ghosh R, Lerman K (2008) Community detection using a measure of global influence. In: Proceedings of SNAKDD 2008: KDD Workshop on Social Network Mining and Analysis
Girvan M and Newman MEJ (2002) Community Structure in Social and Biological Networks, Proceedings of the National Academy of Sciences, Vol. 99, No. 12
Gonzalez MC, Hidalgo CA et al (2008) Understanding individual human mobility patterns. Nature 453:779–782
Guo G, Jain A, MaWY, and Zhang HJ. (2001) Learning similarity measure for natural image retrieval with relevance feedback. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, Vol 13, No. 4, page 811-820
Hauptmann AG., Lin WH, Yan R, Yang J, Chen MY (2006) Extreme video retrieval: joint maximization of human and computer performance. In: Proceedings of the 14th annual ACM international conference on Multimedia, pp 385–389
He JR, Li MJ, Zhang HJ, Tong HH, and Zhang CS (2004) Manifold-ranking based image retrieval.In: Proceedings of the 12th annual ACM international conference on Multimedia, pp 9–16
Heesch D, Howarth P, Magalhaes J, May A et al. (2004)Video retrieval using search and browsing. In: Proc. TRECVID
Hsu WH, Kennedy LS and Chang SF(2007) Video Search Reranking through Random Walk over Document-Level Context Graph. In: Proceeding of 14th ACM International Conference on Multimedia, page 971–980
Jiang YG, Ngo CW, Yang J (2007) Towards optimal bag-of-features for object categorization and semantic video retrieval. In: CIVR, pp 494–501
Jing YS, Baluja S (2008) VisualRank: applying pagerank to large-scale image search. Transactions on Pattern Analysis and Machine Intelligence
Kamvar SD, Haveliwala TH, Manning CD, Golub GH (2003) Extrapolation methods for accelerating pagerank computations. In International WWW Conference
Kernighan BW, Lin S (1970) An effcient heuristic procedure for partitioning graphs. Bell System Technical Journal
Leskovec J, Lang KJ, Dasgupta A, Mahoney MW (2008) Community structure in large networks: natural cluster sizes and the absence of large well-defined clusters. arxiv;0810.1355
Manevitz L, Yousef M (2004) One-class SVMs for document classification. J Mach Learn Res 2:139–154
Newman M (2006) Modularity and community structure in networks. Proceedings of the National Academy of Sciences of the United States of America 103(23):8577–8582
Newman MEJ (2006) Finding community structure in networks using the eigenvectors of matrices. Physical Review E 74:036104
Newman MEJ, and Girvan M (2004) Finding and evaluating community structure in networks. Physical Review E, 69(2)
Onnela J-P, Saram¨aki J, Hyv¨onen J et al. (2007) Structure and tie strengths in mobile communication networks.Proceedings of the National Academy of Sciences, Vol 104, 7332–7336
Palla G, Deŕenyi I, Farkas I, Vicsek T (2005) Uncovering the overlapping community structure of complex networks in nature and society. Nature 435:814–818
Palla G, Barabási A-L, Vicsek T (2007) Quantifying social group evolution. Nature 446:664–667
Pan J-Y, et al. (2004) Gcap: graph-based automatic image captioning. In International Workshop on Multimedia Data and Document Engineering
Papadopoulos S, Kompatsiaris Y, Vakali A (2010) A graph-based clustering scheme for identifying related tags in folksonomies. In DaWaK 2010
Papadopoulos S, Zigkolis C, et al (2010) Image clustering through community detection on hybrid image similarity graphs. In ICIP 2010
Pons P, Latapy M (2006) Computing communities in large networks using random walks. Journal of Graph Algorithms and Applications 10(2):191–218
Porter MA, Onnela JP, Mucha PJ (2009) Communities in Networks. Notices of the American Mathematical Society, Vol. 56, No. 9
Rautiainen M, Ojala T, Seppänen T (2003) Cluster-temporal video browsing with semantic filtering. In :Proc. Advanced Concepts for Intelligent Vision Systems, pp 116–123
Rautianen M, Ojala T, and Seppänen T (2004) Cluster-temporal browsing of large news video databases. In: IEEE ICME, pp 751–754
Snoek CGM, Worring M, Gemert JV, Geusebroek JM et al. (2005) Mediamill: exploring news video archives based on learned semantics. In: ACM Multimedia, pp 225–226
Tong S. and Chang E. (2001) Support vector machine active learning for image retrieval. In: Proc. 9th ACM Int. Conf. on Multimedia, pp 107–118
TREC Video Retrieval (TRECVID). http://www-nlpir.nist.gov/projects/trecvid/.
Wang M, Mei T, Yuan X, Song Y, Dai LR(2007)Video annotation by graph-based learning with neighborhood similarity. In: Proc. of ACM Multimedia, page 325–328
White S and Smyth P. (2005) A spectral clustering approach to finding communities in graphs. In: Proceedings of the 5th SIAM International Conference on Data Mining, pp 76–84.
Worring M, Snoek CGM, Rooij O, Nguyen GP, and Smeulders A(2007)The MediaMill semantic video search engine. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing
Zhou H and Lipowsky R.(2004) Network Brownian Motion: A New Method to Measure Vertex-Vertex Proximity and to Identify Communities and Subcommunities. LNCS, 3038, 1062–1069.
Zhu X (2005) Semi-supervised learning literature survey. Technical report, University of Wisconsin-Madison
Zhu X, and Ghahramani Z (2002). Learning from labeled and unlabeled data with label propagation (Technical Report CMU-CALD-02-107). Carnegie Mellon University.
Acknowledgments
This work was supported by the National Basic Research Program of China (973 Program, 2007CB311100), National Nature Science Foundation of China (60902090), Beijing New Star Project on Science & Technology (2007B071), Co-building Program of Beijing Municipal Education Commission.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pang, L., Cao, J., Bao, L. et al. Towards hierarchical context: unfolding visual community potential for interactive video retrieval. Multimed Tools Appl 55, 151–178 (2011). https://doi.org/10.1007/s11042-010-0605-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-010-0605-0