Article

Video search re-ranking via multi-graph propagation

Authors:

Xian-Sheng Hua,

Shipeng LiAuthors Info & Claims

MM '07: Proceedings of the 15th ACM international conference on Multimedia

Pages 208 - 217

https://doi.org/10.1145/1291233.1291279

Published: 29 September 2007 Publication History

Abstract

This paper¹ is concerned with the problem of multimodal fusion in video search. First, we employ an object-sensitive approach to query analysis to improve the baseline result of text-based video search. Then, we propose a PageRank-like graph-based approach to text-based search result re-ranking. To better exploit the underlying relationship between video shots, the proposed re-ranking scheme simultaneously leverages textual relevancy, semantic concept relevancy, and low-level-feature-based visual similarity. In this PageRank-like scheme, we construct a set of graphs with the video shots as vertexes, and the conceptual and visual similarity between video shots as "hyperlinks". A modified topic-sensitive PageRank algorithm is then applied on these graphs to propagate the relevance scores through all related video shots. Experimental results verify the effectiveness of the graph-based propagation approach combined with the object-sensitive query analysis approach, which brings significant improvement to the baseline of text-based video search. Our experimental analysis also indicates that the proposed re-ranking method is highly generic and independent of different query classes, training data, and human interference.

References

[1]

Amir, A. et al. IBM Research TRECVID-2005 video retrieval system. In TRECVID Workshop, Washington DC, 2005.

[2]

Alias-i. Lingpipe named entity tagger. In http://www.alias-i.com/lingpipe/.

[3]

Brown, Peter F., deSouza, Peter V., Mercer, Robert L., Della Pietra, Vincent J., Lai, Jenifer C. Class-Based n-gram Models of Natural Language. Computational Linguistics, 1992

Digital Library

[4]

Chang, Shih-Fu et al. Columbia University TRECVID-2006 Video Search and High-Level Feature Extraction. In TRECVID Workshop, 2006.

[5]

Chang, Shih-Fu et al. Columbia University TRECVID-2005 Video Search and High-Level Feature Extraction. In TRECVID Workshop, 2005.

[6]

Chang, T.-C et al. TRECVID 2004 Search and Feature Extraction Task by NUS PRIS. In TRECVID Workshop, Washington DC, 2004.

[7]

Campbell, M., Haubold, A., Ebadollahi, S., Naphade, Milind R., Natsev, A., Smith, John R., Tesic, J., Xie, L. IBM Research TRECVID-2006 Video Retrieval System. In TRECVID Workshop, 2006.

[8]

CLAWS part-of-speech tagger for English. http://www.comp.lancs.ac.uk/computing/research/ucrel/claws/

[9]

Donald, K. M. and Smeaton, A. F. A Comparison of Score, Rank and Probability-Based Fusion Methods for Video Shot Retrieval. International Conference on Image and Video Retrieval (CIVR), 2005.

Digital Library

[10]

El-Yaniv, R., and Gerzon, L. Effective Transductive Learning via PAC-Bayesian Model Selection, Technical Report CS-2004-05, Technion-Israel Institute of Technology, 2004.

[11]

Fellbaum, Christiane. WordNet: an Electronic Lexical Database, MIT Press, 1998.

[12]

Hsu, W. H. and Chang, S.-F. Topic Tracking Across Broadcast News Videos with Visual Duplicates and Semantic Concepts. In International Conference on Image Processing (ICIP), Atlanta, GA, USA, 2006.

[13]

Hauptmann, A. G. and Christel, M. G. Successful Approaches in the TREC Video Retrieval Evaluations. ACM Multimedia 2004.

Digital Library

[14]

Hauptmann, A.G., Chen, M.-Y., Christel, M., Lin, W.-H., Yan, R., Yang, J. Multi-Lingual Broadcast News Retrieval. TRECVID2006.

[15]

Haveliwala, Taher H. Topic Sensitive PageRank, International World Wide Web Conference (WWW), 2002.

Digital Library

[16]

Hua, X. S., Mei, T., Lai, W., Wang, M., Tang, J., Qi, G. J., Li, L., Gu, Z. Microsoft Research Asia TRECVID 2006 High-Level Feature Extraction and Rushes Exploitation. TRECVID 2006.

[17]

Hauptmann, Alexander G., Lin, W. H., Yan, R., Yang, J. and Chen, M. Y. Extreme Video Retrieval: Joint Maximization of Human and Computer Performance, ACM Multimedia 2006.

Digital Library

[18]

Hsu. W. H. and Chang. S.-F. Visual Cue Cluster Construction via Information Bottleneck Principle and Kernel Density Estimation. In the International Conference on Image and Video Retrieval (CIVR), Singapore, 2005.

Digital Library

[19]

Hsu. Winston H., Kennedy, Lyndon S., Chang, Shih-Fu. Video Search Reranking via Information Bottleneck Principle. ACM Multimedia 2006.

Digital Library

[20]

Herbrich, R., Graepel, T., and Obermayer, K. Support Vector Learning for Ordinal Regression. In Proc. of the 9th International Conference on Artificial Neural Networks, 1999.

[21]

Iyengar, et al. Joint Visual-Text Modeling for Automatic Retrieval of Multimedia Documents, ACM Multimedia 2006.

Digital Library

[22]

Joachims, Thorsten. 2004. SVMlight - Support Vector Machine. http://svmlight.joachims.org/.

[23]

Kennedy, L., Natsev, P., and Chang, S.-F. Automatic Discovery of Query Class Dependent Models for Multimodal Search. In ACM Multimedia, Singapore, 2005.

Digital Library

[24]

Kurland, Oren and Lee, Lillian. PageRank Without Hyperlinks: Structural Re-Ranking Using Links Induced by Language Models, Proceedings of the 28th ACM SIGIR conference on Research and Development in Information Retrieval, 2005.

Digital Library

[25]

Kurland, Oren and Lee, Lillian. Respect My Authority! HITS Without Hyperlinks, Utilizing Cluster-Based Language Models.SIGIR'06

Digital Library

[26]

Liu, J., Hua, X.S., Li, S. Object-Sensitive Query Analysis for Video Search. Multimedia Signal Processing, 2007.

[27]

Lin, Dekang. Automatic Retrieval and Clustering of Similar Words. COLING-ACL'98.

Digital Library

[28]

LSCOM Lexicon Definitions and Annotations (Version 1.0). DTO Challenge Workshop on Large Scale Concept Ontology for Multimedia. http://www.ee.columbia.edu/ln/dvmm/lscom/

[29]

Natsev, A., Naphade, M. R., and Tesic, J. Learning the Semantics of Multimedia Queries and Concepts from a Small Number of Examples. In ACM Multimedia, Singapore, 2005.

Digital Library

[30]

Nie, L., Davison, Brian D., Qi, X. Topical Link Analysis for Web Search. SIGIR'06

Digital Library

[31]

Page, L., Brin, S., Motwani, R., and Winograd, T. The Pagerank Citation Ranking: Bringing Order to the web. Technical report, Stanford University, Stanford, CA, 1998.

[32]

Robertson, S. E. Overview of the Okapi Projects, Journal of Documentation, Vol. 53, No. 1, pp. 3--7, 1997.

[33]

Richardson, M. and Domingos, P. The Intelligent Surfer: Probabilistic Combination of Link and Content Information in PageRank. In Advances in Neural Information Processing Systems 14. MIT Press, 2002.

[34]

Robertson, S. E., and Sparck Jones, K. Relevance Weighting of Search Terms. Journal of the American Society of Information Science, 1976.

[35]

Salton, G., Wong, A., and Yang, C. S. A Vector Space Model for Automatic Indexing, Communications of the ACM, 1975

Digital Library

[36]

Shipman, F., Girgensohn, A., and Wilcox, L. Hyper-Hitchcock: Towards the Easy Authoring of Interactive Video. Human-computer Interaction INTERACT '03, IOS Press, pp. 33--40, September 1, 2003.

[37]

Shipman, F., Girgensohn, A., and Wilcox, L. Combining Spatial and Navigational Structure in the Hyper-Hitchcock Hypervideo Editor. Proceedings of Hypertext '03, pp. 124--125, August 26, 2003.

Digital Library

[38]

Shipman, F., Girgensohn, A., and Wilcox, L. Creating Navigable Multi-Level Video Summaries IEEE International Conference on Multimedia and Expo, v. II, pp. 753--756, 2003.

Digital Library

[39]

Snoek, Cees G. M., Worring, M., van Gemert, Jan C., Geusebroek, J. M., and Smeulders, Arnold W. M. The Challenge Problem for Automated Detection of 101 Semantic Concepts in Multimedia. ACM Multimedia 2006.

Digital Library

[40]

TRECVID. TREC Video Retrieval Evaluation. In http://www-nlpir.nist.gov/projects/trecvid/.

[41]

Ukkonen, E. Algorithms for Approximate String Matching. Information and Control, 1985.

Digital Library

[42]

Vapnik, V. N. Statistical Learning Theory, John Wiley & Sons, 1995.

[43]

Wu, Y., Tseng, Belle. L., Smith, John R. Ontology-based Multi-Classification Learning for Video Concept Detection. International Conference on Multimedia & Expo (ICME), 2004.

[44]

Wang, C., Jing, F., Zhang, L., Zhang, H.J. Image Annotation Refinement using Random Walk with Restarts, ACM Multimedia 2006.

Digital Library

[45]

Zhou, D., Bousquet, O., Lal, T., Weston, J., and Scholkopf, B. Learning with Local and Global Consistency, in Proc. Advances in Neural Information Processing System, 2004.

Cited By

Gasmi KAouadi HTorjmen M(2023)Link-Driven Study to Enhance Text-Based Image Retrieval: Implicit Links Versus Explicit LinksIEEE Access10.1109/ACCESS.2023.330746411(90526-90537)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3307464
Aouadi HKhemakhem MJemaa M(2019)Uncovering Hidden Links Between Images Through Their Textual ContextEnterprise Information Systems10.1007/978-3-030-26169-6_18(370-395)Online publication date: 28-Jul-2019
https://doi.org/10.1007/978-3-030-26169-6_18
Sabetghadam SLupu MBierig RRauber A(2017)A faceted approach to reachability analysis of graph modelled collectionsInternational Journal of Multimedia Information Retrieval10.1007/s13735-017-0145-87:3(157-171)Online publication date: 16-Dec-2017
https://doi.org/10.1007/s13735-017-0145-8
Show More Cited By

Index Terms

Video search re-ranking via multi-graph propagation
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Improving Ranking Consistency for Web Search by Leveraging a Knowledge Base and Search Logs
CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management

In this paper, we propose a new idea called ranking consistency in web search. Relevance ranking is one of the biggest problems in creating an effective web search system. Given some queries with similar search intents, conventional approaches typically ...
Optimizing video search reranking via minimum incremental information loss
MIR '08: Proceedings of the 1st ACM international conference on Multimedia information retrieval

This paper is concerned with video search reranking - the task of reordering the initial ranked documents (video shots) to improve the search performance - in an optimization framework. Conventional supervised reranking approaches empirically convert ...
Video search reranking through random walk over document-level context graph
MM '07: Proceedings of the 15th ACM international conference on Multimedia

Multimedia search over distributed sources often result in recurrent images or videos which are manifested beyond the textual modality. To exploit such contextual patterns and keep the simplicity of the keyword-based search, we propose novel reranking ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '07: Proceedings of the 15th ACM international conference on Multimedia

September 2007

1115 pages

ISBN:9781595937025

DOI:10.1145/1291233

General Chairs:
Rainer Lienhart
University of Augsburg, Germany
,
Anand R. Prasad
DoCoMo Euro-Labs,Germany
,
Program Chairs:
Alan Hanjalic
Delft University of Technology, The Netherlands
,
Sunghyun Choi
Seoul National University, South Korea
,
Brian Bailey
University of Illinois at Urbana-Champaign
,
Nicu Sebe
University of Amsterdam, The Netherlands

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 September 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM07

Sponsor:

MM07: The 15th ACM International Conference on Multimedia 2007

September 25 - 29, 2007

Augsburg, Germany

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
941
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gasmi KAouadi HTorjmen M(2023)Link-Driven Study to Enhance Text-Based Image Retrieval: Implicit Links Versus Explicit LinksIEEE Access10.1109/ACCESS.2023.330746411(90526-90537)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3307464
Aouadi HKhemakhem MJemaa M(2019)Uncovering Hidden Links Between Images Through Their Textual ContextEnterprise Information Systems10.1007/978-3-030-26169-6_18(370-395)Online publication date: 28-Jul-2019
https://doi.org/10.1007/978-3-030-26169-6_18
Sabetghadam SLupu MBierig RRauber A(2017)A faceted approach to reachability analysis of graph modelled collectionsInternational Journal of Multimedia Information Retrieval10.1007/s13735-017-0145-87:3(157-171)Online publication date: 16-Dec-2017
https://doi.org/10.1007/s13735-017-0145-8
Zhang BQu YPeng JFan J(2017)An automatic image-text alignment method for large-scale web image retrievalMultimedia Tools and Applications10.1007/s11042-016-4059-x76:20(21401-21421)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1007/s11042-016-4059-x
Zhang LHong RNie LHong C(2016)A Biologically Inspired Automatic System for Media Quality AssessmentIEEE Transactions on Automation Science and Engineering10.1109/TASE.2015.241822313:2(894-902)Online publication date: Apr-2016
https://doi.org/10.1109/TASE.2015.2418223
Takehara DHarakawa ROgawa THaseyama M(2016)Hierarchical content group detection from different social media platforms using Web link structure2016 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP.2016.7532403(479-483)Online publication date: Sep-2016
https://doi.org/10.1109/ICIP.2016.7532403
Zoidi OFotiadou ENikolaidis NPitas I(2015)Graph-Based Label Propagation in Digital MediaACM Computing Surveys10.1145/270038147:3(1-35)Online publication date: 1-Apr-2015
https://dl.acm.org/doi/10.1145/2700381
Qu YZhang BFan JHauptmann ANgo CXue XJiang YSnoek CVasconcelos N(2015)Parallel AP Clustering and Re-ranking for Automatic Image-Text Alignment and Large-Scale Web Image SearchProceedings of the 5th ACM on International Conference on Multimedia Retrieval10.1145/2671188.2749294(451-454)Online publication date: 22-Jun-2015
https://dl.acm.org/doi/10.1145/2671188.2749294
Yin YYu YZimmermann R(2015)On Generating Content-Oriented Geo Features for Sensor-Rich Outdoor Video SearchIEEE Transactions on Multimedia10.1109/TMM.2015.245804217:10(1760-1772)Online publication date: Oct-2015
https://doi.org/10.1109/TMM.2015.2458042
Zhou NFan J(2015)Automatic image-text alignment for large-scale web image indexing and retrievalPattern Recognition10.1016/j.patcog.2014.07.00148:1(205-219)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.1016/j.patcog.2014.07.001
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten