research-article

Sparse transfer learning for interactive video search reranking

Authors:

Yong RuiAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 8, Issue 3

Article No.: 26, Pages 1 - 19

https://doi.org/10.1145/2240136.2240139

Published: 06 August 2012 Publication History

Abstract

Visual reranking is effective to improve the performance of the text-based video search. However, existing reranking algorithms can only achieve limited improvement because of the well-known semantic gap between low-level visual features and high-level semantic concepts. In this article, we adopt interactive video search reranking to bridge the semantic gap by introducing user's labeling effort. We propose a novel dimension reduction tool, termed sparse transfer learning (STL), to effectively and efficiently encode user's labeling information. STL is particularly designed for interactive video search reranking. Technically, it (a) considers the pair-wise discriminative information to maximally separate labeled query relevant samples from labeled query irrelevant ones, (b) achieves a sparse representation for the subspace to encodes user's intention by applying the elastic net penalty, and (c) propagates user's labeling information from labeled samples to unlabeled samples by using the data distribution knowledge. We conducted extensive experiments on the TRECVID 2005, 2006 and 2007 benchmark datasets and compared STL with popular dimension reduction algorithms. We report superior performance by using the proposed STL-based interactive video search reranking.

References

[1]

Bian, W. and Tao, D. 2011. Max-Min distance analysis by using sequential SDP relaxation for dimension reduction. IEEE Trans. Patt. Anal. Mach. Intell. 33, 5, 1037--1050.

Digital Library

[2]

Cai, D., He, X., and Han, J. 2005. Using graph model for face analysis. Tech. rep., Computer Science Department, University of Illinois at Urbana-Champaign.

[3]

Cai, D., He, X., and Han, J. 2007a. Semi-supervised discriminant analysis. In Proceedings of the IEEE International Conference on Computer Vision. 1--8.

[4]

Cai D., HE X., and Han J. 2007b. Spectral regression: A unified approach for sparse subspace learning. In Proceedings of ICDM. 73--82.

Digital Library

[5]

Carneiro, G., Chan, A. B., Moreno, P. J., and Vasconcelos, N. 2007. Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans. Patt. Anal. Mach. Intell. 394--410.

Digital Library

[6]

Chang, T. and Kuo, C.-C. 1993. Texture analysis and classification with tree-structured wavelet transform. IEEE Trans. Image Process. 2, 429--441.

Digital Library

[7]

Chang, H. S., Sull, S., and Lee, S. U. 1999. Efficient video indexing scheme for content-based retrieval. IEEE Trans. Circuits Syst. Video Tech. 1269--1279.

Digital Library

[8]

Efron, B., Hastie, T., and Tibshirani, R. 2004. Least angle regression. Ann. Stat. 32, 68--73.

[9]

Fisher, R. A. 1936. The use of multiple measurements in taxonomic problems. Ann. Eugen. 179--188.

[10]

Guan, N., Tao, D., Luo, Z., and Yuan, B. 2011. Manifold Regularized Discriminative Non-negative Matrix Factorization with Fast Gradient Descent. IEEE Trans. Image Proc.

Digital Library

[11]

He, X. and Niyogi, P. 2003. Locality preserving projections. In Adv. Neural Inf. Proc. Syst.

[12]

Hotteling, H. 1933. Analysis of a complex of statistical variables into principal components. J. Educat. Psych. 24, 417--441.

[13]

Hsu, W. H., Kennedy, L. S., and Chang, S.-F. 2006. Video search reranking via information bottleneck principle. In Proceedings of the ACM International Conference on Multimedia. 35--44.

Digital Library

[14]

Hsu, W. H., Kennedy, L. S., and Chang, S.-F. 2007. Video search reranking through random walk over document-level context graph. In Proceedings of the ACM International Conference on Multimedia. 971--980.

Digital Library

[15]

Huang, J., Kumar, S.-R., Mitra, M., Zhu, W.-J., and Zabih, R. 1997. Image indexing using color correlograms. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. 762--768.

Digital Library

[16]

Jeon J., Lavrenko V., and Manmatha R. 2003. Automatic image annotation and retrieval using cross-media relevance models. ACM Special Interest Group on Information Retrieval. 119--126.

[17]

Jing, Y. and Baluja, S. 2008. Visualrank: Applying pagerank to large-scale image search. IEEE Trans. Patt. Anal. Mach. Intell. 30, 1877--1890.

Digital Library

[18]

Kotoulas, L. and Andreadis, I. 2003. Color histogram content based image retrieval and hardware implementation. IEE Proc. Circuits Devices Syst. 387--93.

[19]

Lew, M. S., Sebe, N., Djeraba, C., and Jain R. 2006. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimed. Comput. Commun. Appl. 2, 1--19.

Digital Library

[20]

Lin, Y.-Y., Liu, T.-L., and Chen, H.-T. 2005. Semantic manifold learning for image retrieval. In Proceedings of the ACM International Conference on Multimedia. 06--11.

Digital Library

[21]

Liu, J., Lai, W., Hua, X.-S., Huang, Y., and Li, S. 2007. Video search re-ranking via multi-graph propagation. In Proceedings of the ACM International Conference on Multimedia. 208--217.

Digital Library

[22]

Liu, L., Rui, Y., Sun, L.-F., Yang, B., Zhang, J., and Yang, S.-Q. 2008. Topic mining on web-shared videos. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 2145--2148.

[23]

Ma, W.-Y. and Zhang, H.-J. 1998. Benchmarking of image features for content-based retrieval. In Proceedings of the Record of the 32nd Asilomar Conference on Signals, Systems & Computers, vol. 1, 253--257.

[24]

Neumaier, A. 1998. Solving ill-conditioned and singular linear systems: a tutorial on regularization. SIAM Rev. 40, 636--666.

Digital Library

[25]

Natsev, A., Haubold, A., Tesic, J., Xie, L., and Yan, R. 2007. Semantic concept-based query expansion and re-ranking for multimedia retrieval. In Proceedings of the ACM International Conference on Multimedia. 991--1000.

Digital Library

[26]

Nguyen, G. P. and Worring, M. 2008. Optimization of interactive visual-similarity-based search. ACM Trans. Multimed. Comput. Communicat. Appl. 4.

Digital Library

[27]

Qi, G.-J., Hua, X.-S., Rui, Y., Tang, J., Mei, T., Wang, M., and Zhang, H.-J. 2008. Correlative multilabel video annotation with temporal kernels. ACM Trans. Multimed. Comput. Commun. Appl. 5.

Digital Library

[28]

Robertson, S. E., Walker, S., Hancock-Beaulieu, M., Gatford, M., and Payne A. 1997. Simple, proven approaches to text retrieval. Cambridge University Computer Laboratory, Tech. rep. TR356.

[29]

Rocchio, J. J. 1971. Relevance feedback in information retrieval. In SMART Retrieval System - Experiments in Automatic Document Processing. Prentice-Hall, 313--323.

[30]

Roweis, S. T. and Saul, L. K. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323--2326.

[31]

Rui, Y., Huang, T.-S., and Mehrotra, S. 1997. Content-based image retrieval with relevance feedback in MARS. In Proceedings of the IEEE International Conference on Image Processing. 815--818.

[32]

Rui, Y., Huang, T.-S., Ortega, M., and Mehrotra, S. 1998. Relevance feedback: A power tool in interactive content-based image retrieval. IEEE Trans. Circuits Syst. Video Technol. Special Issue on Segmentation, Description, and Retrieval of Video Content, 8, 644--655.

Digital Library

[33]

Si, S., Tao, D., and Geng, B. 2010. Bregman divergence based regularization for transfer subspace learning. IEEE Trans. Knowl. Data Eng. 22, 7, 929--942.

Digital Library

[34]

Slonim, N. and Tishby, N. 1999. Agglomerative information bottleneck. In Advances in Neural Information Processing Systems.

[35]

Smeulders, A. W., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Trans. Patt. Anal. Mach. Intell. 22, 1349--1380.

Digital Library

[36]

Tao, D., Tang, X., Li, X., and Rui, Y. 2006. Direct kernel biased discriminant analysis: A new content-based image retrieval relevance feedback algorithm. IEEE Trans. Multimed. 716--727.

Digital Library

[37]

Tenenbaum, J. B., de Silva, V., and Langford, J. C. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319--2323.

[38]

TRECVID.TREC Video Retrieval Evaluation. http://www-nlpir.nist.gov/projects/trecvid. Trec measures. In Proceedings of Trec-10. Appendix on common evaluation measures. http://trec.nist.gov/pubs/trec10/appendices/meaures.pdf.

[39]

Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., and Hua, X.-S. 2008. Bayesian video search reranking. In Proceedings of the ACM International Conference on Multimedia. 131--140.

Digital Library

[40]

Tian, X., Tao, D., Hua, X.-S., and Wu, X. 2010. Active reranking for Web image search. IEEE Trans. Image Process. 19, 3, 805--820.

Digital Library

[41]

Yan, S. and Hauptmann, A. G. 2004. Co-retrieval: a boosted reranking approach for video retrieval. In Proceedings of the ACM International Conference on Content-Based Image and Video Retrieval. 60--69.

[42]

Yan, S., Hu, Y., Xu, D., Zhang, B., Zhang, H., and Cheng, Q. 2007. Nonlinear discriminant analysis on embedded manifold. IEEE Trans. Circ. Syst. Video Tech. 17, 4, 468--477.

Digital Library

[43]

Yang, B. 2010. DSI: A model for distributed multimedia semantic indexing and content integration. ACM Trans. Multi. Computing Communi. Appl. 6.

Digital Library

[44]

Zhang, T., Tao, D., Li, X., and Yang, J. 2008. A unifying framework for spectral analysis based dimensionality reduction. In Proceedings of the IEEE International Conference on Neural Networks. 1670--1677.

[45]

Zhang, Z. and Zha, H. 2004. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J. Sci. Comput. 26, 313--338.

Digital Library

[46]

Zhou, X. S. and Huang, T. S. 2001. Small sample learning during multimedia retrieval using biasmap. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Vol. 1. 11--17.

[47]

Zhou, T., Tao, D., and Wu, X. 2011. Manifold elastic net: A unified framework for sparse dimension reduction. Data Mining Knowl. Disc. 22, 3, 340--371.

Digital Library

[48]

Zou, H. and Hastie, T. 2005. Regularization and variable selection via the elastic net. J. Roy. Stat. Soc. B, 67, 301--320.

Cited By

Lv CZhang DGeng SWu ZHuang H(2024)Color Transfer for Images: A SurveyACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363515220:8(1-29)Online publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1145/3635152
Dong SLi YXie MJia QZhang Z(2024)Oil film identification via windspeed interference estimation using hyperspectral dataInternational Journal of Remote Sensing10.1080/01431161.2024.230453745:3(814-847)Online publication date: 28-Jan-2024
https://doi.org/10.1080/01431161.2024.2304537
Jiang JGuo Z(2023)Unsupervised Adversarial Domain Adaptation Regression for Rate of Penetration PredictionSPE Journal10.2118/214680-PA28:05(2604-2618)Online publication date: 5-Apr-2023
https://doi.org/10.2118/214680-PA
Show More Cited By

Index Terms

Sparse transfer learning for interactive video search reranking
1. Information systems
  1. Information retrieval

Recommendations

Bayesian video search reranking
MM '08: Proceedings of the 16th ACM international conference on Multimedia

Content-based video search reranking can be regarded as a process that uses visual content to recover the "true" ranking list from the noisy one generated based on textual information. This paper explicitly formulates this problem in the Bayesian ...
Reranking search results for sparse queries
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management

It is well known that clickthrough data can be used to improve the effectiveness of search results: broadly speaking, a query's past clicks are a predictor of future clicks on documents. However, when a new or unusual query appears, or when a system is ...
Multimodal Fusion for Video Search Reranking

Analysis on click-through data from a very large search engine log shows that users are usually interested in the top-ranked portion of returned search results. Therefore, it is crucial for search engines to achieve high accuracy on the top-ranked ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 8, Issue 3

July 2012

143 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/2240136

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 August 2012

Accepted: 01 March 2011

Revised: 01 December 2010

Received: 01 June 2010

Published in TOMM Volume 8, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

96
Total Citations
View Citations
858
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lv CZhang DGeng SWu ZHuang H(2024)Color Transfer for Images: A SurveyACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363515220:8(1-29)Online publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1145/3635152
Dong SLi YXie MJia QZhang Z(2024)Oil film identification via windspeed interference estimation using hyperspectral dataInternational Journal of Remote Sensing10.1080/01431161.2024.230453745:3(814-847)Online publication date: 28-Jan-2024
https://doi.org/10.1080/01431161.2024.2304537
Jiang JGuo Z(2023)Unsupervised Adversarial Domain Adaptation Regression for Rate of Penetration PredictionSPE Journal10.2118/214680-PA28:05(2604-2618)Online publication date: 5-Apr-2023
https://doi.org/10.2118/214680-PA
Jia QGuo JYang PYang Y(2023)A holistic multi-source transfer learning approach using wearable sensors for personalized daily activity recognitionComplex & Intelligent Systems10.1007/s40747-023-01218-w10:1(1459-1471)Online publication date: 13-Sep-2023
https://doi.org/10.1007/s40747-023-01218-w
Nasfi RBouguila N(2022)A novel feature selection method using generalized inverted Dirichlet-based HMMs for image categorizationInternational Journal of Machine Learning and Cybernetics10.1007/s13042-022-01529-313:8(2365-2381)Online publication date: 14-Mar-2022
https://doi.org/10.1007/s13042-022-01529-3
Ai XSun BChen X(2022)Construction of small sample seismic landslide susceptibility evaluation model based on Transfer Learning: a case study of Jiuzhaigou earthquakeBulletin of Engineering Geology and the Environment10.1007/s10064-022-02601-681:3Online publication date: 23-Feb-2022
https://doi.org/10.1007/s10064-022-02601-6
YOSHIDA SMUNEYASU MOGAWA THASEYAMA M(2020)Heterogeneous-Graph-Based Video Search Reranking Using Topic RelevanceIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences10.1587/transfun.2020SMP0023E103.A:12(1529-1540)Online publication date: 1-Dec-2020
https://doi.org/10.1587/transfun.2020SMP0023
Zheng LLiu GYan CJiang CZhou MLi M(2020)Improved TrAdaBoost and its Application to Transaction Fraud DetectionIEEE Transactions on Computational Social Systems10.1109/TCSS.2020.30170137:5(1304-1316)Online publication date: Oct-2020
https://doi.org/10.1109/TCSS.2020.3017013
Moon JKum SLee S(2019)A Heterogeneous IoT Data Analysis Framework with Collaboration of Edge-Cloud Computing: Focusing on Indoor PM10 and PM2.5 Status PredictionSensors10.3390/s1914303819:14(3038)Online publication date: 10-Jul-2019
https://doi.org/10.3390/s19143038
Wang ZLi YHong RTian X(2019)Eigenvector-Based Distance Metric Learning for Image Classification and RetrievalACM Transactions on Multimedia Computing, Communications, and Applications10.1145/334026215:3(1-19)Online publication date: 20-Aug-2019
https://dl.acm.org/doi/10.1145/3340262
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents