Abstract
Multimedia news may be organized by the keywords and categories for exploration and retrieval applications, but it is very difficult to integrate the relation and visual information into the traditional category browsing and keyword-based search framework. This paper propose a new semantic model that can integrate keyword, relation and visual information in a uniform framework. Based on this semantic representation framework, the news exploration and retrieval applications can be organized by not only keywords and categories but also relations and visual properties. We also proposed a set of algorithms to automatically extract the proposed semantic model automatically from large collection of multimedia news reports.
Similar content being viewed by others
Notes
Data cited herein has been extracted from the British National Corpus Online service, managed by Oxford University Computing Services on behalf of the BNC Consortium. All rights in the texts cited are reserved. Please visit http://www.natcorp.ox.ac.uk/XMLedition/ for more information.
References
Barzilay R, Elhadad N, McKeown KR (2002) Inferring strategies for sentence ordering in multidocument news summarization. J Artif Intell Res 17:35–55
Bollegala D, Okazakia N, Ishizukaa M (2010) A bottom-up approach to sentence ordering for multi-document summarization. Inf Process Manag 46(1):89–109
Bou B (2005) Hyperbolic tree engine, generator, browser. http://treebolic.sourceforge.net/
Carson C, Thomas M, Belongie S, Hellerstein JM, Malik J (1999) Blobworld: a system for region-based image indexing and retrieval. In: International conference on visual information systems, pp 509–516
Chang C-C, Lin C-J (2001) LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/∼cjlin/libsvm
Chang S, Chen W, Sundaram H (1998) Semantic visual templates: linking visual features to semantics. In: IEEE workshop on content based video search and retrieval, Chicago, IL, pp 531–535
Chen Y, Wang JZ (2002) A region-based fuzzy feature matching approach to content-based image retrieval. IEEE Trans Pattern Anal Mach Intell 24(9):1252–1267
Chen Y, Wang JZ (2004) Image categorization by learning and reasoning with regions. J Mach Learn Res 5:913–939
Comanicu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619
Fan J, Gao Y, Luo H (2008) Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation. IEEE Trans Image Process 17:407–426
Fauqueur J, Boujemaa N (2004) Region-based image retrieval: fast coarse segmentation and fine color description. J Vis Lang Comput 15(1):69–95
Finkel JR, Grenager T, Manning C (2005) Incorporating non-local information into information extraction systems by gibbs sampling. In: The 43rd annual meeting of the Association for Computational Linguistics (ACL 2005), pp 363–370
Goh K, Li B, Chang EY (2005) Semantics and feature discovery via confidence-based ensemble. ACM TOMCCAP 1(2):168–189
Gong W, Luo H, Fan J (2009) Extracting informative images from web news pages via imbalanced classification. In: ACM multimedia grand challenge, pp 1123–1124
Gupta A, Jain R (1997) Visual information retrieval. Commun ACM 40(5):70–79
Harris J (2004) Tenbyten. http://tenbyten.org/10x10.html
Havre S, Hetzler B, Nowell L (2002) Themeriver: visualizing thematic changes in large document collections. IEEE Trans Vis Comput Graph 8(1):9–20
He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21:1263–1284
Hetzler EG, Whitney P, Martucci L, Thomas J (1998) Multi-faceted insight through interoperable visual information analysis paradigms. In: IEEE symposium on information visualization, p 137
Hoiem D, Sukthankar R, Schneiderman H, Huston L (2004) Object-based image retrieval using the statistical structure of images. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 490–497
In the news (2004) http://stamen.com/projects/inthenews
Khoshgoftaar TM, Van Hulse J, Napolitano A (2007) Experimental perspectives on learning from imbalanced data. In: ICML, vol 227. ACM, New York, pp 935–942
Joachims T (2002) Learning to classify text using support vector machines. Kluwer, Dordrecht
Jones KS (1972) A statistical interpretation of term specificity and its application in retrieval. J Doc 28:11–21
Lafferty JD, McCallum A, Pereira FCN (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: International conference on machine learning, pp 282–289
Lamping J, Rao R (1996) The hyperbolic browser: a focus+context technique based on hyperbolic geometry for visualizing large hierarchies. J Vis Lang Comput 7(1):33–55
Li B, Goh K (2003) Confidence-based dynamic ensemble for image annotation and semantic discovery. In: ACM multimedia, pp 195–206
Louchnikova T, Marchand-Maillet S (2002) Flexible image decomposition for multimedia indexing and retrieval. In: SPIE internet imaging, pp 203–211
Luo H, Fan J (2004) Concept-oriented video skimming and adaptation via semantic classification. In: ACM multimedia workshop on multimedia information retrieval (MIR), pp 213–220
Luo H, Fan J, Yang J, Ribarsky W, Satoh S (2007) Analyzing large-scale news video databases to support knowledge visualization and intuitive retrieval. In: IEEE symposium on visual analytics science and technology
Luo H, Gao Y, Xue X, Peng J, Fan J (2008) Incorporating feature hierarchy and boosting for concept-oriented video summarization and skimming. ACM TOMCCAP 4(1):1–25
Madnani N, Passonneau R, Ayan NF , Conroy JM, Dorr BJ, Klavans JL, O’Leary DP, Schlesinger JD (2007) Measuring variability in sentence ordering for news summarization. In: Eleventh European workshop on natural language generation
McCallum A, Li W (2003) Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In: Natural language learning at HLT-NAACL, pp 188–191
McEnery T, Xiao R (2004) The Lancaster corpus of Mandarin Chinese. http://www.lancs.ac.uk/fass/projects/corpus/LCMC/
Mehler A, Bao Y, Li X, Wang Y, Skiena S (2006) Spatial analysis of news sources. IEEE Trans Vis Comput Graph 12(5):765–772
Radev D, Otterbacher J, Winkel A, Blair-Goldensohn S (2005) Newsinessence: summarizing online news topics. Commun ACM 48(10):95–98
Rubner Y, Tomasi C (1999) Texture-based image retrieval without segmentation. In: IEEE international conference on computer vision (ICCV), pp 1018–1024
Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905
Snoek CGM, Worring M, Hauptmann AG (2006) Learning rich semantics from news video archives by style analysis. ACM TOMCCAP 2:91–108
Swan R, Jensen D (2000) Timemines: constructing timelines with statistical models of word. In: ACM SIGKDD, pp 73–80
Vapnik V (1995) The nature of statistical learning theory. Springer, Berlin
Wagstaff J (2005) On news visualization. http://www.loosewireblog.com/2005/05/on_news_visuali.html
Walter JA, Ritter H (2002) On interactive visualization of high-dimensional data using the hyperbolic plane. In: ACM SIGKDD
Wang JZ, Li J, Gray RM, Wiederhold G (2001) Unsupervised multiresolution segmentation for images with low depth of field. IEEE Trans Pattern Anal Mach Intell 23(1):85–90
Weskamp M (2004) Newsmap. http://www.marumushi.com/apps/newsmap/index.cfm
Wise JA, Thomas JJ, Pennock K, Lantrip D, Pottier M, Schur A, Crow V (1995) Visualizing the non-visual: spatial analysis and interaction with information from text documents. In: IEEE symposium on information visualization (InfoVis), pp 51–58
Yuan J, Li J, Zhang B (2006) Learning concepts from large scale imbalanced data sets using support cluster machines. In: The 14th annual ACM international conference on multimedia, pp 441–450
Zhu S, Yuille AL (1996) Region competition: unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Trans Pattern Anal Mach Intell 18(9):884–900
Author information
Authors and Affiliations
Corresponding author
Additional information
This work is supported by Shanghai Pujiang Program under 08PJ1404600, NSF-China under 60803077, Shanghai leading academic discipline project under B412 and East China Normal University Science Innovation Fund.
Rights and permissions
About this article
Cite this article
Luo, H., Fan, J. & Zhou, Y. Multimedia news exploration and retrieval by integrating keywords, relations and visual features. Multimed Tools Appl 51, 625–648 (2011). https://doi.org/10.1007/s11042-010-0639-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-010-0639-3