Skip to main content
SpringerLink
Log in

Multimedia news exploration and retrieval by integrating keywords, relations and visual features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Multimedia news may be organized by the keywords and categories for exploration and retrieval applications, but it is very difficult to integrate the relation and visual information into the traditional category browsing and keyword-based search framework. This paper propose a new semantic model that can integrate keyword, relation and visual information in a uniform framework. Based on this semantic representation framework, the news exploration and retrieval applications can be organized by not only keywords and categories but also relations and visual properties. We also proposed a set of algorithms to automatically extract the proposed semantic model automatically from large collection of multimedia news reports.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. Data cited herein has been extracted from the British National Corpus Online service, managed by Oxford University Computing Services on behalf of the BNC Consortium. All rights in the texts cited are reserved. Please visit http://www.natcorp.ox.ac.uk/XMLedition/ for more information.

References

  1. Barzilay R, Elhadad N, McKeown KR (2002) Inferring strategies for sentence ordering in multidocument news summarization. J Artif Intell Res 17:35–55

    MATH  Google Scholar 

  2. Bollegala D, Okazakia N, Ishizukaa M (2010) A bottom-up approach to sentence ordering for multi-document summarization. Inf Process Manag 46(1):89–109

    Article  Google Scholar 

  3. Bou B (2005) Hyperbolic tree engine, generator, browser. http://treebolic.sourceforge.net/

  4. Carson C, Thomas M, Belongie S, Hellerstein JM, Malik J (1999) Blobworld: a system for region-based image indexing and retrieval. In: International conference on visual information systems, pp 509–516

  5. Chang C-C, Lin C-J (2001) LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/∼cjlin/libsvm

  6. Chang S, Chen W, Sundaram H (1998) Semantic visual templates: linking visual features to semantics. In: IEEE workshop on content based video search and retrieval, Chicago, IL, pp 531–535

  7. Chen Y, Wang JZ (2002) A region-based fuzzy feature matching approach to content-based image retrieval. IEEE Trans Pattern Anal Mach Intell 24(9):1252–1267

    Article  Google Scholar 

  8. Chen Y, Wang JZ (2004) Image categorization by learning and reasoning with regions. J Mach Learn Res 5:913–939

    Google Scholar 

  9. Comanicu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  10. Fan J, Gao Y, Luo H (2008) Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation. IEEE Trans Image Process 17:407–426

    Article  MathSciNet  Google Scholar 

  11. Fauqueur J, Boujemaa N (2004) Region-based image retrieval: fast coarse segmentation and fine color description. J Vis Lang Comput 15(1):69–95

    Article  Google Scholar 

  12. Finkel JR, Grenager T, Manning C (2005) Incorporating non-local information into information extraction systems by gibbs sampling. In: The 43rd annual meeting of the Association for Computational Linguistics (ACL 2005), pp 363–370

  13. Goh K, Li B, Chang EY (2005) Semantics and feature discovery via confidence-based ensemble. ACM TOMCCAP 1(2):168–189

    Article  Google Scholar 

  14. Gong W, Luo H, Fan J (2009) Extracting informative images from web news pages via imbalanced classification. In: ACM multimedia grand challenge, pp 1123–1124

  15. Gupta A, Jain R (1997) Visual information retrieval. Commun ACM 40(5):70–79

    Article  Google Scholar 

  16. Harris J (2004) Tenbyten. http://tenbyten.org/10x10.html

  17. Havre S, Hetzler B, Nowell L (2002) Themeriver: visualizing thematic changes in large document collections. IEEE Trans Vis Comput Graph 8(1):9–20

    Article  Google Scholar 

  18. He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21:1263–1284

    Article  Google Scholar 

  19. Hetzler EG, Whitney P, Martucci L, Thomas J (1998) Multi-faceted insight through interoperable visual information analysis paradigms. In: IEEE symposium on information visualization, p 137

  20. Hoiem D, Sukthankar R, Schneiderman H, Huston L (2004) Object-based image retrieval using the statistical structure of images. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 490–497

  21. In the news (2004) http://stamen.com/projects/inthenews

  22. Khoshgoftaar TM, Van Hulse J, Napolitano A (2007) Experimental perspectives on learning from imbalanced data. In: ICML, vol 227. ACM, New York, pp 935–942

    Google Scholar 

  23. Joachims T (2002) Learning to classify text using support vector machines. Kluwer, Dordrecht

    Google Scholar 

  24. Jones KS (1972) A statistical interpretation of term specificity and its application in retrieval. J Doc 28:11–21

    Article  Google Scholar 

  25. Lafferty JD, McCallum A, Pereira FCN (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: International conference on machine learning, pp 282–289

  26. Lamping J, Rao R (1996) The hyperbolic browser: a focus+context technique based on hyperbolic geometry for visualizing large hierarchies. J Vis Lang Comput 7(1):33–55

    Article  Google Scholar 

  27. Li B, Goh K (2003) Confidence-based dynamic ensemble for image annotation and semantic discovery. In: ACM multimedia, pp 195–206

  28. Louchnikova T, Marchand-Maillet S (2002) Flexible image decomposition for multimedia indexing and retrieval. In: SPIE internet imaging, pp 203–211

  29. Luo H, Fan J (2004) Concept-oriented video skimming and adaptation via semantic classification. In: ACM multimedia workshop on multimedia information retrieval (MIR), pp 213–220

  30. Luo H, Fan J, Yang J, Ribarsky W, Satoh S (2007) Analyzing large-scale news video databases to support knowledge visualization and intuitive retrieval. In: IEEE symposium on visual analytics science and technology

  31. Luo H, Gao Y, Xue X, Peng J, Fan J (2008) Incorporating feature hierarchy and boosting for concept-oriented video summarization and skimming. ACM TOMCCAP 4(1):1–25

    Article  Google Scholar 

  32. Madnani N, Passonneau R, Ayan NF , Conroy JM, Dorr BJ, Klavans JL, O’Leary DP, Schlesinger JD (2007) Measuring variability in sentence ordering for news summarization. In: Eleventh European workshop on natural language generation

  33. McCallum A, Li W (2003) Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In: Natural language learning at HLT-NAACL, pp 188–191

  34. McEnery T, Xiao R (2004) The Lancaster corpus of Mandarin Chinese. http://www.lancs.ac.uk/fass/projects/corpus/LCMC/

  35. Mehler A, Bao Y, Li X, Wang Y, Skiena S (2006) Spatial analysis of news sources. IEEE Trans Vis Comput Graph 12(5):765–772

    Article  Google Scholar 

  36. Radev D, Otterbacher J, Winkel A, Blair-Goldensohn S (2005) Newsinessence: summarizing online news topics. Commun ACM 48(10):95–98

    Article  Google Scholar 

  37. Rubner Y, Tomasi C (1999) Texture-based image retrieval without segmentation. In: IEEE international conference on computer vision (ICCV), pp 1018–1024

  38. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  39. Snoek CGM, Worring M, Hauptmann AG (2006) Learning rich semantics from news video archives by style analysis. ACM TOMCCAP 2:91–108

    Article  Google Scholar 

  40. Swan R, Jensen D (2000) Timemines: constructing timelines with statistical models of word. In: ACM SIGKDD, pp 73–80

  41. Vapnik V (1995) The nature of statistical learning theory. Springer, Berlin

    MATH  Google Scholar 

  42. Wagstaff J (2005) On news visualization. http://www.loosewireblog.com/2005/05/on_news_visuali.html

  43. Walter JA, Ritter H (2002) On interactive visualization of high-dimensional data using the hyperbolic plane. In: ACM SIGKDD

  44. Wang JZ, Li J, Gray RM, Wiederhold G (2001) Unsupervised multiresolution segmentation for images with low depth of field. IEEE Trans Pattern Anal Mach Intell 23(1):85–90

    Article  Google Scholar 

  45. Weskamp M (2004) Newsmap. http://www.marumushi.com/apps/newsmap/index.cfm

  46. Wise JA, Thomas JJ, Pennock K, Lantrip D, Pottier M, Schur A, Crow V (1995) Visualizing the non-visual: spatial analysis and interaction with information from text documents. In: IEEE symposium on information visualization (InfoVis), pp 51–58

  47. Yuan J, Li J, Zhang B (2006) Learning concepts from large scale imbalanced data sets using support cluster machines. In: The 14th annual ACM international conference on multimedia, pp 441–450

  48. Zhu S, Yuille AL (1996) Region competition: unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Trans Pattern Anal Mach Intell 18(9):884–900

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Shanghai Key Laboratory of Trustworthy Computing, East China Normal University, Shanghai, China

    Hangzai Luo

  2. Department of Computer Science, University of North Carolina-Charlotte, Charlotte, NC, USA

    Jianping Fan & Youjie Zhou

Authors
  1. Hangzai Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianping Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Youjie Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hangzai Luo.

Additional information

This work is supported by Shanghai Pujiang Program under 08PJ1404600, NSF-China under 60803077, Shanghai leading academic discipline project under B412 and East China Normal University Science Innovation Fund.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Luo, H., Fan, J. & Zhou, Y. Multimedia news exploration and retrieval by integrating keywords, relations and visual features. Multimed Tools Appl 51, 625–648 (2011). https://doi.org/10.1007/s11042-010-0639-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-010-0639-3

Keywords

Search

Navigation