Skip to main content
SpringerLink
Log in

Enhancement of Textual Images Classification Using Segmented Visual Contents for Image Search Engine

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper deals with the use of the dependencies between the textual indexation of an image (a set of keywords) and its visual indexation (colour and shape features). Experiments are realized on a corpus of photographs of a press agency (EDITING) and on another corpus of animals and landscape photographs (COREL). Both are manually indexed by keywords. Keywords of the news photos are extracted from a hierarchically structured thesaurus. Keywords of Corel corpus are semantically linked using WordNet database. A semantic clustering of the photos is constructed from their textual indexation. We use two different visual segmentation schemes. One is based on areas of interest, the other one on blobs of homogenous colour. Both segmentation schemes are used to evaluate the performance of a content-based image retrieval system combining textual and visual descriptions. Results of visuo-textual classifications show an improvement of 50% against classification using only textual information. Finally, we show how to apply this system in order to enhance a web image search engine. To this purpose, we illustrate a method allowing selecting only accurate images resulting from a textual query.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R. Baeza-Yates and B. Ribeiro-Neto Modern Information Retrieval, Addison Wesley, 1999.

  2. K. Barnard 2003, http://vision.cs.arizona.edu/kobus/research/data/jmlr

  3. K. Barnard, P. Duygulu, N. de Freitas, D. Forsyth, D. Blei, and M.I. Jordan “Matching words and pictures,” in Journal of Machine Learning Research, Vol. 3, pp. 1107–1135, 2003.

    Google Scholar 

  4. M. Bouet and A. Khenchaf “Traitement de l’information multimédia: Recherche de média image,” Ingénierie des systèmes d’information (RSTI série ISI-NIS), Vol. 7, Nos. 5/6, pp. 65–90, 2002.

    Google Scholar 

  5. E. Bruno, J. Le Maitre, and E. Murisasco “Indexation et interrogation de photos de presse décrites en MPEG-7 et stockées dans une base de données XML, Ingénierie des systèmes d’information (RSTI série ISI-NIS), Vol. 7, No. 5/6, pp. 69–186, 2002.

    Google Scholar 

  6. J. Canny, A computational approach to edge detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-8, No. 6, pp. 679–698, 1986.

  7. M.L. Cascia, S. Sethi, and S. Sclaroff “Combining textual and visual cues for content-based image retrieval on the world wide web,” Technical Report 1998 004, 9, 1998.

  8. V. Castelli and L.D. Bergman (Eds.), Image Databases, John Wiley & Sons, 2002.

  9. C. Fellbaum (Ed.), WordNet—An Electronic Lexical Database, Bradford Books, 1998.

  10. H. Glotin “Elaboration et étude comparative de systèmes adaptatifs multi-flux de reconnaissance robuste de la parole : incorporation d’indices de voisement et de localisation,” Phd thesis, ICP/Institut National Polytechnique de Grenoble & IDIAP/EPF Lausanne, Grenoble, 2001.

  11. G.N. Lance and W.T. Williams “A general theory of classificatory sorting strategies: I. hierarchical systems,” Computer Journal, Vol. 9, pp. 373–380, 1967.

    Google Scholar 

  12. Y. Li, C.C. Jay Kuo, and X. Wan “Introduction to content-based image retrieval overview of key techniques,” in V. Castelli and L. D. Bergman, (eds.), Image Databases, chapter 10, John Wiley & Sons, 2002, pp. 261–284.

  13. B.S. Manjunath, P. Salembier, and T. Sikora (eds), Introduction to MPEG-7. John Wiley & Sons, 2002.

  14. J. Martinet, Y. Chiaramella, and P. Mulhem “Un modèle vectoriel étendu de recherche d’informations adapté aux images,” Actes du XXème Congrès INFORSID, pp. 337–348, 4-7 juin 2002.

  15. H. Müller, S. Marchand-Maillet, and T. Pun “The truth about corel—evaluation in image retrieval,” in The Challenge of Image and Video Retrieval (CIVR2002), 2002.

  16. C. Nastar “Indexation d’images par le contenu : un état de l’art,” Actes de CORESA’97, 1997.

  17. W. Niblack “The QBIC project: Querying images by content using color, texture and shape,” in Proceedings SPIE: Storage and Retrieval for Image and Video Database, 1993, pp. 173–181.

  18. G. Salton, The SMART Retrieval System; Experiments in Automatic Document Processing,” Englenwood Cliffs: Prenctice-Hall, New Jersey, 1971.

  19. G. Salton and C. Buckley “Term-weighting approaches in automatic retrieval,” Information Processing and Management, Vol. 24, No. 5, pp. 513–523, 1988.

    Google Scholar 

  20. G. Salton and M.J. Lesk “Computer evaluation of indexing and text-processing,” Journal of the ACM, Vol. 15, No. 1, pp. 8–36, 1968.

    Google Scholar 

  21. J. Shi and J. Malik “Normalized cuts and image segmentation,” IEEE Transactions on Pattern Analysis and machine Intelligence, Vol. 22, No. 8, pp. 888–905, 2000.

    Google Scholar 

  22. S. Tollari, H. Glotin, and J. Le Maitre “Rehaussement de la classification textuelle d’images par leur contenu visuelle, Actes du 14ième Congrès Francophone AFRIF-AFIA de Reconnaissance des Formes et Intelligence Artificielle, 2004.

  23. X.S. Zhou and T.S. Huang “Unifying keywords and visual contents in image retrieval,” IEEE Multimedia, 2002.

Download references

Author information

Authors and Affiliations

  1. Laboratoire SIS - Equipe informatique, Université du Sud Toulon-Var, Bâtiment R, BP 20132, F-83957, LA GARDE cedex, France

    Sabrina Tollari, Hervé Glotin & Jacques Le Maitre

Authors
  1. Sabrina Tollari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hervé Glotin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jacques Le Maitre
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sabrina Tollari.

Additional information

Sabrina Tollari received the B.S. and M.S. degrees in computer science, respectively, from the University of Toulon in 2001 and from the University of Marseilles in 2003. She is currently a Ph.D. student at the SIS laboratory (Systèmes-Information-Signal). Her main current research interests are multimedia information retrieval.

Hervé Glotin received his Bachelor in computer science from the University of Paris VI. He got his Ph.D. in 2001 in Cognitive Sciences from the National Polytechnic Institute of Grenoble, France, dealing with robust automatic audiovisual speech recognition and computational auditory scene analysis. He shared his Ph.D. between two laboratories: IDIAP (EPFL-CH), and ICP (CNRS-Grenoble). In 2000 he was invited to the CSLP summer workshop as an expert working with the human language IBM team on ‘via voice’ audiovisual system. In 2001 he was involved as a time life engineer/researcher at CNRS in ERSS laboratory, Toulouse-France, specialized in semantic and syntax language analysis. In 2002, he was invited to the NATO Advances Studies in dynamic of speech perception and production.

In 2003 he joined as a permanent researcher and assistant professor the computer science team of SIS Lab, at the University of Toulon, France. He is currently conducting research on content based information retrieval and automatic speech recognition systems. He is the author, or co-author of 35 conference or journal papers about speech or multimedia documents processing.

Jacques Le Maitre is a professor in computer science at the University of Toulon in France where he leads the SIS laboratory. His main current research interests include query languages for XML databases and multimedia information retrieval.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tollari, S., Glotin, H. & Maitre, J.L. Enhancement of Textual Images Classification Using Segmented Visual Contents for Image Search Engine. Multimed Tools Appl 25, 405–417 (2005). https://doi.org/10.1007/s11042-005-6543-6

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-005-6543-6

Keywords

Search

Navigation