Skip to main content
Springer Nature Link
Log in

A Neural Network to Retrieve Images from Text Queries

  • Conference paper
Artificial Neural Networks – ICANN 2006 (ICANN 2006)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 4132))

Included in the following conference series:

  • 1331 Accesses

Abstract

This work presents a neural network for the retrieval of images from text queries. The proposed network is composed of two main modules: the first one extracts a global picture representation from local block descriptors while the second one aims at solving the retrieval problem from the extracted representation. Both modules are trained jointly to minimize a loss related to the retrieval performance. This approach is shown to be advantageous when compared to previous models relying on unsupervised feature extraction: average precision over Corel queries reaches 26.2% for our model, which should be compared to 21.6% for PAMIR, the best alternative.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Barnard, K., Duygulu, P., Forsyth, D., de Freitas, N., Blei, D.M., Jordan, M.I.: Matching words and pictures. J. of Machine Learning Research 3 (2003)

    Google Scholar 

  2. Grangier, D., Bengio, S.: A discriminative approach for the retrieval of images from text queries. Technical report, IDIAP Research Institute (2006)

    Google Scholar 

  3. Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: ACM Special Interest Group on Information Retrieval (2003)

    Google Scholar 

  4. Monay, F., Gatica-Perez, D.: PLSA-based image auto-annotation: constraining the latent space. In: ACM Multimedia (2004)

    Google Scholar 

  5. Duygulu, P., Barnard, K., de Freitas, N., Forsyth, D.: Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: European Conf. on Computer Vision (2002)

    Google Scholar 

  6. Tieu, K., Viola, P.: Boosting image retrieval. Intl. J. of Computer Vision 56 (2004)

    Google Scholar 

  7. Wu, H., LuE, H., Ma, S.: A practical SVM-based algorithm for ordinal regression in image retrieval. In: ACM Multimedia (2003)

    Google Scholar 

  8. LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Handwritten digit recognition with a back-propagation network. In: Conf. on Advances in Neural Information Processing Systems (1989)

    Google Scholar 

  9. Garcia, C., Delakis, M.: Convolutional face finder: A neural architecture for fast and robust face detection. T. on Pattern Analysis and Machine Intelligence 26 (2004)

    Google Scholar 

  10. Joachims, T.: Optimizing search engines using clickthrough data. In: Intl. Conf. on Knowledge Discovery and Data Mining (2002)

    Google Scholar 

  11. Grangier, D., Bengio, S.: Exploiting hyperlinks to learn a retrieval model. In: NIPS Workshop on Learning to Rank (2005)

    Google Scholar 

  12. Quelhas, P., Monay, F., Odobez, J.M., Gatica-Perez, D., Tuytelaars, T., Gool, L.J.V.: Modeling scenes with local descriptors and latent aspects. In: Intl. Conf. on Computer Vision (2005)

    Google Scholar 

  13. Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison Wesley, Harlow (1999)

    Google Scholar 

  14. LeCun, Y., Bottou, L., Orr, G.B., Mueller, K.R.: Efficient backprop. In: Orr, G.B., Mueller, K.R. (eds.) Neural Networks: Trick of the Trade, Springer, Heidelberg (1998)

    Google Scholar 

  15. Takala, V., Ahonen, T., Pietikainen, M.: Block-based methods for image retrieval using local binary patterns. In: Scandinavian Conf. on Image Analysis (2005)

    Google Scholar 

  16. Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to rank using gradient descent. In: Intl. Conf. on Machine Learning (2005)

    Google Scholar 

  17. Rice, J.: Rice, Mathematical Statistics and Data Analysis. Duxbury Press, Belmont (1995)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IDIAP Research Institute, Martigny, Switzerland

    David Grangier & Samy Bengio

  2. Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland

    David Grangier

Authors
  1. David Grangier
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samy Bengio
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Electrical and Computer Engineering, Image, Video and Multimedia Systems Laboratory, National Technical University of Athens, 157 80, Zographou, GR, Greece

    Stefanos Kollias

  2. Department of Electrical and Computer Engineering, National Technical University of Athens, 15780, Zographou, Greece

    Andreas Stafylopatis

  3. Department of Informatics, Nicolaus Copernicus University, Toruń, Poland

    Włodzisław Duch

  4. Adaptive Informatics Research Centre, Helsinki University of Technology, P.O. Box 5400, 02015, HUT, Finland

    Erkki Oja

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Grangier, D., Bengio, S. (2006). A Neural Network to Retrieve Images from Text Queries. In: Kollias, S., Stafylopatis, A., Duch, W., Oja, E. (eds) Artificial Neural Networks – ICANN 2006. ICANN 2006. Lecture Notes in Computer Science, vol 4132. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840930_3

Download citation

  • DOI: https://doi.org/10.1007/11840930_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-38871-5

  • Online ISBN: 978-3-540-38873-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics