Skip to main content

From Text to Image: Generating Visual Query for Image Retrieval

  • Conference paper
Multilingual Information Access for Text, Speech and Images (CLEF 2004)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 3491))

Included in the following conference series:

Abstract

This paper explores the uses of visual features for cross-language access to an image collection. An approach which transforms textual queries into visual representations is proposed. The relationships between text and images are mined. We employ the mined relationships to construct visual queries from textual ones. The retrieval results using textual and visual queries are combined to generate the final ranked list. We conducted English monolingual and Chinese-English cross-language retrieval experiments. The performances are quite good. The average precision of English monolingual textual run is 0.6304. The performance of cross-lingual retrieval is about 70% of monolingual retrieval. Comparatively, the gain of the generated visual query is not significant. If only appropriate query terms are selected to generate visual query, retrieval performance could be increased.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Bian, G.W., Chen, H.H.: Cross Language Information Access to Multilingual Collections on the Internet. Journal of American Society for Information Science 51(3), 281–296 (2000)

    Article  Google Scholar 

  2. Carson, C., Belongie, S., Greenspan, H., Malik, J.: Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(8), 1026–1038 (2002)

    Article  Google Scholar 

  3. Chen, H.H., Ding, Y.W., Tsai, S.C., Bian, G.W.: Description of the NTU System Used for MET2. In: Proceedings of Seventh Message Understanding Conference (1998)

    Google Scholar 

  4. Chen, H.H., Yang, C., Lin, Y.: Learning Formulation and Transformation Rules for Multilingual Named Entities. In: Proceedings of ACL 2003 Workshop on Multilingual and Mixed-language Named Entity Recognition: Combining Statistical and Symbolic Models. Association for Computational Linguistics, pp. 1–8 (2003)

    Google Scholar 

  5. Duygulu, P., Barnard, K., Freitas, N., Forsyth, D.: Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary. In: Proceedings of Seventh European Conference on Computer Vision, vol. 4, pp. 97–112 (2002)

    Google Scholar 

  6. Jeon, J., Lavrenko, V., Manmatha, R.: Automatic Image Annotation and Retrieval using Cross-Media Relevance Models. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2003), pp. 119–126. ACM Press, New York (2003)

    Chapter  Google Scholar 

  7. Lavrenko, V., Manmatha, R., Jeon, J.: A Model for Learning the Semantics of Pictures. In: Proceedings of the Seventeenth Annual Conference on Neural Information Processing Systems (2003)

    Google Scholar 

  8. Lin, W.H., Chen, H.H.: Backward Machine Transliteration by Learning Phonetic Similarity. In: Proceedings of Sixth Conference on Natural Language Learning. Association for Computational Linguistics, pp. 139–145 (2002)

    Google Scholar 

  9. Lin, W.C., Lin, M.S., Chen, H.H.: Cross-Language Image Retrieval via Spoken Query. In: Proceedings of RIAO 2004: Coupling Approaches, Coupling Media and Coupling Languages for Information Retrieval. Le Centre de Hautes Études Internationales d’Informatique Documentaire – C.I.D., pp. 524–536 (2004)

    Google Scholar 

  10. Lin, W.C., Yang, C., Chen, H.H.: Foreign Name Backward Transliteration in Chinese-English Cross-Language Image Retrieval. In: Peters, C., et al. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 611–620. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  11. Mori, Y., Takahashi, H., Oka, R.: Image-to-Word Transformation Based on Dividing and Vector Quantizing Images with Words. In: Proceedings of the First International Workshop on Multimedia Intelligent Storage and Retrieval Management. (1999)

    Google Scholar 

  12. Robertson, S.E., Walker, S., Beaulieu, M.: Okapi at TREC-7: Automatic Ad Hoc, Filtering, VLC and Interactive. In: Proceedings of the Seventh Text REtrieval Conference (TREC-7). National Institute of Standards and Technology, pp. 253–264 (1998)

    Google Scholar 

  13. Shi, J., Malik, J.: Normalized Cuts and Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888–905 (2000)

    Article  Google Scholar 

  14. The Lowlands Team: Lazy Users and Automatic Video Retrieval Tools in (the) Lowlands. In: Proceedings of the Tenth Text REtrieval Conference (TREC 2001). National Institute of Standards and Technology, pp. 159–168 (2002)

    Google Scholar 

  15. Westerveld, T.: Image Retrieval: Content versus Context. In: Proceedings of RIAO 2000. Le Centre de Hautes Études Internationales d’Informatique Documentaire – C.I.D., vol. 1, pp. 276–284 (2000)

    Google Scholar 

  16. Westerveld, T.: Probabilistic Multimedia Retrieval. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002), pp. 437–438. ACM Press, New York (2002)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Lin, WC., Chang, YC., Chen, HH. (2005). From Text to Image: Generating Visual Query for Image Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds) Multilingual Information Access for Text, Speech and Images. CLEF 2004. Lecture Notes in Computer Science, vol 3491. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11519645_65

Download citation

  • DOI: https://doi.org/10.1007/11519645_65

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-27420-9

  • Online ISBN: 978-3-540-32051-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics