Abstract
This paper explores the uses of visual features for cross-language access to an image collection. An approach which transforms textual queries into visual representations is proposed. The relationships between text and images are mined. We employ the mined relationships to construct visual queries from textual ones. The retrieval results using textual and visual queries are combined to generate the final ranked list. We conducted English monolingual and Chinese-English cross-language retrieval experiments. The performances are quite good. The average precision of English monolingual textual run is 0.6304. The performance of cross-lingual retrieval is about 70% of monolingual retrieval. Comparatively, the gain of the generated visual query is not significant. If only appropriate query terms are selected to generate visual query, retrieval performance could be increased.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bian, G.W., Chen, H.H.: Cross Language Information Access to Multilingual Collections on the Internet. Journal of American Society for Information Science 51(3), 281–296 (2000)
Carson, C., Belongie, S., Greenspan, H., Malik, J.: Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(8), 1026–1038 (2002)
Chen, H.H., Ding, Y.W., Tsai, S.C., Bian, G.W.: Description of the NTU System Used for MET2. In: Proceedings of Seventh Message Understanding Conference (1998)
Chen, H.H., Yang, C., Lin, Y.: Learning Formulation and Transformation Rules for Multilingual Named Entities. In: Proceedings of ACL 2003 Workshop on Multilingual and Mixed-language Named Entity Recognition: Combining Statistical and Symbolic Models. Association for Computational Linguistics, pp. 1–8 (2003)
Duygulu, P., Barnard, K., Freitas, N., Forsyth, D.: Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary. In: Proceedings of Seventh European Conference on Computer Vision, vol. 4, pp. 97–112 (2002)
Jeon, J., Lavrenko, V., Manmatha, R.: Automatic Image Annotation and Retrieval using Cross-Media Relevance Models. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2003), pp. 119–126. ACM Press, New York (2003)
Lavrenko, V., Manmatha, R., Jeon, J.: A Model for Learning the Semantics of Pictures. In: Proceedings of the Seventeenth Annual Conference on Neural Information Processing Systems (2003)
Lin, W.H., Chen, H.H.: Backward Machine Transliteration by Learning Phonetic Similarity. In: Proceedings of Sixth Conference on Natural Language Learning. Association for Computational Linguistics, pp. 139–145 (2002)
Lin, W.C., Lin, M.S., Chen, H.H.: Cross-Language Image Retrieval via Spoken Query. In: Proceedings of RIAO 2004: Coupling Approaches, Coupling Media and Coupling Languages for Information Retrieval. Le Centre de Hautes Études Internationales d’Informatique Documentaire – C.I.D., pp. 524–536 (2004)
Lin, W.C., Yang, C., Chen, H.H.: Foreign Name Backward Transliteration in Chinese-English Cross-Language Image Retrieval. In: Peters, C., et al. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 611–620. Springer, Heidelberg (2004)
Mori, Y., Takahashi, H., Oka, R.: Image-to-Word Transformation Based on Dividing and Vector Quantizing Images with Words. In: Proceedings of the First International Workshop on Multimedia Intelligent Storage and Retrieval Management. (1999)
Robertson, S.E., Walker, S., Beaulieu, M.: Okapi at TREC-7: Automatic Ad Hoc, Filtering, VLC and Interactive. In: Proceedings of the Seventh Text REtrieval Conference (TREC-7). National Institute of Standards and Technology, pp. 253–264 (1998)
Shi, J., Malik, J.: Normalized Cuts and Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888–905 (2000)
The Lowlands Team: Lazy Users and Automatic Video Retrieval Tools in (the) Lowlands. In: Proceedings of the Tenth Text REtrieval Conference (TREC 2001). National Institute of Standards and Technology, pp. 159–168 (2002)
Westerveld, T.: Image Retrieval: Content versus Context. In: Proceedings of RIAO 2000. Le Centre de Hautes Études Internationales d’Informatique Documentaire – C.I.D., vol. 1, pp. 276–284 (2000)
Westerveld, T.: Probabilistic Multimedia Retrieval. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002), pp. 437–438. ACM Press, New York (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lin, WC., Chang, YC., Chen, HH. (2005). From Text to Image: Generating Visual Query for Image Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds) Multilingual Information Access for Text, Speech and Images. CLEF 2004. Lecture Notes in Computer Science, vol 3491. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11519645_65
Download citation
DOI: https://doi.org/10.1007/11519645_65
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-27420-9
Online ISBN: 978-3-540-32051-7
eBook Packages: Computer ScienceComputer Science (R0)