Cited By
View all- Qin ZLi ZBendersky MMetzler D(2020)Matching Cross Network for Learning to Rank in Personal SearchProceedings of The Web Conference 202010.1145/3366423.3380046(2835-2841)Online publication date: 20-Apr-2020
Probabilistic Word Embeddings in Neural IR: A Promising Model That Does Not Work as Expected (For Now)
Authors: 
Alberto Purpura, Marco Maggipinto, 
Gianmaria Silvello, 
Gian Antonio SustoAuthors Info & Claims
Alberto Purpura, Marco Maggipinto, Gianmaria Silvello, Gian Antonio SustoAuthors Info & ClaimsPages 3 - 10
Abstract
In this paper, we discuss how a promising word vector representation based on Probabilistic Word Embeddings (PWE) can be applied to Neural Information Retrieval (NeuIR). We illustrate PWE pros for text retrieval, and identify the core issues which prevent a full exploitation of their potential. In particular, we focus on the application of elliptical probabilistic embeddings, a type of PWE, to a NeuIR system (i.e., MatchPyramid). The main contributions of this paper are: (i) an analysis of the pros and cons of PWE in NeuIR; (ii) an in-depth comparison of PWE against pre-trained Word2Vec, FastText and WordNet word embeddings; (iii) an extension of the MatchPyramid model to take advantage of broader word relations information from WordNet; (iv) a topic-level evaluation of the MatchPyramid ranking models employing the considered word embeddings. Finally, we discuss some lessons learned and outline some open research problems to employ PWE in NeuIR systems more effectively.
References
[1]
C. Basu, L. Dietz, and C. Fellbaum. 2018. WordNetContext: Information Retrieval-friendly Access to WordNet Senses. In ProfS/KG4IR/Data:Search@SIGIR (CEUR Workshop Proceedings), Vol. 2127. CEUR-WS.org, 63--64.
[2]
J. P. Callan, W. B. Croft, and J. Broglio. 1995. TREC and TIPSTER experiments with INQUERY. Information Processing & Management, Vol. 31, 3 (1995), 327--343.
[3]
J. Devlin, M. W. Chang, K. Lee, and K. Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.
[4]
C. Fellbaum. 1999. WordNet: An Electronic Lexical Database. Computational Linguistics, Vol. 25, 2 (June 1999), 292--296.
[5]
J. B. Gao, B. W. Zhang, and X. H. Chen. 2015. A WordNet-based semantic similarity measurement combining edge-counting and information content theory. Engineering Applications of Artificial Intelligence, Vol. 39 (2015), 80--88.
[6]
E. Grave, P. Bojanowski, P. Gupta, A. Joulin, and T. Mikolov. 2018. Learning Word Vectors for 157 Languages. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018) .
[7]
J. Guo, Y. Fan, Q. Ai, and W. B. Croft. 2016. A Deep Relevance Matching Model for Ad-hoc Retrieval. In CIKM 2016 . 55--64.
[8]
F. Hill, R. Reichart, and A. Korhonen. 2015. SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation. Computational Linguistics, Vol. 41, 4 (2015), 665--695.
[9]
B. Hu, Z. Lu, H. Li, and Q. Chen. 2014. Convolutional Neural Network Architectures for Matching Natural Language Sentences. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014. 2042--2050.
[10]
P. Huang, X. He, J. Gao, L. Deng, A. Acero, and L. P. Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. In CIKM 2013 . 2333--2338.
[11]
X. Li, L. Vilnis, D. Zhang, M. Boratko, and A. McCallum. 2019. Smoothing the Geometry of Probabilistic Box Embeddings. In ICLR .
[12]
S. Liu, F. Liu, C. T. Yu, and W. Meng. 2004. An effective approach to document retrieval via utilizing WordNet and recognizing phrases. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval . 266--272.
[13]
X. Lu and H. Li. 2013. A Deep Architecture for Matching Short Texts. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013 . 1367--1375.
[14]
R. Mihalcea and D. Moldovan. 2000. Semantic Indexing Using WordNet Senses. In Proceedings of the ACL-2000 Workshop on Recent Advances in Natural Language Processing and Information Retrieval (RANLPIR '00), Vol. 11. Association for Computational Linguistics, Stroudsburg, PA, USA, 35--45.
[15]
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 3111--3119.
[16]
B. Mitra and N. Craswell. 2018. An Introduction to Neural Information Retrieval. Foundations and Trends in Information Retrieval, Vol. 13, 1 (2018), 1--126.
[17]
B. Muzellec and M. Cuturi. 2018. Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributions. In NeurIPS 2018. 10258--10269.
[18]
V. M. Ngo, T. H. Cao, and T. M. V. Le. 2018. WordNet-Based Information Retrieval Using Common Hypernyms and Combined Features. CoRR, Vol. abs/1807.05574 (2018).
[19]
L. Pang, Y. Lan, J. Guo, J. Xu, and X. Cheng. 2016a. A Study of MatchPyramid Models on Ad-hoc Retrieval. CoRR, Vol. abs/1606.04648 (2016).
[20]
L. Pang, Y. Lan, J. Guo, J. Xu, S. Wan, and X. Cheng. 2016b. Text Matching as Image Recognition. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. 2793--2799.
[21]
G. Pass, A. Chowdhury, and C. Torgeson. 2006. A picture of search. In Proceedings of the 1st International Conference on Scalable Information Systems. 1.
[22]
J. Pennington, R. Socher, and C. D. Manning. 2014. GloVe: Global Vectors for Word Representation. In EMNLP. 1532--1543.
[23]
J. M. Ponte and W. B. Croft. 2017. A Language Modeling Approach to Information Retrieval. SIGIR Forum, Vol. 51, 2 (2017), 202--208.
[24]
Rothe S. and H. Schü tze. 2015. AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes. In ACL 2015, Vol. 1. 1793--1803.
[25]
C. Saedi, A. Branco, J. A. Rodrigues, and J. Silva. 2018. WordNet Embeddings. In Proceedings of The Third Workshop on Representation Learning for NLP: Rep4NLP@ACL. 122--131.
[26]
C. Van Gysel, M. De Rijke, and E. Kanoulas. 2018. Neural vector spaces for unsupervised information retrieval. TOIS, Vol. 36, 4 (2018), 38.
[27]
G. Varelas, E. Voutsakis, P. Raftopoulou, E. G. M. Petrakis, and E. E. Milios. 2005. Semantic Similarity Methods in wordNet and Their Application to Information Retrieval on the Web. In Proceedings of the 7th Annual ACM International Workshop on Web Information and Data Management (WIDM '05). ACM, New York, NY, USA, 10--16.
[28]
L. Vilnis and A. McCallum. 2015. Word Representations via Gaussian Embedding. In ICLR .
[29]
E. M. Voorhees. 1994. Query Expansion Using Lexical-Semantic Relations. In Proceedings of the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval. 61--69.
[30]
E. M. Voorhees. 2004. Overview of the TREC 2004 Robust Track. In TREC 2004 .
[31]
Z. Wu and M. Palmer. 1994. Verbs Semantics and Lexical Selection. In Proceedings of the 32nd Annual Meeting of ACL (ACL '94). ACL, Stroudsburg, PA, USA, 133--138.
[32]
Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. Salakhutdinov, and Q. V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding.
[33]
H. Zamani and W. B. Croft. 2017. Relevance-based Word Embedding. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval . 505--514.
[34]
H. Zamani, M. Dehghani, W. B. Croft, E. G. Learned-Miller, and J. Kamps. 2018. From Neural Re-Ranking to Neural Ranking: Learning a Sparse Representation for Inverted Indexing. In CIKM 2018. 497--506.
Index Terms
- Probabilistic Word Embeddings in Neural IR: A Promising Model That Does Not Work as Expected (For Now)
Recommendations
Using Word Embeddings for Information Retrieval: How Collection and Term Normalization Choices Affect Performance
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge ManagementNeural word embedding approaches, due to their ability to capture semantic meanings of vocabulary terms, have recently gained attention of the information retrieval (IR) community and have shown promising results in improving ad hoc retrieval ...
Adaptive Probabilistic Word Embedding
WWW '20: Proceedings of The Web Conference 2020Word embeddings have been widely used and proven to be effective in many natural language processing and text modeling tasks. It is obvious that one ambiguous word could have very different semantics in various contexts, which is called polysemy. Most ...
Word normalization and decompounding in mono- and bilingual IR
AbstractThe present research studies the impact of decompounding and two different word normalization methods, stemming and lemmatization, on monolingual and bilingual retrieval. The languages in the monolingual runs are English, Finnish, German and ...
Comments
Information & Contributors
Information
Published In
September 2019
273 pages
ISBN:9781450368810
DOI:10.1145/3341981
- General Chairs:
- Yi Fang,
- Yi Zhang,
- Program Chairs:
- James Allan,
- Krisztian Balog,
- Ben Carterette,
- Jiafeng Guo
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 September 2019
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
ICTIR '19
Sponsor:
ICTIR '19: The 2019 ACM SIGIR International Conference on the Theory of Information Retrieval
October 2 - 5, 2019
CA, Santa Clara, USA
Acceptance Rates
ICTIR '19 Paper Acceptance Rate 20 of 41 submissions, 49%;
Overall Acceptance Rate 235 of 527 submissions, 45%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 147Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
Cited By
View all- Qin ZLi ZBendersky MMetzler D(2020)Matching Cross Network for Learning to Rank in Personal SearchProceedings of The Web Conference 202010.1145/3366423.3380046(2835-2841)Online publication date: 20-Apr-2020
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in