Rank-at-a-Time Query Processing
Pages 229 - 232
Abstract
Query processing strategies for ranked retrieval have been studied for decades. In this paper we propose a new strategy, which we call rank-at-a-time query processing, that evaluates documents in descending order of quantized scores and is able to directly compute the final document ranking via a sequence of boolean intersections. We show that such a strategy is equivalent to a second-order restricted composition of per-term scores. Rank-at-a-time query processing has the advantage that it is anytime score-safe, which means that the retrieval algorithm can self-adapt to produce an exact ranking given an arbitrary latency constraint. Due to the combinatorial nature of compositions, however, a naive implementation is too slow to be of practical use. To address this issue, we introduce a hybrid variant that is able to reduce query latency to a point that is on par with state-of-the-art retrieval engines.
References
[1]
V. N. Anh, O. de Kretser, and A. Moffat. Vector-space ranking with effective early termination. SIGIR, 2001.
[2]
N. Asadi and J. Lin. Effectiveness/efficiency tradeoffs for candidate generation in multi-stage retrieval architectures. SIGIR, 2013.
[3]
P. Boldi and S. Vigna. MG4J at TREC 2006. TREC, 2006.
[4]
A. Z. Broder, D. Carmel, M. Herscovici, A. Soffer, and J. Zien. Efficient query evaluation using a two-level retrieval process. CIKM, 2003.
[5]
M. Crane, A. Trotman, and R. O'Keefe. Maintaining discriminatory power in quantized indexes. CIKM, 2013.
[6]
J. S. Culpepper and A. Moffat. Efficient set intersection for inverted indexing. TOIS, 29(1), 2010.
[7]
C. Dimopoulos, S. Nepomnyachiy, and T. Suel. Optimizing top-k document retrieval strategies for block-max indexes. WSDM, 2013.
[8]
S. Ding and T. Suel. Faster top-k document retrieval using block-max indexes. SIGIR, 2011.
[9]
J. Lin, M. Crane, A. Trotman, J. Callan, I. Chattopadhyaya, J. Foley, G. Ingersoll, C. Macdonald, and S. Vigna. Toward reproducible baselines: The Open-Source IR Reproducibility Challenge. ECIR, 2016.
[10]
J. Lin and A. Trotman. Anytime ranking for impact-ordered indexes. ICTIR, 2015.
[11]
A. Moffat and J. Zobel. Self-indexing inverted files for fast text retrieval. TOIS, 14(4):349--379, 1996.
[12]
A. Moffat, J. Zobel, and R. Sacks-Davis. Memory efficient ranking. IP&M, 30(6):733--744, 1994.
[13]
D. R. Page. Generalized algorithm for restricted weak composition generation. Journal of Mathematical Modelling and Algorithms in Operations Research, 12(4):345--372, 2013.
[14]
M. Persin, J. Zobel, and R. Sacks-Davis. Filtered document retrieval with frequency-sorted indexes. JASIS, 47(10):749--764, 1996.
[15]
A. Trotman, X. Jia, and M. Crane. Towards an efficient and effective search engine. OSIR Workshop, 2012.
[16]
S. Vigna. Quasi-succinct indices. WSDM, 2013.
Index Terms
- Rank-at-a-Time Query Processing
Recommendations
Efficient Query Processing Infrastructures: A half-day tutorial at SIGIR 2018
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information RetrievalTypically, techniques that benefit effectiveness of information retrieval (IR) systems have a negative impact on efficiency. Yet, with the large scale of Web search engines, there is a need to deploy efficient query processing techniques to reduce the ...
Combining Joint and Semi-Join Operations for Distributed Query Processing
The application of a combination of join and semi-join operations to minimize the amount of data transmission required for distributed query processing is discussed. Specifically, two important concepts that occur with the use of join operations as ...
Interleaving a Join Sequence with Semijoins in Distributed Query Processing
The problem of combining join and semijoin reducers for distributed query processing is studied. An approach based on interleaving a join sequence with beneficial semijoins is proposed. A join sequence is mapped into a join sequence tree first. The join ...
Comments
Information & Contributors
Information
Published In
September 2016
318 pages
ISBN:9781450344975
DOI:10.1145/2970398
- General Chairs:
- Ben Carterette,
- Hui Fang,
- Program Chairs:
- Mounia Lalmas,
- Jian-Yun Nie
Copyright © 2016 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 September 2016
Check for updates
Author Tags
Qualifiers
- Short-paper
Funding Sources
Conference
ICTIR '16
Sponsor:
ICTIR '16: ACM SIGIR International Conference on the Theory of Information Retrieval
September 12 - 16, 2016
Delaware, Newark, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 106Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 07 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in