Abstract
Recent years, a lot of research has focused on how to improve query processing efficiency of large-scale search engines. In this paper, we focus on top-k query processing on document-sorted indexes and the well-known rank-safe dynamic pruning technique called WAND, which can efficiently reduce the hardware computational resources required for the first phase top-k processing in cascade ranking model. Firstly, we carefully analyze the difference of the intrinsic optimization ideas between WAND method and another well-known dynamic pruning method called MaxScore, and provide an updated immediately skipping-over description of WAND (WAND_IS) for faster query processing, which can highly reduce short distance skippings on posting lists. We then propose two key improvements: partial scoring candidates (P.WAND) and less sortings in AND mode (L.WAND) that can leverage the query efficiency of WAND processing. Finally, we perform detailed experiments on TREC GOV2 dataset with self-indexing and Block-Max techniques, which show that our proposals can reduce the query latency by almost 15% on average over the WAND baseline, with a best improvement of about 20%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bortnikov, E., Carmel, D., Golan-Gueta, G.: Top-k query processing with conditional skips. In: Proceedings of the 26th International Conference on World Wide Web Companion, Perth, Australia, 3–7 April 2017, pp. 653–661 (2017)
Broder, A.Z., Carmel, D., Herscovici, M., Soffer, A., Zien, J.Y.: Efficient query evaluation using a two-level retrieval process. In: Proceedings of the 2003 ACM CIKM International Conference on Information and Knowledge Management, pp. 426–434 (2003)
Crane, M., Culpepper, J.S., Lin, J.J., Mackenzie, J., Trotman, A.: A comparison of document-at-a-time and score-at-a-time query evaluation. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 201–210 (2017)
Ding, S., Suel, T.: Faster top-k document retrieval using block-max indexes. In: Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 993–1002 (2011)
Fontoura, M., Josifovski, V., Liu, J., Venkatesan, S., Zhu, X., Zien, J.Y.: Evaluation strategies for top-k queries over memory-resident inverted indexes. PVLDB 4(12), 1213–1224 (2011)
Jonassen, S., Bratsberg, S.E.: Efficient compressed inverted index skipping for disjunctive text-queries. In: Clough, P., et al. (eds.) ECIR 2011. LNCS, vol. 6611, pp. 530–542. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20161-5_53
Macdonald, C., Ounis, I., Tonellotto, N.: Upper-bound approximations for dynamic pruning. ACM Trans. Inf. Syst. 29(4), 17:1–17:28 (2011)
Moffat, A., Petri, M.: Index compression using byte-aligned ANS coding and two-dimensional contexts. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp. 405–413 (2018)
Petri, M., Culpepper, J.S., Moffat, A.: Exploring the magic of WAND. In: The Australasian Document Computing Symposium, ADCS, pp. 58–65 (2013)
Petri, M., Moffat, A., Culpepper, J.S.: Score-safe term-dependency processing with hybrid indexes. In: The 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 899–902 (2014)
Strohman, T., Turtle, H.R., Croft, W.B.: Optimization strategies for complex queries. In: SIGIR 2005: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 219–225 (2005)
Turtle, H.R., Flood, J.: Query evaluation: strategies and optimizations. Inf. Process. Manage. 31(6), 831–850 (1995)
Wang, Q., Dimopoulos, C., Suel, T.: Fast first-phase candidate generation for cascading rankers. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 295–304 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, K., Zhu, L., Sun, Q. (2020). Optimizing Scoring and Sorting Operations for Faster WAND Processing. In: Yang, X., Wang, CD., Islam, M.S., Zhang, Z. (eds) Advanced Data Mining and Applications. ADMA 2020. Lecture Notes in Computer Science(), vol 12447. Springer, Cham. https://doi.org/10.1007/978-3-030-65390-3_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-65390-3_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65389-7
Online ISBN: 978-3-030-65390-3
eBook Packages: Computer ScienceComputer Science (R0)