Skip to main content
Springer Nature Link
Log in

Faster BlockMax WAND with Longer Skipping

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11437))

Included in the following conference series:

  • 2666 Accesses

Abstract

One of the major problems for modern search engines is to keep up with the tremendous growth in the size of the web and the number of queries submitted by users. The amount of data being generated today can only be processed and managed with specialized technologies.

BlockMax WAND and the more recent Variable BlockMax WAND represent the most advanced query processing algorithms that make use of dynamic pruning techniques, which allow them to retrieve the top k most relevant documents for a given query without any effectiveness degradation of its ranking. In this paper, we describe a new technique for the BlockMax WAND family of query processing algorithm, which improves block skipping in order to increase its efficiency. We show that our optimization is able to improve query processing speed on short queries by up to 37% with negligible additional space overhead.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/pisa-engine/pisa/tree/ecir19-ls.

  2. 2.

    http://tika.apache.org.

References

  1. Broder, A.Z., Carmel, D., Herscovici, M., Soffer, A., Zien, J.: Efficient query evaluation using a two-level retrieval process. In: Proceedings of the 12th International Conference on Information and Knowledge Management, pp. 426–434 (2003)

    Google Scholar 

  2. Callan, J., Hoy, M., Yoo, C., Zhao, L.: Clueweb09 data set (2009). http://lemurproject.org/clueweb09/

  3. Daoud, C.M., de Moura, E.S., da Costa Carvalho, A.L., da Silva, A.S., de Oliveira, D.F., Rossi, C.: Fast top-k preserving query processing using two-tier indexes. Inf. Process. Manage. 52, 855–872 (2016)

    Article  Google Scholar 

  4. Daoud, C.M., de Moura, E.S., de Oliveira, D.F., da Silva, A.S., Rossi, C., da Costa Carvalho, A.L.: Waves: a fast multi-tier top-k query processing algorithm. Inf. Retr. J. 20, 292–316 (2017)

    Article  Google Scholar 

  5. Dimopoulos, C., Nepomnyachiy, S., Suel, T.: Optimizing top-k document retrieval strategies for block-max indexes. In: Proceedings of the 6th ACM International Conference on Web Search and Data Mining, pp. 113–122 (2013)

    Google Scholar 

  6. Ding, S., Suel, T.: Faster top-k document retrieval using block-max indexes. In Proceedings of the 34th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 993–1002 (2011)

    Google Scholar 

  7. Kane, A., Tompa, F.W.: Split-lists and initial thresholds for wand-based search. In: Proceedings of the 41st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 877–880 (2018)

    Google Scholar 

  8. Mallia, A., Ottaviano, G., Porciani, E., Tonellotto, N., Venturini, R.: Faster blockmax WAND with variable-sized blocks. In: Proceedings of the 40th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 625–634 (2017)

    Google Scholar 

  9. Ottaviano, G., Venturini, R.: Partitioned Elias-Fano indexes. In: Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 273–282 (2014)

    Google Scholar 

  10. Rojas, O., Gil-Costa, V., Marin, M.: Efficient parallel block-max wand algorithm. In: Proceedings of the 19th International Conference on Parallel Processing, pp. 394–405 (2013)

    Google Scholar 

  11. Silvestri, F.: Sorting out the document identifier assignment problem. In: Proceedings of the 29th European Conference on IR Research, pp. 101–112 (2007)

    Google Scholar 

  12. Vigna, S.: Quasi-succinct indices. In: Proceedings of the 6th ACM International Conference on Web Search and Data Mining, pp. 83–92 (2013)

    Google Scholar 

  13. Yan, H., Ding, S., Suel, T.: Inverted index compression and query processing with optimized document ordering. In: Proceedings of the 18th International Conference on World Wide Web, pp. 401–410 (2009)

    Google Scholar 

  14. Zobel, J., Moffat, A.: Inverted files for text search engines. ACM Comput. Surv. 38(2), 6 (2006)

    Article  Google Scholar 

Download references

Acknowledgments

Antonio Mallia’s research was partially supported by NSF Grant IIS-1718680 “Index Sharding and Query Routing in Distributed Search Engines”.

Author information

Authors and Affiliations

  1. Computer Science and Engineering, New York University, New York, USA

    Antonio Mallia

  2. Sease Ltd., London, UK

    Elia Porciani

Authors
  1. Antonio Mallia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elia Porciani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Antonio Mallia or Elia Porciani .

Editor information

Editors and Affiliations

  1. University of Strathclyde, Glasgow, UK

    Leif Azzopardi

  2. Bauhaus Universität Weimar, Weimar, Germany

    Benno Stein

  3. Universität Duisburg-Essen, Duisburg, Germany

    Norbert Fuhr

  4. GESIS - Leibniz Institute for the Social Sciences, Cologne, Germany

    Philipp Mayr

  5. Delft University of Technology, Delft, The Netherlands

    Claudia Hauff

  6. University of Twente, Enschede, The Netherlands

    Djoerd Hiemstra

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mallia, A., Porciani, E. (2019). Faster BlockMax WAND with Longer Skipping. In: Azzopardi, L., Stein, B., Fuhr, N., Mayr, P., Hauff, C., Hiemstra, D. (eds) Advances in Information Retrieval. ECIR 2019. Lecture Notes in Computer Science(), vol 11437. Springer, Cham. https://doi.org/10.1007/978-3-030-15712-8_52

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-15712-8_52

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-15711-1

  • Online ISBN: 978-3-030-15712-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics