Simon Jonassen
ABSTRACT
Dynamic index pruning techniques are commonly used to speed up query processing in Web search engines. In this work, we propose a linear programming technique which can further improve the performance of the state-of-the-art dynamic index pruning techniques. The experiments we conducted demonstrate that the proposed technique achieves reduction in terms of the disk access, index decompression, and scoring costs compared to the well-known Max-Score technique.
- A. Broder, D. Carmel, M. Herscovici, A. Soffer, and J. Zien. Efficient query evaluation using a two-level retrieval process. In Proceedings of the 12th International Conference on Information and Knowledge Management (CIKM), pages 426--434, 2003. Google Scholar
Digital Library
- B. B. Cambazoglu, E. Varol, E. Kayaaslan, C. Aykanat, and R. Baeza-Yates. Query forwarding in geographically distributed search engines. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 90--97, 2010. Google ScholarDigital Library
- K. Chakrabarti, S. Chaudhuri, and V. Ganti. Interval-based pruning for top-k processing over compressed lists. In Proceedings of the 2011 IEEE 27th International Conference on Data Engineering (ICDE), pages 709--720, 2011. Google ScholarDigital Library
- C. Dimopoulos, S. Nepomnyachiy, and T. Suel. Optimizing top-k document retrieval strategies for block-max indexes. In Proceedings of the Sixth ACM International Conference on Web Search and Data Mining (WSDM), pages 113--122, 2013. Google ScholarDigital Library
- S. Ding and T. Suel. Faster top-k document retrieval using block-max indexes. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 993--1002, 2011. Google ScholarDigital Library
- M. Fontoura, V. Josifovski, J. Liu, S. Venkatesan, X. Zhu, and J. Y. Zien. Evaluation strategies for top-k queries over memory-resident inverted indexes. Proceedings of the VLDB Endowment, 4(12):1213--1224, 2011.Google ScholarDigital Library
- S. Jonassen. Efficient Query Processing in Distributed Search Engines. PhD thesis, Norwegian University of Science and Technology, Department of Computer and Information Science, pages 133--137, 2013.Google Scholar
- S. Jonassen and S. E. Bratsberg. Efficient compressed inverted index skipping for disjunctive text-queries. In Proceedings of the 33rd European Conference on Information Retrieval (ECIR), pages 530--542, 2011. Google ScholarDigital Library
- S. Jonassen and S. E. Bratsberg. Intra-query concurrent pipelined processing for distributed full-text retrieval. In Proceedings of the 34th European Conference on Information Retrieval (ECIR), pages 413--425, 2012. Google ScholarDigital Library
- S. Jonassen and S. E. Bratsberg. Improving the performance of pipelined query processing with skipping - and its comparison to document-wise partitioning. World Wide Web, 17(5):949--967, 2014. Google ScholarDigital Library
- C. Macdonald, I. Ounis, and N. Tonellotto. Upper-bound approximations for dynamic pruning. ACM Transactions on Information Systems, 29(4):17:1--17:28, 2011. Google ScholarDigital Library
- C. Macdonald, N. Tonellotto, and I. Ounis. Learning to predict response times for online query scheduling. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 621--630, 2012. Google ScholarDigital Library
- O. Rojas, V. Gil-Costa and M. Marin. Efficient Parallel Block-Max WAND Algorithm. In Proceedings of the 19th International Conference on Parallel Processing (Euro-Par), pages 394--405, 2013. Google ScholarDigital Library
- D. Shan, S. Ding, J. He, H. Yan, and X. Li. Optimized top-k processing with global page scores on block-max indexes. In Proceedings of the 5th International Conference on Web Search and Data Mining (WSDM), pages 423--432, 2012. Google ScholarDigital Library
- T. Strohman, H. Turtle, and W. B. Croft. Optimization strategies for complex queries. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 219--225, 2005. Google ScholarDigital Library
- H. Turtle and J. Flood. Query evaluation: strategies and optimizations. Information Processing and Management, 31(6):831--850, 1995. Google ScholarDigital Library
Index Terms
- Improving Dynamic Index Pruning via Linear Programming
Recommendations
Transformation of a multi-choice linear programming problem
The aim of this paper is to transform a multi-choice linear programming problem to a standard mathematical programming problem where the right hand side goals of some constraints are 'multi-choice' in nature. For each of the constraint there may exist ...
Branching on hyperplane methods for mixed integer linear and convex programming using adjoint lattices
We present branching-on-hyperplane methods for solving mixed integer linear and mixed integer convex programs. In particular, we formulate the problem of finding a good branching hyperplane using a novel concept of adjoint lattice. We also reformulate ...
A simplex algorithm for piecewise-linear programming I: Derivation and proof
The simplex method for linear programming can be extended to permit the minimization of any convex separable piecewise-linear objective, subject to linear constraints. This three-part paper develops and analyzes a general, computationally practical ...
Comments