Guglielmo Faggioli
ABSTRACT
How to model the performance of a retrieval system before its deploying has puzzled the Information Retrieval (IR)researchers for a long time. Currently, the evaluation of IR systems relies on empirical experiments. Empirical evaluation means that we need experimental collections: building them is expensive both in term of time and money. Exploiting already available collections to predict the performance of a system on new collections, would dramatically reduce such cost. With the research line described in this work,we plan to study the development of predictive models for the performance of the IR systems. In particular, the proposed research line will investigate Generalized Linear Mixed Models and Causal Inference. Furthermore, we highlight the importance of modelling the performance as distributions rather than point estimations.
Supplemental Material
- Denis Charles, Max Chickering, and Patrice Simard. Counterfactual reasoning and learning systems: The example of computational advertising. Journal of Machine Learning Research, 14, 2013.Google Scholar
- G. Faggioli, O. Zendel, J. S. Culpepper, N. Ferro, and Scholer F. An enhanced evaluation framework for query performance prediction. In Proc. 43nd European Conference on IR Research (ECIR 2021), 2021.Google ScholarDigital Library
- Mark D Smucker and Charles LA Clarke. Modeling user variance in time-biased gain. In Proceedings of the Symposium on Human-Computer Interaction and Information Retrieval, pages 1--10, 2012.Google ScholarDigital Library
- Walter W Stroup. Generalized linear mixed models: modern concepts, methods and applications. CRC press, 2012.Google Scholar
Index Terms
- Enabling Performance Prediction in Information Retrieval Evaluation
Recommendations
Incorporating rich features to boost information retrieval performance
Research highlights We propose a regression-based re-ranking framework that can take into account rich features for boosting information retrieval (IR) performance. A set of salient features that may affect IR performance are investigated. Extensive ...
Information Retrieval Based on Pseudo Prediction Query Performance
ETCS '09: Proceedings of the 2009 First International Workshop on Education Technology and Computer Science - Volume 02Prediction query performance (PQP) has recently been recognized by the IR (information retrieval) community as an important capability for IR systems. Put forward pseudo prediction query performance in the base of analyzing PQP.Design a new IR method. ...
Comments