Abstract
Session search is an information retrieval task that involves a sequence of queries for a complex information need. It is characterized by rich user-system interactions and temporal dependency between queries and between consecutive user behaviors. Recent efforts have been made in modeling session search using the Partially Observable Markov Decision Process (POMDP). To best utilize the POMDP model, it is crucial to find suitable definitions for its fundamental elements – States, Actions and Rewards. This paper investigates the best ways to design the states, actions, and rewards within a POMDP framework. We lay out available design options of these major components based on a variety of related work and experiment on combinations of these options over the TREC 2012 & 2013 Session datasets. We report our findings based on two evaluation aspects, retrieval accuracy and efficiency, and recommend practical design choices for using POMDP in session search.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bellman, R.: Dynamic Programming. Princeton University Press (1957)
Chilton, L.B., Teevan, J.: Addressing people’s information needs directly in a web search result page. In: WWW 2011, pp. 27–36
Cormack, G.V., Smucker, M.D., Clarke, C.L.: Efficient and effective spam filtering and re-ranking for large web datasets. Inf. Retr. 14(5), 441–465 (2011)
Fox, S., Karnawat, K., Mydland, M., Dumais, S., White, T.: Evaluating implicit measures to improve web search. ACM Trans. Inf. Syst. 23(2), 147–168
Guan, D., Zhang, S., Yang, H.: Utilizing query change for session search. In: SIGIR 2013, pp. 453–462 (2013)
Hofmann, K., Whiteson, S., de Rijke, M.: Balancing exploration and exploitation in learning to rank online. In: Clough, P., Foley, C., Gurrin, C., Jones, G.J.F., Kraaij, W., Lee, H., Mudoch, V. (eds.) ECIR 2011. LNCS, vol. 6611, pp. 251–263. Springer, Heidelberg (2011)
Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20(4) (October 2002)
Jin, X., Sloan, M., Wang, J.: Interactive exploratory search for multi page search results. In: WWW 2013, pp. 655–666 (2013)
Joachims, T.: A probabilistic analysis of the Rocchio algorithm with TFIDF for text categorization. In: ICML 1997, pp. 143–151 (1997)
Kaelbling, L., Littman, M., Cassandra, A.: Planning and acting in partially observable stochastic domains. Artificial Intelligence 101(1-2), 99–134 (1998)
Kanoulas, E., Carterette, B., Hall, M., Clough, P., Sanderson, M.: Overview of the trec 2012 session track. In: TREC 2012 (2012)
Kanoulas, E., Carterette, B., Hall, M., Clough, P., Sanderson, M.: Overview of the trec, session track. In: TREC 2013 (2013)
Littman, M.L.: The witness algorithm: Solving partially observable Markov decision processes. Technical report, Providence, RI, USA (1994)
Luo, J., Zhang, S., Yang, H.: Win-win search: Dual-agent stochastic game in session search. In: SIGIR 2014 (2014)
Norris, J.R.: Markov Chains. Cambridge University Press (1998)
Robertson, S., Zaragoza, H.: The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr. 3(4), 333–389 (2009)
Salton, G., Buckley, C.: Improving retrieval performance by relevance feedback. Readings in Information Retrieval 24, 5 (1997)
Shen, X., Tan, B., Zhai, C.: Implicit user modeling for personalized search. In: CIKM 2005, pp. 824–831 (2005)
Sondik, E.: The optimal control of partially observable markov processes over the infinite horizon: Discounted cost. Operations Research 26(2), 282–304 (1978)
Yuan, S., Wang, J.: Sequential selection of correlated ads by POMDPs. In: CIKM 2012, pp. 515–524 (2012)
Zhai, C., Lafferty, J.: Two-stage language models for information retrieval. In: SIGIR 2002, pp. 49–56 (2002)
Zhang, S., Luo, J., Yang, H.: A POMDP model for content-free document re-ranking. In: SIGIR 2014 (2014 )
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Luo, J., Zhang, S., Dong, X., Yang, H. (2015). Designing States, Actions, and Rewards for Using POMDP in Session Search. In: Hanbury, A., Kazai, G., Rauber, A., Fuhr, N. (eds) Advances in Information Retrieval. ECIR 2015. Lecture Notes in Computer Science, vol 9022. Springer, Cham. https://doi.org/10.1007/978-3-319-16354-3_58
Download citation
DOI: https://doi.org/10.1007/978-3-319-16354-3_58
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16353-6
Online ISBN: 978-3-319-16354-3
eBook Packages: Computer ScienceComputer Science (R0)