Makoto P. Kato
ABSTRACT
We propose a method of optimizing search result presentation for queries with diverse intents, by selectively presenting query suggestions for leading users to more relevant search results. The optimization is based on a probabilistic model of users who click on query suggestions in accordance with their intents, and modified versions of intent-aware evaluation metrics that take into account the co-occurrence between intents. Showing many query suggestions simply increases a chance to satisfy users with diverse intents in this model, while it in fact requires users to spend additional time for scanning and selecting suggestions, and may result in low satisfaction for some users. Therefore, we measured the loss of time caused by query suggestion presentation by conducting a user study in different settings, and included its negative effects in our optimization problem. Our experiments revealed that the optimization of search result presentation significantly improved that of a single ranked list, and was beneficial especially for patient users. Moreover, experimental results showed that our optimization was effective particularly when intents of a query often co-occur with a small subset of intents.
- Statcounter global stats. Retrieved January 14, 2015, from http://gs.statcounter.com/#desktop+mobile-comparison-ww-monthly-201312-201412.Google Scholar
- M. Ageev, Q. Guo, D. Lagun, and E. Agichtein. Find it if you can: a game for modeling different types of web search success using interaction data. In SIGIR, pages 345--354, 2011. Google ScholarDigital Library
- R. Agrawal, S. Gollapudi, A. Halverson, and S. Ieong. Diversifying search results. In WSDM, pages 5--14, 2009. Google ScholarDigital Library
- A. Anagnostopoulos, L. Becchetti, C. Castillo, and A. Gionis. An optimization framework for query recommendation. In WSDM, pages 161--170, 2010. Google ScholarDigital Library
- F. Baskaya, H. Keskustalo, and K. Järvelin. Time drives interaction: simulating sessions in diverse searching environments. In SIGIR, pages 105--114, 2012. Google ScholarDigital Library
- C. Brandt, T. Joachims, Y. Yue, and J. Bank. Dynamic ranked retrieval. In WSDM, pages 247--256, 2011. Google ScholarDigital Library
- J. Carbonell and J. Goldstein. The use of mmr, diversity-based reranking for reordering documents and producing summaries. In SIGIR, pages 335--336, 1998. Google ScholarDigital Library
- B. Carterette and P. Chandar. Probabilistic models of ranking novel documents for faceted topic retrieval. In CIKM, pages 1287--1296, 2009. Google ScholarDigital Library
- O. Chapelle and Y. Chang. Yahoo! learning to rank challenge overview. In Yahoo! Learning to Rank Challenge, pages 1--24, 2011.Google Scholar
- O. Chapelle, S. Ji, C. Liao, E. Velipasaoglu, L. Lai, and S.-L. Wu. Intent-based diversification of web search results: metrics and algorithms. Information Retrieval, 14(6):572--592, 2011. Google ScholarDigital Library
- O. Chapelle, D. Metlzer, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. In CIKM, pages 621--630, 2009. Google ScholarDigital Library
- H. Chen and D. R. Karger. Less is more: probabilistic models for retrieving fewer relevant documents. In SIGIR, pages 429--436, 2006. Google ScholarDigital Library
- H. Cui, J.-R. Wen, J.-Y. Nie, and W.-Y. Ma. Query expansion by mining user logs. IEEE TKDE, 15(4):829--839, 2003. Google ScholarDigital Library
- Z. Dou, S. Hu, K. Chen, R. Song, and J.-R. Wen. Multi-dimensional search result diversification. In WSDM, pages 475--484, 2011. Google ScholarDigital Library
- M. D. Dunlop. Time, relevance and interaction modelling for information retrieval. In SIGIR Forum, volume 31, pages 206--213, 1997. Google ScholarDigital Library
- J. He, V. Hollink, and A. de Vries. Combining implicit and explicit topic representations for result diversification. In SIGIR, pages 851--860, 2012. Google ScholarDigital Library
- K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM TOIS, 20(4):422--446, 2002. Google ScholarDigital Library
- K. Järvelin, S. L. Price, L. M. L. Delcambre, and M. L. Nielsen. Discounted cumulated gain based evaluation of multiple-query ir sessions. In ECIR, pages 4--15, 2008. Google ScholarDigital Library
- T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In SIGIR, pages 154--161, 2005. Google ScholarDigital Library
- M. P. Kato, T. Sakai, and K. Tanaka. Structured query suggestion for specialization and parallel movement: effect on search behaviors. In WWW, pages 389--398, 2012. Google ScholarDigital Library
- M. P. Kato, T. Yamamoto, H. Ohshima, and K. Tanaka. Investigating users' query formulations for cognitive search intents. In SIGIR, pages 577--586, 2014. Google ScholarDigital Library
- D. Lagun, C.-H. Hsieh, D. Webster, and V. Navalpakkam. Towards better measurement of attention and satisfaction in mobile search. In SIGIR, pages 113--122, 2014. Google ScholarDigital Library
- S. Liang, Z. Ren, and M. de Rijke. Fusion helps diversification. In SIGIR, pages 303--312, 2014. Google ScholarDigital Library
- T.-Y. Liu, J. Xu, T. Qin, W. Xiong, and H. Li. Letor: Benchmark dataset for research on learning to rank for information retrieval. In SIGIR 2007 workshop on learning to rank for information retrieval, pages 3--10, 2007.Google Scholar
- F. Radlinski, R. Kleinberg, and T. Joachims. Learning diverse rankings with multi-armed bandits. In ICML, pages 784--791, 2008. Google ScholarDigital Library
- T. Sakai, Z. Dou, T. Yamamoto, Y. Liu, M. Zhang, R. Song, M. P. Kato, and M. Iwata. Overview of the NTCIR-10 INTENT-2 Task. In NTCIR-10, 2013.Google Scholar
- R. L. Santos, C. Macdonald, and I. Ounis. Exploiting query reformulations for web search result diversification. In WWW, pages 881--890, 2010. Google ScholarDigital Library
- M. D. Smucker and C. L. Clarke. Time-based calibration of effectiveness measures. In SIGIR, pages 95--104, 2012. Google ScholarDigital Library
- R. Song, M. Zhang, T. Sakai, M. P. Kato, Y. Liu, M. Sugimoto, Q. Wang, and N. Orii. Overview of the NTCIR-9 INTENT Task. In the 9th NTCIR Workshop Meeting on Evaluation of Information Access Technologies (NTCIR-9), pages 82--105, 2011.Google Scholar
- H.-T. Yu and F. Ren. Search result diversification via filling up multiple knapsacks. In CIKM, pages 609--618, 2014. Google ScholarDigital Library
- C. X. Zhai, W. W. Cohen, and J. Lafferty. Beyond independent relevance: methods and evaluation metrics for subtopic retrieval. In SIGIR, pages 10--17, 2003. Google ScholarDigital Library
- Y. Zhu, Y. Lan, J. Guo, X. Cheng, and S. Niu. Learning for search result diversification. In SIGIR, pages 293--302, 2014. Google ScholarDigital Library
Index Terms
- To Suggest, or Not to Suggest for Queries with Diverse Intents: Optimizing Search Result Presentation
Recommendations
Search Result Diversification Based on Hierarchical Intents
CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge ManagementA large percentage of queries issued to search engines are broad or ambiguous. Search result diversification aims to solve this problem, by returning diverse results that can fulfill as many different information needs as possible. Most existing intent-...
Learning to suggest: a machine learning framework for ranking query suggestions
SIGIR '12: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrievalWe consider the task of suggesting related queries to users after they issue their initial query to a web search engine. We propose a machine learning approach to learn the probability that a user may find a follow-up query both useful and relevant, ...
Finding dimensions for queries
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge managementWe address the problem of finding multiple groups of words or phrases that explain the underlying query facets, which we refer to as query dimensions. We assume that the important aspects of a query are usually presented and repeated in the query's top ...
Comments