skip to main content
research-article

"Revisiting information retrieval tasks with user behavior models" by Yiqun Liu and Jiaxin Mao with Martin Vesely as coordinator

Published: 13 February 2020 Publication History

Abstract

Analyzing and modeling user behavior in the Information Retrieval (IR) process is important for IR research. In this paper, we propose a new research schema in IR research. In this schema, we first investigate humans' cognitive process when completing a specific IR task and build cognitive models for that task. The findings in the investigation and the proposed cognitive models are then utilized to improve machines' performance in the IR task. Through this research schema, we revisit three IR tasks. In the first study, we carefully analyze users' clicking behavior in mobile search and propose a mobile click model to extract relevance feedback from the mobile search logs. In the second study, we conduct an eye-tracking study to investigate how human assessors read a document during relevance judgment task and adopt the findings in building a novel retrieval model that can better approximate humans' relevance judgment. In the last study, we conduct another eye-tracking study to investigate humans' reading behavior when completing the reading comprehension task. We build a prediction model for user attention and leverage the predicted attention signals to improve the machine reading comprehension model. By successfully adopting this research schema to three different IR tasks, we demonstrate its effectiveness and generalizability.

References

[1]
Chapelle, O. and Zhang, Y. 2009. A dynamic bayesian network click model for web search ranking. In Proceedings of the 18th international conference on World wide web. ACM, 1--10.
[2]
Chuklin, A., Markov, I., and Rijke, M. d. 2015. Click models for web search. Synthesis Lectures on Information Concepts, Retrieval, and Services 7, 3, 1--115.
[3]
Craswell, N., Zoeter, O., Taylor, M., and Ramsey, B. 2008. An experimental comparison of click position-bias models. In Proceedings of the 2008 international conference on web search and data mining. ACM, 87--94.
[4]
Dupret, G. E. and Piwowarski, B. 2008. A user browsing model to predict search engine click data from past observations. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 331--338.
[5]
Fang, H., Tao, T., and Zhai, C. 2004. A formal study of information retrieval heuristics. In Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 49--56.
[6]
Fuhr, N. 2008. A probability ranking principle for interactive information retrieval. Information Retrieval 11, 3, 251--265.
[7]
Joachims, T., Granka, L. A., Pan, B., Hembrooke, H., and Gay, G. 2005. Accurately interpreting clickthrough data as implicit feedback. In Sigir. Vol. 5. 154--161.
[8]
Li, X., Liu, Y., Mao, J., He, Z., Zhang, M., and Ma, S. 2018. Understanding reading attention distribution during relevance judgement. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. ACM, 733--742.
[9]
Li, X., Mao, J., Wang, C., Liu, Y., Zhang, M., and Ma, S. 2019. Teach machine how to read: Reading behavior inspired relevance estimation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM.
[10]
Liu, Z., Liu, Y., Zhou, K., Zhang, M., and Ma, S. 2015. Influence of vertical result in web search examination. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 193--202.
[11]
Mao, J., Liu, Y., Kando, N., Luo, C., Zhang, M., and Ma, S. 2018. Investigating result usefulness in mobile search. In European Conference on Information Retrieval. Springer, 223--236.
[12]
Mao, J., Luo, C., Zhang, M., and Ma, S. 2018. Constructing click models for mobile search. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. SIGIR '18. ACM, New York, NY, USA, 775--784.
[13]
Moffat, A., Thomas, P., and Scholer, F. 2013. Users versus models: What observation tells us about effectiveness metrics. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management. ACM, 659--668.
[14]
Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
[15]
Robertson, S. E. 1977. The probability ranking principle in ir. Journal of documentation 33, 4, 294--304.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGWEB Newsletter
ACM SIGWEB Newsletter  Volume 2019, Issue Autumn
Autumn 2019
39 pages
ISSN:1931-1745
EISSN:1931-1435
DOI:10.1145/3352683
Issue’s Table of Contents
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 February 2020
Published in SIGWEB Volume 2019, Issue Autumn

Check for updates

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 73
    Total Downloads
  • Downloads (Last 12 months)9
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media