Skip to main content
SpringerLink
Log in

Estimating Credibility of User Clicks with Mouse Movement and Eye-Tracking Information

  • Conference paper
Natural Language Processing and Chinese Computing (NLPCC 2014)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 496))

Abstract

Click-through information has been regarded as one of the most important signals for implicit relevance feedback in Web search engines. Because large variation exists in users’ personal characteristics, such as search expertise, domain knowledge, and carefulness, different user clicks should not be treated as equally important. Different from most existing works that try to estimate the credibility of user clicks based on click-through or querying behavior, we propose to enrich the credibility estimation framework with mouse movement and eye-tracking information. In the proposed framework, the credibility of user clicks is evaluated with a number of metrics in which a user in the context of a certain search session is treated as a relevant document classifier. With an experimental search engine system that collects click-through, mouse movement, and eye movement data simultaneously, we find that credible user behaviors could be separated from non-credible ones with a number of interaction behavior features. Further experimental results indicate that relevance prediction performance could be improved with the proposed estimation framework.

This work was supported by National Key Basic Research Program (2015CB358700) and Natural Science Foundation (61472206, 61073071) of China.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aula, A., Majaranta, P., Räihä, K.-J.: Eye-tracking reveals the personal styles for search result evaluation. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 1058–1061. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  2. Chapelle, O., Zhang, Y.: A dynamic bayesian network click model for web search ranking. In: Proceedings of the 18th International Conference on World Wide Web, pp. 1–10. ACM (2009)

    Google Scholar 

  3. Craswell, N., Zoeter, O., Taylor, M., Ramsey, B.: An experimental comparison of click position-bias models. In: Proceedings of the 2008 International Conference on Web Search and Data Mining, pp. 87–94. ACM (2008)

    Google Scholar 

  4. Cutrell, E., Guan, Z.: What are you looking for?: an eye-tracking study of information usage in web search. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 407–416. ACM (2007)

    Google Scholar 

  5. Dou, Z., Song, R., Wen, J.R.: A large-scale evaluation and analysis of personalized search strategies. In: Proceedings of the 16th International Conference on World Wide Web, pp. 581–590. ACM (2007)

    Google Scholar 

  6. Guo, Q., Lagun, D., Agichtein, E.: Predicting web search success with fine-grained interaction data. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management, pp. 2050–2054. ACM (2012)

    Google Scholar 

  7. Hölscher, C., Strube, G.: Web search behavior of internet experts and newbies. Computer Networks 33(1), 337–346 (2000)

    Article  Google Scholar 

  8. Huang, J., White, R., Buscher, G.: User see, user point: gaze and cursor alignment in web search. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1341–1350. ACM (2012)

    Google Scholar 

  9. Huang, J., White, R.W., Buscher, G., Wang, K.: Improving searcher models using mouse cursor activity. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 195–204. ACM (2012)

    Google Scholar 

  10. Huang, J., White, R.W., Dumais, S.: No clicks, no problem: using cursor movements to understand and improve search. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1225–1234. ACM (2011)

    Google Scholar 

  11. Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 133–142. ACM (2002)

    Google Scholar 

  12. Joachims, T., Granka, L., Pan, B., Hembrooke, H., Gay, G.: Accurately interpreting clickthrough data as implicit feedback. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 154–161. ACM (2005)

    Google Scholar 

  13. Just, M.A., Carpenter, P.A.: A theory of reading: From eye fixations to comprehension. Psychological Review 87, 329–354 (1980)

    Article  Google Scholar 

  14. Liu, Y., Zhang, M., Ru, L., Ma, S.: Automatic query type identification based on click through information. In: Ng, H.T., Leong, M.-K., Kan, M.-Y., Ji, D. (eds.) AIRS 2006. LNCS, vol. 4182, pp. 593–600. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  15. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  16. Richardson, M., Dominowska, E., Ragno, R.: Predicting clicks: Estimating the click-through rate for new ads. In: Proceedings of the 16th International Conference on World Wide Web, pp. 521–530. ACM (2007)

    Google Scholar 

  17. Rodden, K., Fu, X., Aula, A., Spiro, I.: Eye-mouse coordination patterns on web search results pages. In: CHI 2008 Extended Abstracts on Human Factors in Computing Systems, pp. 2997–3002. ACM (2008)

    Google Scholar 

  18. Speicher, M., Both, A., Gaedke, M.: Tellmyrelevance!: Predicting the relevance of web search results from cursor interactions. In: Proceedings of the 22nd ACM International Conference on Conference on Information & Knowledge Management, pp. 1281–1290. ACM (2013)

    Google Scholar 

  19. Stehman, S.V.: Selecting and interpreting measures of thematic classification accuracy. Remote Sensing of Environment 62(1), 77–89 (1997)

    Article  Google Scholar 

  20. Wang, C., Liu, Y., Zhang, M., Ma, S., Zheng, M., Qian, J., Zhang, K.: Incorporating vertical results into search click models. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2013, pp. 503–512. ACM, New York (2013), http://doi.acm.org/10.1145/2484028.2484036

    Google Scholar 

  21. White, R.W., Drucker, S.M.: Investigating behavioral variability in web search. In: Proceedings of the 16th International Conference on World Wide Web, pp. 21–30. ACM (2007)

    Google Scholar 

  22. White, R.W., Dumais, S.T., Teevan, J.: Characterizing the influence of domain expertise on web search behavior. In: Proceedings of the Second ACM International Conference on Web Search and Data Mining, pp. 132–141. ACM (2009)

    Google Scholar 

  23. White, R.W., Morris, D.: Investigating the querying and browsing behavior of advanced search engine users. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 255–262. ACM (2007)

    Google Scholar 

  24. Xing, Q., Liu, Y., Zhang, M., Ma, S., Zhang, K.: Characterizing expertise of search engine users. In: Banchs, R.E., Silvestri, F., Liu, T.-Y., Zhang, M., Gao, S., Lang, J. (eds.) AIRS 2013. LNCS, vol. 8281, pp. 380–391. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Intelligent Technology and Systems, Tsinghua University, Beijing, 1000084, China

    Jiaxin Mao, Yiqun Liu, Min Zhang & Shaoping Ma

  2. Tsinghua National Laboratory for Information Science and Technology, Tsinghua University, Beijing, 1000084, China

    Jiaxin Mao, Yiqun Liu, Min Zhang & Shaoping Ma

  3. Department of Computer Science and Technology, Tsinghua University, Beijing, 1000084, China

    Jiaxin Mao, Yiqun Liu, Min Zhang & Shaoping Ma

Authors
  1. Jiaxin Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiqun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shaoping Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, 100190, Beijing, China

    Chengqing Zong

  2. Dept. of Computer Science and Operations Research, University of Montreal, Montreal, Quebec, Canada

    Jian-Yun Nie

  3. Peking University, Beijing, China

    Dongyan Zhao

  4. Institute of Computer Science & Technology, Peking University, 100871, Beijing, China

    Yansong Feng

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Mao, J., Liu, Y., Zhang, M., Ma, S. (2014). Estimating Credibility of User Clicks with Mouse Movement and Eye-Tracking Information. In: Zong, C., Nie, JY., Zhao, D., Feng, Y. (eds) Natural Language Processing and Chinese Computing. NLPCC 2014. Communications in Computer and Information Science, vol 496. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-45924-9_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-45924-9_24

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-45923-2

  • Online ISBN: 978-3-662-45924-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation