Abstract
User behavior information analysis has been shown important for optimization and evaluation of Web search and has become one of the major areas in both information retrieval and knowledge management researches. This paper focuses on users’ searching behavior reliability study based on large scale query and click-through logs collected from commercial search engines. The concept of reliability is defined in a probabilistic notion. The context of user click behavior on search results is analyzed in terms of relevance. Five features, namely query number, click entropy, first click ratio, last click ratio, and rank position, are proposed and studied to separate reliable user clicks from the others. Experimental results show that the proposed method evaluates the reliability of user behavior effectively. The AUC value of the ROC curve is 0.792, and the algorithm maintains 92.8% relevant clicks when filtering out 40% low quality clicks.
Supported by the Chinese National Key Foundation Research & Development Plan (2004CB318108), Natural Science Foundation (60621062, 60503064, 60736044) and National 863 High Technology Project (2006AA01Z141).
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Yates, R., Tiberi, A.: Extracting semantic relations from query logs. In: Proceedings of the 13th ACM SIGKDD international Conference on Knowledge Discovery and Data Mining, pp. 76–85. ACM, New York (2007)
Fuxman, A., Tsaparas, P., Achan, K., Agrawal, R.: Using the wisdom of the crowds for keyword generation. In: Proceeding of the 17th international Conference on World Wide Web, pp. 61–70. ACM, New York (2008)
Joachims, T., Freitag, D., Mitchell, T.: WebWatcher: a tour guide for the world wide Web. In: IJCAI 1997, vol. 1, pp. 770–777. Morgan Kaufmann, San Francisco (1997)
Agichtein, E., Brill, E., Dumais, S., Ragno, R.: Learning user interaction models for predicting web search result preferences. In: Proceedings of the 29th ACM SIGIR Conference on Research and Development in information Retrieval, pp. 3–10. ACM, New York (2006)
Joachims, T., Granka, L., Pan, B., Hembrooke, H., Gay, G.: Accurately interpreting clickthrough data as implicit feedback. In: Proceedings of the 28th ACM SIGIR Conference on Research and Development in information Retrieval, pp. 154–161. ACM, New York (2005)
Tan, P., Kumar, V.: Modeling of web robot navigational patterns. In: Proceedings ACM WebKDD Workshop (2000)
Tan, P., Kumar, V.: Discovery of web robot sessions based on their navigational patterns. Data Mining and Knowledge Discovery 6, 9–35 (2002)
Yates, R., Hurtado, C., Mendoza, M., Dupret, G.: Modeling user search behavior. In: Proceedings of the 3th Latin American Web Congress. LA-WEB, p. 242. IEEE Computer Society, Los Alamitos (2005)
Kammenhuber, N., Luxenburger, J., Feldmann, A., Weikum, G.: Web search clickstreams. In: Proceedings of the 6th ACM SIGCOMM Conference on internet Measurement, pp. 245–250. ACM, New York (2006)
Craswell, N., Zoeter, O., Taylor, M., Ramsey, B.: An experimental comparison of click position-bias models. In: Proceedings of the international Conference on Web Search and Web Data Mining, pp. 87–94. ACM, New York (2008)
Guo, F., Liu, C., Wang, Y.M.: Efficient multiple-click models in web search. In: Proceedings of the 2nd ACM international Conference on Web Search and Data Mining, pp. 124–131. ACM, New York (2009)
Liu, Y., Cen, R., Zhang, M., Ru, L., Ma, S.: Automatic Search Engine Evaluation Based On User Behavior Analysis. Journal of Software 19(11), 3023–3032 (2008)
Agrawal, R., Halverson, A., Kenthapadi, K., Mishra, N., Tsaparas, P.: Generating labels from clicks. In: Baeza-Yates, R., Boldi, P., Ribeiro-Neto, B., Cambazoglu, B.B. (eds.) Proceedings of the 2nd ACM international Conference on Web Search and Data Mining (2009)
Yu, H., Liu, Y., Zhang, M., Ru, L., Ma, S.: Research in Search Engine User Behavior Based on Log Analysis. Journal of Chinese Information Processing 21(1), 109–114 (2007)
Shannon, C.E.: A Mathematical Theory of Communication. Bell System Technical Journal 27, 379–423, 623–656 (1948)
Svore, K., Wu, Q., Burges, C., Raman, A.: Improving Web Spam Classification using Rank-time Features. In: Proceedings of AIRWeb 2007, pp. 9–16. ACM, New York (2007)
Sadagopan, N., Li, J.: Characterizing typical and atypical user sessions in clickstreams. In: Proceedings of the 17th international Conference on World Wide Web, pp. 885–894. ACM, New York (2008)
Agichtein, E., Brill, E., Dumais, S.: Improving web search ranking by incorporating user behavior information. In: Proceedings of the 29th Annual international ACM SIGIR Conference on Research and Development in information Retrieval, pp. 19–26. ACM, New York (2006)
Dou, Z., Song, R., Yuan, X., Wen, J.: Are click-through data adequate for learning web search rankings? In: Proceeding of the 17th ACM Conference on information and Knowledge Management, pp. 73–78. ACM, New York (2008)
Carletta, J.: Assessing Agreement on Classification Tasks: The Kappa Statistic. Computational Linguistics 22(2), 249–254 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cen, R., Liu, Y., Zhang, M., Ru, L., Ma, S. (2009). Study on the Click Context of Web Search Users for Reliability Analysis. In: Lee, G.G., et al. Information Retrieval Technology. AIRS 2009. Lecture Notes in Computer Science, vol 5839. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04769-5_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-04769-5_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04768-8
Online ISBN: 978-3-642-04769-5
eBook Packages: Computer ScienceComputer Science (R0)