Himanshu Sharma
Bernard J. Jansen
ABSTRACT
Measuring the information retrieval effectiveness of Web search engines can be expensive if human relevance judgments are required to evaluate search results. Using implicit user feedback for search engine evaluation provides a cost and time effective manner of addressing this problem. Web search engines can use human evaluation of search results without the expense of human evaluators. An additional advantage of this approach is the availability of real time data regarding system performance. Wecapture user relevance judgments actions such as print, save and bookmark, sending these actions and the corresponding document identifiers to a central server via a client application. We use this implicit feedback to calculate performance metrics, such as precision. We can calculate an overall system performance metric based on a collection of weighted metrics.
- D. Kelly and J. Teevan, "Implicit Feedback for Inferring User Preference: A Bibliography," SIGIR Forum, vol. 37, pp. 18--28, 2003. Google Scholar
Digital Library
- S. Dumais, T. Joachims, K. Bharat, and A. Weigend, "SIGIR 2003 Workshop Report: Implicit measures of User Interests and Preferences," SIGIR Forum, vol. 37, pp. 50--54, 2003. Google ScholarDigital Library
- D. Oard and J. Kim, "Modeling Information Content Using Observable Behavior," in Proceedings of the 64th Annual Meeting of the American Society for Information Science and Technology, Washington, D.C., USA, 2001. pp. 38--45.Google Scholar
- R. Villa, M. Chalmers, "A framework for implicitly tracking data", Proceedings of the Second DELOS Network of Excellence Workshop on Personalisation and Recommender Systems in Digital Libraries, Dublin City University, Ireland, June 2001. pp 18--20.Google Scholar
Index Terms
- Automated evaluation of search engine performance via implicit user feedback
Recommendations
Improving Search Performance: A Lesson Learned from Evaluating Search Engines Using Thai Queries
This study initiates a systematic evaluation of web search engine performance using queries written in Thai. Statistical testing indicates that there are some significant differences in the performance of search engines. In addition to compare the ...
Brand and its effect on user perception of search engine performance
In this research we investigate the effect of search engine brand on the evaluation of searching performance. Our research is motivated by the large amount of search traffic directed to a handful of Web search engines, even though many have similar ...
Comments