Abstract
Web search results enjoy an increasing importance in our daily lives. But what can be said about their quality, especially when querying a controversial issue? The traditional information retrieval metrics of precision and recall do not provide much insight in the case of web information retrieval. In this paper we examine new ways of evaluating quality in search results: coverage and independence. We give examples on how these new metrics can be calculated and what their values reveal regarding the two major search engines, Google and Yahoo. We have found evidence of low coverage for commercial and medical controversial queries, and high coverage for a political query that is highly contested. Given the fact that search engines are unwilling to tune their search results manually, except in a few cases that have become the source of bad publicity, low coverage and independence reveal the efforts of dedicated groups to manipulate the search results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Amento, B., Terveen, L., Hill, W.: Does authority mean quality? Predicting expert quality ratings of web documents. In: Proceedings of the Twenty-Third Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York (2000)
Berenson, A.: On hair-trigger wall street, a stock plunges on fake news. New York Times (2000)
Brin, S., Page, L.: The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems 30(1–7), 107–117 (1998)
Google. The Google API, google, inc. (2003), http://code.google.com/apis/
Graham, L., Metaxas, P.T.: “Of course it’s true; i saw it on the internet!”: Critical thinking in the internet era. Commun. ACM 46(5), 70–75 (2003)
Gyuongyi, Z., Garcia-Molina, H.: Web spam taxonomy. In: Proceedings of the First International Workshop on Adversarial Information Retrieval on the Web, Chiba, Japan (2005)
Manning, C., Raghavan, P., Schultze, H.: Introduction to Information Retrieval, forthcoming edn. Cambridge Press, Cambridge (2008)
Metaxas, P.T., Destefano, J.: Web spam, propaganda and trust. In: Proceedings of the First International Workshop on Adversarial Information Retrieval on the Web, Chiba, Japan (2005)
Moran, M., Hunt, B.: Search Engine Marketing. IBM Press, New Jersey (2006)
Ntoulas, A., Cho, J., Olston, C.: What’s new on the web? the evolution of the web from a search engine perspective. In: Proceedings of the WWW 2004 Conference, New York, NY (2004)
Silverstein, C., Marais, H., Henzinger, M., Moricz, M.: Analysis of a very large web search engine query log. SIGIR Forum 33(1), 6–12 (1999)
Vedder, A.: Misinformation through the internet: Epistemology and ethics. Intersentia, Antwerpen, Gronigen, Oxford (2001)
Yahoo, The Yahoo search API, Yahoo, inc. (2006), http://developer.yahoo.com/search/
Wellness letter, UC Berkeley (June 2003), http://www.berkeleywellness.com/html/ds/dsGrowthHormone.php (retrieved October 10, 2008)
Wikipedia entry on Growth hormone, http://en.wikipedia.org/wiki/Hgh (retrieved October 10, 2008)
Wikipedia entry on ADHD, http://en.wikipedia.org/wiki/Hyperkinetic_conduct_disorder (retrieved October 10, 2008)
Wikipedia entry on Morality and Legality of Abortion, http://en.wikipedia.org/wiki/Morality_legality_of_abortion (retrieved October 10, 2008)
Fagin, R., Kumar, R., Sivakumar, D.: Comparing top k lists. SIAM J. on Discrete Math. 17(1), 134–160 (2003)
Bar-Ilan, J., Mat-Hassan, M., Levene, M.: Methods for comparing rankings of search engine results. Computer Networks 50(10), 1448–1463 (2006)
Metaxas, P.T., and Ivanova, L.: Coverage and Independence - Defining Quality in Web Search Results. In: Proceedings of the International Conference on Web Information Systems and Technologies (WEBIST), Madeira, Portugal (2008)
Online article entitled Google Kills Bushs Miserable Failure Search & Other Google Bombs, http://searchengineland.com/google-kills-bushs-miserable-failure-search-other-google-bom bs-10363.php (retrieved October 10, 2008)
McCown, F., Nelson, M.L.: Agreeing to Disagreeing: Search Engines and their Public Interfaces. In: The Proc. of ACM JCDL 2007, Vancouver, Canada (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Metaxas, P.T., Ivanova, L., Mustafaraj, E. (2009). New Quality Metrics for Web Search Results. In: Cordeiro, J., Hammoudi, S., Filipe, J. (eds) Web Information Systems and Technologies. WEBIST 2008. Lecture Notes in Business Information Processing, vol 18. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01344-7_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-01344-7_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-01343-0
Online ISBN: 978-3-642-01344-7
eBook Packages: Computer ScienceComputer Science (R0)