Skip to main content

Improving the Evaluation of Web Search Systems

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2633))

Included in the following conference series:

  • 1287 Accesses

Abstract

Linkage analysis as an aid to web search has been assumed to be of significant benefit and we know that it is being implemented by many major Search Engines. Why then have few TREC participants been able to scientifically prove the benefits of linkage analysis over the past three years? In this paper we put forward reasons why disappointing results have been found and we identify the linkage density requirements of a dataset to faithfully support experiments into linkage analysis. We also report a series of linkage-based retrieval experiments on a more densely linked dataset culled from the TREC web documents.

The work presented in this paper is based on research undertaken by the first author as a postgraduate student while working on his Ph.D. dissertation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Amento, B., Terveen, L. and Hill, W.: Does ‘Authority’ mean quality? Predicting Expert Quality Ratings of Web Document. Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in IR (2000)

    Google Scholar 

  2. Page L., Brin S., Motwani R. and Winograd T.: The Page Rank Citation Ranking: Bringing Order to the Web. Stanford Digital Libraries working paper (1997) 0072

    Google Scholar 

  3. Brin S. and Page L.: The Anatomy of a Large-Scale Hypertextual Web Search Engine. Proceedings of the 7th International WWW Conference (1998)

    Google Scholar 

  4. Kleinberg, J.: Authorative Sources in a Hyperlinked Environment. Proceedings of the 9th ACM-SIAM Symposium on Discrete Algorithms (1998)

    Google Scholar 

  5. Bharat K. and Henzinger M.: Improved Algorithms for Topic Distillation in a Hyperlinked Environment. Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in IR (1998)

    Google Scholar 

  6. Hawking D., Voorhees E., Craswell N. and Bailey P.: Overview of the TREC-8 Web Track. Proceedings of the 8th Annual TREC Conference”, (1999)

    Google Scholar 

  7. Gurrin, C. and Smeaton, A. F.: Connectivity Analysis Approaches to Increasing Precision in Retrieval from Hyperlinked Documents. Proceedings of the 8th Annual TREC Conference”, November 16–19 (1999)

    Google Scholar 

  8. Hawking D.:-Overview of the TREC-9 Web Track. Proceedings of the 9th Annual TREC Conference”, November 16–19 (2000)

    Google Scholar 

  9. Gurrin, C. and Smeaton, A.F.: Dublin City University Experiments in Connectivity Analysis for TREC-9. Proceedings of the 9th Annual TREC Conference”, (2000)

    Google Scholar 

  10. Bailey, P., Craswell, N. and Hawking, D.: Engineering a multi-purpose test collection for Web retrieval experiments. Information Processing and Management (2001)

    Google Scholar 

  11. Wu, L., Huang, X., Niu, J., Xia, Y., Feng, Z., Zhou, Y.: FDU at TREC 2002: Filtering, Q&A, Web and Video Tasks. Draft Proceedings of the 11th Annual TREC Conference, November 19–22 (2002)

    Google Scholar 

  12. Singhal, A. and Kaszkiel, M.: AT&T at TREC-9. Proceedings of the 9th Annual TREC Conference, November 16–19 (2000)

    Google Scholar 

  13. SOWS III: The Third State of the Web Survey, Available online at URL: http://www.pantos.org/atw/35654-a.html. (last visited November 2002)

  14. Murray B. and Moore A.: Sizing the Internet — A White Paper. Cyveillance, Inc., 2000. Available online at URL: http://www.cyveillance.com/web/corporate/white_papers.htm. (last visited November 2002)

  15. Broder A., Kumar R., Maghoul, F., Raghavan P., Rajagopalan S., Stata R., Tomkins A. and Weiner J.: Graph Structure in the Web. Proceedings of WWW9 (2000)

    Google Scholar 

  16. URouLette Random Web Page Generator, Available online at URL: http://www.uroulette.com. (last visited November 2002)

  17. Soboroff, I.: Does WT10g look like the Web?. Proceedings of the 27rd Annual International ACM SIGIR Conference on Research and Development in IR (2002)

    Google Scholar 

  18. Pennock, D., Flake, G., Lawrence, S., Glover, E. and Giles, C.: Winners don’t take all: Characterising the competition for links on the web. Proceedings of the National Academy of Sciences, Volume 99, Issue 8, (April 2002) 5207–5211

    Article  MATH  Google Scholar 

  19. Mitzenmacher M.: A Brief History of Generative Models for Power Law and Lognormal Distributions. Allerton (2001)

    Google Scholar 

  20. Faloutsos, M., Faloutsos, P., Faloutsos, C.: On Power-Law Relationships of the Internet Topology. Proceedings of ACM SIGCOMM 99 (1999)

    Google Scholar 

  21. Adamic, L. and Humberman B.: The Web’s Hidden Order. Communications of the ACM, Vol. 44, No. 9 (2001)

    Google Scholar 

  22. Adamic, L.: Zipf, Power-laws, and Pareto — a ranking tutorial. Available online at URL: http://www.hpl.hp.com/shl/papers/ranking/. (last visited November 2002)

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Gurrin, C., Smeaton, A.F. (2003). Improving the Evaluation of Web Search Systems. In: Sebastiani, F. (eds) Advances in Information Retrieval. ECIR 2003. Lecture Notes in Computer Science, vol 2633. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36618-0_3

Download citation

  • DOI: https://doi.org/10.1007/3-540-36618-0_3

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-01274-0

  • Online ISBN: 978-3-540-36618-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics