Abstract
In distributed information retrieval systems, document overlaps occur frequently across results from different resources. This is especially the case for meta-search engines which merge results from several web search engines. This paper addresses the problem of merging results exploiting overlaps in order to achieve better performance. New algorithms for merging results are proposed, which take advantage of the use of duplicate documents in two ways: one correlates scores from different results; the other regards duplicates as increasing evidence of being relevant to the given query. An extensive experimentation has demonstrated that these methods are effective.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bowerman, B.L., O’Connell, R.T.: Linear Statistical Models: An Applied Approach. PWS-KENT Publishing Company (1990)
Callan, J.K., Lu, Z., Croft, W.: Searching distributed collections with inference networks. In: Proceedings of the 18th Annual International ACM SIGIR Conference, Seattle, USA, July 1995, pp. 21–28 (1995)
Calve, A.L., Savoy, J.: Database merging strategy based on logistic regression. Information Processing and Management 36(3), 341–359 (2000)
Craswell, N., Bailer, P., Hawking, D.: Server selection on the world wide web. In: Proceedings of the Fifth ACM Conference on Digital Libraries, San Antonio, CA, USA, June 2000, pp. 37–46 (2000)
Dreilinger, D., Howe, A.: Experiences with selectiong search engines using metasearch. ACM Transaction on Information Systems 15(3), 195–222 (1997)
Fox, E.A., Shaw, J.: Combination of multiple searchs. In: The Second Text REtrieval Conference (TREC-2), Gaitherburg, MD, USA (August 1994)
Gauch, S., Wang, G., Gomez, M.: Profusion: Intelligent fusion from multiple, distributed search engines. Journal of Universal Computer Science 2(9), 637–649 (1996)
Howe, A., Dreilinger, D.: Savvysearch: A meta-search engine that learns which search engines to query. AI Magazine 18(2), 19–25 (1997)
Inquirus, http://www.inquirus.com/
Lee, J.H.: Analysis of multiple evidence combination. In: Proceedings of the 20th Annual International ACM SIGIR Conference, Philadelphia, Pennsylvania, USA, July 1997, pp. 267–275 (1997)
Mearf, http://mearf.cs.umn.edu/
Meng, W., Yu, C., Liu, K.: Building efficient and effective metasearch engines. ACM Computing Surveys 34(1), 48–89 (2002)
Metacrawler, http://www.metacrawler.com/
Ogilvie, P., Callan, J.: Experiments using the lemur toolkit. In: Proceedings of the 2001 Text REtrieval Conference, Gaithersburg, Maryland, USA, November 2001, pp. 103–108 (2001)
Oztekin, B., Karypis, G., Kumar, V.: Expert agreement and content baed reranking in a meta search environment using mearf. In: Proceedings of the 11th International World Wide Web Conference, Honolulu, Hawaii, USA, May 2002, pp. 333–344 (2002)
Profusion, http://www.profusion.com/
Selberg, E., Etzioni, O.: Multi-service search and comparison using the metacrawler. In: Proceedings of the 4th International World Wide Web Conference, Paris, France (May 1996)
Selberg, E., Etzioni, O.: The metacrawler architecture for resource aggregion on the web. IEEE Expert 12(1), 8–14 (1997)
Si, L., Callan, J.: Using sampled data and regression to merge search enging results. In: Proceedings of the 25th Annual International ACM SIGIR Conference, Tempere, Finland, August 2002, pp. 19–26 (2002)
Voorhees, E.M., Gupta, N.K., Johnson-Laird, B.: Learning collection fusion strategies. In: Proceedings of the 18th Annual International ACM SIGIR Conference, Seattle, Washington, USA, July 1995, pp. 172–179 (1995)
Vort, C.C., Cottrell, G.A.: A fusion via a linear combination of scores. Information Retrieval 1(3), 151–173 (1999)
Wu, S., Gibb, F., Crestani, F.: Experiments with document archive size detection. In: Proceedings of the 25th European Conference on Information Retrieval Research, Pisa, Italy, April 2003, pp. 294–304 (2003)
Yu, C., Meng, G., Liu, K., Wu, W., Rishe, N.: Efficient and effective metasearch for a large number of text databases. In: Proceedings of the Eighth ACM International Conference on Information and Knowledge Management, Kansas City, USA, November 2001, pp. 217–224 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wu, S., Crestani, F., Gibb, F. (2004). New Methods of Results Merging for Distributed Information Retrieval. In: Callan, J., Crestani, F., Sanderson, M. (eds) Distributed Multimedia Information Retrieval. DIR 2003. Lecture Notes in Computer Science, vol 2924. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24610-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-540-24610-7_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20875-4
Online ISBN: 978-3-540-24610-7
eBook Packages: Springer Book Archive