Abstract
To facilitate both the understanding and the discovery of information, we need to utilize multiple sources of evidence, integrate a variety of methodologies, and combine human capabilities with those of the machine. The Web Information Discovery Integrated Tool (WIDIT) Laboratory at the School of Library and Information Science, Indiana University-Bloomington, houses several projects that employ this idea of multi-level fusion in the areas of information retrieval and knowledge discovery. This paper describes a Web search optimization study by the TREC research group of WIDIT, who explores a fusion-based approach to enhancing retrieval performance on the Web. In the study, we employed both static and dynamic tuning methods to optimize the fusion formula that combines multiple sources of evidence. By static tuning, we refer to the typical stepwise tuning of system parameters based on training data. “Dynamic tuning”, the key idea of which is to combine the human intelligence, especially pattern recognition ability, with the computational power of the machine, involves an interactive system tuning process that facilitates fine-tuning of the system parameters based on the cognitive analysis of immediate system feedback. The rest of the paper is organized as follows. The next section discusses related work in Web information retrieval (IR). Section 3 details the WIDIT approach to Web IR, followed by the description of our experiment using the TREC .gov data in section 4 and the discussion of results in section 5.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Amitay, E., Carmel, D., Darlow, A., Lempel, R., Soffer, A.: Topic Distillation with Knowledge Agents. In: Proceedings of the11th Text Retrieval Conference (TREC 2002), pp. 263–272 (2003)
Bartell, B.T., Cottrell, G.W., Belew, R.K.: Automatic combination of multiple ranked retrieval systems. In: Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval (1994)
Buckley, C., Salton, G., Allan, J., Singhal, A.: Automatic query expansion using SMART: TREC 3. In: Proceeding of the 3rd Text Rerieval Conference (TREC-3), pp. 1–19 (1995)
Buckley, C., Singhal, A., Mitra, M.: Using query zoning and correlation within SMART: TREC 5. In: Proceeding of the 5th Text REtrieval Conference (TREC-5), pp. 105–118 (1997)
Craswell, N., Hawking, D.: Overview of the TREC-2002 Web track. In: Proceedings of the 11th Text Retrieval Conference (TREC 2002), pp. 86–95 (2003)
Craswell, N., Hawking, D., Robertson, S.: Effective site finding using link anchor information. In: Proceedings of the 24th ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 250–257 (2001)
Fox, E.A., Shaw, J.A.: Combination of multiple searches. In: Proceeding of the3rd Text Rerieval Conference (TREC-3), pp. 105–108 (1995)
Frakes, W.B., Baeza-Yates, R. (eds.): Information retrieval: Data structures & algorithms. Prentice Hall, Englewood Cliffs (1992)
Gurrin, C., Smeaton, A.F.: Dublin City University experiments in connectivity analysis for TREC-9. In: Proceedings of the 9th Text Retrieval Conference (TREC-9), pp. 179–188 (2001)
Hawking, D., Craswell, N.: Overview of the TREC-2001 Web track. In: Proceedings of the 10th Text Retrieval Conference (TREC 2001), pp. 25–31 (2002)
Hawking, D., Craswell, N., Thistlewaite, P., Harman, D.: Results and challenges in web search evaluation. In: Proceedings of the 8th WWW Conference, pp. 243–252 (1999)
Hawking, D., Voorhees, E., Craswell, N., Bailey, P.: Overview of the TREC-8 web track. In: Proceedings of the 8th Text Retrieval Conference (TREC-8), pp. 131–148 (2000)
Hölscher, C., Strube, G.: Web search behavior of internet experts and newbies. In: Proceedings of the 9th International WWW Conference (2000)
Kraaij, W., Westerveld, T., Hiemstra, D.: The importance of prior probabilities for entry page search. In: Proceedings of the 25th ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 27–34 (2002)
Lee, J.H.: Analyses of multiple evidence combination. In: Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 267–276 (1997)
MacFarlane, A.: Pliers at TREC 2002. In: Proceedings of the 11th Text Retrieval Conference (TREC 2002), pp. 152–155 (2003)
Robertson, S.E., Walker, S.: Some simple approximations to the 2-Poisson model for probabilistic weighted retrieval. In: Proceedings of the 17th ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 232–241 (1994)
Savoy, J., Picard, J.: Report on the TREC-8 Experiment: Searching on the Web and in Distributed Collections. In: Proceedings of the 8th Text Retrieval Conference (TREC-8), pp. 229–240 (1998)
Savoy, J., Rasolofo, Y.: Report on the TREC-9 experiment: Link-based retrieval and distributed collections. In: Proceedings of the 9th Text Retrieval Conference (TREC-9), pp. 579–516 (2001)
Silverstein, C., Henzinger, M., Marais, H., Moricz, M.: Analysis of a very large AltaVista query log. Technical Report 1998-014, COMPAQ System Research Center (1998)
Singhal, A., Buckley, C., Mitra, M.: Pivoted document length normalization. In: Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 21–29 (1996)
Singhal, A., Kaszkiel, M.: A case study in Web search using TREC algorithms. In: Proceedings of the 11th International WWW Conference, pp. 708–716 (2001)
Tomlinson, S.: Robust, Web and Genomic retrieval with Hummingbird SearchServer at TREC 2003. In: Proceedings of the 12th Text Retrieval Conference (TREC 2003), pp. 254–267 (2003)
Thompson, P.: A combination of expert opinion approach to probabilistic information retrieval, part 1: The conceptual model. Information Processing & Management 26(3), 371–382 (1990)
Voorhees, E., Harman, D.: Overview of the Eighth Text Retrieval Conference. In: Proceedings of the 8th Text Retrieval Conference (TREC-8), pp. 1–24 (2000)
Yang, K.: Combining Text-, Link-, and Classification-based Retrieval Methods to Enhance Information Discovery on the Web (Doctoral Dissertation. University of North Carolina) (2002a)
Yang, K.: Combining Text- and Link-based Retrieval Methods for Web IR. In: Proceedings of the 10th Text Rerieval Conference (TREC 2001), pp. 609–618 (2002b)
Zhang, M., Song, R., Lin, C., Ma, S., Jiang, Z., Jin, Y., Liu, Y., Zhao, L.: THU TREC 2002: Web Track Experiments. In: Proceedings of the 11th Text Retrieval Conference (TREC 2002), pp. 591–594 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yang, K., Yu, N. (2005). WIDIT: Fusion-Based Approach to Web Search Optimization. In: Lee, G.G., Yamada, A., Meng, H., Myaeng, S.H. (eds) Information Retrieval Technology. AIRS 2005. Lecture Notes in Computer Science, vol 3689. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11562382_16
Download citation
DOI: https://doi.org/10.1007/11562382_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29186-2
Online ISBN: 978-3-540-32001-2
eBook Packages: Computer ScienceComputer Science (R0)