Abstract
In this paper we propose a model for relevance feedback. Our model combines evidence from user's relevance assessments with algorithms describing how words are used within documents. We motivate the use of the Dempster-Shafer framework as an appropriate theory for modelling combination of evidence. This model also incorporates the uncertain nature of information retrieval and relevance feedback. We discuss the sources of uncertainty in combining evidence in information retrievel and the importance of combining evidence in relevance feedback. We also present results from a series of experiments that highlight various aspects of our approach and discuss our findings.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barry, C.L. and Schamber, L. (1998). Users' Criteria for Relevance Evaluation: A Cross-Situational Comparison. Information Processing and Management, 34(2/3), 219–236.
Belkin, N.J., Kantor, P., Fox, E.A., and Shaw, J.A. (1995). Combining the Evidence of Multiple Query Representations for Information Retrieval. Information Processing and Management, 31(3), 431–448.
Borlund, P. and Ingerwersen, P. (1997). The Development of a Method for the Evaluation of Interactive Information Retrieval Systems. Journal of Documentation, 53(5), 225–250.
Campbell, I. and van Rijsbergen, C.J. (1996). Ostensive Model of Information Needs. In Proceedings of the Second International Conference on Conceptions of Library and Information Science: Integration in Perspective (CoLIS 2), Copenhagen, pp. 251–268.
Chang, Y.K., Cirillo, C., and Razon, J. (1971). Evaluation of Feedback Retrieval Using Modified Freezing, Residual Collection & Test and Control Groups. In G. Salton (Ed.), The SMART Retrieval System―Experiments in Automatic Document Processing, Ch. 17, pp. 355-370.
Dempster, A.P. (1968). A Generalization of the Bayesian Inference. Journal of Royal Statistical Society, 30,205–447.
Denos, N., Berrut, C., and Mechkour, M. (1997). An Image System Based on theVisualization of System Relevance via Documents. Database and Expert Systems Applications(DEXA'97), 8th International Conference, Toulouse, pp. 214–224.
Ellis, D. (1989). A Behavioural Approach to Information System Design. Journal of Documentation, 45(3), 171–212.
Haines, D. and Croft, W.B. (1993). Relevance Feedback and Inference Networks. In Proceedings of the Sixteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Pittsburgh, pp. 2–11.
Harman, D. (1992). Ranking Algorithms. In W.B. Frakes and R. Baeza-Yates (Eds.), Information Retrieval: Data Structures & Algorithms, Ch. 14, pp. 363–392.
Ingwersen, P. (1994). Polyrepresentation of Information Needs and Semantic Entities: Elements of a Cognitive Theory for Information Retrieval Interaction. In Proceedings of the Seventeenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Dublin, pp. 101–110.
Lalmas, M. and Ruthven, I. (1998). Representing and Retrieving Structured Documents using the Dempster-Shafer Theory of Evidence: Modelling and Evaluation. Journal of Documentation, 54(5), 529–565.
Lee, J.H. (1998). Combining the Evidence of Different Relevance Feedback Methods for Information Retrieval. Information Processing and Management, 34(6), 681–691.
Robertson, S.E. and Sparck Jones, K. (1976). Relevance Weighting of Search Terms. Journal of the American Society of Information Science, 27, 129–146.
Rocchio, J.J. (1971). Relevance Feedback in Information Retrieval. In G. Salton (Ed.), The SMART Retrieval System: Experiments in Automatic Document Processing, Ch. 14, pp. 313–323, Prentice-Hall, Englew.
Ruthven, I. and Lalmas, M. (1999). Selective Relevance Feedback Using Term Characteristics. In Proceedings of the Third International Conference on Conceptions of Library and Information Science, CoLIS 3, Dubrovnik.
Ruthven, I., Lalmas, M., and van Rijsbergen, C.J. (1999). Retrieval Through Explanation: An Abductive Inference Approach to Relevance Feedback. 10th Irish Conference on Artificial Intelligence & Cognitive Science, Cork.
Saffioti, A. (1987). An AI View of the Treatment of Uncertainty. The Knowledge Engineering Review, 2(2), 75–97.
Salton, G. and Buckley, C. (1990). Improving Retrieval Performance by Relevance Feedback. Journal of the American Society for Information Science, 41(4), 288–297.
Schocken, S. and Hummel, R.A. (1993). On the Use of the Dempster Shafer Model in Information Indexing and Retrieval Applications. International Journal of Man-Machine Studies, 39, 1–17.
Shafer, G. (1976). A Mathematical Theory of Evidence. Princeton University Press.
Silva, I., Ribeiro-Neto, B., Calado, P., Moura, E., and Ziviani, N. (2000). Link-Based and Content-Based Evidential Information in a Belief Network Model. In Proceedings of the 23rd Annual InternationalACMSIGIR Conference on Research and Development in Information Retrieval, Athens, pp. 96–103.
Sparck Jones, K. (1972). A Statistical Interpretation of Term Specificity and its Application in Retrieval. Journal of Documentation, 28(1), 11–20.
Teixeira da Silva, W. and Luiz Milidiu, R. (1993). Belief Function Model for Information Retrieval. Journal of the American Society for Information Science, 44(1), 10–18.
Vakkari, P. (2000). Relevance and Contributing Information Types of Searched Documents in Performance. In Proceedings of the 23rd ACM Sigir Conference on Research and Development in Information Retrieval, Athens, pp. 2–9.
van Rijsbergen, C.J. (1979). Information Retrieval. 2nd edn. Stoneham, MA: Butterworths.
van Rijsbergen, C.J. (1992). Probabilistic Retrieval Revisited. The Computer Journal, 35(3), 291–298.
Voorhees, E.M. and Harman, D. (1996). Overviewof the FifthText REtrieval Conference (TREC-5). In Proceedings of the 6th Text Retrieval Conference, Gaitherburg, pp. 1–28. Nist Special Publication 500-238.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ruthven, I., Lalmas, M. Using Dempster-Shafer's Theory of Evidence to Combine Aspects of Information Use. Journal of Intelligent Information Systems 19, 267–301 (2002). https://doi.org/10.1023/A:1020114205638
Issue Date:
DOI: https://doi.org/10.1023/A:1020114205638