Abstract
Multi-agent systems have been attacking the challenges of distributed information retrieval. In this paper, we propose a consensus method-based framework to evaluate the performance of cooperative information retrieval tasks of the agents. Two well-known measurements, precision and recall, are extended to handle consensual closeness (i.e., local and global consensus) between the retrieved results. We show in a motivating example that the proposed criteria are prone to solve the problem of rigidity of classical precision and recall. More importantly, the retrieved results can be ranked with respect to the consensual score.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley, Reading (1999)
Huhns, M.N., Singh, M.P.: Agents on the Web: “Agents are Everywhere!”. IEEE Internet Computing 1(1), 87 (1997)
Oates, T., Prasad, M.V.N., Lesser, V.R.: Cooperative information-gathering: a distributed problem-solving approach. IEE Proceedings - Software 144(1), 72–88 (1997)
Herrera-Viedma, E., Herrera, F., Martínez, L., Herrera, J.C., López, A.G.: Incorporating filtering techniques in a fuzzy linguistic multi-agent model for information gathering on the web. Fuzzy Sets and Systems 148(1), 61–83 (2004)
Lee, R.S.T., Liu, J.N.K.: iJADE Web-Miner: An intelligent agent framework for internet shopping. IEEE Transactions on Knowledge and Data Engineering 16(4), 461–473 (2004)
Jung, J.J.: Ontological framework based on contextual mediation for collaborative information retrieval. Information Retrieval 10(1), 85–109 (2007)
Callan, J.P.: Distributed information retrieval. In: Advances in Information Retrieval, pp. 127–150. Kluwer Academic Publishers, Dordrecht (2000)
Gravano, L., Garcia-Molina, H.: Generalizing GlOSS to vector-space databases and broker hierarchies. In: Proceedings of the 21th International Conference on Very Large Data Bases (VLDB ’95), pp. 78–89. Morgan Kaufmann, San Francisco (1995)
Coulouris, G., Dollimore, J., Kindberg, T.: Distributed Systems - Concepts and Design. Addison-Wesley, Reading (1996)
Nguyen, N.T.: Consensus system for solving conflicts in distributed systems. Information Science 147(1-4), 91–122 (2002)
Nguyen, N.T., Ganzha, M., Paprzycki, M.: A consensus-based multi-agent approach for information retrieval in internet. In: Alexandrov, V.N., van Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds.) ICCS 2006. LNCS, vol. 3993, pp. 208–215. Springer, Heidelberg (2006)
Callan, J.P., Lu, Z., Croft, W.B.: Searching distributed collections with inference networks. In: Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR ’95), pp. 21–28. ACM Press, New York (1995)
Fu, L., Goh, D.H.L., Foo, S.S.B.: Cqe: a collaborative querying environment. In: Marlino, M., Sumner, T., Shipman, F. (eds.) Proceedings of the 2005 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2005), p. 378. ACM Press, New York (2005)
Borlund, P.: The IIR evaluation model: a framework for evaluation of interactive information retrieval systems. Information Research 8(3) (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jung, J.J., Jo, GS. (2007). Consensus-Based Evaluation Framework for Cooperative Information Retrieval Systems. In: Nguyen, N.T., Grzech, A., Howlett, R.J., Jain, L.C. (eds) Agent and Multi-Agent Systems: Technologies and Applications. KES-AMSTA 2007. Lecture Notes in Computer Science(), vol 4496. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72830-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-540-72830-6_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72829-0
Online ISBN: 978-3-540-72830-6
eBook Packages: Computer ScienceComputer Science (R0)