Abstract
Mechanisms for modeling trust and reputation to improve robustness and performance in multi-agent societies make up a growing field of research that has yet to establish unified direction or benchmarks. The trust research community will benefit significantly from the development of a competition testbed; such development is currently in progress under the direction of the Agent Reputation and Trust (art) Testbed initiative. A testbed can serve in two roles: 1) as a competition forum in which researchers can compare their technologies against objective metrics, and 2) as a suite of tools with flexible parameters, allowing researchers to perform easily-repeatable experiments. As a versatile, universal experimentation site, a competition testbed challenges researchers to solve the most prominent problems in the field, fosters a cohesive scoping of trust research problems, identifies successful technologies, and provides researchers with a tool for comparing and validating their approaches. In addition, a competition testbed places trust research in the public spotlight, improving confidence in the technology and highlighting relevant applications. This paper lays the foundation for testbed development by enumerating the important problems in trust and reputation research, describing important requirements for a competition testbed, and addressing necessary parameters for testbed modularity and flexibility. Finally, the art Testbed initiative is highlighted, and future progress toward testbed development is described.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barber, K.S., Fullam, K., Kim, J.: Challenges for trust, fraud, and deception research in multi-agent systems. Trust, Reputation, and Security: Theories and Practice, 8–14 (2003)
Fullam, K., Barber, K.S.: Evaluating approaches for trust and reputation research: Exploring a competition testbed. In: Proceedings of The Workshop on Reputation in Agent Societies at Intelligent Agent Technology (IAT 2004), Beijing, pp. 20–23 (2004)
Sabater, J.: Toward a test-bed for trust and reputation models. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 101–105 (2004)
ART Testbed Team: Agent Reputation and Trust Testbed (2005), http://www.lips.utexas.edu/~kfullam/competition/
Schillo, M., Funk, P., Rovatsos, M.: Using trust for detecting deceitful agents in artificial societites. In: Applied Artificial Intelligence, pp. 825–848 (2000)
Mui, L., Mohtashemi, M., Halberstadt, A.: Notions of reputation in multi-agent systems: A review. In: Proceedings of the first international joint conference on autonomous agents and multiagent systems (AAMAS 2002), Bologna, Italy, pp. 280–287 (2002)
Marsh, S.: Formalising Trust as a Computational Concept. PhD thesis, Department of Mathematics and Computer Science, University of Stirling (1994)
Verhoeff, T.: The trader’s dilemma: A continuous version of the prisoner’s dilemma. Technical report, Computing Science Notes 93/02, Faculty of Mathematics and Computing Science, Technische Universiteit Eindhoven, The Netherlands (1998)
Zacharia, G.: Collaborative reputation mechanisms for online communities. Master’s thesis, Massachusetts Institute of Technology (1999)
Sabater, J., Sierra, C.: Regret: A reputation model for gregarious societies. In: Proceedings of the Fourth Workshop on Deception, Fraud and Trust in Agent Societies, Montreal, Canada, pp. 61–69 (2001)
Carbo, J., Molina, J., Davila, J.: Trust management through fuzzy reputation. Int. Journal in Cooperative Information Systems 12, 135–155 (2002)
Carbo, J., Molina, J., Davila, J.: Comparing predictions of sporas vs. a fuzzy reputation agent system. In: 3rd International Conference on Fuzzy Sets and Fuzzy Systems, Interlaken, pp. 147–153 (2002)
Sabater, J., Sierra, C.: Reputation and social network analysis in multi-agent systems. In: Proceedings of the first international joint conference on autonomous agents and multiagent systems (AAMAS 2002), Bologna, Italy, pp. 475–482 (2002)
Yu, B., Singh, M.P.: Towards a probabilistic model of distributed reputation management. In: Proceedings of the Fourth Workshop on Deception, Fraud and Trust in Agent Societies, Montreal, Canada, pp. 125–137 (2001)
eBay: eBay (2002), http://www.eBay.com
BizRate: BizRate (2002), http://www.bizrate.com
Fullam, K., Barber, K.S.: A temporal policy for trusting information. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 47–57 (2004)
Fullam, K.: An expressive belief revision framework based on information valuation. Master’s thesis, Department of Electrical and Computer Engineering, The University of Texas at Austin (2003)
Klos, T., la Poutre, H.: Using reputation-based trust for assessing agent reliability. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 75–82 (2004)
Whitby, A., Josang, A., Indulska, J.: Filtering out unfair ratings in bayesian reputation systems. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 106–117 (2004)
Ghanea-Hercock, R.: The cost of trust. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 58–64 (2004)
Yamamoto, H., Ishida, K., Ohta, T.: Trust formation in a c2c market: Effect of reputation management system. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 126–136 (2004)
Ding, L., Kolari, P., Ganjugunte, S., Finin, T., Joshi, A.: On modeling and evaluating trust networks inference. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 21–32 (2004)
Barber, K.S., Kim, J.: Belief revision process based on trust: Agent evaluating reputation of information sources. In: Trust in Cyber-societies: Integrating the Human and Artificial Perspectives. LNCS, pp. 73–82. Springer, Heidelberg (2002)
Huynh, D., Jennings, N., Shadbolt, N.: Developing an integrated trust and reputation model for open multi-agent systems. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 65–74 (2004)
Fujimura, K., Tanimoto, N., Iguchi, M.: Calculating contribution in cyberspace community using reputation system ’rumor’. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 40–46 (2004)
Falcone, R., Pezzulo, G., Castelfranchi, C., Calvi, G.: Trusting the agents and the environment leads to successful delegation: A contract net simulation. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 33–39 (2004)
Neville, B., Pitt, J.: A simulation study of social agents in agent mediated e-commerce. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Third International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, pp. 83–91 (2004)
Biswas, A.S., Sen, S., Debnath, S.: Limiting deception in groups of social agents. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at Autonomous Agents, Seattle, Washington, pp. 21–28 (1999)
Barber, K.S., Fullam, K.: Applying reputation models to continuous belief revision. In: Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at The Second International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2003), Melbourne, Australia, pp. 6–15 (2003)
Sen, S., Sajja, N.: Robustness of reputation-based trust: Boolean case. In: Proceedings of the first international joint conference on autonomous agents and multiagent systems (AAMAS 2002), Bologna, Italy, pp. 288–293 (2002)
UCI: UCI (2004), http://www.ics.uci.edu/~mlearn/MLRepository.html
Castelfranchi, C., Falcone, R.: Towards a theory of delegation for agent-based systems. Robotics and Autonomous Systems 24, 141–157 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fullam, K.K., Sabater-Mir, J., Barber, K.S. (2005). A Design Foundation for a Trust-Modeling Experimental Testbed. In: Falcone, R., Barber, S., Sabater-Mir, J., Singh, M.P. (eds) Trusting Agents for Trusting Electronic Societies. TRUST TRUST 2004 2003. Lecture Notes in Computer Science(), vol 3577. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11532095_6
Download citation
DOI: https://doi.org/10.1007/11532095_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28012-5
Online ISBN: 978-3-540-31859-0
eBook Packages: Computer ScienceComputer Science (R0)