Skip to main content
SpringerLink
Log in

An efficient trust estimation model for multi-agent systems using temporal difference learning

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In multi-agent system (MAS) applications, teamwork among the agents is essential as the agents are required to collaborate and pool resources to execute the given tasks and complete the objectives successfully. A vital part of the collaboration is sharing of information and resources in order to optimize their efforts in achieving the given objectives. Under such collaborative environment, trust among the agents plays a critical role to ensure efficient cooperation. This study looks into developing a trust evaluation model that can empirically evaluate the trust of one agent on the other. The proposed model is developed using temporal difference learning method, incorporating experience gained through interactions into trust evaluation. Simulation experiments are conducted to evaluate the performance of the developed model against some of the most recent models reported in the literature. The results of the simulation experiments indicate that the proposed model performs better than the comparison models in estimating trust more effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Basheer GS, Ahmad MS, Tang AY, Graf S (2015) Certainty, trust and evidence: towards an integrative model of confidence in multi-agent systems. Comput Hum Behav 45:307–315

    Article  Google Scholar 

  2. Yugang L, Nejat G, Vilela J (2013) ‘Learning to cooperate together: a semi-autonomous control architecture for multi-robot teams in urban search and rescue’, Safety, Security, and Rescue Robotics (SSRR), 2013 IEEE International Symposium on, pp 1–6

  3. Mohammed M, Lim C, Quteishat A (2014) A novel trust measurement method based on certified belief in strength for a multi-agent classifier system. Neural Comput and Appl 24(2):421–429

    Article  Google Scholar 

  4. Fullam KK, Klos TB, Muller G, Sabater J, Schlosser A, Topol Z, Barber KS, Rosenschein JS, Vercouter L, Voss M (2005) ‘A specification of the Agent Reputation and Trust (ART) testbed: experimentation and competition for trust in agent societies’, Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, pp 512–518

  5. Han Y, Zhiqi S, Leung C, Chunyan M, Lesser VR (2013) A survey of multi-agent trust management systems. IEEE Access 1:35–50

    Article  Google Scholar 

  6. Busoniu L, Babuska R, De Schutter B (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern Part C Appl Rev 38(2):156–172

    Article  Google Scholar 

  7. Aref A, Tran T (2014) ‘Using fuzzy logic and Q-learning for trust modeling in multi-agent systems’, Computer Science and Information Systems (FedCSIS), 2014 Federated Conference on, pp 59–66

  8. Das A, Islam MM (2012) SecuredTrust: a dynamic trust computation model for secured communication in multiagent systems. IEEE Trans Dependable Secure Comput 9(2):261–274

    Article  Google Scholar 

  9. Miao G, Ma Q, Liu Q (2015) Consensus problems for multi-agent systems with nonlinear algorithms. Neural Comput Appl 1–10. doi:10.1007/s00521-015-1936-6

  10. Kurniawati H, Yanzhu D, Hsu D, Lee WS (2010) Motion planning under uncertainty for robotic tasks with long time horizons. Int J Robot Res 30(3):308–323

    Article  MATH  Google Scholar 

  11. Kurniawati H, Bandyopadhyay T, Patrikalakis NM (2012) Global motion planning under uncertain motion, sensing, and environment map. Auton Robots 33(3):255–272

    Article  Google Scholar 

  12. Gorbunov R, Barakova E, Rauterberg M (2013) Design of social agents. Neurocomputing 114:92–97

    Article  Google Scholar 

  13. Rosaci D, Sarné GML, Garruzzo S (2012) Integrating trust measures in multiagent systems. Int J Intell Syst 27(1):1–15

    Article  Google Scholar 

  14. Sabater J, Sierra C (2005) Review on computational trust and reputation models. Artif Intell Rev 24(1):33–60

    Article  MATH  Google Scholar 

  15. Vanderelst D, Ahn RM, Barakova EI (2008) Simulated Trust-Towards robust social learning. In: ALIFE, pp 632–639

  16. Vanderelst D, Ahn RM, Barakova EI (2009) Simulated trust: a cheap social learning strategy. Theor Popul Biol 76(3):189–196

    Article  MATH  Google Scholar 

  17. Huynh TD, Jennings NR, Shadbolt NR (2006) An integrated trust and reputation model for open multi-agent systems. Auton Agent Multi-Agent Syst 13(2):119–154

    Article  Google Scholar 

  18. Yu H, Shen Z, Miao C, An B, Leung C (2014) Filtering trust opinions through reinforcement learning. Decis Support Syst 66:102–113

    Article  Google Scholar 

  19. Jøsang A (2001) A logic for uncertain probabilities. Int J Uncertain Fuzziness Knowl-Based Syst 9(3):279–311

    Article  MathSciNet  MATH  Google Scholar 

  20. Zhou P, Gu X, Zhang J, Fei M (2015) A priori trust inference with context-aware stereotypical deep learning. Knowl-Based Syst 88:97–106

    Article  Google Scholar 

  21. Fan W, Perros H (2014) A novel trust management framework for multi-cloud environments based on trust service providers. Knowl-Based Syst 70:392–406

    Article  Google Scholar 

  22. Jin-Hee C, Chan K, Mikulski D (2014) Trust-based information and decision fusion for military convoy operations, Military Communications Conference (MILCOM), 2014 IEEE, pp 1387–1392

  23. Wang Y, Singh MP (2007) Formal trust model for multiagent systems, In Proceedings of the 20th international joint conference on Artifical intelligence, pp 1551–1556

  24. Wang Y, Singh MP (2010) Evidence-based trust: a mathematical model geared for multiagent systems. ACM Trans Auton Adapt Syst 5(4):1–28

    Article  Google Scholar 

  25. Wang Y, Hang C-W, Singh MP (2011) A probabilistic approach for maintaining Trust.pdf. J Artif Intell Res 40:47

    Google Scholar 

  26. Mantel KT, Clark CM (2012) Trust networks in multi-robot communities, In Robotics and Biomimetics (ROBIO), 2012 IEEE International Conference on, pp 2114–2119

  27. Jøsang A, Haller J (2007) ‘Dirichlet Reputation Systems’, Availability, Reliability and Security, ARES 2007. The Second International Conference on, pp 112–119

  28. Namin AS, Ruizhong W, Weiming S, Ghenniwa H (2006) An efficient trust model for multi-agent systems, Computer Supported Cooperative Work in Design, 2006. CSCWD ‘06. 10th International Conference on, pp 1–6

  29. Peng M, Xu Z, Pan S, Li R, Mao T (2012) AgentTMS: a MAS trust model based on agent social relationship. J Comput 7(6):1535–1542

    Article  Google Scholar 

  30. Mui L, Mohtashemi M, Halberstadt A (2002) ‘A computational model of trust and reputation’, System Sciences, 2002. HICSS. Proceedings of the 35th Annual Hawaii International Conference on, pp 2431–2439

  31. Griffiths N (2005) Task delegation using experience-based multi-dimensional trust, In Proceedings of the fourth international joint conference on autonomous agents and multiagent systems, pp 489-496

  32. Fullam KK, Barber KS (2007) Dynamically learning sources of trust information: experience versus reputation, In Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, pp 1–8

  33. Russell SJ, Norvig P (2003) Artificial intelligence a modern approach (Pearson Education, Inc., 2003, Second edn. 2003)

  34. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT press, Cambridge

    Google Scholar 

  35. Jsang A, Ismail R (2002) The beta reputation system, In Proceedings of the 15th bled electronic commerce conference, vol 5, pp 2502–2511

  36. Gautam A, Mohan S (2012) A review of research in multi-robot systems’, Industrial and Information Systems (ICIIS), 2012 7th IEEE International Conference on, pp 1–5

  37. Nagatani K, Kiribayashi S, Okada Y, Otake K, Yoshida K, Tadokoro S, Nishimura T, Yoshida T, Koyanagi E, Fukushima M, Kawatsuma S (2013) Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots. J Field Robot 30(1):44–63

    Article  Google Scholar 

  38. Burdakov O, Doherty P, Holmberg K, Kvarnstrom J, Olsson PM (2010) Relay positioning for unmanned aerial vehicle surveillance. Int J Robot Res 29(8):1069–1087

    Article  Google Scholar 

Download references

Acknowledgments

This research is funded by e-Science grant provided by Ministry of Science, Technology and Innovation (MOSTI), Malaysia, Project Number: 03-02-10-SF0200.

Author information

Authors and Affiliations

  1. Advanced Engineering Platform and School of Engineering, Monash University Malaysia, Jalan Lagoon Selatan, Bandar Sunway, 47500, Subang Jaya, Selangor, Malaysia

    G. Rishwaraj & S. G. Ponnambalam

  2. Universiti Malaya, Jalan Universiti, 50603, Kuala Lumpur, Wilayah Persekutuan Kuala Lumpur, Malaysia

    Loo Chu Kiong

Authors
  1. G. Rishwaraj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. G. Ponnambalam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Loo Chu Kiong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. G. Ponnambalam.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rishwaraj, G., Ponnambalam, S.G. & Kiong, L.C. An efficient trust estimation model for multi-agent systems using temporal difference learning. Neural Comput & Applic 28 (Suppl 1), 461–474 (2017). https://doi.org/10.1007/s00521-016-2354-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-016-2354-0

Keywords

Search

Navigation