Abstract
Agent evaluation is an important topic in collaborative systems. Agent evaluation is to accurately assess the ability of the agent to complete a task. The ability of the agent is demonstrated by the result of the task, and the ability changes over time. Thus, this paper studies the evaluation of agent in collaborative systems. Considering the independent and interaction abilities of an agent for a role, as well as the historical and current evaluation value, we propose a trust-based evaluation approach that promotes the effectiveness of evaluation. The experimental results show that this approach can obtain more accurate results when the agent capability changes fast and there are only a few known evaluation values.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhu, H.: Maximizing group performance while minimizing budget. IEEE Trans. Syst. Man Cybern. Syst. 1–13 (2017). https://doi.org/10.1109/tsmc.2017.2735300
Haiyan, G.U., Chi, X.U.: Cooperative operational technologies for manned/unmanned teaming. Command Inf. Syst. Technol. 8(6), 37–45 (2017). https://doi.org/10.15908/j.cnki.cist.2017.06.006
Gershoff, A.D., Mukherjee, A.: Few ways to love, but many ways to hate: attribute ambiguity and the positivity effect in agent evaluation. J. Consum. Res. 33(4), 499–505 (2007). https://doi.org/10.1086/510223
Jensen, J.: Employee evaluation: it’s a dirty job, but somebody’s got to do it. Grantsmanship Center News 8(4), 36 (1980)
Moore, J.P., Inder, R., Chung, P.W.H., Macintosh, A., Stader, J.: Who does what? Matching agents to tasks in adaptive workflow. In: International Conference on Enterprise Information Systems, pp. 181–185 (2000)
Neely, A., Richards, H., Mills, J., Platts, K., Bourne, M.: Designing performance measures: a structured approach. Int. J. Oper. Prod. Manag. 17(11), 1131–1152 (1997). https://doi.org/10.1108/01443579710177888
Zhu, H., Feng, L., Pickering, R.: Agent evaluation in distributed adaptive systems. In: Proceedings of the 2013 IEEE International Conference on Systems, Man, and Cybernetics, pp. 752–757 (2013). https://doi.org/10.1109/smc.2013.133
Osman, N., Sierra, C., Mcneill, F., Pane, J., Debenham, J.: Trust and matching algorithms for selecting suitable agents. ACM Trans. Intell. Syst. Technol. 5(1), 1–39 (2013). https://doi.org/10.1145/2542182.2542198
Wang, Y., Singh, M.P.: Evidence-based trust: a mathematical model geared for multiagent systems. ACM Trans. Auton. Adapt. Syst. 5(4), 1–28 (2010). https://doi.org/10.1145/1867713.1867715
Shi, F.L., Yang, F., Xu, Y.P.: Power index method for operational capability evaluation of weapon equipment based on ANP and simulation. Syst. Eng.-Theory Pract. 31(6), 1086–1094 (2011). https://doi.org/10.1090/S0002-9939-2011-10775-5
Yao, C.M., Wang, Q.Y., Xie, R.S.: Multi-target threat assessment method for tank. Command Inf. Syst. Technol. 9(1), 68–72 (2018). https://doi.org/10.15908/j.cnki.cist.2018.01.012
Xu, T., Xie, B., Liu, B., Jin, X.: Effectiveness evaluation for collaborative planning pattern across services and arms. Command Inf. Syst. Technol. 10(2), 29–33 (2019)
Zhu, H., Zhou, M.C.: Roles in information systems: a survey. IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 38(3), 377–396 (2008). https://doi.org/10.1109/tsmcc.2008.919168
Castelfranchi, C., Falcone, R., Marzo, F.: Being trusted in a social network: trust as relational capital. In: Stølen, K., Winsborough, W.H., Martinelli, F., Massacci, F. (eds.) iTrust 2006. LNCS, vol. 3986, pp. 19–32. Springer, Heidelberg (2006). https://doi.org/10.1007/11755593_3
Acknowledgement
The authors wish to thank Natural Science Foundation of the Jiangsu Higher Education Institutions of China under Grant 18KJB520007 and 18KJB520018, the High-Level Talent Foundation of Jinling Institute of Technology under Grant JIT-B-201703, the High-Level Talent Foundation of Jiangsu Open University under Grant 19RC-2.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sheng, Y., Hu, W., Zhu, X. (2019). Trust-Based Agent Evaluation in Collaborative Systems. In: Sun, Y., Lu, T., Yu, Z., Fan, H., Gao, L. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2019. Communications in Computer and Information Science, vol 1042. Springer, Singapore. https://doi.org/10.1007/978-981-15-1377-0_21
Download citation
DOI: https://doi.org/10.1007/978-981-15-1377-0_21
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1376-3
Online ISBN: 978-981-15-1377-0
eBook Packages: Computer ScienceComputer Science (R0)
