research-article

Controlling Negative Emergent Behavior by Graph Analysis at Runtime

Authors:

Melanie Kauder,

Richard Scharrer,

Sarah Edenhofer,

Christian Müller-SchloerAuthors Info & Claims

ACM Transactions on Autonomous and Adaptive Systems (TAAS), Volume 11, Issue 2

Article No.: 7, Pages 1 - 34

https://doi.org/10.1145/2890507

Published: 06 June 2016 Publication History

Abstract

Self-organized systems typically consist of distributed autonomous entities. An increasing part of such systems is characterized by openness and heterogeneity of participants. For instance, open desktop computing grids provide a framework for unrestrictedly joining in. However, openness and heterogeneity present severe challenges to the overall system’s stability and efficiency since uncooperative and even malicious participants are free to join. A promising solution for this problem is to introduce technical trust as a basis; however, in turn, the utilization of trust opens space for negative emergent behavior. This article introduces a system-wide observation and control loop that influences the self-organized behavior to provide a performant and robust platform for benevolent participants. Thereby, the observation part is responsible for gathering information and deriving a system description. We introduce a graph-based approach to identify groups of suspicious or malicious agents and demonstrate that this clustering process is highly successful for the considered stereotype agent behaviors. In addition, the controller part guides the system behavior by issuing norms that make use of incentives and sanctions. We further present a concept for closing the control loop and show experimental results that highlight the potential benefit of establishing such a control loop.

References

[1]

David P. Anderson and Gilles Fedak. 2006. The computational and storage potential of volunteer computing. In Proceedings of the 2006 CCGRID Conference. IEEE, Los Alamitos, CA, 73--80.

Digital Library

[2]

Cosimo Anglano, John Brevik, Massimo Canonico, Dan Nurmi, and Rich Wolski. 2006. Fault-aware scheduling for bag-of-tasks applications on desktop grids. In Proceedings of the 2006 GRID Conference. IEEE, Los Alamitos, CA, 56--63.

Digital Library

[3]

Cosimo Anglano, Massimo Canonico, Marco Guazzone, Marco Botta, Sergio Rabellino, Simone Arena, and Guglielmo Girardi. 2008. Peer-to-peer desktop grids in the real world: The ShareGrid project. In Proceedings of the 2008 CCGrid Conference. 609--614.

Digital Library

[4]

Alexander Artikis and Jeremy Pitt. 2009. Specifying open agent systems: A survey. In Engineering Societies in the Agents World IX, A. Artikis, G. Picard, and L Vercouter (Eds.). Lecture Notes in Computer Science, Vol. 5485. Springer, 29--45.

Digital Library

[5]

Tina Balke, Celia da Costa Pereira, Frank Dignum, Emiliano Lorini, Antonino Rotolo, Wamberto Vasconcelos, and Serena Villata. 2013. Norms in MAS: Definitions and related concepts. In Normative Multi-Agent Systems. Dagstuhl Follow-Ups, Vol. 4. Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 1--31.

[6]

Jon C. R. Bennett and Hui Zhang. 1996. WF2Q: Worst-case fair weighted fair queueing. In Proceedings the 1996 IEEE INFOCOM Conference, Vol. 1. IEEE, Los Alamitos, CA, 120--128.

Digital Library

[7]

Yvonne Bernard, Lukas Klejnowski, Emre Çakar, Jörg Hähner, and Christian Müller-Schloer. 2011. Efficiency and robustness using trusted communities in a trusted desktop grid. In Proceedings of the 2011 SASO Workshops. IEEE, Los Alamitos, CA, 21--26.

Digital Library

[8]

David Billington. 1993. Defeasible logic is stable. Journal of Logic and Computation 3, 4, 379--400.

[9]

Vincent D. Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. 2008. Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment 2008, 10, P10008.

[10]

Guido Boella, Gabriella Pigozzi, and Leendert van der Torre. 2009. Normative systems in computer science: Ten guidelines for normative multiagent systems. In Normative Multi-Agent Systems, G. Boella, P. Noriega, G. Pigozzi, and H. Verhagen (Eds.). Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 1--21.

[11]

Sergey Brin and Lawrence Page. 1998. The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems 30, 1, 107--117.

Digital Library

[12]

Christiano Castelfranchi and Rino Falcone. 2010. Trust Theory: A Socio-Cognitive and Computational Model. Vol. 18. John Wiley & Sons, Chichester, UK.

Digital Library

[13]

M. Emre Celebi and Hassan A. Kingravi. 2012. Deterministic initialization of the k-means algorithm using hierarchical clustering. International Journal of Pattern Recognition and Artificial Intelligence 26, 07, 1--25.

[14]

Arjav J. Chakravarti, Gerald Baumgartner, and Mario Lauria. 2004. Application-specific scheduling for the organic grid. In Proceedings of the 2004 GRID Workshops. IEEE, Los Alamitos, CA, 146--155.

Digital Library

[15]

Sungjin Choi, Rajkumar Buyya, Hongsoo Kim, and Eunjoung Byun. 2008. A Taxonomy of Desktop Grids and Its Mapping to State of the Art Systems. Technical Report. Grid Computing and Distributed Systems Laboratory, University of Melbourne, Melbourne, Australia.

[16]

Sung-Jin Choi, Hong-Soo Kim, Eun-Joung Byun, Maeng-Soon Baik, Sung-Suk Kim, Chan-Yeol Park, and Chong-Sun Hwang. 2007. Characterizing and classifying desktop grid. In Proceedings of the 2007 CCGRID Conference. IEEE, Los Alamitos, CA, 743--748.

Digital Library

[17]

Rosaria Conte, Cristiano Castelfranchi, and Frank Dignum. 1999. Autonomous norm acceptance. In Intelligent Agents V: Agents Theories, Architectures, and Languages, J. P. Müller, A. S. Rao, and M. P. Singh (Eds.). Lecture Notes in Computer Science, Vol. 1555. Springer, 99--112.

Digital Library

[18]

Alan Demers, Srinivasan Keshav, and Scott Shenker. 1989. Analysis and simulation of a fair queueing algorithm. In Proceedings of the 2009 SIGCOMM Conference. ACM, New York, NY, 1--12.

Digital Library

[19]

Patricio Domingues, Bruno Sousa, and Luis Moura Silva. 2007. Sabotage-tolerance and trust management in desktop grid computing. Future Generation Computer Systems 23, 7, 904--912.

Digital Library

[20]

Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the 1996 KDD Conference, Vol. 96. 226--231.

[21]

Ernst Fehr. 2004. Human behaviour: Don’t lose your reputation. Nature 432, 7016, 449--450.

[22]

Guido Governatori and Antonino Rotolo. 2008. BIO logical agents: Norms, beliefs, intentions in defeasible logic. Autonomous Agents and Multi-Agent Systems 17, 1, 36--69.

Digital Library

[23]

Garrett Hardin. 1968. The tragedy of the commons. Science 162, 3859, 1243--1248.

[24]

Carl Hewitt. 1991. Open information systems semantics for distributed artificial intelligence. Artificial Intelligence 47, 1, 79--106.

Digital Library

[25]

Alexander Hinneburg and Hans-Henning Gabriel. 2007. DENCLUE 2.0: Fast clustering based on kernel density estimation. In Advances in Intelligent Data Analysis VII. Springer, Berlin, Germany, 70--80.

Digital Library

[26]

Christopher D. Hollander and Annie S. Wu. 2011. The current state of normative agent-based systems. Journal of Artificial Societies and Social Simulation 14, 2, 6.

[27]

Raj Jain, Gojko Babic, Bhavana Nagendra, and Chi-Chung Lam. 1996. Fairness, call establishment latency and other performance metrics. ATM-Forum 96, 1173, 1--6.

[28]

Nicholas Jardine and Robin Sibson. 1971. Mathematical Taxonomy. John Wiley & Sons, Chichester, UK.

[29]

Jan Kantert, Sarah Edenhofer, Sven Tomforde, Jörg Hähner, and Christian Müller-Schloer. 2015b. Detecting and isolating inconsistently behaving agents using an intelligent control loop. In 2015 INSTICC Conference. 246--253.

Digital Library

[30]

Jan Kantert, Sarah Edenhofer, Sven Tomforde, Jörg Hähner, and Christian Müller-Schloer. 2015c. Norm-based system control in distributed low-power sensor networks. In Proceedings of the Poster Session of the 2015 ARCS Conference. 13--14.

[31]

Jan Kantert, Sarah Edenhofer, Sven Tomforde, and Christian Müller-Schloer. 2015a. Distributed rendering in an open self-organised trusted desktop grid. In Proceedings of the 2015 ICAC Conference. IEEE, Los Alamitos, CA, 267--272.

Digital Library

[32]

Jan Kantert, Hannes Scharf, Sarah Edenhofer, Sven Tomforde, Jörg Hähner, and Christian Müller-Schloer. 2014. A graph analysis approach to detect attacks in multi-agent-systems at runtime. In Proceedings of the 2014 SASO Conference. IEEE, Los Alamitos, CA, 80--89.

Digital Library

[33]

Jan Kantert, Sergej Wildemann, Georg von Zengen, Sarah Edenhofer, Sven Tomforde, Lars Wolf, Jörg Hähner, and Christian Müller-Schloer. 2015d. Improving reliability and endurance using end-to-end trust in distributed low-power sensor networks. In Architecture of Computing Systems—ARCS 2015. LNCS Lecture Notes in Computer Science, Vol. 9017. Springer, 135--145.

[34]

Mohamed A. Khamsi and William A. Kirk. 2011. An Introduction to Metric Spaces and Fixed Point Theory. Vol. 53. John Wiley & Sons, Chichester, UK.

[35]

Jon M. Kleinberg. 1999. Authoritative sources in a hyperlinked environment. Journal of the ACM 46, 5, 604--632.

Digital Library

[36]

Lukas Klejnowski. 2014. Trusted Community: A Novel Multiagent Organisation for Open Distributed Systems. Ph.D. Dissertation. Leibniz Universität Hannover. http://edok01.tib.uni-hannover.de/ edoks/e01dh11/668667427.pdf.

[37]

Janus B. Kristensen. 2014. Big Buck Bunny 3D Rendering Exploration. Retrieved May 2, 2016, from http://bbb3d.renderfarming.net/explore.html.

[38]

Stuart Lloyd. 1982. Least squares quantization in PCM. IEEE Transactions on Information Theory 28, 2, 129--137.

Digital Library

[39]

Mark E. J. Newman. 2003. The structure and function of complex networks. SIAM Review 45, 2, 167--256.

Digital Library

[40]

Mark E. J. Newman. 2004. Analysis of weighted networks. Physical Review E 70, 5, Article No. 5.

[41]

Andreas Noack. 2007. Unified Quality Measures for Clusterings, Layouts, and Orderings of Graphs, and Their Application as Software Design Criteria. Ph.D. Dissertation. Brandenburg University of Technology.

[42]

Donald Nute. 1988. Defeasible reasoning: A philosophical analysis in prolog. In Aspects of Artificial Intelligence. Springer, Berlin, Germany, 251--288.

[43]

Donald Nute. 1994. Defeasible logic. In Handbook of Logic in Artificial Intelligence and Logic Programming. Oxford University Press, New York, NY, 353--395.

Digital Library

[44]

Donald Nute. 2003. Defeasible logic. In Proceedings of the 2003 INAP Conference. 151--169.

Digital Library

[45]

Joshua O’Madadhain, Danyel Fisher, Padhraic Smyth, Scott White, and Yan-Biao Boey. 2005. Analysis and visualization of network data using JUNG. Journal of Statistical Software 10, 2, 1--35.

[46]

Elinor Ostrom. 1990. Governing the Commons: The Evolution of Institutions for Collective Action. Cambridge University Press, Cambridge, MA.

[47]

Jeremy Pitt, Julia Schaumeier, and Alexander Artikis. 2011. The axiomatisation of socio-economic principles for self-organising systems. In Proceedings of the 2011 SASO Conference. IEEE, Los Alamitos, CA, 138--147.

Digital Library

[48]

Jeffrey S. Rosenschein and Gilad Zlotkin. 1994. Rules of Encounter: Designing Conventions for Automated Negotiation Among Computers. MIT Press, Cambridge, MA.

Digital Library

[49]

Giovanni Sartor. 2005. Legal Reasoning: A Cognitive Approach to Law. Springer, Berlin, Germany.

[50]

Bastin Tony Roy Savarimuthu and Stephen Cranefield. 2011. Norm creation, spreading and emergence: A survey of simulation models of norms in multi-agent systems. Multiagent and Grid Systems 7, 1, 21--54.

Digital Library

[51]

Satu Elisa Schaeffer. 2007. Graph clustering. Computer Science Review 1, 1, 27--64.

Digital Library

[52]

Hartmut Schmeck, Christian Müller-Schloer, Emre Çakar, Moez Mnif, and Urban Richter. 2010. Adaptivity and self-organization in organic computing systems. ACM Transactions on Autonomous and Adaptive Systems 5, 3, Article No. 10. http://doi.acm.org/10.1145/1837909.1837911

Digital Library

[53]

Gholamhosein Sheikholeslami, Surojit Chatterjee, and Aidong Zhang. 2000. WaveCluster: A wavelet-based clustering approach for spatial data in very large databases. VLDB Journal 8, 3--4, 289--304.

Digital Library

[54]

Munindar P. Singh. 1999. An ontology for commitments in multiagent systems. Artificial Intelligence and Law 7, 1, 97--113.

Digital Library

[55]

Munindar P. Singh, Matthew Arrott, Tina Balke, Amit K. Chopra, Rob Christiaanse, Stephen Cranefield, Frank Dignum, et al. 2013. The uses of norms. In Normative Multi-Agent Systems. Dagstuhl Follow-Ups, Vol. 4. Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 191--229.

[56]

Jan-Philipp Steghöfer, Gerrit Anders, Wolfgang Reif, Jan Kantert, and Christian Müller-Schloer. 2014. An effective implementation of norms in trust-aware open self-organising systems. In Proceedings of the 2014 SASOW Conference Workshops. IEEE, Los Alamitos, CA, 76--77.

Digital Library

[57]

Jan-Philipp Steghöfer, Rolf Kiefhaber, Karin Leichtenstern, Yvonne Bernard, Lukas Klejnowski, Wolfgang Reif, Theo Ungerer, Elisabeth André, Jörg Hähner, and Christian Müller-Schloer. 2010. Trustworthy organic computing systems: Challenges and perspectives. In Proceedings of the 2010 ATC Conference. 62--76.

Digital Library

[58]

Daniel Terdiman. 2011. New technology revs up Pixar’s ‘Cars 2.’ CNet. Retrieved May 2, 2016, from http://www.cnet.com/news/new-technology-revs-up-pixars-cars-2/.

[59]

Sven Tomforde, Holger Prothmann, Jürgen Branke, Jörg Hähner, Moez Mnif, Christian Müller-Schloer, Urban Richter, and Hartmut Schmeck. 2011. Observation and control of organic systems. In Organic Computing—A Paradigm Shift for Complex Systems. Birkhäuser, Basel, Switzerland, 325--338.

[60]

Raimo Tuomela and Maj Bonnevier-Tuomela. 1995. Norms and agreements. European Journal of Law, Philosophy and Computer Science 5, 41--46.

[61]

Andreea Urzică and Cristian Gratie. 2013. Policy-based instantiation of norms in MAS. In Intelligent Distributed Computing VI. Studies in Computational Intelligence, Vol. 446. Springer, 287--296.

[62]

Stijn Marinus Van Dongen. 2001. Graph Clustering by Flow Simulation. Ph.D. Dissertation. Utrecht University. http://dspace.library.uu.nl/bitstream/handle/1874/848/full.pdf.

[63]

Asimina Vasalou, Astrid Hopfensitz, and Jeremy V. Pitt. 2008. In praise of forgiveness: Ways for repairing trust breakdowns in one-off online interactions. International Journal of Human-Computer Studies 66, 6, 466--480.

Digital Library

[64]

Georg Henrik von Wright. 1963. Norms and Action: A Logical Enquiry. Routledge & Kegan Paul, London, UK.

[65]

Y. Wang and J. Vassileva. 2004. Trust-based community formation in peer-to-peer file sharing networks. In Proceedings of the Web Intelligence Conference. IEEE, Los Alamitos, CA, 341--348.

Digital Library

[66]

Jos Warmer and Anneke Kleppe. 1999. The Object Constraint Language: Precise Modeling with UML. Addison Wesley Longman, Boston, MA.

Digital Library

[67]

Stanley Wasserman. 1994. Social Network Analysis: Methods and Applications. Vol. 8. Cambridge University Press, Cambridge, MA.

[68]

Duncan J. Watts and Steven H. Strogatz. 1998. Collective dynamics of ‘small-world’ networks. Nature 393, 440--442.

[69]

Mark Weiser. 1991. The computer for the 21st century. Scientific American 265, 3, 66--75.

Digital Library

[70]

Rui Xu and Donald Wunsch. 2005. Survey of clustering algorithms. IEEE Transactions on Neural Networks 16, 3, 645--678.

Digital Library

[71]

Tian Zhang, Raghu Ramakrishnan, and Miron Livny. 1996. BIRCH: An efficient data clustering method for very large databases. ACM SIGMOD Record 25, 2, 103--114.

Digital Library

Cited By

Kröher CGerling LSchmid K(2023)Control Action Types -Patterns of Applied Control for Self-adaptive Systems2023 IEEE/ACM 18th Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS)10.1109/SEAMS59076.2023.00015(32-43)Online publication date: May-2023
https://doi.org/10.1109/SEAMS59076.2023.00015
Landauer CBellman K(2017)Integration by Negotiated Behavior Restrictions2017 IEEE 2nd International Workshops on Foundations and Applications of Self* Systems (FAS*W)10.1109/FAS-W.2017.130(117-121)Online publication date: Sep-2017
https://doi.org/10.1109/FAS-W.2017.130
Kantert JTomforde SScharrer RWeber SEdenhofer SMller-Schloer C(2017)Identification and classification of agent behaviour at runtime in open, trust-based organic computing systemsJournal of Systems Architecture: the EUROMICRO Journal10.1016/j.sysarc.2017.02.00375:C(68-78)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1016/j.sysarc.2017.02.003
Show More Cited By

Index Terms

Controlling Negative Emergent Behavior by Graph Analysis at Runtime
1. Computing methodologies
  1. Artificial intelligence
    1. Distributed artificial intelligence
      1. Multi-agent systems

Recommendations

Identifying norms of behaviour in open multi-agent societies
ADS '11: Proceedings of the 2011 Workshop on Agent-Directed Simulation

Norms have an obvious role in the coordinating, regulating, controlling and predicting agents' behaviours in software agents' societies. Most researchers assume that agents in their societies already know the norms as protocols or some other form. Some ...
Norms of Behaviour and Their Identification and Verification in Open Multi-Agent Societies

Norms have an obvious role in the coordinating and predicting behaviours in societies of software agents. Most researchers assume that agents already know the norms of their societies beforehand at design time. Others assume that norms are assigned by a ...
An architecture for identifying emergent behavior in multi-agent systems
AAMAS '14: Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems

Multi-agent systems exhibit unexpected, emergent behavior as a result of the complexity of agent behaviors and their interactions. Despite significant research interest in the past decades, computational methods to identify and analyze emergence as it ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Autonomous and Adaptive Systems

ACM Transactions on Autonomous and Adaptive Systems Volume 11, Issue 2

Special Section on Best Papers from SASO 2014 and Regular Articles

July 2016

267 pages

ISSN:1556-4665

EISSN:1556-4703

DOI:10.1145/2952298

Editors:
Manish Parashar
Rutgers University, USA
,
Franco Zambonelli
University of Modena e Reggio Emilia, Italy

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 June 2016

Accepted: 01 February 2016

Revised: 01 October 2015

Received: 01 February 2015

Published in TAAS Volume 11, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

German Research Foundation (DFG)
OC-Trust research unit

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
245
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kröher CGerling LSchmid K(2023)Control Action Types -Patterns of Applied Control for Self-adaptive Systems2023 IEEE/ACM 18th Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS)10.1109/SEAMS59076.2023.00015(32-43)Online publication date: May-2023
https://doi.org/10.1109/SEAMS59076.2023.00015
Landauer CBellman K(2017)Integration by Negotiated Behavior Restrictions2017 IEEE 2nd International Workshops on Foundations and Applications of Self* Systems (FAS*W)10.1109/FAS-W.2017.130(117-121)Online publication date: Sep-2017
https://doi.org/10.1109/FAS-W.2017.130
Kantert JTomforde SScharrer RWeber SEdenhofer SMller-Schloer C(2017)Identification and classification of agent behaviour at runtime in open, trust-based organic computing systemsJournal of Systems Architecture: the EUROMICRO Journal10.1016/j.sysarc.2017.02.00375:C(68-78)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1016/j.sysarc.2017.02.003
Müller-Schloer CTomforde SMüller-Schloer CTomforde S(2017)ApplicationsOrganic Computing – Technical Systems for Survival in the Real World10.1007/978-3-319-68477-2_8(429-547)Online publication date: 28-Dec-2017
https://doi.org/10.1007/978-3-319-68477-2_8

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents