Abstract
The use of simulation modeling in computational analysis of organizations is becoming a prominent approach in social science research. However, relying on simulations to gain intuition about social phenomena has significant implications. While simulations may give rise to interesting macro-level phenomena, and sometimes even mimic empirical data, the underlying micro and macro level processes may be far from realistic. Yet, this realism may be important to infer results that are relevant to existing theories of social systems and to policy making. Therefore, it is important to assess not only predictive capability but also explanation accuracy of formal models in terms of the degree of realism reflected by the embedded processes. This paper presents a process-centric perspective for the validation and verification (V&V) of agent-based computational organization models. Following an overview of the role of V&V within the life cycle of a simulation study, emergent issues in agent-based organization model V&V are outlined. The notion of social contract that facilitates capturing micro level processes among agents is introduced to enable reasoning about the integrity and consistency of agent-based organization designs. Social contracts are shown to enable modular compositional verification of interaction dynamics among peer agents. Two types of consistency are introduced: horizontal and vertical consistency. It is argued that such local consistency analysis is necessary, but insufficient to validate emergent macro processes within multi-agent organizations. As such, new formal validation metrics are introduced to substantiate the operational validity of emergent macro-level behavior.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aho VA (1990) Algorithms for finding patterns in strings. Handbook of Theoretical Computer Science, Volume A: Algorithms and complexity, North Holland, Amsterdam, Elsevier Science Publishers B. V (North Holland), Amsterdam, pp 255–400
Axelrod R (1997) Advancing the art of simulation in the social sciences. In: Conte Rosario, Hegselmann Rainer, and Terna Pietro (eds) Simulating Social Phenomena. Springer, Berlin, pp 21–40
Axtell R (2003) The new coevolution of information science and social science, Working Paper, Brookings Institution 2003
Balci O, Nance RE (1985) Formulated problem verification as an explicit requirement of model credibility. Simulation 45(2):76–86
Balci O (1987) Credibility and assessment of simulation results: The State of the Art. In: Balci O (ed) Methodology and Validation proceedings of the conference on methodology and validation, 6–9 April. Orlando Fl. San Diego, CA, Society for Computer Simulation
Banks J, Gerstein S, Searles SP (1987) Modeling processes, validation, and verification of complex simulations: A survey. In: Balci O (ed) Methodology and Validation Proceedings of the conference on methodology and validation, Orlando Fl, San Diego, CA, Society for Computer Simulation, pp. 13–18
Barnsley MF et al (1988) The science of fractal images. Springer-Verlag, New York
Carley MK, Newell A (1994) The nature of the social agent. J Math Sociol 19(4): 221–262
Carley MK, Butts C (1997) An algorithmic approach to the comparison of partially labeled graphs. In: Proceedings of the 1997 International Symposium on Command and Control Research and Technology, Washington, DC
Carley MK, Gasser L (1999) Computational organization theory. In: G. Weiss (ed.) Distributed artificial intelligence, MIT Press, Cambridge, MA, (Ch. 7), pp 206–220
Carley MK (2002) Computational organizational science and organizational engineering. Simulation Practice and Theory. (In press)
Clarke EM, Grumberg O, Long DE (1994) Model checking and abstraction. ACM Transa Programming Lang Sys 16(5):1512–1542
Dalal PN, Kamath M, Kolarik WJ, Sivaraman E (2004) Toward an integrated framework for modeling enterprise processes. Communications of the ACM, 47(3):83–87
Derry R, Landry M, Banville C (1993) Revisiting the issue of model validation in OR: An epistemological view. Eur J Oper Res 66(2):168–184
Ferber J (1999) Multi agent systems: An introduction to distributed Artificial Intelligence. Addison Wesley
Epstein J, Axtell R (1996) Growing artificial societies: Social science from the Bottom Up, MIT Press Brookings, MA
Gasser L, Hulhage I, Leverich B, Lieb J, Majchrzak A (1993) Organization as Complex Dynamic Design Problems, Progress in Artificial Intelligence. Lecture Notes in Artificial Intelligence 727. Springer-Verlag
Jain R (1991) The art of computer systems performance analysis. Addison-Wesley
Johnson E (1994) Miller and freuds probability and statistics for engineers, 5th Edition, Englewood-Cliffs, N.J, Prentice Hall
Kaufmann S (1996) At home in the universe: The search for laws of Self-Organization and Complexity, Oxford Univ. Press, Oxford, England
Kerlinger F (1973) Foundations of behavioral research. Reinhart & Winston, New York
Landry M, Malouin JL, Oral M (1983) Model validation in operations research. Eur J Oper Res 14(3):207–220
Law AM, Kelton WD (1991) Simulation modeling and analysis. 2nd edition. McGraw-Hill, New York, NY
Liskov B, Wing MJ (1997) A behavioral notion of subtyping. ACM TOPLAS 16(6):1811–1841
MacKenzie G, Schulmeyer G, Yilmaz L (2002) Verification technology potential with different modeling and simulation development and implementation Paradigms, Foundations for V & V in the 21st Century Workshop (Foundations ′02) The Johns Hopkins University Applied Physics Laboratory Laurel, Maryland, USA
Nance ER (1987) The conical methodology: A framework for simulation model development. Methodology and validation, SCS, San Diego, CA, pp 38–43
Naylor TH, Finger JM (1963) Verification of computer simulation models. Management Science 14(2)
Nazareth LD (1989) Issues in the verification of knowledge in rule-based systems. Inter J Man-Machine Studies 30:255–271
O’Keefe MR, O’Leary DE (1993) Expert system verification and validation. Artif Intell Rev 7(3):3–42
Oral M, Kettani O (1993) The facets of the modeling and validation process in operations research. Eur J Oper Res 66(2):216–234
Pace Dale K (2000) Simulation conceptual model issues: Development methods (Part 1), Interaction with Simulation requirements (Part 2), and simulation development costs and V & V Costs (Part 3). In: Proceedings of the 2000 Summer Computer Simulation Conference, July 16–20, 2000, Vancouver, British Columbia
Pace KD (2003) The roles of metrics in simulation verification and validation. In: Proceedings of the Spring 2003 Simulation Interoperability Workshop, March 30–April 4, 2003, Kissimmee, FL, CD
Pearl J (1999) Causality: Models, Reasoning, and Inference. Cambridge University Press this month (March 2000)
Sankof D, Kruskal JB (1983) Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison. Addison-Wesley Publishing Company
Sargent GR (1996) Verifying and validating simulation models. In: Proceedings of the 1996 Winter Simulation Conference, pp 55–64
Sargent RG (1999) Validation and verification of simulation models. In: Farrington PA, Nembhard HB, Sturrock DT, and Evans GW, (eds), Proceedings of the 1999 Winter Simulation Conference, IEEE, Piscataway, NJ, pp 39–48
Spriet JA, Vansteenkiste GC (1982) Computer-Aided Modelling and Simulation. Academic Press, London, England)
Shoham Y (1993) Agent oriented programming. Artif Intell 60:51–92
Stephen AG (1994). String searching algorithms. World Scientific Publishing, Singapore
Waldrop MM (1993) Complexity: The Emerging science at the edge of order and chaos. Touchstone books
Wing MJ, Farahani MV (1995) Model checking software systems: A case study. SIGSOFT Software Engineering Notes 20(4):128–39
Author information
Authors and Affiliations
Corresponding author
Additional information
Levent Yilmaz is Assistant Professor of Computer Science and Engineering in the College of Engineering at Auburn University and co-founder of the Auburn Modeling and Simulation Laboratory of the M&SNet. Dr. Yilmaz received his Ph.D. and M.S. degrees from Virginia Polytechnic Institute and State University (Virginia Tech). His research interests are on advancing the theory and methodology of simulation modeling, agent-directed simulation (to explore dynamics of socio-technical systems, organizations, and human/team behavior), and education in simulation modeling. Dr. Yilmaz is a member of ACM, IEEE Computer Society, Society for Computer Simulation International, and Upsilon Pi Epsilon. URL: http://www.eng.auburn.edu/~yilmaz
Rights and permissions
About this article
Cite this article
Yilmaz, L. Validation and verification of social processes within agent-based computational organization models. Comput Math Organiz Theor 12, 283–312 (2006). https://doi.org/10.1007/s10588-006-8873-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10588-006-8873-y