Abstract
Simulations provide a useful methodological approach for studying the behavior of complex socio-technical information systems (IS), in which humans and IT artifacts interact to process information. However, the use of simulations is relatively new in IS research and the current presence and impact of simulation-based studies is still limited. Furthermore, simulation-based research is quite different from other approaches, making it difficult to position and evaluate it adequately. Therefore, this paper first analyses the epistemic particularities of simulation-based IS research. Based on this analysis, a structured literature review of the status quo of simulation-based IS research was conducted, to understand how IS scholars currently employ simulation. A comparison of the epistemic particularities of simulation-based research with its status quo in IS literature allows to critically examine epistemic inferences in the respective research process. The results provide guidance for prospective simulation-based IS research through discussing the theory-based derivation of simulation models, as well as different simulation techniques, validation techniques, and simulation uses.
Similar content being viewed by others
Notes
An earlier version of this literature review was previously presented at the 36th International Conference on Information Systems (Beese et al. 2015). In the paper at hand we not only significantly extend the literature review and use a completely different coding approach, but also focus on the additional insights that follow from the discussion of the epistemic particularities of simulation-based IS research.
References
Adner R, Pólos L, Ryall M, Sorenson O (2009) The case for formal theory. Acad Manag Rev 34:201–208. https://doi.org/10.5465/AMR.2009.36982613
Aminof B, Murano A, Rubin S, Zuleger F (2016) Automatic verification of multi-agent systems in parameterised grid-environments. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems. Richland, pp 1190–1199
Arlot S, Celisse A (2010) A survey of cross-validation procedures for model selection. Stat Surv 4:40–79. https://doi.org/10.1214/09-SS054
Ashish AK, Aghav J (2013) Automated techniques and tools for program analysis. In: Proceedings of the 4th International Conference on Computing, Communications and Networking Technologies. pp 1–7
Axelrod R (1997) Advancing the art of simulation in the social sciences. In: Conte R, Hegselmann R, Terna P (eds) Simulating social phenomena. Springer, Heidelberg, pp 21–40
Axtell R, Axelrod R, Epstein JM, Cohen MD (1996) Aligning simulation models: a case study and results. Comput Math Organ Theory 1:123–141. https://doi.org/10.1007/BF01299065
Becker J, Niehaves B, Klose K (2005) A framework for epistemological perspectives on simulation. JASSS 8(4):1–16
Becker J, Brocke J, vom Heddier M, Seidel S (2015) In search of information systems (grand) challenges. Bus Inf Syst Eng 57(6):377–390. https://doi.org/10.1007/s12599-015-0394-0
Bednar P, Sadok M (2015) A socio-technical toolbox for business systems analysis and design. In: 1st International Workshop on Socio-Technical Perspective in IS Development, pp 29–31
Beese J, Haki K, Aier S (2015) On the conceptualization of information systems as socio-technical phenomena in simulation-based research. In: Proceedings of the 36th International Conference on Information Systems. Fort Worth
Bélanger F, Cefaratti M, Carte T, Markham S (2014) Multilevel research in information systems: concepts, strategies, problems, and pitfalls. J Assoc Inf Syst 15(9):614–650
Bichler M, Frank U, Avison D, Malaurent J, Fettke P, Hovorka D, Krämer J, Schnurr D, Mueller B, Suhl L, Thalheim B (2016) Theories in business and information systems engineering. Bus Inf Syst Eng 58(4):291–319
Birdsey L, Szabo C (2014) An architecture for identifying emergent behavior in multi-agent systems. In: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems. Richland, pp 1455–1456
Boero R, Squazzoni F (2005) Does empirical embeddedness matter? Methodological issues on agent-based models for analytical social science. JASSS 8(4)
Bonabeau E (2002) Agent-based modeling: methods and techniques for simulating human systems. Proc Natl Acad Sci 99:7280–7287. https://doi.org/10.1073/pnas.082080899
Bostrom RP, Heinen JS (1977) MIS problems and failures: a socio-technical perspective. Part I: the causes. MIS Q 1:17–32. https://doi.org/10.2307/248710
Bratley P, Fox BL, Schrage LE (2011) A guide to simulation. Springer, New York
Burton RM, Obel B (2011) Computational modeling for what-is, what-might-be, and what-should-be studies—and triangulation. Organ Sci 22:1195–1202. https://doi.org/10.1287/orsc.1100.0635
Carley KM (2001) Computational approaches to sociological theorizing. In: Turner JH (ed) Handbook of sociological theory. Springer, US, pp 69–83
Chang P-C, Lai C-Y, Lai KR (2006) A hybrid system by evolving case-based reasoning with genetic algorithm in wholesaler’s returning book forecasting. Decis Support Syst 42:1715–1729. https://doi.org/10.1016/j.dss.2006.02.014
Coleman JS (1994) Foundations of social theory. Harvard University Press, Cambridge
Curşeu PL (2006) Emergent states in virtual teams: a complex adaptive systems perspective. J Inf Technol 21:249–261. https://doi.org/10.1057/palgrave.jit.2000077
D’Silva V, Kroening D, Weissenbacher G (2008) A survey of automated techniques for formal software verification. IEEE Trans Comput Aided Des Integr Circuits Syst 27:1165–1178. https://doi.org/10.1109/TCAD.2008.923410
Davis JP, Eisenhardt KM, Bingham CB (2007) Developing theory through simulation methods. Acad Manag Rev 32:480–499. https://doi.org/10.5465/AMR.2007.24351453
Doan TT, Yao Y, Alechina N, Logan B (2014) Verifying heterogeneous multi-agent programs. In: Proceedings of the 2014 International Conference on autonomous agents and multi-agent systems. Richland, pp 149–156
Dooley K (2002) Simulation research methods. In: Baum J (ed) Companion to organizations. Blackwell, London, pp 829–848
Dowling D (1999) Experimenting on theories. Sci Context 12:261–273. https://doi.org/10.1017/S0269889700003410
Edmonds B, Hales D (2003) Replication, replication and replication: some hard lessons from model alignment. JASSS 6(4):1–4
Epstein LG (1999) A definition of uncertainty aversion. Rev Econ Stud 66(3):579–608
Epstein JM (2008) Why model? JASSS 11(4):1–12
Frank U (2011) Multi-perspective enterprise modelling: background and terminological foundation. ICB-Research Report, No. 46. University Duisburg-Essen, Essen
Frank U (2014) Multilevel modeling. Bus Inf Syst Eng 6(6):319–337. https://doi.org/10.1007/s12599-014-0350-4
Frank U, Troitzsch KG (2005) Epistemological perspectives on simulation. JASSS 8(4):1–5
Frank U, Strecker S, Fettke P, Brocke J, vom Becker J, Sinz E (2014) The research field “modeling business information systems”. Bus Inf Syst Eng 6(1):39–43. https://doi.org/10.1007/s12599-013-0301-5
Frigg R, Hartmann S (2017) Models in science. In: Zalta EN (ed) The Stanford encyclopedia of philosophy, Spring, 2017th edn. Metaphysics Research Lab, Stanford University, Stanford
Frigg R, Reiss J (2008) The philosophy of simulation: hot new issues or same old stew? Synthese 169:593–613. https://doi.org/10.1007/s11229-008-9438-z
Graupe D (2013) Principles of artificial neural networks, 3rd edn. World Scientific, Singapore
Gregor S, Hovorka D (2011) Causality: the elephant in the room in information systems epistemology. In: Proceedings of the 19th European Conference on Information Systems. Helsinki
Gregoriades A, Sutcliffe A (2008) A socio-technical approach to business process simulation. Decis Support Syst 45:1017–1030. https://doi.org/10.1016/j.dss.2008.04.003
Grüne-Yanoff T, Weirich P (2010) The philosophy and epistemology of simulation: a review. Simul Gaming 41:20–50. https://doi.org/10.1177/1046878109353470
Harrison JR, Lin Zhiang, Carroll GR, Carley KM (2007) Simulation modeling in organizational and management research. Acad Manag Rev 32:1229–1245. https://doi.org/10.5465/AMR.2007.26586485
Hedström P, Ylikoski P (2010) Causal mechanisms in the social sciences. Annu Rev Social 36:49–67. https://doi.org/10.1146/annurev.soc.012809.102632
Houy C, Fettke P, Loos P (2012) Understanding understandability of conceptual models—what are we actually talking about? Conceptual modeling. Springer, Heidelberg, pp 64–77
Houy C, Fettke P, Loos P (2015) Stylized Facts as an instrument for literature review and cumulative information systems research. Commun AIS 37:1
Hua Z, Sun Y, Xu X (2011) Operational causes of bankruptcy propagation in supply chain. Decis Support Syst 51:671–681. https://doi.org/10.1016/j.dss.2011.03.007
Humphreys P (1990) Computer simulations. In: Proceedings of the Biennial Meeting of the Philosophy of Science Association, pp 497–506
Humphreys P (2008) The philosophical novelty of computer simulation methods. Synthese 169:615–626. https://doi.org/10.1007/s11229-008-9435-2
Jamroga W, Konikowska B, Penczek W (2016) Multi-valued verification of strategic ability. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems. Richland, pp 1180–1189
Johnson CF (1996) Deductive versus inductive reasoning: a closer look at economics. Soc Sci J 33:287–299. https://doi.org/10.1016/S0362-3319(96)90024-5
Johnson SLF, Samer F, Kudaravalli S (2014) Emergence of power laws in online communities: the role of social mechanisms and preferential attachment. MIS Q 38:795–808 (A13)
Kim Y, Street WN (2004) An intelligent system for customer targeting: a data mining approach. Decis Support Syst 37:215–228. https://doi.org/10.1016/S0167-9236(03)00008-3
Kochanowicz J, Ah-Hwee T, Thalmann D (2013) Enabling generative, emergent artificial culture. In: Proceedings of the 2013 International conference on autonomous agents and multi-agent systems. Richland, pp 789–792
Kulik BW, Baker T (2008) Putting the organization back into computational organization theory: a complex Perrowian model of organizational action. Comput Math Organ Theory 14:84–119. https://doi.org/10.1007/s10588-008-9022-6
Kuo RJ, Liao JL, Tu C (2005) Integration of ART2 neural network and genetic K-means algorithm for analyzing web browsing paths in electronic commerce. Decis Support Syst 40:355–374. https://doi.org/10.1016/j.dss.2004.04.010
Küppers G, Lenhard J (2005) Validation of simulation: patterns in the social and natural sciences. JASSS 8(4):1–3
Lacy S, Riffe D (1996) Sampling error and selecting intercoder reliability samples for nominal content categories. J Mass Commun Q 73:963–973. https://doi.org/10.1177/107769909607300414
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174. https://doi.org/10.2307/2529310
Law AM (2008) How to build valid and credible simulation models. In: Proceedings of the 40th conference on winter simulation. Winter Simulation Conference, Miami, pp 39–47
Lee AS (2010) Retrospect and prospect: information systems research in the last and next 25 years. J Inf Technol 25:336–348. https://doi.org/10.1057/jit.2010.24
Lee J-S, Filatova T, Ligmann-Zielinska A, Hassani-Mahmooei B, Stonedahl F, Lorscheid I, Voinov A, Polhill JG, Sun Z, Parker DC (2015) The complexities of agent-based modeling output analysis. JASSS 18(4):1–4
Lim S, Saldanha T, Malladi S, Melville N (2013) Theories used in information systems research: insights from complex network analysis. JITTA 14(2):5–46
Lombard M, Snyder-Duch J, Bracken CC (2002) Content analysis in mass communication: assessment and reporting of intercoder reliability. Hum Commun Res 28:587–604. https://doi.org/10.1111/j.1468-2958.2002.tb00826.x
Loos P, Mettler T, Winter R, Goeken M, Frank U, Winter A (2013) Methodological pluralism in business and information systems engineering? Bus Inf Syst Eng 5(6):453–460. https://doi.org/10.1007/s12599-013-0299-8
Lowry PB, Moody GD, Gaskin J, Galletta DF, Humphreys S, Barlow JB, Wilson D (2013) Evaluating journal quality and the association for information systems (AIS) senior scholars’ journal basket via bibliometric measures: do expert journal assessments add value?. Social Science Research Network, Rochester
Luna-Reyes LF, Zhang J, Ramón Gil-García J, Cresswell AM (2005) Information systems development as emergent socio-technical change: a practice approach. Eur J Inf Syst 14:93–105. https://doi.org/10.1057/palgrave.ejis.3000524
Lyytinen K, Newman M (2008) Explaining information systems change: a punctuated socio-technical change model. Eur J Inf Syst 17:589–613. https://doi.org/10.1057/ejis.2008.50
Macal CM, North MJ (2009) Agent-based modeling and simulation. In: Winter Simulation Conference. Winter Simulation Conference, Austin, pp 86–98
Manzo G (2007) Variables, mechanisms, and simulations: can the three methods be synthesized? Revue française de sociologie 48:35–71
McKelvey B (2002) Model-centered organization science epistemology. Companion to organizations. Oxford University Press, New York, pp 752–780
McLeod L, Doolin B (2012) Information systems development as situated socio-technical change: a process approach. Eur J Inf Syst 21:176–191. https://doi.org/10.1057/ejis.2011.43
Montali M, Calvanese D, De Giacomo G (2014) Verification of data-aware commitment-based multiagent system. In: Proceedings of the 2014 international conference on autonomous agents and multi-agent systems. Richland, pp 157–164
Morgan MS, Morrison M (1999) Models as mediators: perspectives on natural and social science. Cambridge University Press, Cambridge
Nan N (2011) Capturing bottom-up information technology use processes: a complex adaptive systems model. MIS Q 35:505–532 (A7)
Neuendorf KA (2016) The content analysis guidebook. SAGE, London
Olson DL, Delen D, Meng Y (2012) Comparative analysis of data mining methods for bankruptcy prediction. Decis Support Syst 52:464–473. https://doi.org/10.1016/j.dss.2011.10.007
Orlikowski WJ (1992) The duality of technology: rethinking the concept of technology in organizations. Organ Sci 3:398–427. https://doi.org/10.1287/orsc.3.3.398
Orlikowski WJ, Iacono CS (2001) Research commentary: desperately seeking the “IT” in IT research—a call to theorizing the IT artifact. Inf Syst Res 12:121–134. https://doi.org/10.1287/isre.12.2.121.9700
Parunak HVD, Savit R, Riolo RL (1998) Agent-based modeling vs. equation-based modeling: a case study and users’ guide. In: Sichman JS, Conte R, Gilbert N (eds) Multi-agent systems and agent-based simulation. Springer, Heidelberg, pp 10–25
Pias C (2011) On the epistemology of computer simulation. Zeitschrift für Medien- und Kulturforschung 2011(1):29–54
Poile C, Safayeni F (2016) Using Computational modeling for building theory: a double edged sword. JASSS 19(3):1–20. https://doi.org/10.18564/jasss.3137
Prat N, Comyn-Wattiau I, Akoka J (2015) A taxonomy of evaluation methods for information systems artifacts. J Manag Inf Syst 32:229–267. https://doi.org/10.1080/07421222.2015.1099390
Ripley BD (2009) Stochastic simulation. Hoboken, NJ, USA
Rivkin JW (2001) Reproducing knowledge: replication without imitation at moderate complexity. Organ Sci 12:274–293. https://doi.org/10.1287/orsc.12.3.274.10106
Roberts N, Deal R, Shaffer W, Garet M, Anderson D (1994) Introduction to computer simulation; a system dynamics approach. Productivity Press Inc. CRC press
Rohrlich F (1990) Computer simulation in the physical sciences. In: Proceedings of the Biennial Meeting of the Philosophy of Science Association, pp 507–518
Rowe F (2014) What literature review is not: diversity, boundaries and recommendations. Eur J Inf Syst 23:241–255. https://doi.org/10.1057/ejis.2014.7
Saldaña J (2013) The coding manual for qualitative researchers. SAGE, London
Sargent RG (2005) Verification and validation of simulation models. In: Proceedings of the 37th conference on winter simulation. Orlando, pp 130–143
Sidorova A, Evangelopoulos N, Valacich J, Ramakrishnan T (2008) Uncovering the intellectual core of the information systems discipline. MIS Q 32:467–482
Spagnoletti P, Za S, Winter R (2013) Exploring foundations for using simulations in IS research. In: Proceedings of the 34th international conference on information systems. Milan
Sterman JD (2002) Systems dynamics modeling: tools for learning in a complex world. IEEE Eng Manag Rev 30:42. https://doi.org/10.1109/EMR.2002.1022404
Straub D (2012) Editor’s comments: does MIS have native theories? MIS Q 36:iii–xii
Sun Z-L, Choi T-M, Au K-F, Yu Y (2008) Sales forecasting using extreme learning machine with applications in fashion retailing. Decis Support Syst 46:411–419. https://doi.org/10.1016/j.dss.2008.07.009
Tiwana A, Bush AA (2007) A comparison of transaction cost, agency, and knowledge-based predictors of IT outsourcing decisions: a US-Japan cross-cultural field study. J Manag Inf Syst 24:259–300. https://doi.org/10.2753/MIS0742-1222240108
Trier M (2008) Research note—towards dynamic visualization for understanding evolution of digital communication networks. Inf Syst Res 19:335–350. https://doi.org/10.1287/isre.1080.0191
vom Brocke J, Simons A, Riemer K, Niehaves B, Plattfaut R, Cleven A (2015) Standing on the shoulders of giants: challenges and recommendations of literature search in information systems research. Commun AIS 37(1):205–223
Wang C-H, Chuang J-J (2016) Integrating decision tree with back propagation network to conduct business diagnosis and performance simulation for solar companies. Decis Support Syst 81:12–19. https://doi.org/10.1016/j.dss.2015.10.004
Webster J, Watson RT (2002) Analyzing the past to prepare for the future: writing a literature review. MIS Q 26:xiii–xxiii
Weinhardt JM, Vancouver JB (2012) Computational models and organizational psychology: opportunities abound. Organ Psychol Rev 2:267–292. https://doi.org/10.1177/2041386612450455
Whitley D (1994) A genetic algorithm tutorial. Stat Comput 4:65–85. https://doi.org/10.1007/BF00175354
Winsberg E (1999) Sanctioning models: the epistemology of simulation. Sci Context 12:275–292. https://doi.org/10.1017/S0269889700003422
Winsberg E (2001) Simulations, models, and theories: complex physical systems and their representations. Philos Sci 68:S442–S454
Winsberg E (2003) Simulated experiments: methodology for a virtual world. Philos Sci 70:105–125. https://doi.org/10.1086/367872
Winsberg E (2009) Computer simulation and the philosophy of science. Philos Compass 4:835–845. https://doi.org/10.1111/j.1747-9991.2009.00236.x
Wöhner T, Köhler S, Peters R (2015) Managed wikis—a new approach for web 2.0. Bus Inf. Syst Eng 57(3):155–166
Woodard CJ, Clemons E (2014) Modeling the evolution of generativity and the emergence of digital ecosystems. In: Proceedings of the 35th international conference on information systems. Auckland
Wu PP-Y, Fookes C, Pitchforth J, Mengersen K (2015) A framework for model integration and holistic modelling of socio-technical systems. Decis Support Syst 71:14–27. https://doi.org/10.1016/j.dss.2015.01.006
Xiao S, Dong M (2015) Hidden semi-Markov model-based reputation management system for online to offline (O2O) e-commerce markets. Decis Support Syst 77:87–99. https://doi.org/10.1016/j.dss.2015.05.013
Zhang M, Gable G (2014) Rethinking the value of simulation methods in the Inf Syst Res field: a call for reconstructing contribution for a broader audience. In: Proceedings of the 35th international conference on information systems. Auckland
Zhuge H (2006) Knowledge flow network planning and simulation. Decis Support Syst 42:571–592. https://doi.org/10.1016/j.dss.2005.03.007
Author information
Authors and Affiliations
Corresponding author
Additional information
Accepted after two revisions by Prof. Dr. Frank.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Beese, J., Haki, M.K., Aier, S. et al. Simulation-Based Research in Information Systems. Bus Inf Syst Eng 61, 503–521 (2019). https://doi.org/10.1007/s12599-018-0529-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12599-018-0529-1