Abstract
This paper presents an overview of uncertainty assessment in agent-based simulations, mainly related to land use and cover change. Almost every multiagent-based simulation review has expressed the need for statistical methods to evaluate the certainty of the results. Yet these problems continue to be underestimated and often neglected. This work aims to review how uncertainty is being portrayed in agent-based simulation and to perform an exploratory study to use statistical methods to estimate uncertainty. MASE, a Multi-Agent System for Environmental simulation, is the system under study. We first identified the most sensitive parameters using Morris One-at-a-Time sensitivity analysis. The efforts to assess agent-based simulation through statistical methods are paramount to corroborate and improve the level of confidence of the research that has been made in land use simulation.
This papers has already been published in: \(\copyright \) Springer International Publishing AG 2017 G. Sukthankar and J. A. Rodriguez-Aguilar (Eds.): AAMAS 2017 Best Papers, LNCS 10642, pp. 36–50, 2017. https://doi.org/10.1007/978-3-319-71682-4_3
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
Software Availability: http://mase.cic.unb.br/.
References
Albrecht, S.V., Ramamoorthy, S.: Are you doing what I think you are doing? Criticising uncertain agent models. In: Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence, Amsterdam, Netherlands, p. 10 (2015)
Herd, B., Miles, S., McBurney, P., Luck, M.: MC\(^\mathtt{2}\)MABS: a Monte Carlo model checker for multiagent-based simulations. In: Gaudou, B., Sichman, J.S. (eds.) MABS 2015. LNCS (LNAI), vol. 9568, pp. 37–54. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31447-1_3
Bommel, P.: Foreword. In: Adamatti, D.F. (ed.) Multi-Agent Based Simulations Applied to Biological and Environmental Systems, pp. xv–xviii. IGI Global, Hershey (2017)
Coelho, C.G., Abreu, C.G., Ramos, R.M., Mendes, A.H., Teodoro, G., Ralha, C.G.: MASE-BDI: agent-based simulator for environmental land change with efficient and parallel auto-tuning. Appl. Intell. 45(3), 904–922 (2016)
Campolongo, F., Braddock, R.: The use of graph theory in the sensitivity analysis of the model output: a second order screening method. Reliab. Eng. Syst. Saf. 64(1), 1–12 (1999). https://doi.org/10.1016/S0951-8320(98)00008-8
Casti, J.L.: Complexification: Explaining a Paradoxical World through the Science of Surprise. HarperCollins, New York (1995). (Reprint edn.)
Intergovernmental Panel on Climate Change: Climate Change 2013 The Physical Science Basis: Working Group I Contribution to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press (2014)
Gan, Y., Duan, Q., Gong, W., Tong, C., Sun, Y., Chu, W., Ye, A., Miao, C., Di, Z.: A comprehensive evaluation of various sensitivity analysis methods: a case study with a hydrological model. Environ. Model. Softw. 51, 269–285 (2014)
Goldsman, D., Tokol, G.: Output analysis procedures for computer simulations. In: Joines, J., Barton, R.R., Kang, K., Fishwick, P. (eds.) Proceedings of the 2000 Winter Simulation Conference, pp. 39–45 (2000)
Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S.K., Huse, G.: A standard protocol for describing individual-based and agent-based models. Ecol. Model. 198(1–2), 115–126 (2006). http://linkinghub.elsevier.com/retrieve/pii/S0304380006002043
Grimm, V., Berger, U., DeAngelis, D.L., Polhill, J.G., Giske, J., Railsback, S.F.: The ODD protocol: a review and first update. Ecol. Model. 221(23), 2760–2768 (2010). http://linkinghub.elsevier.com/retrieve/pii/S030438001000414X
Heath, B., Hill, R., Ciarallo, F.: A survey of agent-based modeling practices (January 1998 to July 2008). JASSS 12(4), 1–49 (2009)
Houghton, J., Filho, L.M., Callander, B., Harris, N., Kattenberg, A., Maskell, K. (eds.): Climate Change 1995 The Science of Climate Change. The Intergovernmental Panel on Climate Change (1996)
Iooss, B., Lemaître, P.: A review on global sensitivity analysis methods. In: Dellino, G., Meloni, C. (eds.) Uncertainty Management in Simulation-Optimization of Complex Systems. ORSIS, vol. 59, pp. 101–122. Springer, Boston (2015). https://doi.org/10.1007/978-1-4899-7547-8_5
Kelly (Letcher), R.A., Jakeman, A.J., Barreteau, O., Borsuk, M.E., ElSawah, S., Hamilton, S.H., Henriksen Jr., H., Kuikka, S., Maier, H.R., Rizzoli, A.E., van Delden, H., Voinov, A.A.: Selecting among five common modelling approaches for integrated environmental assessment and management. Environ. Model. Softw. 47, 159–181 (2013)
Kleijnen, J.P., Sanchez, S.M., Lucas, T.W., Cioppa, T.M.: A user’s guide to the brave new world of designing simulation experiments. INFORMS J. Comput. 17(3), 263–289 (2005). https://harvest.nps.edu/papers/UserGuideSimExpts.pdf
Le, Q.B., Seidl, R., Scholz, R.W.: Feedback loops and types of adaptation in the modelling of land-use decisions in an agent-based simulation. Environ. Model. Softw. 27–28, 83–96 (2012)
Lee, J.S., Filatova, T., Ligmann-Zielinska, A., Hassani-Mahmooei, B., Stonedahl, F., Lorscheid, I., Voinov, A., Polhill, G., Sun, Z., Parker, D.C.: The complexities of agent-based modeling output analysis. JASSS 18(4), 1–25 (2015)
Levy, S., Steinberg, D.M.: Computer experiments: a review. AStA Adv. Stat. Anal. 94(4), 311–324 (2010)
Li, J.D., Duan, Q.Y., Gong, W., Ye, A.Z., Dai, Y.J., Miao, C.Y., Di, Z.H., Tong, C., Sun, Y.W.: Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis. Hydrol. Earth Syst. Sci. Discuss. 10(2), 2243–2286 (2013)
Lorscheid, I., Heine, B.O., Meyer, M.: Opening the ‘Black Box’ of simulations: increased transparency and effective communication through the systematic design of experiments. Comput. Math. Organ. Theory 18(1), 22–62 (2012)
Marks, R.E.: Validating simulation models: a general framework and four applied examples. Comput. Econ. 30(3), 265–290 (2007)
Mastrandrea, M.D., Field, C.B., Stocker, T.F., Edenhofer, O., Ebi, K.L., Frame, D.J., Held, H., Kriegler, E., Mach, K.J., Matschoss, P.R., Plattner, G.K., Yohe, G.W., Zwiers, F.W.: Guidance note for lead authors of the IPCC fifth assessment report on consistent treatment of uncertainties. In: Intergovernmental Panel on Climate Change (IPCC), pp. 1–7 (2010)
McKay, M.D., Morrison, J.D., Upton, S.C.: Evaluating prediction uncertainty in simulation models. Comput. Phys. Commun. 117(1–2), 44–51 (1999)
Morris, M.D.: Factorial sampling plans for preliminary computational experiments. Technometrics 33(2), 161–174 (1991)
Paegelow, M., Camacho Olmedo, M.T., Mas, J.F., Houet, T.: Benchmarking of LUCC modelling tools by various validation techniques and error analysis. Cybergeo 701(online), 29 (2014)
Pontius, R.G., Boersma, W., Castella, J.C., Clarke, K., Nijs, T., Dietzel, C., Duan, Z., Fotsing, E., Goldstein, N., Kok, K., Koomen, E., Lippitt, C.D., McConnell, W., Mohd Sood, A., Pijanowski, B., Pithadia, S., Sweeney, S., Trung, T.N., Veldkamp, A.T., Verburg, P.H.: Comparing the input, output, and validation maps for several models of land change. Ann. Reg. Sci. 42(1), 11–37 (2008)
Ralha, C.G., Abreu, C.G.: A multi-agent-based environmental simulator. In: Adamatti, D.F. (ed.) Multi-Agent Based Simulations Applied to Biological and Environmental Systems, Chap. 5, pp. 106–127. IGI Global, Hershey (2017)
Ralha, C.G., Abreu, C.G., Coelho, C.G., Zaghetto, A., Macchiavello, B., Machado, R.B.: A multi-agent model system for land-use change simulation. Environ. Model. Softw. 42, 30–46 (2013)
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., Tarantola, S.: Global Sensitivity Analysis: The Primer. Wiley, Hoboken (2008)
Smajgl, A., Brown, D.G., Valbuena, D., Huigen, M.G.A.: Empirical characterisation of agent behaviours in socio-ecological systems. Environ. Model. Softw. 26(7), 837–844 (2011). https://doi.org/10.1016/j.envsoft.2011.02.011
Tong, C.: PSUADE Short Manual (Version 1.7). Lawrence Livermore National Laboratory (LLNL), Livermore (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Abreu, C.G., Ralha, C.G. (2018). Uncertainty Assessment in Agent-Based Simulation: An Exploratory Study. In: Dimuro, G., Antunes, L. (eds) Multi-Agent Based Simulation XVIII. MABS 2017. Lecture Notes in Computer Science(), vol 10798. Springer, Cham. https://doi.org/10.1007/978-3-319-91587-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-91587-6_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91586-9
Online ISBN: 978-3-319-91587-6
eBook Packages: Computer ScienceComputer Science (R0)