Abstract
Conceptual model comprehensibility has attracted the interest of many experimental researchers over the past decades. Several studies have employed a variety of definitions and operationalizations of the comprehensibility construct as well as procedures for measuring it on a variety of model types. Intuitive comprehensibility is a specialization of the construct, referring to model or language comprehensibility exhibited by partially trained users. We present an experimental design for measuring the intuitive comprehensibility of a proposed extension to a goal modeling language as a means for reviewing experimental practices we have followed for similar studies in the past. Through such review, we hope to demonstrate the possibility of experimental design and technique reusability and its role as a motivating factor for more experimentation within the conceptual modeling research community.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alothman, N., Zhian, M., Liaskos, S.: User perception of numeric contribution semantics for goal models: an exploratory experiment. In: Mayr, H.C., Guizzardi, G., Ma, H., Pastor, O. (eds.) ER 2017. LNCS, vol. 10650, pp. 451–465. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69904-2_34
Amyot, D., Mussbacher, G.: User requirements notation: the first ten years, the next ten years (invited paper). J. Softw. 6(5), 747–768 (2011)
Bork, D., Schrüffer, C., Karagiannis, D.: Intuitive understanding of domain-specific modeling languages: proposition and application of an evaluation technique. In: Laender, A.H.F., Pernici, B., Lim, E.-P., de Oliveira, J.P.M. (eds.) ER 2019. LNCS, vol. 11788, pp. 311–319. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33223-5_26
Crump, M.J.C., McDonnell, J.V., Gureckis, T.M.: Evaluating Amazon’s mech. Turk as a tool for experimental behavioral research. PLoS One 8(3), 1–18 (2013)
Dalpiaz, F., Franch, X., Horkoff, J.: iStar 2.0 language guide. The Computing Research Repository (CoRR) abs/1605.0 (2016)
Evans, J.S.B.T.: Dual-processing accounts of reasoning, judgment, and social cognition. Annu. Rev. Psychol. 59(1), 255–278 (2008)
Falessi, D., et al.: Empirical software engineering experts on the use of students and professionals in experiments. Empir. Softw. Eng. 23(1), 452–489 (2017). https://doi.org/10.1007/s10664-017-9523-3
Gonçalves, E., Almendra, C., Goulão, M., Araújo, J., Castro, J.: Using empirical studies to mitigate symbol overload in iStar extensions. Softw. Syst. Model. 19(3), 763–784 (2019). https://doi.org/10.1007/s10270-019-00770-9
Guizzardi, G.: Ontological foundations for structural conceptual models. Ph.D. thesis, University of Twente (2005)
Hadar, I.: When intuition and logic clash: the case of the object-oriented paradigm. Sci. Comput. Program. 78(9), 1407–1426 (2013)
Houy, C., Fettke, P., Loos, P.: Understanding understandability of conceptual models – what are we actually talking about? In: Atzeni, P., Cheung, D., Ram, S. (eds.) ER 2012. LNCS, vol. 7532, pp. 64–77. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34002-4_5
Jošt, G., Huber, J., Heričko, M., Polančič, G.: An empirical investigation of intuitive understandability of process diagrams. Comput. Stand. Interfaces 48, 90–111 (2016)
Krogstie, J., Sindre, G., Jørgensen, H.: Process models representing knowledge for action: a revised quality framework. Eur. J. Inf. Syst. 15(1), 91–102 (2006)
Liaskos, S., Dundjerovic, T., Gabriel, G.: Comparing alternative goal model visualizations for decision making: an exploratory experiment. In: Proceedings of the 33rd ACM Symposium on Applied Computing (SAC 2018), Pau, France, pp. 1272–1281 (2018)
Liaskos, S., Jaouhar, I.: Towards a framework for empirical measurement of conceptualization qualities. In: Dobbie, G., Frank, U., Kappel, G., Liddle, S.W., Mayr, H.C. (eds.) ER 2020. LNCS, vol. 12400, pp. 512–522. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-62522-1_38
Liaskos, S., Khan, S.M., Soutchanski, M., Mylopoulos, J.: Modeling and reasoning with decision-theoretic goals. In: Ng, W., Storey, V.C., Trujillo, J.C. (eds.) ER 2013. LNCS, vol. 8217, pp. 19–32. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41924-9_3
Liaskos, S., Ronse, A., Zhian, M.: Assessing the intuitiveness of qualitative contribution relationships in goal models: an exploratory experiment. In: Proceedings of the 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2017), Toronto, Ontario, pp. 466–471 (2017)
Liaskos, S., Tambosi, W.: Factors affecting comprehension of contribution links in goal models: an experiment. In: Laender, A.H.F., Pernici, B., Lim, E.-P., de Oliveira, J.P.M. (eds.) ER 2019. LNCS, vol. 11788, pp. 525–539. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33223-5_43
Mair, P., Wilcox, R.: Robust statistical methods in R using the WRS2 package. Behav. Res. Methods 52(2), 464–488 (2019). https://doi.org/10.3758/s13428-019-01246-w
Maxwell, S.E., Delaney, H.D.: Designing Experiments and Analyzing Data, 2nd edn. Taylor and Francis Group, LLC, New York (2004)
Moody, D.L.: The “Physics” of notations: toward a scientific basis for constructing visual notations in software engineering. IEEE Trans. Softw. Eng. 35(6), 756–779 (2009)
Roelens, B., Bork, D.: An evaluation of the intuitiveness of the PGA modeling language notation. In: Nurcan, S., Reinhartz-Berger, I., Soffer, P., Zdravkovic, J. (eds.) BPMDS/EMMSAD -2020. LNBIP, vol. 387, pp. 395–410. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49418-6_27
Rosnow, R.L., Rosenthal, R.: Beginning Behavioral Research: A Conceptual Primer, 6th edn. Pearson Prentice Hall, Upper Saddle River (2008)
Wand, Y., Weber, R.: On the ontological expressiveness of information systems analysis and design grammars. Inf. Syst. J. 3(4), 217–237 (1993)
Yu, E.S.K.: Towards modelling and reasoning support for early-phase requirements engineering. In: Proceedings of the 3rd IEEE International Symposium on Requirements Engineering (RE 1997), Annapolis, MD, pp. 226–235 (1997)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Liaskos, S., Zhian, M., Jaouhar, I. (2020). Experimental Practices for Measuring the Intuitive Comprehensibility of Modeling Constructs: An Example Design. In: Grossmann, G., Ram, S. (eds) Advances in Conceptual Modeling. ER 2020. Lecture Notes in Computer Science(), vol 12584. Springer, Cham. https://doi.org/10.1007/978-3-030-65847-2_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-65847-2_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65846-5
Online ISBN: 978-3-030-65847-2
eBook Packages: Computer ScienceComputer Science (R0)