Skip to main content
Log in

Exploring how users engage with hybrid process artifacts based on declarative process models: a behavioral analysis based on eye-tracking and think-aloud

Software and Systems Modeling Aims and scope Submit manuscript

Abstract

Process design artifacts have been increasingly used to guide the modeling of business processes. To support users in designing and understanding process models, different process artifacts have been combined in several ways leading to the emergence of the so-called “hybrid process artifacts”. While many hybrid artifacts have been proposed in the literature, little is known about how they can actually support users in practice. To address this gap, this work investigates the way users engage with hybrid process artifacts during comprehension tasks. In particular, we focus on a hybrid representation of DCR Graphs (DCR-HR) combining a process model, textual annotations and an interactive simulation. Following a qualitative approach, we conduct a multi-granular analysis exploiting process mining, eye-tracking techniques, and verbal data analysis to scrutinize the reading patterns and the strategies adopted by users when being confronted with DCR-HR. The findings of the coarse-grained analysis provide important insights about the behavior of domain experts and IT specialists and show how user’s background and task type change the use of hybrid process artifacts. As for the fine-grained analysis, user’s behavior was classified into goal-directed and exploratory and different strategies of using the interactive simulation were identified. In addition, a progressive switch from an exploratory behavior to a goal-directed behavior was observed. These insights pave the way for an improved development of hybrid process artifacts and delineate several directions for future work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Notes

  1. http://www.dcrgraphs.net.

  2. https://ecoknow.org.

  3. http://english.sm.dk/media/14900/consolidation-act-on-social-services.pdf.

  4. http://andaloussi.org/sosym2019/design/.

  5. See http://andaloussi.org/sosym2019/design/Physical_assessment_questionnaire.xlsx.

  6. See http://andaloussi.org/sosym2019/design/Background_expertise_questionnaire.xlsx.

  7. See https://www.tobiipro.com/product-listing/tobii-pro-x3-120/.

  8. Atlas.ti, a qualitative data analysis tool. See https://atlasti.com.

  9. See http://andaloussi.org/sosym2019/analysis/.

  10. Quote translated from Danish.

References

  1. Hevner Von Alan, R., March, S.T., Park, J., Ram, S.: Design science in information systems research. MIS Q. 28(1), 75–105 (2004)

    Article  Google Scholar 

  2. Dengler, F., Vrandečić, D.: Wiki-Based Maturing of Process Descriptions, pp. 313–328 (2011)

  3. Pinggera, J., Porcham, T., Zugal, S., Weber, B.: LiProMo-Literate process modeling. In: CEUR Workshop Proceedings, vol. 855, pp. 163–170 (2012)

  4. Zugal, S., Pinggera, J., Weber, B.: Creating declarative process models using test driven modeling suite. In: Nurcan, S. (ed.) IS Olympics: Information Systems in a Diverse World, pp. 16–32. Springer, Berlin (2012)

    Google Scholar 

  5. Zugal, S., Pinggera, J., Weber, B.: Toward enhanced life-cycle support for declarative processes. J. Softw. Evol. Process 24(3), 285–302 (2012)

    Article  Google Scholar 

  6. De Smedt, J., De Weerdt, J., Serral, E., Vanthienen, J.: Improving understandability of declarative process models by revealing hidden dependencies. In: Nurcan, S., Soffer, P., Bajec, M., Eder, J. (eds.) Advanced Information Systems Engineering, pp. 83–98. Springer, Cham (2016)

    Chapter  Google Scholar 

  7. De Smedt, J., De Weerdt, J., Serral, E., Vanthienen, J.: Discovering hidden dependencies in constraint-based declarative process models for improving understandability. Inf. Syst. 74, 40–52 (2018)

    Article  Google Scholar 

  8. López, H.A., Debois, S., Hildebrandt, T.T., Marquard, M.: The process highlighter: from texts to declarative processes and back. In: BPM (Dissertation/Demos/Industry), pp. 66–70 (2018)

  9. Andaloussi, A.A., Buch-Lorentsen, J., López, H.A., Slaats, T., Weber, B.: Exploring the modeling of declarative processes using a hybrid approach. In: Laender, A.H.F., Pernici, B., Lim, E.-P., de Oliveira, J.P.M. (eds.) Conceptual Modeling, pp. 162–170. Springer, Cham (2019)

    Chapter  Google Scholar 

  10. Andaloussi, A.A., Burattin, A., Slaats, T., Petersen, A.C.M., Hildebrandt, T.T., Weber, B.: Exploring the understandability of a hybrid process design artifact based on DCR graphs. In: Reinhartz-Berger, I., Zdravkovic, J., Gulden, J., Schmidt, R. (eds.) Enterprise, Business-Process and Information Systems Modeling, pp. 69–84. Springer, Cham (2019)

    Chapter  Google Scholar 

  11. Andaloussi, A.A., Burattin, A., Slaats, T., Kindler, E., Weber, B.: On the declarative paradigm in hybrid business process representations: a conceptual framework and a systematic literature study. Inf. Syst. 91, 101505 (2020)

    Article  Google Scholar 

  12. Reichert, M., Weber, B.: Enabling Flexibility in Process-Aware Information Systems. Springer, Berlin (2012)

    Book  MATH  Google Scholar 

  13. Zugal, S., Pinggera, J., Weber, B.: The impact of testcases on the maintainability of declarative process models. In: Halpin, T., Nurcan, S., Krogstie, J., Soffer, P., Proper, E., Schmidt, R., Bider, I. (eds.) Enterprise, Business-Process and Information Systems Modeling, pp. 163–177. Springer, Berlin (2011)

    Chapter  Google Scholar 

  14. De Smedt, J., De Weerdt, J., Vanthienen, J., Poels, G.: Mixed-paradigm process modeling with intertwined state spaces. Bus. Inf. Syst. Eng. 58(1), 19–29 (2016)

    Article  Google Scholar 

  15. Pesic, M., Schonenberg, H., van der Aalst, W.M.P.: DECLARE: full support for loosely-structured processes. In: 11th IEEE International Enterprise Distributed Object Computing Conference (EDOC 2007), pp. 287–287. IEEE (2007)

  16. Debois, S., Hildebrandt, T., Marquard, M., Slaats, T.: Hybrid process technologies in the financial sector. In: Proceedings of the Industry Track at the 13th International Conference on Business Process Management 2015, vol. 4, pp. 107–119 (2015)

  17. Hildebrandt, T.T., Mukkamala, R.R.: Declarative event-based workflow as distributed dynamic condition response graphs. Electron. Proc. Theor. Comput. Sci. 69, 59–73 (2011)

    Article  Google Scholar 

  18. Marquard, M., Shahzad, M., Slaats, T.: Web-based modelling and collaborative simulation of declarative processes. In: Motahari-Nezhad, H.R., Recker, J., Weidlich, M. (eds.) Business Process Management, pp. 209–225. Springer, Cham (2016)

    Google Scholar 

  19. Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., van de Weijer, J.: Eye Tracking: A Comprehensive Guide to Methods and Measures. OUP, Oxford (2011)

    Google Scholar 

  20. Bergstrom, J.R., Schall, A.: Eye Tracking in User Experience Design. Elsevier, Amsterdam (2014)

    Google Scholar 

  21. Buscher, G., Dengel, A., Biedert, R., Elst, L.V.: Attentive documents: eye tracking as implicit feedback for information retrieval and beyond. ACM Trans. Interact. Intell. Syst. (TiiS) 1(2), 9 (2012)

    Google Scholar 

  22. Van Der Aalst, W.M.P.: Process Mining: Discovery, Conformance, and Enhancement of Business Processes. Springer, Berlin (2010)

    MATH  Google Scholar 

  23. Blascheck, T., Kurzhals, K., Raschke, M., Burch, M., Weiskopf, D., Ertl, T.: Visualization of eye tracking data: a taxonomy and survey. In: Computer Graphics Forum, vol. 36, pages 260–284. Wiley Online Library, London (2017)

  24. Janiszewski, C.: The influence of display characteristics on visual exploratory search behavior. J. Consum. Res. 25(3), 290–301 (1998)

    Article  Google Scholar 

  25. Pfeiffer, J., Meißner, M., Prosiegel, J., Pfeiffer, T.: Classification of goal-directed search and exploratory search using mobile eye-tracking. In: Proceedings of the International Conference on Information Systems (2014)

  26. Maalej, W., Nayebi, M., Johann, T., Ruhe, G.: Toward data-driven requirements engineering. IEEE Softw. 33(1), 48–54 (2016)

    Article  Google Scholar 

  27. Slaats, T.: Declarative and hybrid process discovery: recent advances and open challenges. J. Data Semant. 9, 1–18 (2020)

    Article  Google Scholar 

  28. Pesic, M.: Constraint-based workflow management systems: shifting control to users. Ph.D. thesis, TU Eindhoven (2008)

  29. Fahland, D., Lübke, D., Mendling, J., Reijers, H., Weber, B., Weidlich, M., Zugal, S.: Declarative versus imperative process modeling languages: the issue of understandability. In: Proceedings of EMMSAD, pp. 353–366. Springer, Berlin (2009)

  30. Fahland, D., Mendling, J., Reijers, H.A., Weber, B., Weidlich, M., Zugal, S.: Declarative versus imperative process modeling languages: the issue of maintainability. In: Rinderle-Ma, S., Sadiq, S., Leymann, F. (eds.) Business Process Management Workshops, pp. 477–488. Springer, Berlin (2010)

    Chapter  Google Scholar 

  31. Miller, G.A.: The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 63(2), 81 (1956)

    Article  Google Scholar 

  32. Debois, Søren, Hildebrandt, T., Marquard, M., Slaats, T.: Bridging the valley of death: a success story on danish funding schemes paving a path from technology readiness level 1 to 9. In: Proceedings of the Second International Workshop on Software Engineering Research and Industrial Practice, Piscataway, NJ, USA, pp. 54–57. IEEE Press (2015)

  33. Slaats, T.: Flexible process notations for cross-organizational case management systems. Ph.D. thesis, IT University of Copenhagen (2015)

  34. Debois, S., Hildebrandt, T.T., Marquard, M., Slaats, T.: The DCR graphs process portal. In: BPM (Demos), pp. 7–11 (2016)

  35. Kurzhals, K., Fisher, B., Burch, M., Weiskopf, D.: Eye tracking evaluation of visual analytics. Inf. Vis. 15(4), 340–358 (2016)

    Article  Google Scholar 

  36. Holmqvist, K., Nyström, M., Mulvey, F.: Eye tracker data quality: what it is and how to measure it. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 45–52. ACM, London (2012)

  37. Gulden, J., Burattin, A., Andaloussi, A.A., Weber, B.: From analytical purposes to data visualizations: a decision process guided by a conceptual framework and eye tracking. Softw. Syst. Model. 19, 531–554 (2020)

    Article  Google Scholar 

  38. Kurzhals, K., Fisher, B., Burch, M., Weiskopf, D.: Evaluating visual analytics with eye tracking. In: Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization, BELIV ’14, pp. 61–69. ACM, New York (2014)

  39. Zhao, F., Schnotz, W., Wagner, I., Gaschler, R.: Eye tracking indicators of reading approaches in text-picture comprehension. Frontline Learn. Res. 2(5), 46–66 (2014)

    Google Scholar 

  40. Zugal, S.: Applying cognitive psychology for improving the creation, understanding and maintenance of business process models. Ph.D. Thesis, University of Innsbruck (2013)

  41. Figl, K.: Comprehension of procedural visual business process models. Bus. Inf. Syst. Eng. 59(1), 41–67 (2017)

    Article  Google Scholar 

  42. Garcia, F., Ruiz, F., Visaggio, C.A.: A proposal and empirical validation of metrics to evaluate the maintainability of software process models. In: 2006 IEEE Instrumentation and Measurement Technology Conference Proceedings, pp. 1093–1097 (2006)

  43. Claes, J., Vanderfeesten, I., Gailly, F., Grefen, P., Poels, G.: The structured process modeling theory (SPMT) a cognitive view on why and how modelers benefit from structuring the process of process modeling. Inf. Syst. Front. 17(6), 1401–1425 (2015)

    Article  Google Scholar 

  44. Zugal, S., Haisjackl, C., Pinggera, J., Weber, B.: Empirical evaluation of test driven modeling. Int. J. Inf. Syst. Model. Des. 4(2), 23–43 (2013)

    Article  Google Scholar 

  45. Duchowski, A.T.: Eye Tracking Methodology: Theory and Practice. Springer, Berlin (2007)

    MATH  Google Scholar 

  46. Petrusel, R., Mendling, J.: Eye-tracking the factors of process model comprehension tasks. In: Salinesi, C., Norrie, M.C., Pastor, Ó. (eds.) Advanced Information Systems Engineering, pp. 224–239. Springer, Berlin (2013)

    Google Scholar 

  47. Object Management Group OMG. Business Process Modeling Notation V 2.0. (2006)

  48. Bera, P., Soffer, P., Parsons, J.: Using eye tracking to expose cognitive processes in understanding conceptual models. MIS Q. Manag. Inf. Syst. 43, 1105–1126 (2019)

    Google Scholar 

  49. Keller, G., Nüttgens, M., Scheer, A.W.: Semantische Prozessmodellierung auf der Grundlage “ereignisgesteuerter Prozessketten (EPK)”. Veröffentlichungen des Instituts für Wirtschaftsinformatik. Inst. für Wirtschaftsinformatik, Saarbrücken (1992)

    Google Scholar 

  50. Wang, W., Indulska, M., Sadiq, S., Weber, B.: Effect of linked rules on business process model understanding. In: Business Process Management, pp. 200–215. Springer, Cham (2017)

  51. Sapkota, B., Sinderen, M.V.: Exploiting rules and processes for increasing flexibility in service composition. In: 2010 14th IEEE International Enterprise Distributed Object Computing Conference Workshops, pp. 177–185 (2010)

  52. Zimoch, M., Pryss, R., Schobel, J., Reichert, M.: Eye tracking experiments on process model comprehension: lessons learned. In: Reinhartz-Berger, I., Gulden, J., Nurcan, S., Guédria, W., Bera, P. (eds.) Enterprise, Business-Process and Information Systems Modeling, pp. 153–168. Springer, Cham (2017)

    Chapter  Google Scholar 

  53. Haisjackl, C., Barba, I., Zugal, S., Soffer, P., Hadar, I., Reichert, M., Pinggera, J., Weber, B.: Understanding declare models: strategies, pitfalls, empirical results. Softw. Syst. Model. 15(2), 325–352 (2016)

    Article  Google Scholar 

  54. Haisjackl, C., Soffer, P., Lim, S.Y., Weber, B.: How do humans inspect BPMN models: an exploratory study. Softw. Syst. Model. 17(2), 655–673 (2018)

    Article  Google Scholar 

  55. Pinggera, J.: The process of process modeling. Ph.D. Thesis, Department of Computer Science (2014)

  56. Pinggera, J., Soffer, P., Fahland, D., Weidlich, M., Zugal, S., Weber, B., Reijers, H.A., Mendling, J.: Styles in business process modeling: an exploration and a model. Softw. Syst. Model. 14(3), 1055–1080 (2015)

    Article  Google Scholar 

  57. Caldiera, V.R.B.G., Rombach, H.D.: The goal question metric approach. In: Encyclopedia of Software Engineering, pp. 528–532 (1994)

  58. Charmaz, K.: Constructing Grounded Theory. Introducing Qualitative Methods Series. SAGE Publications, London (2014)

    Google Scholar 

  59. Richardson, D.C., Dale, R.: Looking to understand: the coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cogn. Sci. 29(6), 1045–1060 (2005)

    Article  Google Scholar 

  60. Olsen, A.: The Tobii i-vt Fixation Filter. Tobii Technology, Danderyd (2012)

    Google Scholar 

  61. IEEE Standard for eXtensible Event Stream (XES) for Achieving Interoperability in Event Logs and Event Streams. IEEE Std 1849-2016, pp. 1–50 (2016)

  62. Process mining with the heuristics miner algorithm: Weijters, A.J.M.M., van der Aalst, W.M.P., Alves De Medeiros, A.K. TU/e Tech. Report 166, 1–34 (2006)

  63. Coombes, K.R., Brock, G., Abrams, Z.B.: Polychrome: creating and assessing qualitative palettes with many colors. J. Stat. Softw. (2019)

  64. Gulden, J.: Visually comparing process dynamics with rhythm-eye views. In: BPM Workshops. LNBIP, pp. 474–485. Springer, Berlin (2016)

  65. Saldana, J.: The Coding Manual for Qualitative Researchers. SAGE Publications, London (2012)

    Google Scholar 

  66. Friese, S.: Qualitative Data Analysis with ATLAS ti. SAGE Publications Limited, London (2019)

    Google Scholar 

  67. Ehmke, C., Wilson, S.: Identifying web usability problems from eye-tracking data. In: Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI... But Not as We Know It-Volume 1, pp. 119–128. British Computer Society (2007)

  68. Dumas, M., La Rosa, M., Mendling, J., Reijers, H.A.: Fundamentals of Business Process Management. Springer, Berlin (2018)

    Book  Google Scholar 

  69. Glaholt, M.G., Mei-Chun, W., Reingold, E.M.: Evidence for top-down control of eye movements during visual decision making. J. Vis. 10(5), 15–15 (2010)

    Article  Google Scholar 

  70. Burattin, A., Kaiser, M., Neurauter, M., Weber, B.: Learning process modeling phases from modeling interactions and eye tracking data. Data Knowl. Eng. 121, 1–17 (2019)

    Article  Google Scholar 

  71. Ottensooser, A., Fekete, A., Reijers, H.A., Mendling, J., Menictas, C.: Making sense of business process descriptions: an experimental comparison of graphical and textual notations. J. Syst. Softw. 85(3), 596–606 (2012). Novel approaches in the design and implementation of systems/software architecture

  72. Curtis, B., Sheppard, S.B., Kruesi-Bailey, E., Bailey, J., Boehm-Davis, D.A.: Experimental evaluation of software documentation formats. J. Syst. Softw. 9(2), 167–207 (1989)

    Article  Google Scholar 

  73. Roth, W.-M., Bowen, G.M.: When are graphs worth ten thousand words? An expert-expert study. Cogn. Instr. 21(4), 429–473 (2003)

    Article  Google Scholar 

  74. Moher, T.G., Mak, K.H., Blumenthal, B.B., Levanthal, L.M.: Comparing the comprehensibility of textual and graphical programs (1993)

  75. Haisjackl, C., Zugal, S.: Investigating differences between graphical and textual declarative process models. In: Iliadis, L., Papazoglou, M., Pohl, K. (eds.) Advanced Information Systems Engineering Workshops, pp. 194–206. Springer, Cham (2014)

    Chapter  Google Scholar 

  76. Larkin, J.H., Simon, H.A.: Why a diagram is (sometimes) worth ten thousand words. Cogn. Sci. 11(1), 65–100 (1987)

    Article  Google Scholar 

  77. Moody, D.: The “physics” of notations: toward a scientific basis for constructing visual notations in software engineering. IEEE Trans. Softw. Eng. 35(6), 756–779 (2009)

    Article  Google Scholar 

  78. Siau, K.: Informational and computational equivalence in comparing information modeling methods. J. Database Manag. 15, 73–86 (2004)

    Article  Google Scholar 

  79. Paivio, A.: Dual coding theory: retrospect and current status (1991)

  80. Many, C.W.W.N., Sufficient, Y.W.W.: Estimating the number of subjects needed for a thinking aloud test. Int. J. Hum. Comput. Stud. 41, 385–397 (1994)

    Article  Google Scholar 

  81. Costain, G.: Cognitive support during object-oriented software development: the case of UML diagrams. Ph.D. thesis, ResearchSpace@ Auckland (2008)

  82. Kalyuga, S., Ayres, P., Chandler, P., Sweller, J.: The expertise reversal effect. Educ. Psychol. 38(1), 23–31 (2003)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Amine Abbad Andaloussi.

Additional information

Communicated by Jens Gulden and Rainer Schmidt.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Work supported by the Innovation Fund Denmark project EcoKnow (7050-00034A); the fourth author additionally by the Danish Council for Independent Research project Hybrid Business Process Management Technologies (DFF-6111-00337).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abbad Andaloussi, A., Zerbato, F., Burattin, A. et al. Exploring how users engage with hybrid process artifacts based on declarative process models: a behavioral analysis based on eye-tracking and think-aloud. Softw Syst Model 20, 1437–1464 (2021). https://doi.org/10.1007/s10270-020-00811-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-020-00811-8

Keywords

Navigation