Skip to main content
SpringerLink
Log in

An investigation of the relationship between joint visual attention and product quality in collaborative business process modeling: a dual eye-tracking study

  • Regular Paper
  • Published:
Software and Systems Modeling Aims and scope Submit manuscript

Abstract

Collaborative business process modeling is a collective activity where team members jointly discuss, design, and document business processes. During such activities, team members need to communicate with each other to coordinate the modeling activities, propose and justify changes, and negotiate common terms and definitions. Throughout this process, stakeholders should be aware of when and what kind of changes have been made by each team member on the shared space so that they can discuss design ideas and build on each other’s work. Joint visual attention has a fundamental role in establishing and maintaining common ground among interlocutors in such cooperative work settings. In addition to this, the co-constructed model's quality is often considered a key evaluation outcome measure to assess the success of collaboration. However, process and outcome measures of collaboration have been prone to difficulties due to challenges in devising measures that can adequately capture the complex dynamics of collaborative work. This study explored the relationship between a popularly used outcome measure in the business process modeling literature and a process measure approximating the level of joint visual attention present among the participants based on the degree of gaze cross-recurrence among the team members over a shared task space. The results suggest that joint visual attention as operationalized in terms of gaze cross-recurrence was a strong predictor of the syntactic, semantic, and pragmatic qualities of collaboratively produced business process models. Moreover, the collaboration process was subjected to qualitative analysis to probe further into the interactional organization of the modeling activity, which identified communication, coordination, awareness, group decision making, and motivation dimensions as key factors contributing to the quality of collaboration among group members. The results indicated strong relationships between the distribution of quality factors and the degree of gaze cross-recurrence and the final models' syntactic and semantic quality scores. Given the increasing availability of affordable eye trackers and the low resolution, practical nature of the employed analysis methodology, the proposed approach can be fruitfully employed to evaluate team performance and test the effectiveness of software interfaces designed to support collaborative work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ambler, S.W.: The elements of UML (TM) 2.0 style. Cambridge University Press, Cambridge (2005)

    Book  Google Scholar 

  2. Andaloussi AA, Slaats T, Burattin A, Hildebrandt TT, Weber B (2018) Evaluating the understandability of hybrid process model representations using eye tracking: first insights. In: international conference on business process management 432:475–481 Springer Sydney. doi:https://doi.org/10.1007/978-3-030-11641-5_37

  3. Antikainen, M., Mäkipää, M., Ahonen, M.: Motivating and supporting collaboration in open innovation. Eur. J. Innov. Manag. 13(1), 100–119 (2010)

    Article  Google Scholar 

  4. Baghaei, N., Mitrovic, A., Irwin, W.: Supporting collaborative learning and problem-solving in a constraint-based CSCL environment for UML class diagrams. Int. J. Comput.-Support. Collab. Learn. 2(2–3), 159–190 (2007)

    Article  Google Scholar 

  5. Bakeman, R., Adamson, L.B.: Coordinating attention to people and objects in mother-infant and peer-infant interaction. Child Dev. 55, 1278–1289 (1984)

    Article  Google Scholar 

  6. Bannon, L.J., Schmidt, K.: CSCW-four characters in search of a context. DAIMI Rep. Ser. 18(289), 1–20 (1989). https://doi.org/10.7146/dpb.v18i289.6667

    Article  Google Scholar 

  7. Barron, B.: When smart groups fail. J. Learn. Sci. 12(3), 307–359 (2003)

    Article  Google Scholar 

  8. Basheri M (2010) Collaborative learning of UML-state diagrams using multi-touch technology. TR-TEL-10–05, Durham University

  9. Battistella, C., Nonino, F.: Open innovation web-based platforms: the impact of different forms of motivation on collaboration. Innovation 14(4), 557–575 (2012). https://doi.org/10.5172/impp.2012.14.4.557

    Article  Google Scholar 

  10. Bera, P., Soffer, P., Parsons, J.: Using eye tracking to expose cognitive processes in understanding conceptual Models. MIS Q. 43(4), 1105–1126 (2019). https://doi.org/10.25300/MISQ/2019/14163

    Article  Google Scholar 

  11. Brennan, S.E., Chen, X., Dickinson, C.A., Neider, M.B., Zelinsky, G.J.: Coordinating cognition: the costs and benefits of shared gaze during collaborative search. Cognition 106(3), 1465–1477 (2008)

    Article  Google Scholar 

  12. Brooks, R., Meltzoff, A.N.: Infant gaze following and pointing predict accelerated vocabulary growth through two years of age: a longitudinal, growth curve modeling study. J. Child Lang. 35(01), 207–220 (2008)

    Article  Google Scholar 

  13. Brown, R., Recker, J., West, S.: Using virtual worlds for collaborative business process modeling. Bus. Process. Manag. J. 17(3), 546–564 (2011). https://doi.org/10.1108/14637151111136414

    Article  Google Scholar 

  14. Cardoso J (2005) How to measure the control-flow complexity of web processes and workflows. In Workflow Handbook edited by L. Ficher, pp. 199–212

  15. Carpenter M, Liebal K, Seemann A (2011). Joint attention, communication, and knowing together in infancy. Joint attention: new developments in psychology, philosphy of mind, and social neuroscience, pp. 159–182

  16. Charman, T., Baron-Cohen, S., Swettenham, J., Baird, G., Cox, A., Drew, A.: Testing joint attention, imitation, and play as infancy precursors to language and theory of mind. Cogn. Dev. 15(4), 481–498 (2000)

    Article  Google Scholar 

  17. Cherubini M, Nüssli M, Dillenbourg P (2008) Deixis and gaze in collaborative work at a distance (over a shared map): a computational model to detect misunderstandings. In: proceedings of the 2008 symposium on eye tracking research & applications 1: 173–180. doi:https://doi.org/10.1145/1344471.1344515

  18. Cherubini, M., Nüssli, M., Dillenbourg, P.: This is it!: indicating and looking in collaborative work at distance. J. Eye Movement Res. 3(5), 1–20 (2010). https://doi.org/10.16910/jemr.3.5.3

    Article  Google Scholar 

  19. Clark, H.H., Brennan, S.E.: Grounding in communication. In: Resnick, L.B., Levine, J., Behrend, S.D. (eds.) Perspectives on socially shared cognition, pp. 127–149. Morgan Kaufman, San Mateo, CA (1991)

    Chapter  Google Scholar 

  20. Courgeon, M., Rautureau, G., Martin, J.-C., Grynszpan, O.: Joint attention simulation using eye-tracking and virtual humans. IEEE Trans. Affect. Comput. 5(3), 238–250 (2014)

    Article  Google Scholar 

  21. Davis, R., Brabander, E.: ARIS design platform: getting started with BPM. Springer, Berlin (2007)

    Google Scholar 

  22. Dollmann T, Houy C, Fettke CP, Loos P (2011) Collaborative business process modeling with CoMoMod - a toolkit for model integration in distributed cooperation environments. In: proceedings of the 20th IEEE international conference on collaboration technologies and infrastructures, edited by S. Reddy and S. Tata, 217–222. doi:https://doi.org/10.1109/WETICE.2011.36

  23. Dourish P, Bellotti V (1992) Awareness and coordination in shared workspaces. In: proceedings of the ACM conference on computer-supported cooperative work pp. 107–114 ACM Toronto. doi:https://doi.org/10.1145/143457.143468

  24. Duchowski, A.: Eye tracking methodology. Theory and practice. Springer, London (2007)

    MATH  Google Scholar 

  25. Dumas, M., La Rosa, M., Mendling, J., Reijers, H.A.: Fundamentals of business process management. Springer, Berlin Heidelberg (2013)

    Book  Google Scholar 

  26. Fiebich, A., Gallagher, S.: Joint attention in joint action. Philos. Psychol. 26(4), 571–587 (2013)

    Article  Google Scholar 

  27. Field, A.: Discovering statistics using SPSS Third edition (and sex and drugs and rock “n” roll). Sage (2009)

    Google Scholar 

  28. Figl, K.: Comprehension of procedural visual business process models. Bus. Inf. Syst. Eng. 59(1), 41–67 (2017)

    Article  Google Scholar 

  29. Figl K, Laue R (2011) Cognitive complexity in business process modeling. In: international conference on advanced information systems engineering, pp. 452–466. Springer, Berlin, Heidelberg. doi: https://doi.org/10.1007/978-3-642-21640-4_34

  30. Forster S, Pinggera J, Weber B (2013) Toward an understanding of the collaborative process of process modeling. In: CAiSE forum, pp. 98–105

  31. Frederiks, P.J.M., Van der Weide, T.P.: Information modeling: The process and the required competencies of its participants. Data Knowl. Eng. 58(1), 4–20 (2006). https://doi.org/10.1016/j.datak.2005.05.007

    Article  Google Scholar 

  32. Gulden, J., Burattin, A., Andaloussi, A.A., Weber, B.: From analytical purposes to data visualizations: a decision process guided by a conceptual framework and eye tracking. Softw. Syst. Model. 19(3), 531–554 (2020). https://doi.org/10.1007/s10270-019-00742-z

    Article  Google Scholar 

  33. Gürbüz Ö (2011) An automated quality measurement approach for business process models. Dissertation, Middle East Technical University

  34. Hahn C, Recker J, Mendling J (2011) An exploratory study of it-enabled collaborative process modeling. In: Business process management workshops, edited by M. Muehlen and J. Su, 61–72. Springer Berlin Heidelberg. doi:https://doi.org/10.1007/978-3-642-20511-8_6

  35. Haisjackl, C., Soffer, P., Lim, S.Y., Weber, B.: How do humans inspect BPMN models: an exploratory study. Softw. Syst. Model. 17(2), 655–673 (2018). https://doi.org/10.1007/s10270-016-0563-8

    Article  Google Scholar 

  36. Hinsz, V.B., Tindale, R.S., Vollrath, D.A.: The emerging conceptualization of groups as information processors. Psychol. Bull. 121(1), 43–64 (1997). https://doi.org/10.1037/0033-2909.121.1.43

    Article  Google Scholar 

  37. Hogrebe F, Gehrke N, Nüttgens M (2011) Eye tracking experiments in business process modeling: agenda setting and proof of concept. In: EMISA pp. 183–188

  38. Holmqvist, K., Nyström, N., Andersson, R., Dewhurst, R., Jarodzka, H., Van de Weijer, J.: Eye tracking: a comprehensive guide to methods and measures. University Press, Oxford (2011)

    Google Scholar 

  39. Hoppenbrouwers, S., Rouwette, E.: A dialogue game for analysing group model building: framing collaborative modelling and its facilitation. Int. J. Organ. Des. Eng. 2(1), 19–40 (2012). https://doi.org/10.1504/IJODE.2012.045905

    Article  Google Scholar 

  40. Hoppenbrouwers S, Thijssen R, Vogels J (2013) Operationalizing dialogue games for collaborative modeling. In: Nolte, A. (Ed.). In: proceedings of the international workshop on models and their role in collaboration at the ECSCW 2013 (MoRoCo 2013), pp. 41–48

  41. Jermann P, Nüssli M (2012) Effects of sharing text selections on gaze cross-recurrence and interaction quality in a pair programming task. In: proceedings of the ACM 2012 conference on computer supported cooperative work, pp. 1125–1134. doi:https://doi.org/10.1145/2145204.2145371

  42. Kahrimanis G, Meier A, Chounta IA, Voyiatzaki E, Spada H, Rummel N, Avouris N (2009) Assessing collaboration quality in synchronous CSCL problem-solving activities: Adaptation and empirical evaluation of a rating scheme. In: learning in the synergy of multiple disciplines vol 5794. pp. 267–272 Springer Nice. doi: https://doi.org/10.1007/978-3-642-04636-0_25

  43. Kaplan, F., Hafner, V.V.: The challenges of joint attention. Interact. Stud. 7(2), 135–169 (2006)

    Article  Google Scholar 

  44. Kolb J, Benjamin R, Reichert M (2012) Towards gesture-based process modeling on multi-touch devices. In: international conference on advanced information systems engineering pp. 280–293. Springer, Berlin, Heidelberg. doi: https://doi.org/10.1007/978-3-642-31069-0_24

  45. Krippendorff, K.: Reliability in content analysis. Hum. Commun. Res. 30(3), 411–433 (2004). https://doi.org/10.1111/j.1468-2958.2004.tb00738.x

    Article  Google Scholar 

  46. Malone, T.W., Crowston, K.: The interdisciplinary study of coordination. ACM Comput. Surv. 26(1), 87–119 (1994). https://doi.org/10.1145/174666.174668

    Article  Google Scholar 

  47. Meier, A., Spada, H., Rummel, N.: A rating scheme for assessing the quality of computer-supported collaboration processes. Int. J. Comput.-Support. Collab. Learn. 2(1), 63–86 (2007). https://doi.org/10.1007/s11412-006-9005-x

    Article  Google Scholar 

  48. Mendling, J.: Metrics for process models: empirical foundations of verification, error prediction, and guidelines for correctness. Springer, Berlin (2008)

    Book  Google Scholar 

  49. Mendling, J., Recker, J.C., Wolf, J.: Collaboration features in current BPM tools. EMISA Forum 32(1), 48–65 (2012)

    Google Scholar 

  50. Moody DL, Sindre G., Brasethvik T, Sølvberg A (2003) Evaluating the quality of information models: empirical testing of a conceptual model quality framework. In: proceedings of the 25th international conference on software engineering, pp. 295–305. IEEE Computer Society. doi: https://doi.org/10.1109/ICSE.2003.1201209

  51. Moore, C., Dunham, P.J.: Joint attention: its origins and role in development. Psychology Press, United Kingdom (1995)

    Google Scholar 

  52. Mundy, P., Jarrold, W.: Infant joint attention, neural networks and social cognition. Neural Netw. 23(8), 985–997 (2010). https://doi.org/10.1016/j.neunet.2010.08.009

    Article  Google Scholar 

  53. Mundy, P., Newell, L.: Attention, joint attention, and social cognition. Curr. Dir. Psychol. Sci. 16(5), 269–274 (2007). https://doi.org/10.1111/j.1467-8721.2007.00518.x

    Article  Google Scholar 

  54. Nüssli MA (2011) Dual eye-tracking methods for the study of remote collaborative problem solving. Dissertation, École Polytechnique Fédérale De Lausanne

  55. Nüssli MA, Jermann P, Sangin M, Dillenbourg P (2009) Collaboration and abstract representations: towards predictive models based on raw speech and eye-tracking data. In: proceedings of the 9th international conference on computer supported collaborative learning, 1:78–82

  56. Oppl, S.: Evaluation of collaborative modeling processes for knowledge articulation and alignment. IseB 15(3), 717–749 (2016). https://doi.org/10.1007/s10257-016-0324-9

    Article  Google Scholar 

  57. Petrusel R, Mendling J (2013) Eye-Tracking the factors of process model comprehension tasks. In: international conference on advanced information systems engineering, pp. 224–239 Berlin, Heidelberg.

  58. Petrusel, R., Mendling, J., Reijers, H.A.: Task-specific visual cues for improving process model understanding. Inf. Softw. Technol. 79, 63–78 (2016)

    Article  Google Scholar 

  59. Petrusel, R., Mendling, J., Reijers, H.A.: How visual cognition influences process model comprehension. Decis. Support Syst. 96, 1–16 (2017)

    Article  Google Scholar 

  60. Pfeiffer, U.J., Vogeley, K., Schilbach, L.: From gaze cueing to dual eye-tracking: novel approaches to investigate the neural correlates of gaze in social interaction. Neurosci. Biobehav. Rev. 37(10), 2516–2528 (2013)

    Article  Google Scholar 

  61. Pinggera J, Furtner M, Martini M, Sachse P, Reiter K, Zugal S, Weber B (2013) Investigating the process of process modeling with eye movement analysis. In: business process management workshops pp. 438–450 Springer

  62. Pinggera, J., Soffer, P., Fahland, D., Weidlich, M., Zugal, S., Weber, B., Reijers, H.A., Mendling, J.: Styles in business process modeling: an exploration and a model. Softw. Syst. Model. 14(3), 1055–1080 (2015). https://doi.org/10.1007/s10270-013-0349-1

    Article  Google Scholar 

  63. Poole, A., Ball, L.J.: Eye tracking in human-computer interaction and usability research: current status and future prospects. In: Ghaoui, C. (ed.) Encyclopedia of human computer interaction, pp. 211–219. Idea Group, Hershey, PA (2006)

    Chapter  Google Scholar 

  64. Recker, J., Mendling, J., Christopher, H.: How collaborative technology supports cognitive processes in collaborative process modeling: a capabilities-gains-outcome model. Inf. Syst. 38(8), 1031–1045 (2013). https://doi.org/10.1016/j.is.2013.04.001

    Article  Google Scholar 

  65. Richardson, D.C., Dale, R.: Looking to understand: the coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cogn. Sci. 29(6), 1045–1060 (2005). https://doi.org/10.1207/s15516709cog0000_29

    Article  Google Scholar 

  66. Richardson, D.C., Dale, R., Kirkham, N.Z.: The art of conversation is coordination common ground and the coupling of eye movements during dialogue. Psychol. Sci. 18(5), 407–413 (2007)

    Article  Google Scholar 

  67. Richardson, D.C., Dale, R., Tomlinson, J.M.: Conversation, gaze coordination, and beliefs about visual context. Cogn. Sci. 33(8), 1468–1482 (2009)

    Article  Google Scholar 

  68. Ridings CM, Gefen D (2004) Virtual community attraction: Why people hang out online. J Comput Mediat Commun 10(1): JCMC10110

  69. Riemer K, Holler J, Indulska M (2011) Collaborative process modelling - tool analysis and design implications. In: European conference on information systems 39 Helsinki, Finland

  70. Risko, E.F., Kingstone, A.: Eyes wide shut: implied social presence, eye tracking and attention. Atten. Percept. Psychophys. 73(2), 291–296 (2011)

    Article  Google Scholar 

  71. Rittgen, P.: Negotiating models. In: Krogstie, J., Opdahl, A.L., Sindre, G. (eds.) Advanced Information systems engineering – CAiSE 2007, pp. 561–573. Springer, Trondheim (2007)

    Google Scholar 

  72. Rittgen P (2008) COMA: a tool for collaborative modeling. In: CEUR workshop proceedings pp. 61-64 Montpellier, France

  73. Rittgen, P.: Success factors of e-Collaboration in business process modeling. In: Pernici, B. (ed.) Advanced information systems engineering CAiSE 2010, pp. 24–37. Springer, Berlin (2010)

    Google Scholar 

  74. Roschelle, J., Teasley, S.D.: The construction of shared knowledge in collaborative problem solving. In: O’Malley, C.E. (ed.) Computer-supported collaborative learning, pp. 69–97. Springer, Berlin (1995)

    Chapter  Google Scholar 

  75. Roscoe, J.T.: Fundamental research statistics for the behavioral sciences, 2nd edn. Holt, Rinehart and Winston, New York (1975)

    Google Scholar 

  76. Roser S, Bauer B (2005) A categorization of collaborative business process modeling techniques. In: E-commerce technology workshops, pp. 43–51. IEEE, Munich, Germany. doi: https://doi.org/10.1109/CECW.2005.1

  77. Roth, W.M.: Gestures: their role in teaching and learning. Rev. Educ. Res. 71(3), 365–392 (2001)

    Article  Google Scholar 

  78. Rummel, N., Spada, H.: Learning to collaborate: An instructional approach to promoting collaborative problem solving in computer-mediated settings. J. Learn. Sci. 14(2), 201–241 (2005)

    Article  Google Scholar 

  79. Sangin M, Molinari G, Nüssli MA, Dillenbourg, P (2008) How learners use awareness cues about their peer’s knowledge?: Insights from synchronized eye-tracking data. In: proceedings of the 8th international conference on international conference for the learning sciences 2:287–294. International society of the learning sciences

  80. Sánchez-González, L., García, F., Ruiz, F., Mendling, J.: Quality indicators for business process models from a gateway complexity perspective. Inf. Softw. Technol. 54(11), 1159–1174 (2012). https://doi.org/10.1016/j.infsof.2012.05.001

    Article  Google Scholar 

  81. Schneider, B., Pea, R.: Real-time mutual gaze perception enhances collaborative learning and collaboration quality. Int. J. Comput.-Support. Collab. Learn. 8(4), 375–397 (2013). https://doi.org/10.1007/s11412-013-9181-4

    Article  Google Scholar 

  82. Schwartz, D.L.: The emergence of abstract representations in dyad problem solving. J. Learn. Sci. 4(3), 321–354 (1995)

    Article  Google Scholar 

  83. Sebanz, N., Bekkering, H., Knoblich, G.: Joint action: bodies and moinds moving together. Trends Cogn. Sci. 10(2), 70–76 (2006). https://doi.org/10.1016/j.tics.2005.12.009

    Article  Google Scholar 

  84. Sharafi, Z., Soh, Z., Guéhéneuc, Y.G.: A systematic literature review on the usage of eye-tracking in software engineering. Inf. Softw. Technol. 67, 79–107 (2015)

    Article  Google Scholar 

  85. Sharma, K., Olsen, J.K., Aleven, V., Rummel, N.: Measuring causality between collaborative and individual gaze metrics for collaborative problem-solving with intelligent tutoring systems. J. Comput. Assist. Learn. 37(1), 51–68 (2020)

    Article  Google Scholar 

  86. Spada H, Meier A, Rummel N, Hauser S. (2005) A new method to assess the quality of collaborative process in CSCL. In: Koschmann T, Suthers DD, Chan T (Eds) Proceedings of the international conference on computer supported collaborative learning pp. 622–631 Taipei, Taiwan: International society of the learning sciences. doi: https://doi.org/10.3115/1149293.1149375

  87. Stahl, G.: Studying virtual math teams. Springer, Berlin (2009)

    Book  MATH  Google Scholar 

  88. Stern, D.N.: The first relationship: infant and mother. Harvard University Press, Cambridge (2009)

    Book  Google Scholar 

  89. Tomasello, M.: Joint attention as social cognition. In: Moore, C., Dunham, P.J. (eds.) Joint attention: its origins and role in development, pp. 103–130. Lawrence Erlbaum Associates Inc, Hillsdale, NJ (1995)

    Google Scholar 

  90. Turetken, O., Demirors, O.: Plural: a decentralized business process modeling method. Inf. Manag. 48(6), 235–247 (2011). https://doi.org/10.1016/j.im.2011.06.001

    Article  Google Scholar 

  91. Vanderfeesten I, Reijers HA, Mendling J, Van Der Aalst WMP, Cardoso J (2008) On a quest for good process models: the cross-connectivity metric. In: CAiSE 2008, pp. 480-494. Montpellier, France. doi:https://doi.org/10.1007/978-3-540-69534-9_36

  92. Vanderhaegen, F.: Pedagogical learning supports based on human–systems inclusion applied to rail flow control. Cogn. Technol. Work 23, 193–202 (2019). https://doi.org/10.1007/s10111-019-00602-2

    Article  Google Scholar 

  93. Wang W, Indulska M, Sadiq S, Weber B (2017) Effect of linked rules on business process model understanding. In: international conference on business process management vol 10445, pp. 200-215 doi:https://doi.org/10.1007/978-3-319-65000-5_12

  94. Zhang Y, Wildemuth BM (2009) Qualitative analysis of content. In B. Wildemuth (Ed.) Applications of social research methods to questions in information and library pp. 308–319 Westport, CT: Libraries Unlimited

  95. Zimoch M, Pryss R, Layher G, Neumann H, Probst T, Schlee W, Reichert M (2018) Utilizing the capabilities offered by eyetracking to foster novices’ comprehension of business process models. In: International conference on cognitive computing ICCC 2018, pp. 155–163 Seattle. doi: https://doi.org/10.1007/978-3-319-94307-7_12

  96. Zimoch, M., Pryss, R., Schobel, J., Reichert, M.: Eye tracking experiments on process model comprehension: lessons learned. In: Reinhartz-Berger, I., Gulden, J., Nurcan, S., Guédria, W., Bera, P. (eds.) Enterprise, business-process and information systems modeling, pp. 153–168. Springer, Cham (2017)

    Chapter  Google Scholar 

Download references

Funding

This work was supported by [The Scientific and Technological Research Council of Turkey] under grant [TUBITAK 2211-A].

Author information

Authors and Affiliations

  1. Faculty of Economics and Administrative Sciences, Ataturk University, Erzurum, Turkey

    Duygu Fındık-Coşkunçay

  2. Informatics Institute, Middle East Technical University, Ankara, Turkey

    Duygu Fındık-Coşkunçay & Murat Perit Çakır

  3. METU-TAF Modeling & Simulation R&D Center, Middle East Technical University, Ankara, Turkey

    Murat Perit Çakır

Authors
  1. Duygu Fındık-Coşkunçay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Murat Perit Çakır
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors made substantial contributions to the design of the work.

Corresponding author

Correspondence to Duygu Fındık-Coşkunçay.

Ethics declarations

Conflict of interest

The authors have no conflict of interest to declare that are relevant to the content of this article.

Data availability

Dataset is shared as supplementary file.

Additional information

Communicated by Dragan Milicev.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix I

The following illustration exemplifies the evaluation of syntactic and semantic qualities of collaboratively produced process models shown in Fig. 

Fig. 8
figure 8

Model 1

Full size image

8 (Process Definition 1: Taking semester leave due to military service) and Fig. 

Fig. 9
figure 9

Model 2

Full size image

9 (Process Definition 2: Course exemption process).

Two experts used the following rubric, including the rules for the syntactic quality evaluation:

Evaluation criteria

Model 1

Model 2

Expert_1 Review

Score

Expert_1 Review

Score

Does the model have at least one start event/initial state and one end event/final state? (10 points)

Start and end nodes were used appropriately

10

The model has a start event element and three end event elements. However, there is a function that needs an end event

8.5

Are the model elements named conveniently? (i.e., the function model elements named with Verb + Noun and the event model elements named with Noun + Verb convention) (15 points)

The model elements were named conveniently

15

The model elements were named conveniently

15

Are the model elements assigned grammatically correct labels? (10 points)

There are no grammatical errors on the labels

10

There are no grammatical errors on the labels

10

Do the model elements have a single incoming and outgoing connection? (10 points)

The model elements have single incoming and outgoing connections

10

The model elements have single incoming and outgoing connections

10

Are the input and output resources used appropriately in the model? (15 points)

The resources are connected to the functions appropriately

15

The resources are connected to the functions appropriately

15

Are the control points (or operator rules) used correctly? (20 points)

For UML Activity Diagram, consider the following controls [1];

*Decision is similar to the if-else statement. After the incoming edge is executed, only one edge is followed out of the decision node

*Merge includes several incoming edges and one outgoing edge. Merge node is used to accept one incoming edge among several alternate flows

*Join is a black bar with several incoming edges and one outgoing edge. Join node denotes the end of parallel processing

*Fork is a black bar with one incoming edge and several outgoing edges, denoting the beginning of parallel activity

*The guards, which are the conditions that must be true in order for an activity edge to be traversed, should be depicted

For eEPC consider the following operator rules [21];

*XOR-split model element represents a choice between one of several alternative branches. XOR-join model element merges alternative branches

*OR-split model element triggers one, two, or up to all of the multiple branches based on conditions. OR-join model element synchronizes all active incoming branches

*AND-split model element activates all subsequent branches in concurrency. AND-join model element waits for all incoming branches to complete and then propagates the control of the coming EPC element

The modelers used the decision and guards appropriately

20

The operator rule is used in the correct place, but the XOR operation rule should be used instead of OR

3

Does the model have a deadlock condition? (i.e., deadlock is a condition used to describe a process instance will not be able to progress any further) (10 points)

There is no deadlock condition

10

There is no deadlock condition

10

Does the model have a livelock condition? (i.e., Livelock can occur in an inappropriate loop structure) (10 points)

There is no deadlock condition

10

There is no livelock condition

10

Total Score for the Syntactic Quality:

 

100

 

81.5

Two experts used the following rubric for the semantic quality evaluation.

Model 1

Model 2

Evaluation criteria

Expert_1 Review

Score

Evaluation Criteria

Expert_1 Review

Score

Has the student filled the Registration Suspension Application Form to suspend registration due to military service? (10 points)

The model reflects the student's filling of the Registration Suspension Application Form. However, a note can be created for the output produced by this activity

7

Has the student submitted the Course Substitution Form to the Course Advisor for the course to be substituted? (10 points)

Yes

10

Has the Head of the Department evaluated the application form? (10 points)

The model reflects the evaluation of the application form by the Head of the department. However, a note can be created for the input used by this activity

7

Has the advisor evaluated the course substitution request? (10 points)

Yes

10

Has the Head of the Department made one of the decisions of “Accept” or “Reject” regarding the registration suspension request? (10 points)

Yes

10

Has the advisor made one of the decisions of “Accepted” or “Consultation with the Head of the Department was required” regarding the course substitution request? (10 points)

Although the decision point is correct, the model does not reflect the correct definition. Based on this model, the advisor may follow either the “Accept” or “Consultation” decision, or the advisor may follow both of them. However, this is not correct because the advisor must follow only “Accept” or “Consultation” decisions

6

If the Head of the Department rejected the request, did the process end? (10 points)

Yes

10

If the advisor has made an admission decision, has he/she approved the course substitution form? (10 points)

Yes

10

If the Head of the Department accepted, did the student write a petition for the suspension of registration and receive a Military Referral Certificate from the Department of Recruitment? (10 points)

The model does not reflect any output note related to the petition. The model reflects that the Department of Recruitment will prepare the Military Referral Certificate, but it does not reflect that this document will be received by the student

2

If the advisor was not sure of the equivalence of the course and decided to consult the Head of the Department, did the Head of the Department evaluate the relevant course using the “Course Equivalence Form”? (10 points)

Yes

10

Did the Institute Administrative Board evaluate the registration suspension request by using the Registration Suspension Application Form, Registration Suspension Petition, and Military Referral Document and make one of the decisions “Accept” or “Reject?” (10 points)

The model reflects the Institute Administrative Board's evaluation, but it does not provide input on which documents to use

5.5

Did the Head of the Department make one of the decisions “Course equivalency is valid” or “Course equivalence is invalid” at the end of the evaluation? (10 points)

Although the decision point is correct, the model does not reflect the correct definition. Based on this model, the Head of the Department may follow either “valid” or “invalid” decisions, or both. It is not possible to follow valid and invalid decisions at the same time

5

Has the Institute Administrative Board sent an official letter to the student affairs department about the suitability of registration suspension if the application is approved? (10 points)

The model does not reflect this information

0

Has the Head of the Department approved the “Course Submission Form” if the course equivalency has been decided valid? (10 points)

Yes

10

Are the activities performed in the correct order? (10 points)

Yes

10

Are the activities performed in the correct order? (10 points)

Yes

10

Is an unnecessary activity added to the model that does not reflect the process definition? (10 points)

No

10

Is an unnecessary activity added to the model that does not reflect the process definition? (10 points)

No

10

Is an unnecessary resource added to the model that does not reflect the process definition? (10 points)

No

10

Is an unnecessary resource added to the model that does not reflect the process definition? (10 points)

No

10

Total score for the semantic quality

71.5

 

91

Appendix II

The following example presents the calculation steps of the CC measure for a collaboratively produced process model. Figure 8 illustrates the model with eleven tasks (i.e. T = {A, B, C, D, E, F, G, H, I, J, K}), four connectors (i.e. C = {OR1, OR2, XOR1, XOR2}), and sixteen directed arcs (i.e. A = {a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, a13, a14, a15, a16}) (Fig. 

Fig. 10
figure 10

A sample collaboratively produced model tasks and connectors

Full size image

10).

Firstly, the weight for each node is calculated (in Table

Table 7 Degrees and weights for the nodes in the process model of Fig. 8
Full size table

7) with the following formula [91];

“Let a process model be given as a graph consisting of a set of nodes (n1, n2, …Є N) and a set of directed arcs (a1, a2, … Є A). A node can be one of two types: (i) task, e.g., t1, t2 Є T, and (ii) connector, e.g., c1, c2 Є C. Thus, N = T U C. The weight of a node n, \(w\)(n), is defined as follows:

figure c

with d the degree of the node (i.e., the total number of ingoing and ongoing arcs of the node).”

Secondly, the weight for each arc is calculated with the following formula [91];

“Let a process model be given by a set of nodes (N) and a set of directed arcs (A). Each directed arc (a) has a source node (denoted by src (a) and a destination node (denoted by dest (a)).


The weight of arc a, W (a) is defined as follow:

W(a) = w(src(a). w(dest(a)))”


The weight for each arc:

W(a1) = w(A). w(B) = 1.1 = 1

W(a2) = w(B). w(C) = 1.1 = 1

W(a3) = w(C). w(OR1) = 1.3/7 = 3/7

W(a4) = w(OR1). w(D) = 3/7.1 = 3/7

W(a5) = w(OR1). w(E) = 3/7.1 = 3/7

W(a6) = w(D). w(XOR1) = 1.1/3 = 1/3

W(a7) = w(E). w(F) = 1.1 = 1

W(a8) = w(F). w(OR2) = 1.3/7 = 3/7

W(a9) = w(XOR1). w(G) = 1/3.1 = 1/3

W(a10) = w(OR2). w(H) = 3/7.1 = 3/7

W(a11) = w(OR2). w(I) = 3/7.1 = 3/7

W(a12) = w(H). w(XOR1) = 1.1/3 = 1/3

W(a13) = w(I). w(J) = 1.1 = 1

W(a14) = w(J). w(XOR2) = 1.1/3 = 1/3

W(a15) = w(G). w(XOR2) = 1.1/3 = 1/3

W(a16) = w(XOR2). w(K) = 1/3.1 = 1/3


Thirdly, the value of a path is calculated with the following formula [91];

“Let a process model be given by a set of nodes (N) and a set of directed arcs (A). A path p from node n1 to n2 is given by the sequence of directed arcs that should be followed from n1 to n2: p =  < a1, a2,…, ax > . The value for a path p, v(p), is the product of the weights of all arcs in the path:

v(p) = W(a1). W(a2). …W(ax)”


Fourthly, the value of a connection is examined with the following formula [91];

“Let a process model be given by a set of nodes (N) and a set of directed arcs (A) and let Pn 1 ,n 2 be the set of paths from node n 1 to n 2 . The value of the connection from n 1 to n 2 , V (n 1 , n 2 ), is the maximum value of all paths connecting n 1 and n 2 :

$$ V\left( {n_{1} ,n_{2} } \right) = \mathop {\max }\limits_{{p\epsilon Pn1,n2}} v\left( p \right) $$

If no path exists between node n1 and n2, then V (n1; n2) = 0. Also note that loops in a path should not be considered more than once, since the value of the connection will not be higher if the loop is followed more than once in the particular path.”

All values are provided in Table

Table 8 Connections between all pairs of nodes
Full size table

8. Finally, the CC value is computed with the following formula [91];

“Let a process model be given by a set of nodes (N) and a set of directed arcs (A). The Cross-Connectivity metric is then defined as follows:”

$$ {\text{CC}} = \frac{{\sum\nolimits_{{n1,n2 \in N}} {V\left( {n1,n2} \right)} }}{{\left| N \right| \cdot \left( {\left| N \right| - 1} \right)}} $$
$$ CC = \frac{{1 + 1 + \frac{39}{{49}} + \frac{4}{9} + 1 + \frac{39}{{49}} + \frac{4}{9} + \frac{4}{9} + 1 + \frac{4}{9} + 0 + \frac{6}{7} + \frac{1}{3} + \frac{6}{7} + \frac{1}{3}}}{15*14} = 0,046 $$

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fındık-Coşkunçay, D., Çakır, M.P. An investigation of the relationship between joint visual attention and product quality in collaborative business process modeling: a dual eye-tracking study. Softw Syst Model 21, 2429–2460 (2022). https://doi.org/10.1007/s10270-022-00974-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-022-00974-6

Keywords

Search

Navigation