How visual cognition influences process model comprehension

doi:10.1016/j.dss.2017.01.005

Decision Support Systems

Volume 96, April 2017, Pages 1-16

https://doi.org/10.1016/j.dss.2017.01.005 Get rights and content

Highlights

•
We use Visual Cognition research model to explain understanding of process models
•
It has better explanatory power than model based on Complexity and Reader Expertise
•
Visual Cognition mediates Comprehension Performance (e.g. Correctness, Task time)
•
We test hypotheses using an experiment that makes use of eye-tracking technology

Abstract

Process analysts and other professionals extensively use process models to analyze business processes and identify performance improvement opportunities. Therefore, it is important that such models can be easily and properly understood. Previous research has mainly focused on two types of factors that are important in this context: (i) properties of the model itself, and (ii) properties of the model reader. The work in this paper aims at determining how the performance of subjects varies across different types of comprehension tasks, which is a new angle. To reason about the complexity of comprehension tasks we take a theoretical perspective that is grounded in visual cognition. We test our hypotheses using a free-simulation experiment that incorporates eye-tracking technology. We find that model-related and person-related factors are fully mediated by variables of visual cognition. Moreover, in comparison, visual cognition variables provide a significantly higher explanatory power for the duration and efficiency of comprehension tasks. These insights shed a new perspective on what influences sense-making of process models, shifting the attention from model and reader characteristics to the complexity of the problem-solving task at hand. Our work opens the way to investigate and develop effective strategies to support readers of process models, for example through the context-sensitive use of visual cues.

Introduction

Business process models play an important role in different phases of the business process management lifecycle [1]: These models structure the overall process landscape, they serve as input for analysis, and they can be used as blueprints for process implementation. Business process models (or process models for short) are created and utilized collaboratively by process analysts, process owners, process participants, and senior management. They should be presented and designed in such a way that these different stakeholders can best utilize them for the respective tasks at hand.

A prerequisite for an effective usage of process models is that stakeholders can readily understand them. Recent research has investigated process model comprehension by evaluating different types of factors, including model complexity [2], [3] as well as model reader characteristics [4], [5], [6]. What if we now consider the same model and the same model reader while the comprehension tasks differ? Existing work does not provide any explanation why certain comprehension tasks appear to be easy to solve and others difficult [7]. Yet, understanding the reasons why certain comprehension tasks are difficult bears the potential to support modeling in a more effective way. First of all, based on such insights, tool features can be designed to help the model viewer in reading and understanding a model. Second, modelers can be directed to those parts of their model that are likely to be difficult to understand by the intended readership.

In this paper, we address this research gap from a theoretical angle. We analyze the comprehension process from the perspective of visual cognition in order to build hypotheses of comprehension task performance in relation to process models. We test our hypotheses using a free-simulation experimental design [8] in order to integrate visual cognition data from an eye-tracking device. The results underline the importance of visual cognition for process model comprehension. Factors associated with visual cognition explain a good share of the overall variance in comprehension performance and mediate classical factors such as model complexity and personal differences. This has implications for designing process models in practice and for research on conceptual models altogether.

The rest of the paper is structured as follows. Section 2 summarizes prior research on process model comprehension and develops hypotheses based on visual cognition. Section 3 presents the design of our study, and Section 4 provides the results. Section 5 discusses implications of this research. Section 6 concludes the paper and points to directions of future research.

Section snippets

Background

In this section, we present the background of our research. First, we summarize prior research on process model comprehension. Then, we discuss visual cognition and its link to the notion of a relevant region. Finally, we present our research question along with corresponding hypotheses.

Research method

To examine the role of visual cognition in process model comprehension, we designed a free-simulation experiment [8] based on an eye-tracking observation method. Free-simulation experiments are different from traditional factorial experiment designs in that subjects are confronted with tasks and asked to respond to them. Therefore, we can use them to uncover connections between variables that are not exactly binary factors. Our research is specifically focused on visual cognition efficiency and

Results

This section presents the results of our experiment, following the recommendations of [35]. First, we provide an overview of the data by summarizing descriptive statistics and screen it for correlations. Second, for testing H1, H2, we evaluate regression models linking Expertise and Model independent variables to Visual Cognition. Then, Hypothesis H3 is tested by comparing the explanatory power of regression models including the state-of-the-art literature model factors (i.e. Elements and

Discussion

This section discusses the implications of our research findings. Section 5.1 summarizes the results. Section 5.2 discusses implications for research and practice. Section 5.3 clarifies potential threats to the validity of our study.

Conclusions

In this paper, we found that visual cognition variables outperform model complexity and personal knowledge, in terms of familiarity, in explaining the variance of Task Duration and efficiency. Furthermore, we find that the statistical effect of model complexity and familiarity is fully mediated by visual cognition variables for all considered performance variables.

Our findings have strong implications for future research on process model comprehension. It emphasizes the relevance of

Acknowledgement

This work was supported by UBB-GTC grant no. 34018/2013. We would also like to thank all the people who participated in this study.

Razvan Petrusel is an Associate Professor with the Department of Business Information Systems of the Faculty of Economics and Business Administration in Babeș-Bolyai University of Cluj-Napoca, Romania. He received his Ph.D. in Cybernetics and Statistics in 2008 from Babeș-Bolyai University. His research interests, as well as teaching, include Business Process Management, Conceptual Modeling, Decision Modeling and Mining, and Process Mining. He published and presented over 40 research papers in

References (40)

M. Dumas et al.
Fundamentals of Business Process Management
(2013)
H.A. Reijers et al.
A study into the factors that influence the understandability of business process models
Syst. Man Cybern. Part A Syst. Hum. IEEE Trans.
(2011)
J. Mendling et al.
Factors of process model comprehension—findings from a series of experiments
Decis. Support. Syst.
(2012)
J.C. Recker et al.
The effects of content presentation format and user characteristics on novice developers' understanding of process models
CAIS
(2011)
J. Recker
Continued use of process modeling grammars: the impact of individual difference factors
Eur. J. Inf. Syst.
(2010)
A. Burton-Jones et al.
The effects of decomposition quality and multiple forms of information on novices' understanding of a domain from a conceptual model
JAIS
(2008)
K. Figl et al.
Influence factors for local comprehensibility of process models
Int. J. Hum. Comput. Stud.
(2015)
H.L. Fromkin et al.
Laboratory Experimentation
(1972)
D.L. Moody
Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions
DKE
(2005)
J. Recker et al.
Process model comprehension: the effects of cognitive abilities, learning style, and strategy
CAIS
(2014)

A. Gemino et al.

A framework for empirical evaluation of conceptual modeling techniques

Requir. Eng.

(2004)

J. Mendling et al.

What makes process models understandable?

J. Cardoso

Evaluating workflows and web process complexity

L. Sanchez Gonzalez et al.

Measurement in business processes: a systematic review

Bus. Process. Manag. J.

(2010)

A. Polyvyanyy et al.

Structuring acyclic process models

Inf. Syst.

(2012)

K. Figl et al.

The influence of notational deficiencies on process model comprehension

J. Assoc. Inf. Syst.

(2013)

J. Recker et al.

Do ontological deficiencies in modeling grammars matter?

MIS Q.

(2011)

M. Zur Muehlen et al.

How much language is enough? Theoretical and practical use of the business process modeling notation

P. Bera

Does cognitive overload matter in understanding bpmn models?

J. Comput. Inf. Syst.

(2012)

H.A. Reijers et al.

Syntax highlighting in business process models

Decis. Support. Syst.

(2011)

Cited by (55)

An amalgamation of cognitive aspects in software engineering: A content analysis
2024, Expert Systems with Applications
Understanding the way, developers perform different software development and management tasks with objective measures can support improving potency and performance. Exploring the cognitive skills of programmers in software engineering has been the focus of immense and ongoing research for beyond 30 years. This work aims to combine and analyze different cognitive aspects such as psychology, understanding of program comprehension, and emotions of developers in the software engineering field, by conducting a Systematic Literature Review (SLR). We collected more than 300 articles from 7 major digital libraries and narrowed them down by selecting the 80 most relevant articles. Moreover, we framed six research questions and justified each question with the help of tables and graphs. Furthermore, we generated a classification scheme for each article, based on the justification of 10 quality assessment queries. Our analysis based on research questions shows that most of the research articles (12/80) published in the field of cognitive aspects are after the year 2017. In our SLR 39/80 articles are published in reputed journals. To measure developers’ cognitive abilities, 48/80 studies used particular bio measures. Most of the articles (19/80) employed student datasets in their experiments. In the case of the research methodology used, 23/80 of the articles are validation studies and 22/80 are experimental lab studies. In exploring the sensors, 37 % of the studies in which sensors were used to analyze cognitive features have applied multimodal sensors, while the rest of the studies used individual sensors according to their requirement such as Eye trackers, fMRI, fNIRS, etc. Exploring the range of the participants engaged in the experiments in various studies, 18 % of the total studies involved participants in the range of 11–20 and 27 % of studies did not mention the number of participants. The research implications of this review work will be the assessment of the understanding level of a programmer while developing or comprehending a program. This combined analysis will be helpful for the professionals or field experts to perform a better analysis of different cognitive features of developers in the software engineering field and for that analysis, they will be able to select more optimal bio measures, sensors, and methodologies over different conditions.
The effects of modular process models on gaze patterns - A follow-up investigation about modularization in process model literacy
2024, Expert Systems with Applications
Advanced organizational processes today require many process-relevant artifacts to be formally represented in process models. As these processes become more complex, process designers face the challenge of maintaining high comprehensibility while adequately visualizing process-relevant information. Modularization has been introduced to address the increasing complexity and information overload of process models, tackling this issue. While research acknowledges the importance of process model comprehensibility in order to benefit from the merits of such models, there still needs to be more clarity about the benefits of process model modularization. Recent research has focused on the human visual system and investigated visual behavior in the context of process model literacy. Continuing this branch of research, this paper analyses gaze behavior when comprehending modularized process models. A follow-up eye-tracking study was conducted to comprehend differently modularized BPMN 2.0 process models (i.e., vertical, horizontal, orthogonal). The results indicated that process model readers comprehend modularization approaches differently. Vertical modularization was perceived as the best comprehensible approach, whereas horizontal received the least acceptance. Considered eye movement performance measures confirm prior observation that attention is deployed more effectively in vertical modularization. Based upon the findings, the paper proposes a revised theoretical model from prior work that relates visual tasks to the execution of visual routines, explaining comprehension and information processing in modularized process models. The theoretical model elaborates how the visual system process and comprehends information obtained from modularized process models. Furthermore, the paper derived implications for modeling and practical applications, pointing out solid representational points and shortcomings of each modularization approach regarding how modularized process models are read and comprehended.
Conducting eye-tracking studies on large and interactive process models using EyeMind
2023, SoftwareX
The understandability of process models has been subject to extensive research in which eye-tracking has demonstrated great capability to deliver meaningful insights. However, the full potential of this technology is not fully exploited due to the complexity of using dynamic stimuli in experiments (i.e., large and interactive process models) and the common use of static stimuli (i.e., small non-interactive models) as a cheap alternative limiting the ecological validity of the used experimental setting and the generalizability of the results. This paper presents EyeMind, a solution to overcome this limitation by supporting the whole experimental workflow using dynamic stimuli and offering a comprehensive analysis toolkit of eye-tracking data. All these features facilitate experiments on large and interactive process models as well as the extraction of meaningful insights.
Defining gaze patterns for process model literacy – Exploring visual routines in process models with diverse mappings
2023, Expert Systems with Applications
Citation Excerpt :
Further, as shown by the authors in (Bera, Soffer & Parsons, 2019), different eye movement parameters may constitute indicators for high cognitive load, whereas research in (Petrusel, Mendling & Reijers, 2016) and (Winter, Pryss, Probst & Reichert, 2021) demonstrated that visual cues support the reading and comprehension of process models. Finally, the authors in (Petrusel, Mendling & Reijers, 2017) provide insights into how visual cognition affects process model comprehension. In summary, despite all conducted works in the context of process model literacy, no work has addressed so far visual information processing in terms of visual routines and elemental operations during the comprehension of process models.
Process models represent important information artifacts for organizations for documentation, communication and collaboration. Proper understanding of such models is critical for effective application in practice (e.g., to have the right information in the right place at the right time). A key aspect of understanding process models is how visual information represented in process models is extracted and processed by the human visual system. For such visuospatial tasks, the visual system employs a set of elementary operations, the compilation of which gives rise to various visual routines. However, which visual routines and related operations are used in understanding process models has not yet been investigated. To address this question, this paper presents findings from an exploratory eye-tracking study in which visual routines were considered during process model comprehension. Specifically, n = 29 participants were asked to understand m = 18 process models expressed in Business Process Model and Notation 2.0, reflecting three different mapping types (i.e., straight, upward, downward) and complexity levels (i.e., easy, medium, hard). The performance measures showed that even less complex process models were challenging to understand properly. The upward mapping presented greater challenges to par-ticipants’ attention, whereas the downward mapping was better understood. Three gaze patterns were derived from the recorded eye movements to be applied during model comprehension. A general model is then proposed that summarizes a set of applied visual routines and elementary operations during process model comprehension. Finally, implications for practice as well as for research and future work are discussed.
Business process and rule integration approaches—An empirical analysis of model understanding
2022, Information Systems
Citation Excerpt :
Eye tracking equipment measures the biometric condition and movements of the eye, and has been widely used in IS research, especially in the field of conceptual models [78,79,94,95]. For example, eye fixation duration has been shown to be a valid behavioral measurement of cognitive load [96–99], and visual attention [100,101], visual cognition efficiency, and intensity [95,102] were found to be correlated with high task performance. Eye movements consist of fixations and saccades [103].
Business process models are widely used in organizations by information systems analysts to represent complex business requirements. They are also used by business users to understand business operations and constraints. This understanding is extracted from graphical process models as well as business rules. Prior research advocated integrating business rules and business process models to improve the effectiveness of various organizational activities, such as developing a shared understanding of practices, process improvement, and mitigating risks of compliance and policy breaches. However, whether such integrated modeling can improve the understanding of business processes, which is a fundamental benefit of integrated modeling, has not been empirically evaluated. In this paper, first, we report on an experiment investigating whether rule linking, a representative integrated modeling method, can improve understanding performance. We use eye tracking technology to understand the cognitive process by which model readers use models to perform understanding tasks. Our results show that rule linking outperforms separated modeling in terms of understanding effectiveness, efficiency, perceived mental effort, and visual attention. Further, cognitive process analysis reveals that the form of rule representation does not affect the extent of deep processing, but rule linking significantly decreases the occurrence of rule scanning and screening processes. Moreover, our results show that rule linking leads to an increase of visual association suggesting improved information integration, leading to improved task performance.
Process model abstraction for rapid comprehension of complex business processes
2022, Information Systems
Citation Excerpt :
Process model comprehension is a research area that attracts a lot of attention by the research community. A vast body of research focuses on studying the factors that influence process model comprehension [35–38] and methods to get new insights on process model comprehension [39–41]. Also, techniques have been proposed that enrich process models to foster their comprehension [42–44].
Business process management has been widely adopted by many organizations, resulting in the accumulation of large collections of process models. The majority of these models are rather large and complex. Even though such models constitute a great source of knowledge, they cannot be easily understood by all process stakeholders. Process model abstraction techniques have been proven effective in generating easy to comprehend, high-level views on business process models; thus, such techniques change the way that detailed process models may be utilized within an organization. Although much attention has been given to abstract activities of process models, to the best of our knowledge, there are no research works that deliver abstract process model views, by considering as candidates for abstraction not only activities but also other process model elements. In this paper, we present an abstraction approach that simplifies existing process models by focusing not only on the abstraction of activities, but also on the abstraction of data, roles, messages and artifacts. The proposed approach exploits both model structure and element properties, while it is grounded on a set of abstraction rules. A prototype tool has been implemented as a proof of concept; this tool has been used for validating the proposed approach by automatically applying the suggested abstraction rules to different sets of real-world process models. A number of process stakeholders have also been involved in this validation. Thus, it is empirically proved that the presented work is an effective process model abstraction method that increases the usability of complex business process models, as it enables their rapid comprehension by process stakeholders.

View all citing articles on Scopus

Jan Mendling is a Full Professor with the Institute for Information Business at Wirtschaftsuniversität Wien (WU Vienna), Austria. His research interests include various topics in the area of business process management and information systems. He has published > 250 research papers and articles, among others in ACM Transactions on Software Engineering and Methodology, IEEE Transaction on Software Engineering, Information Systems, Data & Knowledge Engineering, and Decision Support Systems. He is member of the editorial board of seven international journals, member of the board of the Austrian Society for Process Management (http://prozesse.at), one of the founders of the Berlin BPM Community of Practice (http://www.bpmb.de), organizer of several academic events on process management, and member of the IEEE Task Force on Process Mining. His Ph.D. thesis has won the Heinz-Zemanek-Award of the Austrian Computer Society and the German Targion-Award for dissertations in the area of strategic information management.

Hajo Reijers is a full professor in Business Informatics at the Department of Computer Science of the Vrije Universiteit Amsterdam (VU), where he is head of the Business Informatics group. Furthermore, he is a part-time full professor in the AIS group of the Department of Mathematics and Computer Science of Eindhoven University of Technology (TU/e). His research and teaching focus on business process management, workflow technology, business process improvement, and conceptual modeling. On these and related topics, he published over 150 research papers, book chapters and professional publications. He is also the managing director of the European BPM Round Table, an initiative to connect researchers and practitioners in the area of Business Process Management («http://www.bpmroundtable.eu/»). He is closely cooperating with companies from the services and healthcare domains, as well as with various international scholars.

View full text

How visual cognition influences process model comprehension

Highlights

Abstract

Introduction

Section snippets

Background

Research method

Results

Discussion

Conclusions

Acknowledgement

Fundamentals of Business Process Management

A study into the factors that influence the understandability of business process models

Syst. Man Cybern. Part A Syst. Hum. IEEE Trans.

Factors of process model comprehension—findings from a series of experiments

Decis. Support. Syst.

The effects of content presentation format and user characteristics on novice developers' understanding of process models

CAIS

Continued use of process modeling grammars: the impact of individual difference factors

Eur. J. Inf. Syst.

The effects of decomposition quality and multiple forms of information on novices' understanding of a domain from a conceptual model

JAIS

Influence factors for local comprehensibility of process models

Int. J. Hum. Comput. Stud.

Laboratory Experimentation

Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions

DKE

Process model comprehension: the effects of cognitive abilities, learning style, and strategy

CAIS

A framework for empirical evaluation of conceptual modeling techniques

Requir. Eng.

What makes process models understandable?

Evaluating workflows and web process complexity

Measurement in business processes: a systematic review

Bus. Process. Manag. J.

Structuring acyclic process models

Inf. Syst.

The influence of notational deficiencies on process model comprehension

J. Assoc. Inf. Syst.

Do ontological deficiencies in modeling grammars matter?

MIS Q.

How much language is enough? Theoretical and practical use of the business process modeling notation

Does cognitive overload matter in understanding bpmn models?

J. Comput. Inf. Syst.

Syntax highlighting in business process models

Decis. Support. Syst.