Keywords

1 Introduction

A very common way to visualize causal relationships is using graphs, in particular, node-link diagrams. However, not much has been done to evaluate the best ways of depicting causality and the associated strength and direction in graph visualizations [1], neither do we know very much in general about how people read graphs [2, 3]. Bae et al. [4] provided a set of guidelines to draw graphs representing causal relations with associated direction, strength, and significance. They highlighted that brightness exhibits higher accuracy and understandability in user ratings when it is used to depict trustworthiness. Moreover, Bae et al. [5] showed that sequential graph layout does not play a crucial role when analyzing causal relationships.

Following the results presented in [4, 5], we analyze in this paper the factors that help people to identify a root cause or a derived effect given a causal graph, and we compare the results obtained between groups using different sequential layouts.

In a complex biological pathway study [6], researchers acknowledged that topological ordering of closely related nodes helped to understand causality in biological pathways. Inspired by this work, we use the obesity influence map [7] to investigate the effectiveness and importance of sequencing the cause and effect relationship with influence values. The obesity influence map links 108 obesity-driving-factors to depict a web of causal relationships which was reviewed by over 300 experts in various disciplines. However, we take only a part of the whole map to fit the causal graph in our study.

The paper is structured as follows: Sect. 2 presents previous work regarding causality visualization, recommendations for graph depiction and relevant studies on graph reading. Section 3 outlines the study background and motivation, followed by Sects. 4 and 5 with the study design and results. General discussion and limitations are listed in Sect. 6, and conclusions in Sect. 7.

2 Related Work

A commonly used visual representation for correlation and causal relations among variables is node-link diagrams [1, 8, 9]. It depicts a collection of elements (vertices or nodes) and a set of relations between them (edges). Edges often indicate a weight (such as the strength and significance), as well as the direction of a relationship between the nodes.

Guo et al. [1] evaluated user perception of undirected edges which encoded two variables at the same time: strength and certainty (i.e., causality was omitted from their study). Different combinations of visual variables (such as hue, width, fuzziness, etc.) were assessed for different tasks, and a list of design recommendations was suggested based on their results. These include, for example, the usage of brightness, fuzziness and grain to depict correlations, but also that the effects of the combinations of the different visual variables need to be carefully investigated together with the task to be conducted.

In spite of the increasing use of graphs in everyday life, little is known about how people read graphs [2, 3], especially in causal relationships with several attributes. A series of experiments presented in [3, 10] tried to understand how people read graphs through eye tracking. To a larger extent, the results of these studies focused on how people deal with edge crossings, however, there are some interesting implications for our experiments. First, the authors showed that people have geodesic-path tendency (people tend to follow links first that lead toward the target node), and second, in performing path search tasks, when eyes encounter a node that has more than one link, links that go towards the target node are more likely to be searched first.

Another result related to graph reading that deserves our attention is localization. Waldmann et al. [11] proposed a minimal rational model that was sufficient to judge participants’ causal model. Participants perceived a causal relationship between a pair of variables locally without considering other dependencies. In another study on locality, Fernback and Sloman [12] found that people have difficulties in inferring chains, especially with additional links in between. The results showed that participants inferred a causal link individually (local) and ignored the rest of the relationships (global). Bramley et al. [13] claim that people maintain only a single hypothesis about the global causal model, rather than a distribution over all possibilities. In [12], the authors asked participants to make judgments following observations of several pre-selected interventions in causal learning. They found that participants were particularly bad at inferring chains, often inferring spurious additional links from the root to the sink node. Fernbach and Sloman [12] proposed that this was a consequence of participants inferring causal relationships through local rather than global computations.

Fig. 1.
figure 1

Sequential layout. Darker red and darker green marked objects indicate more selections from participants for cause and effect objects–nodes, respectively. (Color figure online)

Full size image
Fig. 2.
figure 2

Non-sequential layout. Darker red and darker green marked objects indicate more selections from participants for cause and effect objects–nodes, respectively. (Color figure online)

Full size image

3 Study Background and Motivation

In this paper, we applied visual cues to represent direction, strength, and trustworthiness (significance) associated with edges from a recent study [4]. Given a casual diagram with direction, strength and significance values, we investigate the reasons why people consider a certain object to be a root cause and a derived effect in the causal graph. We aim to explore which potential factors – including context – influence people to decide which node is a root cause or a derived effect in causal relationships.

4 Study Design

Using the obesity map as our dataset, we utilize sequential and non-sequential graph layouts in our user study. We showed the sequential layout using graphviz dot [14] and showed the non-sequential layout using Fruchterman and Reingold’s algorithm [15]. Three researchers from the research group at our university investigated the obesity influence map and agreed upon which objects to put into our stimuli since some labels were difficult to understand at a glance. Our stimuli consisted of 18 objects and 20 edges. The edges depicted different strength and trustworthiness levels of the causal relationships, based on the approaches by Guo [1] and Bae [4], using width for strength and brightness for trustworthiness. The edges were depicted in three levels of strength (width 78, 48, 18 px for high, middle, and low) and trustworthiness (brightness controlled as from a previous study [1] with hue 240 in HSB model for low, middle, and high). The strength-trustworthiness combinations used for the 20 edges consisted of three of each H-L, H-M, M-H, M-L, and four of each L-H and L-Ms (H: high, M: medium, L: low). In order to show all combinations, we added two additional objects to the stimuli – “family/environment support” (we may use “family support” instead in the tables) as a cause of “education”, and “cultural background” as a cause of “individualism”.

Figures 1 and 2 (without the colored circle marks) illustrate the stimuli we used. We explain the colored circle marks in the results Sect. 5.3. Figure 1 displays the sequential layout with objects “family/environment support” and “cultural background” located at the top and 9 leaves at the bottom. Figure 2 shows a force-directed layout where “family/environment support” is located at the left bottom, but “cultural background” at the top right. Most of the leaf objects are located around the boundary of the diagram.

Our online test environment displayed a stimulus on the left, and root cause and derived effect questions on the right. The participants were able to choose from 19 objects – including no reply –, and were requested to write their reason for selecting a particular node in a text box. The same process was applied for the derived effect. To remove the effects of fatigue and learning, we counterbalanced the order of the two layouts.

Table 1. Descriptions of root cause reasons and defined categories.
Full size table

We had 44 participants recruited online with normal or corrected-to-normal vision. We obtained confirmation of consent from the participants before they started the online study, and asked them to read the instructions for the experiment. Participants performed the strength and trustworthiness measurement tasks first but only the last two trials asked four questions regarding to the root cause and the derived effect given a certain layout. The root cause questions we asked were, “What do you think the root cause is from the overall relationships?” (with dropbox options), and “Why do you think so?” (with text box), and the same with the “derived effect” by changing the terms “root cause”. We collected 44 root cause objects, their reasons of selection, derived effect objects, and their reasons of selections, including no replies (n/a). Three researchers examined the participants’ reasoning replies and grouped them into the best-fit category (Fig. 5). The mapping between the categories and the individual’s reasoning descriptions are described in Table 1 for the root cause and in Table 2 for the derived effect.

5 Results

Figures 3 and 4 display the participants’ selections on cause objects and effect objects by descending total number of selections. Almost half of the participants selected “family/environment support” and “cultural background” as causes. Many opted not to respond to the derived effect question, but second and third chosen were “perceived lack of time” and “tendency to graze (eat out)”. We provide detailed information regarding dominant objects selected in one of our later sections.

Table 2. Descriptions of derived effect reasons and defined categories.
Full size table
Fig. 3.
figure 3

Root cause objects selected by the participants (in descending order of total selections).

Full size image
Fig. 4.
figure 4

Derived effect objects selected by the participants (in descending order of total selections).

Full size image
Fig. 5.
figure 5

Reasons of why participants chose a certain object as a cause or an effect (by descending total number of selections). Root cause reasons on the left and derived effect reasons on the right.

Full size image

Table 1 shows the mapping between our defined categories and participants’ reasons for selecting a cause object in descending order of total number of selections in both sequential and non-sequential layouts. Most participants described their reason of the cause selection as a “starting point” (38%) followed by “global influence” (28%) and “personal comments” (13%). Table 2 shows the mapping between our defined categories and participants’ reasons for selecting a derived effect object in the descending order of total number of selections in both layouts. The selection was distributed almost evenly in percentage. The most selected was “personal comments” (17%) followed by “n/a” replies (16%) and “higher strength and trustworthiness” (15%).

Our Fisher’s exact test revealed that selecting a root cause object (p = 0.0827, Cramer’s V = 0.4457, Fisher’s exact test) is marginally significantly affected by sequential and non-sequential layouts. However, describing its reason is not affected by the different layouts (p = 0.4499 FET, Cramer’s V = 0.2586).

Moreover, our analysis showed that a derived effect’s object selection is not affected by the different layouts (p = 0.5983 FET, Cramer’s V = 0.2926). However, the effect’s selection reasoning is marginally significantly different by the different sequential layouts (p = 0.0899 FET, Cramer’s V = 0.3671). “Higher strength and trustworthiness” was selected more as an effect decision reason in the sequential layout while “there are too many to choose” was selected more in the non-sequential layout.

Table 3. Reason-cause and reason-effect pairs by descending total number of selections (displays four top ranking pairs).
Full size table

5.1 Pairs of Root Cause Objects and Perceived Reasons

Table 3(a) describes the root cause object and its perceived reason by descending total number of selections. We find that “family/environment support” is selected 20 times in total because it is a starting point. It is also selected 18 times more because participants perceived it to have a global influence. We find that there are certain nodes chosen as a cause object in the sequential layout while others in the non-sequential diagram. For example, “individualism” was never chosen in the sequential, but was chosen four times in the non-sequential diagram (special mark at “individualism” in Fig. 2). The top four responses from the participants regarding selecting a certain root cause object and its perceived reason indicated a non-significant difference from the different layouts (p = 0.4571 FET, Cramer’s V = 0.2342).

5.2 Pairs of Derived Effect Objects and Perceived Reasons

In contrast to the cause object-reasoning pairs, the top four derived effect object-reasoning pairs were significantly affected by sequential and non-sequential layouts (p = 0.0279 FET, Cramer’s V = 0.5593, Table 3(b)). “Perceived lack of time” and “tendency to graze” were selected as derived effects more times in the sequential layout due to “affected by both roots” and “higher strength/trustworthiness”, respectively. Many chose “n/a” due to “there are too many to choose” which is six times more in the non-sequential than in the sequential diagram.

5.3 Dominant Selections

The colored circles in Figs. 1 and 2 show the dominance of participants’ selections for both cause (red circles) and effect (green circles) objects.

In the sequential layout (Fig. 1), three objects positioned at top such as “family/environment support” (20 selections), “cultural background” (10 selections), and “education” (5 selections) were chosen in total 40% as a root cause. Although we did not perform a statistical analysis on each node’s position, it is likely that participants selected root causes more from the top locations and derived effect objects more from the bottom of the diagram.

In contrast, the non-sequential layout (Fig. 2) shows that “family/environment support” is still dominantly selected disregarding of its position, but “cultural background” and “education” are lesser selected compared to the sequential layout. The star mark on “individualism” stands for an odd selection which was only selected as a root cause in the non-sequential layout, never in the sequential layout. In fact, it seemed that participants had a harder time to decide a cause or effect object in the non-sequential layout. More than 25% refused to respond to the cause and effect object selection task in the non-sequential layout (only 2% in the sequential layout).

5.4 Pairs of Cause Object-Effect Object and Cause Reason-Effect Reason

We investigated whether certain pairs of cause-effect objects or cause-effect reasons were selected depending on the different layouts. Table 4 shows these pairs in descending order of the total number of selections. From Table 4(a), we see that many chose “family/environment support” as a root cause but the effects are scattered to “perceived lack of time”, “n/a”, and “tendency to graze”. However, we did not find evidence that the cause-effect object pairs are significantly different based on the two layouts (p = 0.2241 by FET, Cramer’s V = 0.4676). Likewise, the difference of cause-effect reasons between different sequential layouts was not significant (p = 0.1349 by FET, Cramer’s V = 0.481).

Table 4. Cause-effect object pairs and cause-effect reason pairs by descending total number of selections (displays 8 top ranking pairs).
Full size table

5.5 Other Factors

As we examined participants’ replies, we found that there are other factors that need further investigation. For instance, the size of a graph (number of objects - nodes and edges), the number of connections on each object (a degree of a node), the context of the causal relationship, the importance of each object given the content, the strength and significance information in the causal relationship, and the position of the object (top, bottom, left, right) may affect the understanding and selection of root causes and derived effects in causal diagrams.

Although the frequency of selecting many-degree objects compared to few-degree objects was not significantly different in our experiment (for both cause and effect cases), we need to explore further if the number of degrees is a factor that affects viewers’ decision. The number of sources and sinks may also influence selecting a root cause or a derived effect.

Looking at contextual information, we found that several participants used their own experiences and background knowledge when interpreting the causal relations of the obesity factors (marked in personal comments in Tables 1 and 2). Some examples of the comments given are “family support is important”, “cultural background molds individuals”, “media consumption is a basic problem”, and “individualism is human’s psychology”.

According to the participants’ responses, combinations of the factors described above may influence the results as well; this needs to be explored in future experimental studies.

We additionally examined whether being a root or a leaf node matters. However, the statistical results suggest that sequential layouts do not influence the selection of a cause object because it was a root node, \({\chi }^2\)(1,N = 88)=0.7857, p = 0.3754. Likewise, there was no evidence that participants selected an effect object because it was a leaf node in a specific sequential layout, \({\chi }^2\)(1,N = 88)=1.65, p = 0.199.

6 Discussion

Numerical outcomes of our investigations exhibit a marginal significant difference in cause object selection and effect reason selection between sequence and non-sequence layouts. In addition, there is a significant difference in the top four effect reason-effect object pairs for these groups. Clearly, we need to investigate this further, using, e.g. eye tracking sensors, with additional participants.

As illustrated by the sequential layout (Fig. 1), “family/environment support” and “cultural background” were selected over 50% of the time as a root cause, thus, it was likely that participants selected root causes more from the top locations and derived effect objects more from the bottom of the diagram. This result seems to match results discussed in the related-work section, since they show geodesic-path tendencies, as discussed as well in [3, 10].

The results also show that participant’s own beliefs, experiences and knowledge might influence graph interpretation. Looking at the reasons given by the participants for selecting root causes and derived effects, several used the information shown in the graph to match their own beliefs and experience, i.e. they confirmed their own hypothesis, e.g. “family support is important”, “cultural background molds individuals”, etc.

Two interesting aspects were found from analyzing the personal comments: the weight or importance of each object (factor or node) and the fact that context matters. In fact, participants tend to measure the importance of each object and think how it affects or relates to their individual life. This brings up the issue of visualizing the importance weight of each object in a causal relationship, which can be subjective and controversial. It would be interesting to see how people perceive the obesity map with the importance of each obesity factor. Statistically measuring the strength and trustworthiness of causal relationships is still an on-going research topic. Since the obesity map selected did not provide the strength/trustworthiness of the relationships between the obesity factors, we had to manually make up combinations of strength and trustworthiness in our examples.

In some occasions, it has been a challenge to analyze and categorize the reasons regarding the cause and effects selected by the participants, since some did not fit well in the categories defined. In such occasions, we decided to classify them as “personal comments”. Furthermore, we discovered that a lot of participants refused to reply what an effect is when there were too many options. There were a larger number of categories for the derived effect reasons compared with the root cause reasons (Fig. 5). Finally, since we carried out an online experiment, we did not further confirm whether participants’ reasoning regarding the categorizations made hold.

7 Conclusion and Future Work

In this paper, we investigated factors that influence the process of finding a root cause and a derived effect in causal node-link graphs with associated strength and significance depictions. For doing so, we carried out an online study with 44 participants that read sequential and non-sequential node-link graphs. The results indicate that participants show geodesic-path tendencies when selecting causes and derived effects, and that context matters, i.e., participant’s own beliefs, experiences and knowledge influence graph interpretation.

These results complement our previous findings presented in [4, 5], which provided a set of guidelines for drawing graphs representing causal relations with associated direction, strength, and significance, and studied the role of sequence in graph reading.

In future work, we plan to perform similar studies in other domains and with other data sets, involving expert analysts as well.