A causal discovery algorithm based on the prior selection of leaf nodes

Abstract

In recent years, Linear Non-Gaussian Acyclic Model (LiNGAM) has been widely used for the discovery of causal network. However, solutions based on LiNGAM usually yield high computational complexity as well as unsatisfied accuracy when the data is high-dimensional or the sample size is too small. Such complexity or accuracy problems here are often originated from their prior selection of root nodes when estimating a causal ordering. Thus, a causal discovery algorithm termed as GPL algorithm (the LiNGAM algorithm of Giving Priority to Leaf-nodes) under a mild assumption is proposed in this paper. It assigns priority to leaf nodes other than root nodes. Since leaf nodes do not affect others in a structure, we can directly estimate a causal ordering in a bottom-up way without performing additional operations like data updating process. Corresponding proofs for both feasibility and superiority are offered based on the properties of leaf nodes. Aside from theoretical analyses, practical experiments are conducted on both synthetic and real-world data, which confirm that GPL algorithm outperforms the other two state-of-the-art algorithms in computational complexity and accuracy, especially when dealing with high-dimensional data (up to 200) or small sample size (down to 100 for the dimension of 70).

Introduction

Causality study has thrived and captured special attention in various fields for a long time, including statistics, artificial intelligence, and cognitive science (Janzing et al., 2017, Pearl, 2000, Spirtes et al., 2000). By deriving causal relationships from factual knowledge, causality discovers connections between variables and seeks possible explanations for observed events. One popular branch of causality focuses on continuous variables under the frameworks of structural equation models (Bollen, 1989) and Bayesian networks (Pearl, 2000, Spirtes et al., 2000). Since the recent study has shown that the non-Gaussianity of noise do benefit the causal discovery (Shimizu, 2014, Shimizu et al., 2006), in this work we focus on the framework of the Linear non-Gaussian Acyclic Model (LiNGAM), which is also a typical model to estimate causal networks from observational data.

The problems of estimating LiNGAM have been extensively studied in the literature, and there are three main frameworks, i.e., Independent Component Analysis (ICA)-based framework (Hyvärinen, 1999, Hyvärinen et al., 2004, Shimizu et al., 2006, Zhang and Chan, 2006) (e.g. ICA-LiNGAM algorithm), Score-based framework (Cai et al., 2018, Hoyer and Hyttinen, 2009) (e.g. BayesLiNGAM algorithm) and Root-based framework (Hyvärinen and Smith, 2013, Shimizu et al., 2011, Sogawa et al., 2011) (e.g. DirectLiNGAM algorithm and its improved version Pairwise-LiNGAM algorithm). The first two frameworks convert LiNGAM into a function optimization problem, whose approaches are sensitive to initial values (Hyvärinen et al., 2004) and often lead to a local optima (Himberg, Hyvärinen, & Esposito, 2004). Compared with the first two frameworks, Root-based framework is a nonparametric estimation and it directly estimates causal networks in a finite number of steps. It contains two main phases: (1) identify a root node; (2) perform data updating process. This data updating process is applied because the selected root node must affect other nodes, and before we select the next root node, their influences on other nodes need to be removed. However, with a limited number of samples, errors may occur during the data updating process, which means that using this updated data with errors causes the wrong identification of the next root node. Moreover, when the data is high-dimensional, challenges of high complexity as well as cascading errors will be met.

Reasons why the Root-based framework is sensitive to the sample size, especially with high dimensional data, are as follows. When the sample is not adequate, it cannot provide enough information to conduct the data updating process. That is to say, it cannot assure that the estimated coefficients are all correct or at least approximately equal to the real coefficients. See a simple example in Fig. 1. As depicted in Fig. 1(a-1) and (a-2), coefficients like −5.08, 3.04, −9.85 and 5.89 should have been at least approximately equal to −5, 3, −10 and 6 so as to avoid errors. When the data is in higher dimensionality, methods under the Root-based framework have to carry out more data updating processes iteratively, which renders higher complexity.

To address the above problems, we propose a Leaf-based framework to solve LiNGAM under the causal tree assumption (The description of this assumption can be seen in Section 4). Specifically, owing to the property of the leaves, we can remove one of them from the whole structure, which does not affect the causal structure of other nodes. Hence we can, without performing data updating process, identify the leaf node iteratively and estimate a complete causal network in a bottom-up way within finite steps. Besides, we also propose an algorithm that gives priority to leaf nodes, namely the GPL algorithm under the Leaf-based framework. In order to identify the leaf node from the observed data, we measure the independence between a variable and its residuals, and find the variable which is dependent with all its residuals as the leaf node. In this way, GPL avoids the cascading errors and owns significant advantages in accuracy and especially computational complexity, with corresponding proofs provided to verify its effectiveness.

The major contributions of this paper are listed as follows:

(1) A new framework to estimate LiNGAM, based on the prior selection of leaf nodes (Leaf-based framework), is proposed.

(2) An effective algorithm to identify leaf nodes is proposed and detailed theoretical proofs are given under the causal tree assumption, from the aspects of sufficiency and necessity.

(3) Theoretical analyses demonstrate that our framework can quickly and efficiently estimate LiNGAM under the causal tree assumption, especially when dealing with the high-dimensional data (up to 200) or small sample size data (down to 100).

(4) Experiments on both synthetic and real-world data, including the causal tree assumption violated data, are conducted, which verify the availability of our algorithm.

The remainder of this paper is organized as follows. Related work on LiNGAM and its variant algorithms will be introduced in Section 2. In Section 3, a rapid review of LiNGAM will be undertaken. Subsequently, in Section 4, properties of leaf nodes with corresponding theoretical proofs will be studied, and a detailed GPL algorithm which gives priority to leaf nodes will be held. After assessing GPL’s performance with experiments in Section 5, a brief conclusion will be drawn in Section 6.