Abstract
Free full text
Exploring neural directed interactions with transfer entropy based on an adaptive kernel density estimator
Abstract
This paper aims at estimating causal relationships between signals to detect flow propagation in autoregressive and physiological models. The main challenge of the ongoing work is to discover whether neural activity in a given structure of the brain influences activity in another area during epileptic seizures. This question refers to the concept of effective connectivity in neuroscience, i.e. to the identification of information flows and oriented propagation graphs. Past efforts to determine effective connectivity rooted to Wiener causality definition adapted in a practical form by Granger with autoregressive models. A number of studies argue against such a linear approach when nonlinear dynamics are suspected in the relationship between signals. Consequently, nonlinear nonparametric approaches, such as transfer entropy (TE), have been introduced to overcome linear methods limitations and promoted in many studies dealing with electrophysiological signals. Until now, even though many TE estimators have been developed, further improvement can be expected. In this paper, we investigate a new strategy by introducing an adaptive kernel density estimator to improve TE estimation.
I. Introduction
In neuroscience, recent works have been devoted to detecting effective connectivity [1] defined as a causal influence of the dynamics of a first system on the dynamics of a second one. In this context, two questions are commonly addressed: (i) how to choose a formal quantitative definition of effective connectivity and (ii) how to provide corresponding estimators defined as functions of signals recorded in both systems. Nowadays two approaches contrast. The first one does not rely on an underlying physiological model while the second one, namely dynamical causal modeling, does. In this contribution, we are only concerned with the first approach including linear and nonlinear methodologies, and we consider nonlinear nonparametric entropic characterization of this connectivity using the so-called transfer entropy (TE). When computed on a stationary bivariate time series (X, Y), this quantity measures the amount of information transferred from channel X (resp. Y) to channel Y (resp. X) and is denoted TEx→y (resp. TEy→x) hereafter. It was introduced by Schreiber [2] and defined as the Kullback-Leibler divergence between two different predictive probability distributions of Yn. The first distribution is defined conditionally to amplitudes of Xn′, n′ < n and Yn′, n′ < n, at time instants prior to n, and the second one is only defined conditionally to Yn′, n′ < n. A simple exchange of X and Y leads to the definition of TEy→x. Formal definition is given by
with
The choice of k and l can impact drastically on theoretical TE value and, without a priori information on the hidden nonlinear dynamics generating (X, Y), this issue is not trivial and is not discussed in this paper. Given the theoretical index and an N point observation (X, Y)n, n = 1..N, we have to determine an estimation procedure to compute
If all probability densities are known, the trivial Monte Carlo estimator could be:
Since the densities are unknown, a first method consists in replacing each density pU by an estimation U possibly obtained by a fixed size kernel estimation approach as proposed in [3]. A second method computing estimations
II. Methods and Materials
A. Kernel methods
In order to reconstruct a probability density pU from N observed states un, the general form of a fixed kernel density estimator (FKDE) of bandwidth h is given by:
where K denotes a kernel function. For joint density probability estimated at (y, yk, xl), we write:
where Δ = {m / max(k, l) < m ≤ N − 1, |m − n| > τc}, and τc is the decorrelation time defined as the minimum delay leading to a correlation coefficient equal to 0.1. Parameters hx, hy are the respective kernel bandwidths for signals x and y which are normalized. A fixed bandwidth (independent of m) is unable to deal satisfactorily with the tails of the distribution without over smoothing the main part of this distribution. To avoid this issue, two methods help in estimating a density f at a point x :
The first method (7) uses a x dependent bandwidth, this bandwidth being unchanged for different points xm. One example of this method is a KNN estimator [6]. The second method (8) uses a xm dependent bandwidth, which does not depend on x, leading to the adaptive kernel density estimator (AKDE) [7] we adopted.
B. Adaptive kernel density estimator
AKDE is an improved alternative to FKDE. Given an initial bandwidth h and a first FKDE based estimation 0, Abramson [8] adapted the bandwidth according to these initial quantities:
In [9], Hwang extended this procedure:
where g is the geometric mean of (0 (xm))m, i.e.
r is an user defined sensitivity parameter generally satisfying 0 < r < 1. For different dimensions, the value of r should change. However, it is difficult to choose the proper r for four different probability density estimations. We suggest to compute
Step 1: for nh (10) values vi of h, 0 < v1 <..<vnh ≤3, compute fixed kernel density estimations at
and (yn+1, ) ;Step 2: compute hm using (10) only to update
leaving the other densities unchanged, and search the set Vh of values vi leading to unimodality of for vi {v1,.., vnh}hence eliminating monotonic curves (no multimodal curve was observed);Step 3: finally retain the maximum value
.
Experimentally, the selected value hs in the last step is often close to the initial bandwidth h0. Clearly, the computation time is increased with AKDE (multiplied by 20) for updating the density in step 2.
III. Experimental Results
We tested our method with Gaussian kernels to compute TEx→ y and TEy →x on two kinds of signals. The first kind included two toy linear AutoRegressive (AR) models and the second one was a realistic EEG model. Predictor dimensions k and l were chosen equal to the corresponding AR models orders estimated by the generalized Bayesian Information Criterion as in [10]. For AR models, the decorrelation time was τc=20 and experiments were repeated 200 times on 1024-point signals to get averaged values.
A. Unidirectional linear model
For the first linear stochastic system (model 1), the following two signals were generated:
where e1 and e2 were independent white Gaussian noises with zero means and unit variances.
Fig. 1 displays TE computed with a fixed bandwidth h. Experimental transfer entropy
TABLE I
Estimator | X → Y | Y → X |
---|---|---|
GC/2 | 0.4146 | 0 |
TE (fixed h) | 0.3242 (0.0158) | 0 (0) |
TE (AKDE) | 0.4063 (0.0179) | 0.01267(0.0045) |
Trentool | 0.3484 (0.0115) | −0.0158 (0.0070) |
B. Bidirectional linear model
For the second linear stochastic system (model 2), we generated the following signals:
where e1 and e2 were as in (12). Fig. 3 and Fig. 4 allow to compare TE values using either a fixed bandwidth (Fig. 3) or AKDE (Fig. 4, hs = 0.45). For this model, the exact value of TE from signal x to signal y (resp. from signal y to signal x) given in Table II is represented by a solid grey line (resp. a dashed grey line) in Fig. 3 and and4.4. Focusing on Fig. 4, the bidirectional flow propagation was correctly detected using AKDE, the mean values of TE being close to the exact ones (see also Table II). This figure reveals that the bias of AKDE estimator is negligible. As for TE estimated with a fixed bandwidth (Fig. 3 and Table II), its values remain lower than the exact ones, similarly as those estimated with Trentool toolbox. For all estimators tested, the standard deviation is 5 to 10 times lower than the corresponding mean value.
TABLE II
Estimator | X → Y | Y → X |
---|---|---|
GC/2 | 0.1511 | 0.0630 |
TE (fixed h) | 0.1118 (0.0123) | 0.0422 (0.0083) |
TE (AKDE) | 0.1457 (0.0133) | 0.0689 (0.0087) |
Trentool | 0.1120 (0.0091) | 0.0446 (0.0079) |
C. Physiology based model
We simulated EEG signals with a nonlinear SDE (stochastic differential equation) model [11] of order 20 to represent the activities of two neuronal populations PopX and PopY:
where the line vectors
TABLE III
Estimator | X → Y | Y → X |
---|---|---|
GC/2 | 0.011 (0.0193) | 0.0028 (0.0009) |
TE (AKDE) | 0.2521 (0.1143) | 0.1249 (0.0615) |
Trentool | 0.0049 (0.3182) | 0.0091 (0.0083) |
IV. Conclusion
In this paper, we focused on information propagation between two observations using TE and introduced an adaptive kernel density estimator to improve fixed kernel TE estimator. Results on simulated AR models revealed a very low bias with AKDE approach and proved the relevance of this new method in detecting uni/bi-directional propagation flows. Using a fixed bandwidth or Trentool approach led to much more biased values. For physiological signals, even if we had no ground-truth, the causal effects were perfectly identified and allowed characterizing the driving system and the responding one. In the future, the AKDE method will be tested on real EEG signals and on more complex scenarios including stronger nonlinearities and/or multivariate observations. A validation phase including statistical hypothesis tests based on surrogate data will complete this work.
References
Full text links
Read article at publisher's site: https://doi.org/10.1109/embc.2013.6610507
Read article for free, from open access legal sources, via Unpaywall: https://europepmc.org/articles/pmc3898961
HAL Open Archive
http://hal.archives-ouvertes.fr/hal-00881459
Citations & impact
Impact metrics
Article citations
Benchmarking Transfer Entropy Methods for the Study of Linear and Nonlinear Cardio-Respiratory Interactions.
Entropy (Basel), 23(8):939, 23 Jul 2021
Cited by: 5 articles | PMID: 34441079 | PMCID: PMC8394114
Similar Articles
To arrive at the top five similar articles we use a word-weighted algorithm to compare words from the Title and Abstract of each citation.
Wiener-Granger causality: a well established methodology.
Neuroimage, 58(2):323-329, 02 Mar 2010
Cited by: 304 articles | PMID: 20202481
The influence of filtering and downsampling on the estimation of transfer entropy.
PLoS One, 12(11):e0188210, 17 Nov 2017
Cited by: 9 articles | PMID: 29149201 | PMCID: PMC5693301
Estimation of direct nonlinear effective connectivity using information theory and multilayer perceptron.
J Neurosci Methods, 229:53-67, 19 Apr 2014
Cited by: 8 articles | PMID: 24751646
Nonlinear System Identification of Neural Systems from Neurophysiological Signals.
Neuroscience, 458:213-228, 11 Dec 2020
Cited by: 11 articles | PMID: 33309967 | PMCID: PMC7925423
Review Free full text in Europe PMC
Non-linear algorithms for processing biological signals.
Comput Methods Programs Biomed, 51(1-2):51-73, 01 Oct 1996
Cited by: 13 articles | PMID: 8894391
Review