Post-hoc Saliency Methods Fail to Capture Latent Feature Importance in Time Series Data

Abstract

Saliency methods provide visual explainability for deep image processing models by highlighting informative regions in the input images based on feature-wise (pixels) importance scores. These methods have been adopted to the time series domain, aiming to highlight important temporal regions in a sequence. This paper identifies, for the first time, the systematic failure of such methods in the time series domain when underlying patterns (e.g., dominant frequency or trend) are based on latent information rather than temporal regions. The latent feature importance postulation is highly relevant for the medical domain as many medical signals, such as EEG signals or sensor data for gate analysis, are commonly assumed to be related to the frequency domain. To the best of our knowledge, no existing post-hoc explainability method can highlight influential latent information for a classification problem. Hence, in this paper, we frame and analyze the problem of latent feature saliency detection. We first assess the explainability quality of multiple state-of-the-art saliency methods (Integrated Gradients, DeepLift, Kernel SHAP, Lime) on top of various classification methods (LSTM, CNN, LSTM and CNN trained via saliency-guided training) using simulated time series data with underlying temporal or latent space patterns. In conclusion, we identify that Integrated Gradients and DeepLift, if redesigned, could be potential candidates for latent saliency scores.

M. Schröder and A. Zamanian—Authors contributed equally.

A Appendix

1.1 A.1 Synthetic Data Generation

Based on the Fourier series latent model, a time series $x_t, t=1,...,T$ is modeled as

$$\begin{aligned} x_t&= a_0 + \sum _{n=1}^{\infty } a_n \cos (\omega _nt) + \sum _{n=1}^{\infty } b_n \sin (\omega _nt) \\&= a_0 + \sum _{n=1}^{\infty } A_n \cos (\omega _nt + \phi _n)\\&= a_0 + \sum _{n=1}^{\infty } A_n \sin (\omega _nt + \phi _n + \frac{\pi }{2}). \end{aligned}$$

To simulated data, let $\tilde{n}$ represent the number of amplitudes present in the series, i.e. $\forall i > \tilde{n}, A_i = 0$. For simplicity, we consider centered stationary periodic time series in the data generation process, i.e. $a_0 = 0$. In this case, the value at every time step t is calculated as

$$\begin{aligned} x_t = \sum _{i=1}^{\tilde{n}} A_i \sin (\omega _i t + \phi _i + \frac{\pi }{2}). \end{aligned}$$

(1)

We refer to the notions amplitude A, frequency $\omega$, phase shift $\phi$ as concepts. The separate Fourier coefficients $A_i, \omega _i, \phi _i$ for $i=1,...,\tilde{T}$ are referred to as latent features. The latent features frequency $\omega _i$ and phase shift $\phi _i$ are each sampled from a uniform distribution. The sampling intervals are chosen with respect to the specific intention in the experiment design. To simulate the amplitude parameters $A_i$, a dominant amplitude $A_1$ is sampled. The next amplitudes are calculated considering an exponential decay with a fixed rate dec:

$$\begin{aligned} A_i = A_1 \exp (-i \cdot dec), \quad i=1,...,\tilde{n}. \end{aligned}$$

This makes the first frequency i.e. $\omega _1$ to be the dominant frequency of the Fourier series. Throughout the experiments, all time series were generated with an equal length of 300 time steps. i.e. $T=300$.

For assigning class labels to the time series samples, we consider the following two scenarios.

Scenario 1: Label based on the presence of a shapelet

For assigning shape-based labels to the time series, a shapelet is inserted at a random or fixed position into all time series $X \in D$ belonging to one class. The shapelet is a second simulated Fourier series of length $l \le T$, where $l = \text {window-ratio} \cdot T$ for a chosen window ratio. We define the sampling intervals for the latent features of the shapelet to be non-intersecting with the sampling intervals of the latent features of the original time series X. The resulting shapelet replaces the original time series in the interval $[j, j+l]$, where

$$\begin{aligned} j \sim \mathcal {U}(1, T-l). \end{aligned}$$

Table 2. Label-making features per experiment. The overlapping ranges refer to the sampling intervals for frequency and phase shift.

Table 3. Overview of simulation parameters of the Fourier series. If two entries are present in one cell, each the classes were sampled from different distributions. The first entry in each cell corresponds to the sampling parameter of class 0, the second entry to class 1.

Scenario 2: Label based on differences in the latent features

Following the investigation of the effectiveness of explainability methods for latent features, we introduce a second simulation scenario where the labels depend on a difference in the sampling distribution of latent features of the time series. This scenario highlights the main focus of this project and represents our novel view of explainability methods for time series. Similar to the first scenario, the time series are sampled as discretized Fourier series with latent variables $\omega , A$ and $\phi$. The latent dependency is induced as follows:

• 1. Two normal distributions with different means (based on Table 3) are selected for classes 0 and 1. For positive parameters, the distributions are log-normal.

• 2. Per each class, N/2 Fourier parameters are sampled from the given distributions.

• 3. The rest of the parameters are sampled from the same distribution for both classes.

• 4. Sampled parameters are given to the deterministic Fourier series in Eq. 1 to generate the temporal samples. Rows are then labeled with the associated class, from the corresponding distribution of which the informative parameters are sampled.

1.2 A.2 Data Set Description

Based on the data generation method described above, we design ten different mechanisms for binary classification of univariate time series. Table 2 lists the parameters and algorithms for assigning labels to each sample. In Table 3 the parameters used for sampling the Fourier series are presented. The complete simulation code base can be found in the GitHub repository at https://github.com/m-schroder/TSExplainability.