Elsevier

Information Sciences

Volume 607, August 2022, Pages 477-492
Information Sciences

Predicting high-dimensional time series data with spatial, temporal and global information

https://doi.org/10.1016/j.ins.2022.06.021Get rights and content

Abstract

In the field of time series forecasting, deep learning and dynamics-based methods are two main research directions. The former focuses on the temporal information of the data while the latter emphasizes on the spatial information of the data, and rare methods combine the two information properly. In order to make better use of the information in the data, we propose the STSM (spatiotemporal skip-connection model) based on the dynamics framework, which contains a temporal module composed of CNN and a spatial module composed of fully connected layers, as well as a skip connection to the original input to fuse temporal, spatial, and global information in the data. To predict the future value of the target variable, STSM is required to learn a mapping from original attractors to delay attractors in an end-to-end framework. The results of ablation and contrast experiments on one simulated dataset and seven real-world datasets show that STSM not only performs better than a separate temporal or spatial module, but also predicts more accurately than other traditional methods. Besides, we verify the robustness of the model in different scenarios through several experiments.

Introduction

In today’s highly information-based era, a huge amount of time series data has been generated in various disciplines of scientific research [1]. For example, in the field of biology, driven by gene sequencing technology, high-throughput biological data has grown rapidly [2], such situation also appears in atmospheric science[3] and intelligent transportation[4]. In this case, many researchers are facing huge analysis needs for high-dimensional time series data.

Historically, many theories have been proposed to solve the problem of time series prediction. Early methods are often based on parametric models, like autoregressive models and moving average models as well as their variants [5], [6], [7]. The disadvantage of such models is that they cannot model the relationship between high-dimensional data and have strong constraints on the stability of time series data. With the development of machine learning and deep learning, related methods have been widly applied in the field of time series prediction, such as support vector regression [8], [9], [10], gaussian process [11], feedforward neural network [12], [13], [14], convolutional neural network [15], recursive neural network [16], [17], [18] and so on. Compared with previous methods, they have fewer constraints on the data and enjoy a higher degree of approximation. However, the interpretability is not enough due to the inherent property of black box models, and the information from time and space cannot be combined well. Some methods under the dynamics framework [19], [20], [21] have better interpretability. But the shortcomings are obvious, too much attention is paid to spatial information while ignoring temporal information, and it also has high requirements for the chaos of the system.

In order to properly combine the spatial and temporal information on high-dimensional data to make better predictions, we propose a new model called Spatiotemporal skip-connection model (STSM for short) in this paper, which can simultaneously make temporal transformation, make spatial transformation and keep global information with temporal module, spatial module and skip connection respectively, and combine the spatial, temporal, and global information to make predictions. Before describing STSM, we need to introduce the concept of attractor: under the dynamics framework, high-dimensional variables and their changing process over time constitute a dynamic system, and all variables in the system constitute a phase space, the development of phase space tends to a relatively stable state called attractor. And in this study, we consider the local sampling of high-dimensional data as original attractors, the continuation of the target variable in time as delay attractors. Based on the Takens theorm [22], STSM tries to reconstruct delay attractors from original attractors by solving a mapping between them, so that the dynamic information in the high-dimensional data can be converted into the temporal information of the target variable. Specifically, we repeatedly sample the training data to get several pairs of original and delay attractors, take them as input and output of STSM respectively to learn the mapping. Although it seems that STSM can only make short-term predictions, we can get long-term prediction through an iterative process, in which the output of the previous prediction is used as the input of the next step. The results of contrast, ablation and robustness experiments on synthetic and real-world datasets show that STSM model can make more accurate predictions with strong robustness than traditional methods. In all, we make following contributions in this paper.

  • Rather than utilizing the spatial information from high-dimensional data to make predictions directly like traditional dynamic methods, we want to combine the spatial information and temporal information properly to get better results. To prove the feasibility of our idea, we propose a model called STSM based on Takens theorem to predict target variable from high-dimensional data, which contains a temporal module, a spatial module and a skip connection to the original input to combine spatial, temporal and global information.

  • We conduct several experiments on different datasets to test the ability of our model. It can be seen from the figures and indicators of contrast experiments that STSM surpasses other traditional time series prediction models in terms of predictive effects. And ablation experiments show that each module of our model is indispensable. Besides, the robustness experiments indicate the potential of STSM in combating noise, adapting to different initialization parameters as well as hyperparameters, dealing with time variability and making long-term prediction

The rest of this paper is organized as follows. In Section 2, we summarize related work from the perspective of predicting low-dimensional and high-dimensional data, the advantages and disadvantages of different models are compared in a fair way at the same time. Section 3 gives a definition of the prediction problem and explains how to apply Takens theorem to solve it under the dynamics framework, then our STSM model is introduced from the whole to the part. In Section 4, we do several experiments on one simulated Lorentz dataset and seven real-world datasets, the results of these contrast experiments, ablation experiments and robustness experiments are carefully analyzed. Finally, Section 5 draws the conclusion, analyses the advantages and disadvantages of STSM, and points out the direction for improvements in the future.

Section snippets

Related work

This chapter aims to summarize and compare the traditional models and algorithms for time series forecasting, show the advantages and disadvantages of different models, so as to highlight the main improvements of this work. The following contents will be divided into two parts to introduce models for predicting low-dimensional (univariate) and high-dimensional (multivariate) time series respectively.

Problem Formulation

In this work, we focus on inferring the evolution trend of a single variable from the overall dynamics of a high-dimensional system. Suppose one system contains N variables including x1,x2,,xN, when N is large enough, the system is high-dimensional. We make M continuous observations to get a original attractor O. If the observation starts at time t, the interval between observations is τ, then the time point of the i-th observation can be written as ti=t+iτ, and the value of xj at ti is xj(ti)

Experiments

This chapter mainly introduces some details of our experiments, including information of datasets, experimental setup, evaluation indicators, experimental results and corresponding analysis. The contrast experiments and ablation experiments are carried out on synthetic datasets and real-world datasets, and we do several robustness experiments to test the stability of STSM in the end, they will be introduced respectively below.

Conclusion

In this paper, we introduce a new model called STSM to make accurate predictions for high dimensional time series data, which can help to meet some real-life needs such as precise weather forecast, timely traffic warning, stock price prediction and prevention for the rapid spread of infectious diseases like COVID-19. Different from traditional dynamic methods, STSM combines the temporal, spatial, and global information in the data properly through structural innovation. We prove the efficiency

CRediT authorship contribution statement

Jining Wang: Methodology, Software, Writing - original draft, Writing - review & editing. Chuan Chen: Conceptualization, Methodology, Writing - original draft, Writing - review & editing. Zibin Zheng: Resources, Supervision, Project administration, Funding acquisition. Luonan Chen: Conceptualization, Methodology, Resources. Yuren Zhou: Supervision, Validation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

The research is supported by the National Key R&D Program of China (2020YFB1006001), the National Natural Science Foundation of China (62176269), the Guangdong Basic and Applied Basic Research Foundation (2019A1515011043) and the Tencent Wechat Rhino-bird project (2021321).

References (31)

  • H. Akaike

    Fitting autoregressive models for prediction

    Annals of the institute of Statistical Mathematics

    (1969)
  • R.J. Hyndman et al.

    Forecasting: principles and practice

    OTexts

    (2018)
  • R.S. Tsay

    An introduction to analysis of financial data with R

    (2014)
  • Y. Wang et al.

    A knn based kalman filter gaussian process regression

  • J.T. Turner

    Time series analysis using deep feed forward neural networks

    (2014)
  • Cited by (2)

    • Predicting time series by data-driven spatiotemporal information transformation

      2023, Information Sciences
      Citation Excerpt :

      Making accurate predictions of future system behavior using only observed time-series data of a nonlinear dynamical system, especially short-term data, is of interest in many fields, including computational biology, ecology, geoscience, and econometrics [1–11].

    View full text