Embracing Data Irregularities in Multivariate Time Series with Recurrent and Graph Neural Networks

Abstract

Data collection in many engineering fields involves multivariate time series gathered from a sensor network. These sensors often display differing sampling rates, missing data, and various irregularities. To manage these issues, complex preprocessing mechanisms are required, which become coupled with any statistical model trained with the transformed data. Modeling the motion of seabed-anchored floating platforms from measurements is a typical example for that. We propose and analyze a model that uses both recurrent and graph neural networks to handle irregularly sampled multivariate time series, while maintaining low computational cost. In this model, each time series is represented as a node in a heterogeneous graph, where edges depict the relationships between each measured variable. The time series are encoded using independent recurrent neural networks. A graph neural network then propagates information across the time series using attention layers. The outcome is a set of updated hidden representations used by the recurrent neural networks to create forecasts in an autoregressive manner. This model can generate forecasts for all input time series simultaneously while remaining lightweight. We argue that this architecture opens up new possibilities as the model can be integrated into low-capacity systems without needing expensive GPU clusters for inference.

We gratefully acknowledge the support from ANP/PETROBRAS, Brazil (project N. 21721-6), CNPq (grants 310085/2020-9 and 310127/2020-3), CAPES (finance code 001) and C4AI-USP (FAPESP grant 2019/07665-4 and IBM Corporation).