MAMixer: Multivariate Time Series Forecasting via Multi-axis Mixing

Abstract

Sensor data, such as traffic flow monitoring data, constitutes a type of multimedia data. Forecasting sensor data holds significant potential for decision-making. And we can explore its patterns using time series forecasting methods. In the past few years, Transformer-based models have gained popularity in multivariate time series forecasting due to their ability to capture long-range temporal dependencies. Transformer models have quadratic time complexity in the self-attention mechanism, which hinders their efficiency in handling long-term sequences. The recently proposed PatchTST model has addressed this limitation by dividing sequences into a series of patches and using patch-wise tokens as input, which can dramatically reduce the computational complexity in Transformers. However, PatchTST adopts a channel-independent approach, which ignores the cross-channel interaction within multivariate time-series data. In this work, we propose Multi-Axis Mixer(MAMixer), in which we use three layers to capture the spatial, global and local temporal interactions within multivariate time-series data: a Spatial-Interaction Layer to capture the interactions among different channels, a Global-Temporal-Interaction Layer to capture the interactions among different patches of data, and a Local-Temporal-Interaction Layer to capture the interactions within a patch. We conduct extensive experiments on various real-world datasets for time-series forecasting. The experimental results demonstrate that the proposed method has superior performance gains over several state-of-the-art methods for time-series forecasting.