Abstract
Extending the forecasting horizon is a crucial demand for real applications in time series forecasting with multiple exogenous series (TFME). Previous studies adopt Transformer to effectively capture long-term dependency coupling between output and input in a sequence. However, the potential entanglement in multi-dimensional feature space still precludes the application in TFME tasks. In this paper, we propose a dual-branch network named Orthrus to solve this issue by differentiating the processing of target and exogenous sequences. Orthrus takes the long target sequence as input and uses multi-head self-attention mechanism to capture long-term cyclical patterns. Concurrently, it applies the Local Mutual Dependency Analysis module to extract the sub-sequences of the exogenous sequences with the maximum expected information and adopts the Multi-scale Convolutional Neural Network module to capture the dependencies among the sub-sequences and align with the target sequence. In this way, the feature entanglement issue is largely alleviated. Extensive experiments on four real-world datasets verify that Orthrus is superior to all baselines in prediction accuracy, inference efficiency, and memory usage, providing an effective solution to the TFME task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Asteriou, D., Hall, S.G.: Arima models and the Box-Jenkins methodology. Appl. Econometrics 2(2), 265–286 (2011)
Atiya, A.F., El-Shoura, S.M., Shaheen, S.I., El-Sherif, M.S.: A comparison between neural-network forecasting techniques-case study: river flow forecasting. IEEE Trans. Neural Netw. 10(2), 402–409 (1999)
Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv:1803.01271 (2018)
Bengio, Y., Simard, P.Y., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5, 157–166 (1994)
Bouchachia, A., et al.: Ensemble learning for time series prediction. In: The 1st International Workshop on Nonlinear Dynamics and Synchronization (2008)
Box, G.E., Jenkins, G.M.: Some recent advances in forecasting and control. J. R. Stat. Soc. Ser. C Appl. Stat. 17(2), 91–109 (1968)
Chen, S., Wang, X., Harris, C.: NARX-based nonlinear system identification using orthogonal least squares basis hunting. IEEE Trans. Control Syst. Technol. 16, 78–84 (2008)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: EMNLP (2014)
Chung, J., et al.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555 (2014)
Diaconescu, E.: The use of NARX neural networks to predict chaotic time series. WSEAS Trans. Comput. Arch. 3, 182–191 (2008)
Frigola, R., et al.: Integrated pre-processing for Bayesian nonlinear system identification with gaussian processes. In: Conference on Decision and Control (2013)
Gao, Y., Er, M.J.: NARMAX time series model prediction: feedforward and recurrent fuzzy neural network approaches. Fuzzy Sets Syst. 150(2), 331–350 (2005)
Hochreiter, S., et al.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)
Huang, X., et al.: TEALED: a multi-step workload forecasting approach using time-sensitive EMD and auto LSTM encoder-decoder. In: DASFAA 2022. Lecture Notes in Computer Science, vol. 13246, pp. 706–713. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-00126-0_55
Lai, G., Chang, W.C., Yang, Y., Liu, H.: Modeling long- and short-term temporal patterns with deep neural networks. In: International ACM SIGIR Conference on Research and Development in Information Retrieval (2018)
Li, S., et al.: Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Liang, Y., Ke, S., Zhang, J., Yi, X., Zheng, Y.: GeoMAN: multi-level attention networks for geo-sensory time series prediction. In: IJCAI (2018)
Lin, T., Horne, B.G., Tino, P., Giles, C.L.: Learning long-term dependencies in NARX recurrent neural networks. Neural Netw. 7, 1329–1338 (1996)
Matsubara, Y., Sakurai, Y., Van Panhuis, W.G., Faloutsos, C.: FUNNEL: automatic mining of spatially coevolving epidemics. In: ACM SIGKDD, pp. 105–114 (2014)
Qin, Y., Song, D., Cheng, H., Cheng, W., Jiang, G., Cottrell, G.W.: A dual-stage attention-based recurrent neural network for time series prediction. In: International Joint Conference on Artificial Intelligence (2017)
Reshef, D.N., et al.: Detecting novel associations in large data sets. Science 334(6062), 1518–1524 (2011)
Song, H., Rajan, D., Thiagarajan, J., Spanias, A.: Attend and diagnose: clinical time series analysis using attention models. In: AAAI. vol. 32 (2018)
Thomas, M., Joy, A.T.: Elements of Information Theory. Wiley-Interscience, Hoboken (2006)
Vaswani, A., et al.: Attention is all you need. In: Advances in neural information processing systems, pp. 5998–6008 (2017)
Xu, C., et al.: Graph attention networks for new product sales forecasting in e-commerce. In: Jensen, C.S., et al. (eds.) DASFAA 2021. LNCS, vol. 12683, pp. 553–565. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73200-4_39
Zerveas, G., et al.: A transformer-based framework for multivariate time series representation learning. In: ACM SIGKDD (2021)
Zhou, H., et al.: Informer: beyond efficient transformer for long sequence time-series forecasting. In: AAAI (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, Z., Zhou, B., Tang, X., Li, R., Hu, S. (2023). Orthrus: A Dual-Branch Model for Time Series Forecasting with Multiple Exogenous Series. In: Wang, X., et al. Database Systems for Advanced Applications. DASFAA 2023. Lecture Notes in Computer Science, vol 13943. Springer, Cham. https://doi.org/10.1007/978-3-031-30637-2_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-30637-2_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30636-5
Online ISBN: 978-3-031-30637-2
eBook Packages: Computer ScienceComputer Science (R0)