Abstract
For time series forecasting, the weight distribution among multivariables and the long-short-term time dependence are always very important and challenging. Traditional machine forecasting can’t automatically select the effective features of multivariable input and can’t capture the time dependence of sequences. The key to solve this problem is to capture the spatial correlations at the same time, the spatiotemporal relationships at different times and the long-term dependence of the temporal relationships between different series. In this paper, inspired by human attention mechanism including encoder-decoder model, we propose DPAST-based RNN (DPAST-RNN) for long-term time series prediction. Specifically, in the first phase we use attention mechanism to extract relevant features at each time adaptively then we use stacked LSTM units to extract hidden information of time series both from time and space dimensions. In the second phase, we use another attention mechanism to select the related hidden state in encoder to the hidden state of the decoder at the current time to make context vector which is embed into recurrent neural network in decoder. Thorough empirical studies based upon the VM-Power dataset we collected on OpenStack and the NASDAQ 100 Stock dataset demonstrate that the DPAST-RNN can outperform state-of-the-art methods for time series prediction.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Amini, M.H., Kargarian, A., Karabasoglu, O.: ARIMA-based decoupled time series forecasting of electric vehicle charging demand for stochastic power system operation. Electr. Power Syst. Res. 140, 378–390 (2016)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Chakraborty, P., Marwah, M., Arlitt, M., Ramakrishnan, N.: Fine-grained photovoltaic output prediction using a Bayesian ensemble. In: Twenty-Sixth AAAI Conference on Artificial Intelligence (2012)
Cheng, Y., Huang, F., Zhou, L., Jin, C., Zhang, Y., Zhang, T.: A hierarchical multimodal attention-based neural network for image captioning. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 889–892 (2017)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Di Gangi, M.A., Federico, M.: Deep neural machine translation with weakly-recurrent units. arXiv preprint arXiv:1805.04185 (2018)
Fernandez-Fraga, S., Aceves-Fernandez, M., Pedraza-Ortega, J., Ramos-Arreguin, J.: Screen task experiments for EEG signals based on SSVEP brain computer interface. Int. J. Adv. Res. 6(2), 1718–1732 (2018)
Forster, F., Harl, S.: Collectd - the system statistics collection daemon (2012). https://collectd.org
Guo, T., Lin, T.: Multi-variable LSTM neural network for autoregressive exogenous model. arXiv preprint arXiv:1806.06384 (2018)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Hübner, R., Steinhauser, M., Lehle, C.: A dual-stage two-phase model of selective attention. Psychol. Rev. 117(3), 759 (2010)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Liang, Y., Ke, S., Zhang, J., Yi, X., Zheng, Y.: GeoMAN: multi-level attention networks for geo-sensory time series prediction. In: IJCAI, pp. 3428–3434 (2018)
Liu, J., Zio, E.: SVM hyperparameters tuning for recursive multi-step-ahead prediction. Neural Comput. Appl. 28(12), 3749–3763 (2017). https://doi.org/10.1007/s00521-016-2272-1
Müller, M.: Dynamic time warping. In: Müller, M. (ed.) Information Retrieval for Music and Motion, pp. 69–84. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74048-3_4
Plutowski, M., Cottrell, G., White, H.: Experience with selecting exemplars from clean data. Neural Netw. 9(2), 273–294 (1996)
Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., Cottrell, G.: A dual-stage attention-based recurrent neural network for time series prediction. arXiv preprint arXiv:1704.02971 (2017)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
Wu, Y., Hernández-Lobato, J.M., Ghahramani, Z.: Dynamic covariance models for multivariate financial time series. arXiv preprint arXiv:1305.4268 (2013)
Zamora-Martinez, F., Romeu, P., Botella-Rocamora, P., Pardo, J.: On-line learning of indoor temperature forecasting models towards energy efficiency. Energy Build. 83, 162–172 (2014)
Acknowledgments
This work was supported by National Key Research and Development Program of China (2018YFB1003702) and Jiangsu Scientific Research Innovation Practice Project (KYCX20_0760).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Shan, S., Shen, Z., Xia, B., Liu, Z., Li, Y. (2020). DPAST-RNN: A Dual-Phase Attention-Based Recurrent Neural Network Using Spatiotemporal LSTMs for Time Series Prediction. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12534. Springer, Cham. https://doi.org/10.1007/978-3-030-63836-8_47
Download citation
DOI: https://doi.org/10.1007/978-3-030-63836-8_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63835-1
Online ISBN: 978-3-030-63836-8
eBook Packages: Computer ScienceComputer Science (R0)