Abstract
Long Short-Term Memory (LSTM) has been widely applied in time series predictions. Time attributes are important factors in time series prediction. However, existing studies often ignore the influence of time attributes when splitting the time series data, and seldom utilize the time information in the LSTM models. In this paper, we propose a novel method named Position encoding and Time gate LSTM (PT-LSTM). We first propose a position-encoding based time attributes integration method, which obtains the vector representation of time attributes through position encoding, and integrate it with the observed value vectors of the data. Moreover, we propose a LSTM variant by adding a new time gate which is specially designed to process time attributes. Therefore, PT-LSTM can make good use of time attributes in the key phases of data prediction. Experimental results on three public datasets show that our PT-LSTM model outperforms the state-of-the-art methods in time series prediction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ariyo, A.A., Adewumi, A.O., Ayo, C.K.: Stock price prediction using the arima model. In: 2014 UKSim-AMSS 16th International Conference on Computer Modelling and Simulation, pp. 106–112 (2014)
Kong, W., Dong, Z.Y., Jia, Y., Hill, D.J., Xu, Y., Zhang, Y.: Short-term residential load forecasting based on lstm recurrent neural network. IEEE Trans. Smart Grid 10(1), 841–851 (2019)
Tsai, Y., Zeng, Y., Chang, Y.: Air pollution forecasting using RNN with LSTM. In: International Conference on Dependable, Autonomic and Secure Computing, pp. 1074–1079 (2018)
Box, G.E.P., Jenkins, G.M.: Time Series Analysis: Forecasting and Control. Prentice Hall PTR (1994)
Wang, H., Lu, L., Dong, S., Qian, Z., Wei, H.: A novel work zone short-term vehicle-type specific traffic speed prediction model through the hybrid emd-arima framework. Transportmetrica B-Transp. Dyn. 4(3), 159–186 (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
Azzouni, A., Pujolle, G.: A long short-term memory recurrent neural network framework for network traffic matrix prediction. arXiv: Networking and Internet Architecture (2017)
Bao, W., Yue, J., Rao, Y.: A deep learning framework for financial time series using stacked autoencoders and long-short term memory. PLoS One 12(7) (2017)
Wei, W., Honghai, Wu., Ma, H.: An autoencoder and LSTM-based traffic flow prediction method. Sensors 19(13), 2946 (2019)
Qin, D., Yu, J., Zou, G., Yong, R., Zhao, Q., Zhang, B.: A novel combined prediction scheme based on CNN and LSTM for urban pm2.5 concentration. IEEE Access 7, 20050–20059 (2019)
Wang, Yi., Gan, D., Sun, M., Zhang, N., Zongxiang, Lu., Kang, C.: Probabilistic individual load forecasting using pinball loss guided LSTM. Appl. Energy 235, 10–20 (2019)
Choi, E., Bahadori, M.T., Schuetz, A., Stewart, W.F., Sun, J.: Doctor AI: predicting clinical events via recurrent neural networks. In: Machine Learning for Healthcare Conference, pp. 301–318 (2015)
Mou, L., Zhao, P., Xie, H., Chen, Y.: T-LSTM: a long short-term memory neural network enhanced by temporal information for traffic flow prediction. IEEE Access 7, 98053–98060 (2019)
Gers, F.A., Schmidhuber, J., Cummins, F.: Learning to forget: continual prediction with LSTM. Neural Comput. 12(10), 2451–2471 (2000)
Bedi, J., Toshniwal, D.: Empirical mode decomposition based deep learning for electricity demand forecasting. IEEE Access 6, 49144–49156 (2018)
Baytas, I.M., Xiao, C., Zhang, X., Wang, F., Jain, A.K., Zhou, J.: Patient subtyping via time-aware LSTM networks. In: ACM Knowledge Discovery and Data Mining, pp. 65–74 (2017)
Zhu, Y., et al.: What to do next: Modeling user behaviors by time-LSTM. In: International Joint Conferences on Artificial Intelligence, pp. 3602–3608 (2017)
Vaswani, A., et al.: Attention is all you need. In: Annual Conference on Neural Information Processing Systems, pp. 5998–6008 (2017)
Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N.: Convolutional sequence to sequence learning. In: International Conference on Machine Learning, pp. 1243–1252 (2017)
Shaw, P., Uszkoreit, J., Vaswani, A.: Self-attention with relative position representations. In: Annual Conference of the North American Chapter of the Association for Computational Linguistics, vol. 2, pp. 464–468 (2018)
He, K., Zhang, X., Ren, S., Jian, S.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Liang, X., et al.: Assessing beijing’s pm2.5 pollution: severity, weather impact, apec and winter heating. Proc. Roy. Soc. A Math. Phys. Eng. Sci. 471(2182), 20150257 (2015)
Fanaeet, H., Gama, J.: Event labeling combining ensemble detectors and background knowledge. Progress Artif. Intell. 2(2), 113–127 (2014)
AEMO. Aggregated price and demand data (2016). https://www.aemo.com.au/energy-systems/electricity/national-electricity-market-nem/data-nem/aggregated-data
Castro-Neto, M., Jeong, Y.-S., Jeong, M.-K., Han, L.D.: Online-svr for short-term traffic flow prediction under typical and atypical traffic conditions. Expert Syst. Appl. 36(3, Part 2), 6164–6173 (2009)
Wu, C., Ho, J., Lee, D.T.: Travel-time prediction with support vector regression. IEEE Trans. Intell. Transp. Syst. 5(4), 276–281 (2004)
Gers, F.A., Schmidhuber, J.: Recurrent nets that time and count. In: International Joint Conference on Neural Networks, vol. 3, pp. 189–194 (2000)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yu, Y., Xia, X., Lang, B., Liu, H. (2021). PT-LSTM: Extending LSTM for Efficient Processing Time Attributes in Time Series Prediction. In: U, L.H., Spaniol, M., Sakurai, Y., Chen, J. (eds) Web and Big Data. APWeb-WAIM 2021. Lecture Notes in Computer Science(), vol 12858. Springer, Cham. https://doi.org/10.1007/978-3-030-85896-4_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-85896-4_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85895-7
Online ISBN: 978-3-030-85896-4
eBook Packages: Computer ScienceComputer Science (R0)