Skip to main content
SpringerLink
Log in

Deep transition network with gating mechanism for multivariate time series forecasting

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

As an essential task in the machine learning community, multivariate time series forecasting has many real-world applications, such as PM2.5 forecasting, electricity price forecasting, and traffic flow forecasting. In multivariate time series data, one feature is chosen as the target feature and the other is seen as auxiliary information. These features have very complex spatial dependencies, and most of the existing models put all features together for spatial feature extraction, which is an undifferentiated feature extraction process. It can not distinguish the importance of different features to the target feature. Thus, the spatial feature extraction of data is insufficient. To solve the problems mentioned above, based on the Gated Recurrent Neural Network, we design a new architecture named Auxiliary Information-GRU (AI-GRU) to extract deep temporal and spatial features and combined it into Auxiliary Information-deep Transition Network (Al-DTN). AI-GRU added three gating units, auxiliary information gate controls the amount of auxiliary information flowing into AI-DTN, fusion information gate controls the degree of fusion of target feature and auxiliary information, linear transformation gate isolates the target features from the auxiliary information. Experimental results show that our proposed model outperforms the representative baseline models on the two benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Algorithm 2
Fig. 4
Algorithm 3
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The data and materials are available.

Code availability

The codes are also available if reviewers need it.

References

  1. Karevan Z, Suykens JA (2020) Transductive lstm for time-series prediction: an application to weather forecasting. Neural Netw 125:1–9

    Article  Google Scholar 

  2. Mahajan S, Chen L-J, Tsai T-C (2017) An empirical study of pm2. 5 forecasting using neural network. In: 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), pp 1–7. IEEE

  3. Jakasǎ T, Androceč I, Sprcić̌ P (2011) Electricity price forecasting—arima model approach. In: 2011 8th international conference on the European energy market (EEM), pp 222–225. IEEE

  4. Chang Y-Y, Sun F-Y, Wu Y-H, Lin S-D (2018) A memory-network based solution for multivariate time-series forecasting. arXiv:1809.02105

  5. Box GE, Pierce DA (1970) Distribution of residual autocorrelations in autoregressive-integrated moving average time series models. J Am Stat Assoc 65(332):1509–1526

    Article  MathSciNet  MATH  Google Scholar 

  6. Voyant C, Nivet ML, Paoli C, Muselli M, Notton G (2015) Meteorological time series forecasting based on mlp modelling using heterogeneous transfer functions. In: Journal of Physics: Conference Series, vol 574, p 012064. IOP Publishing

  7. Zhang GP (2003) Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50:159–175

    Article  MATH  Google Scholar 

  8. Lütkepohl H (2005) New introduction to multiple time series analysis

  9. Roberts S, Osborne M, Ebden M, Reece S, Gibson N, Aigrain S (2013) Gaussian processes for time-series modelling. Phil Trans R Soc A: Math Phys Eng Sci 371(1984):20110550

    Article  MathSciNet  MATH  Google Scholar 

  10. Frigola R (2015) Bayesian time series learning with gaussian processes. PhD thesis, University of Cambridge

  11. Connor JT, Atlas LE, Martin RD (1991) Recurrent networks and NARMA modeling. In: Moody JE, Hanson SJ, lippmann R (eds) Advances in neural information processing systems 4, [NIPS Conference, Denver, Colorado, USA, December 2-5, 1991], pp 301–308

  12. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  13. Chung J, Gülçehre Ç, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555

  14. Lai G, Chang W-C, Yang Y, Liu H (2018) Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st international ACM SIGIR conference on research & development in information retrieval, pp 95–104

  15. Assaf R, Giurgiu I, Bagehorn F, Schumann A (2019) Mtex-cnn: multivariate time series explanations for predictions with convolutional neural networks. In: 2019 IEEE international conference on data mining (ICDM), pp 952–957. IEEE

  16. Lai G, Chang W-C, Yang Y, Liu H (2018) Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st international ACM SIGIR conference on research & development in information retrieval, pp 95–104

  17. Wen C, Liu S, Yao X, Peng L, Li X, Hu Y, Chi T (2019) A novel spatiotemporal convolutional long short-term neural network for air pollution prediction. Sci Total Environ 654:1091–1099

    Article  Google Scholar 

  18. Qin Y, Song D, Chen H, Cheng W, Jiang G, Cottrell G (2017) A dual-stage attention-based recurrent neural network for time series prediction. arXiv:1704.02971

  19. Yin C, Dai Q (2022) A deep multivariate time series multistep forecasting network. Appl Intell 52(8):8956–8974

    Article  Google Scholar 

  20. Box GE, Jenkins GM, Reinsel GC, Ljung GM (2015) Time series analysis: Forecasting and Control

  21. Geetha A, Nasira GM (2016) Time-series modelling and forecasting: modelling of rainfall prediction using arima model. Int J Soc Syst Sci 8(4):361–372

    Google Scholar 

  22. Vapnik V, Golowich S, Smola A (1996) Support Vector method for function approximation, regression estimation and signal processing. Adv Neural Inf Process Syst, 9

  23. Yang H, Huang K, King I, Lyu MR (2009) Localized support vector regression for time series prediction. Neurocomputing 72(10-12):2659–2669

    Article  Google Scholar 

  24. Ong BT, Sugiura K, Zettsu K (2016) Dynamically pre-trained deep recurrent neural networks using environmental monitoring data for predicting pm2. 5. Neural Comput & Applic 27(6):1553–1566

    Article  Google Scholar 

  25. Liu F, Cai M, Wang L, Lu Y (2019) An ensemble model based on adaptive noise reducer and over-fitting prevention lstm for multivariate time series forecasting. IEEE Access 7:26102– 26115

    Article  Google Scholar 

  26. Liu X, Liu Q, Zou Y, Wang G (2018) A self-organizing lstm-based approach to pm2. 5 forecast. In: International conference on cloud computing and security, pp 683–693. Springer

  27. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555

  28. Tan Q, Ye M, Yang B, Liu S, Ma AJ, Yip TC-F, Wong GL-H, Yuen P (2020) Data-gru: Dual-attention time-aware gated recurrent unit for irregular multivariate time series. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 930– 937

  29. Huang G, Li X, Zhang B, Ren J (2021) Pm2. 5 concentration forecasting at surface monitoring sites using gru neural network based on empirical mode decomposition, vol 768

  30. Shih S, Sun F, Lee H (2019) Temporal pattern attention for multivariate time series forecasting. Mach. Learn 108(8-9):1421–1441

    Article  MathSciNet  MATH  Google Scholar 

  31. He H, Zhang Q, Bai S, Yi K, Niu Z (2022) Catn: cross attentive tree-aware network for multivariate time series forecasting

  32. Liu Y, Gong C, Yang L, Chen Y (2020) DSTP-RNN: a dual-stage two-phase attention-based recurrent neural network for long-term and multivariate time series prediction. Expert Syst Appl. 143

  33. Liang Y, Meng F, Zhang J, Xu J, Chen Y, Zhou J (2019) A novel aspect-guided deep transition model for aspect based sentiment analysis. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pp 5568–5579

  34. Chang Y, Sun F, Wu Y, Lin S (2018) A memory-network based solution for multivariate time-series forecasting. arXiv:1809.02105

Download references

Funding

This work was partially supported by the National Natural Science Foundation of China (NSFC) [No.61876071] and Scientific and Technological Developing Scheme of Jilin Province [No.20180201003SF, No.20190701031GH] and Energy Administration of Jilin Province [No.3D516L921421].

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130022, China

    Yimeng Wang, Shi Feng, Bing Wang & Jihong Ouyang

  2. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130022, China

    Yimeng Wang, Shi Feng, Bing Wang & Jihong Ouyang

Authors
  1. Yimeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shi Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jihong Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis are performed by Yiming Wang. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jihong Ouyang.

Ethics declarations

Ethics Approval

This research does not involvehuman participants nor animals.

Consent to participate

Consent to submit this manuscript has been received tacitly from the author’s institution, Jilin University.

Consent for Publication

All authors are consent for publication.

Competing interests

The authors have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Feng, S., Wang, B. et al. Deep transition network with gating mechanism for multivariate time series forecasting. Appl Intell 53, 24346–24359 (2023). https://doi.org/10.1007/s10489-023-04503-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04503-w

Keywords

Search

Navigation