Skip to main content
SpringerLink
Log in

A hybrid modelling method for time series forecasting based on a linear regression model and deep learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Time series forecasting has important theoretical significance and engineering application value. A number of studies have shown that hybrid modelling is very successful in various modelling applications, and both theoretical and empirical findings have shown that hybrid modelling is an effective method to improve the accuracy of time series models. This paper proposes a hybrid model that combines a linear regression (LR) model and deep belief network (DBN) model for the prediction of time series data. In the hybrid model, the linear AR (auto-regression) LR model or ARIMA (auto-regressive integrated moving average) model and the nonlinear DBN model are explored to capture the linear and nonlinear behaviours of a time series, respectively. We first use an LR model to fit the original data and obtain the LR model residuals between the original data and the predicted data of the LR model. Then, the residuals are regarded as the nonlinear component and are used as inputs into the DBN model. The LR model prediction and the output of the DBN model are the final forecasting value for the time series, which takes full advantage of the two models for predicting time series. The proposed hybrid model and other existing models are applied to four well-known time series for comparison, and the results show that the proposed hybrid model has a high prediction accuracy and may be a useful tool for time series forecasting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Babu CN, Reddy BE (2014) A moving-average filter based hybrid Arima–ann model for forecasting time series data. Appl Soft Comput 23(10):27–38

    Article  Google Scholar 

  2. Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  3. Dedinec A, Filiposka S, Dedinec A, Kocarev L (2016) Deep belief network based electricity load forecasting: an analysis of macedonian case. Energy 115:1688–1700

    Article  Google Scholar 

  4. Hrasko R, Pacheco AGC, Krohling RA (2015) Time series prediction using restricted boltzmann machines and backpropagation. Procedia Comput Sci 55:990–999

    Article  Google Scholar 

  5. Vu DH, Muttaqi KM, Agalgaonkar AP, Bouzerdoum A, Vu DH, Muttaqi KM (2017) Short-term electricity demand forecasting using autoregressive based time varying model incorporating representative data adjustment. Appl Energy 205:790–801

    Article  Google Scholar 

  6. Wang JJ, Wang JZ, Zhang ZG, Guo SP (2012) Stock index forecasting based on a hybrid model. Omega 40(6):758–766

    Article  Google Scholar 

  7. Lydia M, Kumar SS, Selvakumar AI, Kumar GEP (2016) Linear and non-linear autoregressive models for short-term wind speed forecasting. Energy Convers Manag 112:115–124

    Article  Google Scholar 

  8. Akouemo HN, Povinelli RJ (2017) Data improving in time series using ARX and ANN models. IEEE Trans Power Syst 32(5):3352–3359

    Article  Google Scholar 

  9. Choubin B, Malekian A (2017) Combined gamma and m-test-based ANN and ARIMA models for groundwater fluctuation forecasting in semiarid regions. Environ Earth Sci 76(15):538

    Article  Google Scholar 

  10. Suk HI, Wee CY, Lee SW, Shen D (2016) State-space model with deep learning for functional dynamics estimation in resting-state fmri. Neuroimage 129:292–307

    Article  Google Scholar 

  11. Pektaş AO, Cigizoglu HK (2013) ANN hybrid model versus ARIMA and ARIMAX models of runoff coefficient. J Hydrol 500(11):21–36

    Article  Google Scholar 

  12. Kuremoto T, Kimura S, Kobayashi K, Obayashi M (2014) Time series forecasting using a deep belief network with restricted boltzmann machines. Neurocomputing 137(15):47–56

    Article  Google Scholar 

  13. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  MATH  Google Scholar 

  14. Shen F, Chao J, Zhao J (2015) Forecasting exchange rate using deep belief networks and conjugate gradient method. Neurocomputing 167:243–253

    Article  Google Scholar 

  15. Huang HB, Li RX, Yang ML, Lim TC, Ding WP (2017) Evaluation of vehicle interior sound quality using a continuous restricted boltzmann machine-based dbn. Mech Syst Signal Process 84:245–267

    Article  Google Scholar 

  16. Abdel-Zaher AM, Eldeib AM (2016) Breast cancer classification using deep belief networks. Expert Syst Appl 46:139–144

    Article  Google Scholar 

  17. Yousefi-Azar M, Hamey L (2017) Text summarization using unsupervised deep learning. Expert Syst Appl 68:93–105

    Article  Google Scholar 

  18. Huang Z, Siniscalchi SM, Lee CH (2016) A unified approach to transfer learning of deep neural networks with applications to speaker adaptation in automatic speech recognition. Neurocomputing 218:448–459

    Article  Google Scholar 

  19. Ma X, Tao Z, Wang Y, Yu H, Wang Y (2015) Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transp Res C 54:187–197

    Article  Google Scholar 

  20. Zheng H, Yuan J, Chen L (2017) Short-term load forecasting using EMD-LSTM neural networks with a Xgboost algorithm for feature importance evaluation. Energies 10(8):1168

    Article  Google Scholar 

  21. Probst M, Rothlauf F, Grahl J (2017) Scalability of using restricted boltzmann machines for combinatorial optimization. Eur J Oper Res 256(2):368–383

    Article  MathSciNet  MATH  Google Scholar 

  22. Dahl GE, Yu D, Deng L, Acero A (2012) Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Trans Audio Speech Lang Process 20(1):30–42

    Article  Google Scholar 

  23. Memisevic R, Hinton GE (2014) Learning to represent spatial transformations with factored higher-order boltzmann machines. Neural Comput 22(6):1473–1492

    Article  MATH  Google Scholar 

  24. Khashei M, Bijari M (2011) A novel hybridization of artificial neural networks and ARIMA models for time series forecasting. Appl Soft Comput J 11(2):2664–2675

    Article  Google Scholar 

  25. Zhang GP (2003) Time series forecasting using a hybrid ARIMA ARIMA and neural network model. Neurocomputing 50(1):159–175

    Article  MATH  Google Scholar 

  26. Gan M, Peng H, Peng X, Chen X, Inoussa G (2010) A locally linear RBF network-based state-dependent AR model for nonlinear time series modeling. Inf Sci 180(22):4370–4383

    Article  MathSciNet  Google Scholar 

  27. Liu H, Tian HQ, Li YF (2012) Comparison of two new Arima ARIMA-ANN and ARIMA-Kalman hybrid methods for wind speed prediction. Appl Energy 98(1):415–424

    Article  Google Scholar 

  28. Zhu B, Chevallier J (2013) Carbon price forecasting with a hybrid Arima and least squares support vector machines methodology. Omega 41(3):517–524

    Article  Google Scholar 

  29. Nourani V, Baghanam AH, Adamowski J, Kisi O (2014) Applications of hybrid wavelet–artificial intelligence models in hydrology: a review. J Hydrol 514:358–377

    Article  Google Scholar 

  30. Shukur OB, Lee MH (2015) Daily wind speed forecasting through hybrid KF-ANN model based on ARIMA. Renew Energy 76:637–647

    Article  Google Scholar 

  31. Barak S, Sadegh SS (2016) Forecasting energy consumption using ensemble ARIMA-ANFIS hybrid algorithm. Int J Electr Power Energy Syst 82:92–104

    Article  Google Scholar 

  32. Qiu X, Ren Y, Suganthan PN, Amaratunga GAJ (2017) Empirical mode decomposition based ensemble deep learning for load demand time series forecasting. Appl Soft Comput 54(C):246–255

    Article  Google Scholar 

  33. Qiu X, Zhang L, Ren Y, Suganthan PN, Amaratunga G (2015) Ensemble deep learning for regression and time series forecasting. Comput Intell Ensemble Learn 1–6

  34. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  35. Haykin S (1999) Neural networks: a comprehensive foundation, International Edition, Prentice Hall

  36. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MATH  Google Scholar 

  37. Ye L, Liu P (2011) Combined model based on EMD-SVM for short-term wind powerprediction. In: Proc. Chinese Society for Electrical Engineering (CSEE) 31:102–108

  38. Liu H, Chen C, Tian H, Li Y (2012) A hybrid model for wind speed prediction usingempirical mode decomposition and artificial neural networks. Renew Energy 48:545–556

    Article  Google Scholar 

  39. Salakhutdinov R, Hinton G (2007) Using deep belief nets to learn covariance kernels for Gaussian processes. International Conference on Neural Information Processing Systems 1249–1256

  40. Jang RJS, Gulley N (2000) Fuzzy logic toolbox user’s guide. The Math Works, Inc., Natick

    Google Scholar 

  41. Shi Z, Tamura Y, Ozaki T (1999) Nonlinear time series modelling with the radial basis function-based state-dependent autoregressive model. Int J Syst Sci 30(7):717–727

    Article  MATH  Google Scholar 

  42. Peng H, Ozaki T, Haggan-Ozaki V, Toyoda Y (2003) A parameter optimization method for radial basis function type models. IEEE Trans Neural Netw 14(2):432–438

    Article  MATH  Google Scholar 

  43. Hipel KW, Mcleod TI (1994) Time series modelling of water resources and environmental systems. J Hydrol 167(1–4):399–400

    Google Scholar 

  44. SIDC (World Data Center for the Sunspot Index), http://sidc.oma.be/index.php3

  45. Ardalani-Farsa M, Zolfaghari S (2010) Chaotic time series prediction with residual analysis method using hybrid Elman–NARX neural networks. Neurocomputing 73(13):2540–2553

    Article  Google Scholar 

  46. Gholipour A, Araabi BN, Lucas C (2006) Predicting chaotic time series using neural and neurofuzzy models: a comparative study. Neural Process Lett 24(3):217–239

    Article  Google Scholar 

  47. Ma QL, Zheng QL, Peng H, Zhong TW, Xu LQ (2007) Chaotic time series prediction based on evolving recurrent neural networks. Int Conf Mach Learn Cybern 3496–3500

  48. Chandra R, Zhang M (2012) Cooperative coevolution of elman recurrent neural networks for chaotic time series prediction. Neurocomputing 86(12):116–123

    Article  Google Scholar 

  49. Teo KK, Wang L, Lin Z (2001) Wavelet packet multi-layer perceptron for chaotic time series prediction: effects of weight initialization. Comput Sci 2074:310–317

    MATH  Google Scholar 

  50. Mcnish AG, Lincoln JV (1949) Prediction of sunspot numbers. Eos Trans Am Geophys Union 30(5):673–685

    Article  Google Scholar 

  51. Sello S (2001) Solar cycle forecasting: a nonlinear dynamics approach. Astron Astrophys 377(1):312–320

    Article  Google Scholar 

  52. Denkmayr K, Cugnon P (1997) About sunspot number medium-term predictions. In: Heckman G et al. (Ed.), Solar-Terrestrial Prediction Workshop V, Hiraiso Solar Terrestrial Research Center, Japan, 103

  53. Koskela T, Lehtokangas M, Saarinen J, Kaski K (1996) Time series prediction with multilayer perceptron, FIR and Elman neural networks. In: Proceedings of the World Congress on Neural Networks 491–496

  54. Lichman M (2013) UCI machine learning repository Irvine. University of California. Irvine, CA, USA. [Online]. http://archive.ics.uci.edu/ml

  55. AEMO, Australian Energy Market Operator (2013). http://www.aemo.com.au/

  56. Qiu X, Suganthan PN, Amaratunga GAJ (2018) Ensemble incremental learning random vector functional link network for short-term electric load forecasting. Knowl-Based Syst 145:182–196

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the editors and referees for their valuable comments and suggestions, which substantially improved the original manuscript. This research was supported by the National Natural Science Foundation of China (61773402, 51575167, 61540037, and 71271215).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Central South University, Changsha, 410083, Hunan, China

    Wenquan Xu, Hui Peng, Xiaoyong Zeng & Xiaoying Tian

  2. School of Physics and Electronic Engineering, Anqing Normal University, Anqing, 246001, Anhui, China

    Wenquan Xu

  3. College of Electronic Information and Electrical Engineering, Changsha University, Changsha, 410003, Hunan, China

    Feng Zhou

  4. College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, Hunan, China

    Xiaoyan Peng

Authors
  1. Wenquan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyong Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoying Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaoyan Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Peng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, W., Peng, H., Zeng, X. et al. A hybrid modelling method for time series forecasting based on a linear regression model and deep learning. Appl Intell 49, 3002–3015 (2019). https://doi.org/10.1007/s10489-019-01426-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01426-3

Keywords

Search

Navigation