Skip to main content
SpringerLink
Log in

A Hybrid Modeling Method Based on Linear AR and Nonlinear DBN-AR Model for Time Series Forecasting

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Improving time series forecasting accuracy is an important work for decision makers. A single model applied on a data series may not obtain satisfactory prediction accuracy. Both theoretical and empirical findings have indicated that integration of linear model and nonlinear model may provide more accurate prediction than an individual linear or nonlinear model. This paper presents a hybrid modeling approach that combines a linear autoregressive (AR) model and a nonlinear deep belief network-based autoregressive (DBN-AR) model for time series forecasting. The proposed modeling approach first applies an AR model to fit time series data, and the error between the original date and the AR model forecast data as a nonlinear component is considered, and then the error is modeled by a DBN-AR model. DBaN-AR model is a modeling method for nonlinear time series, which uses a set of deep belief networks to approximate the state-dependent functional coefficients of state dependent auto-regressive model. The proposed hybrid model can overcome limitation of an individual model and obtain more general and more accurate forecasting result than some existing hybrid models. The experiment results demonstrate that the MSE of modeling residuals using the proposed hybrid model is largely reduced compared with the results of some single prediction models and some hybrid models for one-step-ahead and multistep-ahead forecast.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig.6
Fig. 7
Fig.8
Fig.9
Fig. 10
Fig.11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Khashei M, Bijari M (2011) A novel hybridization of artificial neural networks and ARIMA models for time series forecasting. Appl Soft Comput 11(2):2664–2675

    Article  Google Scholar 

  2. Khashei M, Bijari M, Ardali GAR (2009) Improvement of auto-regressive integrated moving average models using fuzzy logic and artificial neural networks (ANNs). Neurocomputing 72(4–6):956–967

    Article  Google Scholar 

  3. Wang D, Guo H, Luo H, Grunder O, Lin Y (2017) Multi-step-ahead electricity price forecasting using a hybrid model based on two-layer decomposition technique and BP neural network optimized by firefly algorithm. Appl Energy 190:390–407

    Article  Google Scholar 

  4. Lei M, Luan S, Jiang C, Liu H, Yan Z (2009) A review on the forecasting of wind speed and generated power. Renew Sustain Energy Rev 13(4):915–920

    Article  Google Scholar 

  5. Shen S, Shen Y (2016) ARIMA: an applied time series forecasting model for the bovespa stock index. J Comput Commun 5(21):3383–3391

    Google Scholar 

  6. Takeda H, Tamura Y, Sato S (2016) Using the ensemble Kalman filter for electricity load forecasting and analysis. Energy 104:184–198

    Article  Google Scholar 

  7. Bengio Y (2009) Learning deep architectures for AI. Found Trends® Mach Learn 2(1):1–127

    Article  MathSciNet  Google Scholar 

  8. Lv Y, Duan Y, Kang W, Li Z, Wang FY (2015) Traffic flow prediction with big data: a deep learning approach. IEEE Trans Intell Transp Syst 16(2):865–873

    Google Scholar 

  9. Hinton GE, Osindero S, Teh YW (2014) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  Google Scholar 

  10. Qin M, Du Z, Du Z (2017) Red tide time series forecasting by combining ARIMA and deep belief network. Knowl-Based Syst 125:39–52

    Article  Google Scholar 

  11. Qiu X, Ren Y, Suganthan PN, Amaratunga GAJ (2017) Empirical mode decomposition based ensemble deep learning for load demand time series forecasting. Appl Soft Comput 54:246–255

    Article  Google Scholar 

  12. Yin Y, Shang P (2016) Forecasting traffic time series with multivariate predicting method. Appl Math Comput 291:266–278

    MathSciNet  MATH  Google Scholar 

  13. Shen F, Chao J, Zhao J (2015) Forecasting exchange rate using deep belief networks and conjugate gradient method. Neurocomputing 167:243–253

    Article  Google Scholar 

  14. Dong Y, Liu Y, Lian S (2016) Automatic age estimation based on deep learning algorithm. Neurocomputing 187:4–10

    Article  Google Scholar 

  15. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504

    Article  MathSciNet  Google Scholar 

  16. Collobert R, Weston J (2008) A unified architecture for natural language processing: deep neural networks with multitask learning. In: International conference 160–167

  17. Zhou J, Jing S, Gong L (2011) Fine tuning support vector machines for short-term wind speed forecasting. Energy Convers Manag 52(4):1990–1998

    Article  Google Scholar 

  18. Babu CN, Reddy BE (2014) A moving-average filter based hybrid arima–ann model for forecasting time series data. Appl Soft Comput 23(10):27–38

    Article  Google Scholar 

  19. Li S, Wang P, Goel L (2016) A novel wavelet-based ensemble method for short-term load forecasting with hybrid neural networks and feature selection. IEEE Trans Power Syst 31(3):1788–1798

    Article  Google Scholar 

  20. Ardalani-Farsa M, Zolfaghari S (2010) Chaotic time series prediction with residual analysis method using hybrid Elman-NARX neural networks. Neurocomputing 73(13–15):2540–2553

    Article  Google Scholar 

  21. Bai Y, Chen Z, Xie J, Li C (2016) Daily reservoir inflow forecasting using multiscale deep feature learning with hybrid models. J Hydrol 532:193–206

    Article  Google Scholar 

  22. Zhang GP (2003) Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50(1):159–175

    Article  Google Scholar 

  23. Priestley MB (1980) State-dependent models: a general approach to non-linear time series analysis. J Time 1(1):47–71

    MathSciNet  MATH  Google Scholar 

  24. Vesin J (1993) An amplitude-dependent autoregressive signal model based on a radial basis functions expansion. In: Proc. IEEE Int. Conf. on acoustics, speech and signal processing, ICASSP"93, Minneapolis, USA, 3

  25. Shi Z, Tamura Y, Ozaki T (1999) Nonlinear time series modelling with the radial basis function-based state-dependent autoregressive model. Int J Syst Sci 30(7):717–727

    Article  Google Scholar 

  26. Peng H, Ozaki T, Hagganozaki V, Toyoda Y (2003) A parameter optimization method for radial basis function type models. IEEE Trans Neural Networks 14(2):432–438

    Article  Google Scholar 

  27. Gan M, Peng H, Peng X, Chen X, Inoussa G (2010) A locally linear RBF network-based state-dependent AR model for nonlinear time series modeling. Inf Sci 180(22):4370–4383

    Article  MathSciNet  Google Scholar 

  28. Gan M, Peng H, Chen L (2012) A global–local optimization approach to parameter estimation of RBF-type models. Inf Sci 197:144–160

    Article  Google Scholar 

  29. Zhou F, Peng H, Qin Y, Zeng X, Tian X, Xu W (2017) A RBF-ARX model-based robust MPC for tracking control without steady state knowledge. J Process Control 51:42–54

    Article  Google Scholar 

  30. Zhou F, Peng H, Qin Y, Zeng X, Xie W, Wu J (2015) RBF-ARX model-based MPC strategies with application to a water tank system. J Process Control 34:97–116

    Article  Google Scholar 

  31. Xu W, Peng H, Zeng X, Zhou F, Tian X, Peng X (2019) Deep belief network-based AR model for nonlinear time series forecasting. Appl Soft Comput 77:605–621

    Article  Google Scholar 

  32. Chen GY, Gan M, Chen CLP, Li HX (2019) A regularized variable projection algorithm for separable nonlinear least squares problems. IEEE Trans Autom Control 64(2):526–537

    MathSciNet  MATH  Google Scholar 

  33. Gan M, Chen CLP, Li HX, Chen L (2015) Gradient radial basis function based varying-coefficient autoregressive model for nonlinear and nonstationary time series. IEEE Signal Process Lett 22(7):809–812

    Article  Google Scholar 

  34. Gan M, Chen CLP, Chen GY, Chen L (2018) On some separated algorithms for separable nonlinear squares problems. IEEE Trans Cybern 48(10):2866–2874

    Article  Google Scholar 

  35. SIDC (World Data Center for the Sunspot Index). http://side.oma.be/index.php3

  36. AEMO, (2013) Australian energy market operator. http://www.aemo.com.au/

  37. Akaike H (1969) Fitting autoregressive models for prediction. Ann Inst Stat Math 21(1):243–247

    Article  MathSciNet  Google Scholar 

  38. Jang RJS, Gulley N (2000) Fuzzy logic toolbox user’s guide. The Math Works Inc, Natick, MA

    Google Scholar 

  39. Sello S (2001) Solar cycle forecasting: a nonlinear dynamics approach. Astron Astrophys 377(1):312–320

    Article  Google Scholar 

  40. Gholipour A, Araabi BN, Lucas C (2006) Predicting chaotic time series using neural and neurofuzzy models: a comparative study. Neural Process Lett 24(3):217–239

    Article  Google Scholar 

  41. Wang D, Luo H, Grunder O, Lin Y, Guo H (2017) Multi-step ahead electricity price forecasting using a hybrid model based on two layer decomposition technique an BP neural network optimized by firefly algorithm. Appl Energy 190:390–407

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the editors and referees for their valuable comments and suggestions, which substantially improved the original manuscript. This research was supported by the National Natural Science Foundation of China (61773402, 51575167, 61540037, and 71271215), the key projects of natural science research in colleges and universities of Anhui Province (Grant No. KJ2020A0508), and the Anhui Provincial Natural Science Foundation (2008085MF197, 1908085MF195).

Author information

Authors and Affiliations

  1. School of Automation, Central South University, Changsha, 410083, Hunan, China

    Wenquan Xu, Hui Peng, Xiaoyong Zeng & Xiaoying Tian

  2. School of Physics and Electrical Engineering, Anqing Normal University, Anqing, 246001, Anhui, China

    Wenquan Xu

  3. College of Electronic Information and Electrical Engineering, Changsha University, Changsha, 410003, Hunan, China

    Feng Zhou

  4. College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, Hunan, China

    Xiaoyan Peng

Authors
  1. Wenquan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyong Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoying Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaoyan Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Peng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A

Appendix A

1.1 Nomenclature

DBN:

Deep belief network

SD-AR:

State dependent auto-regressive

DBN-AR:

Deep belief network-based autoregressive

AR:

Auto-regressive

AI:

Artificial intelligence

ARMA:

Auto-regressive moving average

ARIMA:

Auto-regressive integrated moving average

ANN:

Artificial neural network

SVM:

Support vector machine

ELM:

Extreme learning machine

DL:

Deep learning

RBM:

Restricted boltzmann machines

NARX:

Nonlinear autoregressive model with eXogenous

RBF:

Radial basis function

LLRBF:

Locally linear RBF

AEMO:

Australian energy market operator

LSM:

Least square method

AIC:

Akaike information criteria

MSE:

Mean squared errors

RMSE:

Root mean square error

NMSE:

Normalized mean squared error

SIDC:

World data center for the sunspot index

AEMO:

Australian energy market operator

VIC:

Victoria

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, W., Peng, H., Zeng, X. et al. A Hybrid Modeling Method Based on Linear AR and Nonlinear DBN-AR Model for Time Series Forecasting. Neural Process Lett 54, 1–20 (2022). https://doi.org/10.1007/s11063-021-10651-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10651-2

Keywords

Search

Navigation