Skip to main content
SpringerLink
Log in

Industrial time series forecasting based on improved Gaussian process regression

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Industrial processes often include shifting operating phases and dynamics, and system uncertainty. Industrial time series data may obey different distributions because of the time-varying characteristic. Therefore, a single global model cannot describe the local characteristics of multiple distributions. In this work, a hybrid GMM-IGPR model is proposed to solve this kind of time series prediction problem by using an improved Gaussian process regression (GPR) based on Gaussian mixture model (GMM) and a variant of the basic particles swarm optimization (PSO). In a first treatment to the time series, different distributions of the original dataset are characterized by adopting the GMM as a cluster method. Then, multiple localized GPR models are built to characterize the different properties between inputs and output within various clusters. In order to optimize the proposed algorithms, this paper utilizes the DEPSO which introduces differential evolution (DE) operator into the basic PSO algorithm to estimate hyperparameters of the GPR model, instead of using the traditional conjugate gradient (CG) method. Lastly, the Bayesian inference strategy is used to estimate the posterior probabilities of the test data with respect to different clusters. The various localized GPR models are integrated through these posterior probabilities as the weightings so that a global predictive model is developed for the final prediction. The effectiveness of the proposed algorithm is verified by means of a numerical example and a real industrial winding process. Statistical tests of experimental results compared with other popular prediction models demonstrate the good performance of the proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Aye SA, Heyns PS (2017) An integrated Gaussian process regression for prediction of remaining useful life of slow speed bearings based on acoustic emission. Mech Syst Signal Process 84:485–498

    Article  Google Scholar 

  • Bastogne T, Noura H, Richard A, Hittinger JM (1997) Application of subspace methods to the identification of a winding process. In: IEEE European control conference (ECC), pp 2168–2173

  • Bhowmik S, Paul A, Panua R, Ghosh SK, Debroy D (2018) Performance-exhaust emission prediction of diesosenol fueled diesel engine: an ANN coupled MORSM based optimization. Energy 153:212–222

    Article  Google Scholar 

  • Bishop CM (2006) Pattern recognition and machine learning. Springer, New York

    MATH  Google Scholar 

  • Cappé O (2011) Online expectation-maximisation. In: Mengersen K, Robert C, Titterington M (eds) Mixtures: estimation and applications. Wiley, New York, pp 1–53

    Google Scholar 

  • Cappé O, Moulines E (2009) On-line expectation-maximization algorithm for latent data models. J R Stat Soc B 71(3):593–613

    Article  MathSciNet  MATH  Google Scholar 

  • DaISy: Database for the Identification of Systems (2006). In: De Moor BLR (ed) Department of Electrical Engineering, ESAT/STADIUS. KU Leuven, Belgium. http://homes.esat.kuleuven.be/~smc/daisy/daisydata.html

  • Demšar J (2006) Statistical comparisions of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  • Elguebaly T, Bouguila N (2015) Simultaneous high-dimensional clustering and feature selection using asymmetric Gaussian mixture models. Image Vis Comput 34:27–41

    Article  Google Scholar 

  • Ferlito S, Adinolfi G, Graditi G (2017) Comparative analysis of data-driven methods online and offline trained to the forecasting of grid-connected photovoltaic plant production. Appl Energy 205:116–129

    Article  Google Scholar 

  • Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32:675–701

    Article  MATH  Google Scholar 

  • Friedman M (1940) A comparison of alternative tests of significance for the problem of \(m\) rankings. Ann Math Stat 11(1):86–92

    Article  MathSciNet  MATH  Google Scholar 

  • General Electric Intelligent Platforms, (2012). The Rise of Industrial Big Data, 2012, White Paper

  • Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Boca Raton

    MATH  Google Scholar 

  • Grbić R, Slišković D, Kadlec P (2013) Adaptive soft sensor for online prediction and process monitoring based on a mixture of Gaussian process models. Comput Chem Eng 58:84–97

    Article  Google Scholar 

  • Gregorčič G, Lightbody G (2009) Gaussian process approach for modelling of nonlinear systems. Eng Appl Artif Intell 22:522–533

    Article  MATH  Google Scholar 

  • He F, Li M, Wang BJ (2016) Mult-mode acid concentration prediction models of cold-rolled strip steel pickling process. J Process Control 24:916–923

    Google Scholar 

  • Herp J, Ramezani MH, Bach-Andersen M, Pedersen NL, Nadimi ES (2018) Bayesian state prediction of wind turbine bearing failure. Renew Energy 116:164–172

    Article  Google Scholar 

  • Higashi N, Iba H (2003) Particle swarm optimization with Gaussian mutation. In: IEEE swarm intelligence symposium, pp 72–79

  • Jin H, Chen X, Wang L, Yang K, Wu L (2015) Adaptive soft sensor development based on online ensemble Gaussian process regression for nonlinear time-varying batch processes. Ind Eng Chem Res 54(30):7320–7345

    Article  Google Scholar 

  • Joachim R (1997) The permutation distribution of the Friedman test. Comput Stat Data Anal 26:83–99

    Article  MathSciNet  MATH  Google Scholar 

  • Kennedy J (1997) The particle swarm: social adaptation of knowledge. In: IEEE international conference on evolutionary computation, pp 303–308

  • Kennedy J, Eberhart RC (1994) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks, Perth, Australia 4:1942–1948

  • Liu Y, Chen JH (2013) Integrated soft sensor using just-in-time support vector regression and probabilistic analysis for quality prediction of multi-grade processes. J Process Control 23:793–804

    Article  Google Scholar 

  • Liu L, Wang Q, Wang J, Liu M (2016) A rolling grey model optimized by particle swarm optimization in economic prediction. Comput Intell 32(3):391–419

    Article  MathSciNet  Google Scholar 

  • López C, Zhong W, Zheng ML (2017) Short-term electric load forecasting based on wavelet neural network, particle swarm optimization and ensemble empirical mode decomposition. Energy Proc 105:3677–3682

    Article  Google Scholar 

  • Martínez F, Frías MP, Pérez-Godoy MD, Rivera AJ (2018) Dealing with seasonality by narrowing the training set in time series forecasting with kNN. Expert Syst Appl 103:38–48

    Article  Google Scholar 

  • Nabney IT (2002) NETLAB algorithms for pattern recognition. Springer, Great Britain

    MATH  Google Scholar 

  • Neal RM, Hinton GE (1998) A view of the EM algorithm that justifies incremental, sparse, and other variants Learning in graphical models. In: Jordan MI (ed) Learning in graphical models, NATO ASI series, vol 89. Springer, Netherlands, pp 355–368

    Chapter  Google Scholar 

  • Nowakowska E, Koronacki J, Lipovetsky S (2015) Clusterability assenssment for Gaussian mixure models. Appl Math Comput 256:591–601

    MathSciNet  MATH  Google Scholar 

  • Pradeepkumar D, Ravi D (2017) Forecasting financial time series volatility using particle swarm optimization trained quantile regression neural network. Appl Soft Comput 58:35–52

  • Ranjan R, Huang B, Fatehi A (2016) Robust Gaussian process modeling using EM algorihtm. J Process Control 42:125–136

    Article  Google Scholar 

  • Rasmussen CE, Nickisch H (2010) Gaussian processes for machine learning (GPML) toolbox. J Mach Learn Res 11:3011–3015

    MathSciNet  MATH  Google Scholar 

  • Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. The MIT Press, Cambridge

    MATH  Google Scholar 

  • Reynolds D (2015) Gaussian mixture models. Encycl Biometrics 741:827–832

    Article  Google Scholar 

  • Sato MA, Ishii S (2000) On-line EM algorithm for the normalized Gaussian network. Neural Comput 12(2):407–432

    Article  Google Scholar 

  • Schenker B, Agarwal M (1995) Prediction of infrequently measurable quantities in poorly modeled processes. J Process Control 5:329–339

    Article  Google Scholar 

  • Scrucca L (2016) Identifying connected components in Gaussian finite mixture models for clustering. Comput Stat Data Anal 93:5–17

    Article  MathSciNet  MATH  Google Scholar 

  • Shi Y, Eberhart RC (1998) Parameter selection in particle swarm optimization. In: International conference on evolutionary programming, pp 591–601

  • Singh P, Borah B (2014) Forecasting stock index price based on M-factors fuzzy time series and particle swarm optimization. Int J Approx Reason 55:812–833

    Article  MathSciNet  MATH  Google Scholar 

  • Sun AY, Wang D, Xu X (2014) Monthly streamflow forecasting using Gaussian process regression. J Hydrol 511:72–81

  • Sun W, Wang CF, Zhang CC (2017) Factor analysis and forecasting of CO\(_{2}\) emissions in Hebei, using extreme learning machine based on particle swarm optimization. J Clean Prod 162:1095–1101

    Article  Google Scholar 

  • Titterington DM (1984) Recursive parameter estimation using incomplete data. J R Stat Soc B 46:257–267

    MathSciNet  MATH  Google Scholar 

  • Tobias P, Malte K, Carl ER (2006) Nonstationary gaussian process regression using a latent extension of the input space. In: ISBA eighth world meeting on Bayesian statistics

  • Xie XF, Zhang WJ, Yang ZL (2002) A dissipative particle swarm optimization. In: IEEE congress on evolutionary computation, pp 1456–1461

  • Xu C, Liu BG, Liu KY, Guo JQ (2011) Intelligent analysis model of landslide displacement time series based on coupling PSO-GPR. Rock Soil Mech 32(6):1669–1675

    Google Scholar 

  • Xu J, Yamada K, Seikiya K, Tanaka R, Yamane Y (2014) Effect of different features to drill-wear prediction with back propagation neural network. Precis Eng 38(4):791–798

    Article  Google Scholar 

  • Yang K, Jin H, Chen X (2016) Soft sensor development for online quality prediction of industrial batch rubber mixing process using ensemble just-in-time Gaussian process regression models. Chemom Intell Lab 155:170–182

    Article  Google Scholar 

  • Yu J (2012) Online quality prediction of nonlinear and non-Gaussian chemical processes with shifting dynamics using finite mixture model based Gaussian process regression approach. Chem Eng Sci 82:22–30

    Article  Google Scholar 

  • Yu J, Chen K, Rashid MM (2013) A Bayesian model averaging based multi-kernel Gaussian process regression framework for nonlinear state estimation and quality prediction of multiphase batch processes with transient dynamics and uncertainty. Chem Eng Sci 93:96–109

    Article  Google Scholar 

  • Yuan X, Tan Q, Lei X, Yuan Y, Wu X (2017) Wind power prediction using hybrid autoregressive fractionally integrated moving average and least square support vector machine. Energy 129:122–137

    Article  Google Scholar 

  • Zhao J, Liu QL, Wang W, Pedrycz W, Cong L (2012) Hybrid neural prediction and optimized adjustment for coke oven gas system in steel industry. IEEE Trans Neural Netw Learn Syst 23:439–450

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the financial support of this research by the Natural Science Foundation of Jiangsu Province (Nos. BK20170500, BK20190876), Natural Science Foundation of China (Grant No.61773118), Natural Science Foundation of the Jiangsu Higher Education Institutions (Nos. 19KJB520063, 18KJB460032).

Author information

Authors and Affiliations

  1. School of Information Engineering, Yangzhou University, Yangzhou, 225127, People’s Republic of China

    Tianhong Liu

  2. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, People’s Republic of China

    Tianhong Liu

  3. Key Laboratory of Measurement and Control of CSE, Ministry of Education, School of Automation, Southeast University, Nanjing, 210096, People’s Republic of China

    Haikun Wei & Kanjian Zhang

  4. School of Mechanical Engineering, Yangzhou University, Yangzhou, 225127, People’s Republic of China

    Sixing Liu

Authors
  1. Tianhong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haikun Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sixing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kanjian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tianhong Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, T., Wei, H., Liu, S. et al. Industrial time series forecasting based on improved Gaussian process regression. Soft Comput 24, 15853–15869 (2020). https://doi.org/10.1007/s00500-020-04916-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-020-04916-6

Keywords

Search

Navigation