Skip to main content
SpringerLink
Log in

A WSFA-based adaptive feature extraction method for multivariate time series prediction

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In recent years, artificial neural networks (ANNs) have been successfully and widely used in multivariate time series prediction, but the accuracy of the prediction is significantly affected by the ANNs’ input. In order to determine the appropriate input for more accurate prediction, a weighted slow feature analysis-based adaptive feature extraction (WSFA-AFE) method is proposed for multivariate time series prediction. Firstly, the weighted SFA (WSFA) algorithm is developed to extract slow features weighted by their contributions. Then, an improved adaptive sliding window algorithm is designed to self-determine the historical information of slow features for input. Finally, the out-of-model performance of the WSFA-AFE method is verified by applying it to different ANN models with several benchmark data sets as well as a practical dataset in wastewater treatment process. The results indicate that a better modeling performance of ANNs for multivariate time series prediction can be obtained by the WSFA-AFE method, which can adaptively extract feature variables from the multivariate time series. Besides, the robustness of the proposed method is demonstrated as well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ragulskis M, Lukoseviciute K (2009) Non-uniform attractor embedding for time series forecasting by fuzzy inference systems. Neurocomputing 72:2618–2626. https://doi.org/10.1016/j.neucom.2008.10.010

    Article  Google Scholar 

  2. Yang C, Qiao J, Wang L, Zhu X (2019) Dynamical regularized echo state network for time series prediction. Neural Comput Appl 31:6781–6794. https://doi.org/10.1007/s00521-018-3488-z

    Article  Google Scholar 

  3. Song W, Fujimura S (2021) Capturing combination patterns of long- and short-term dependencies in multivariate time series forecasting. Neurocomputing 464:72–82. https://doi.org/10.1016/j.neucom.2021.08.100

    Article  Google Scholar 

  4. Li G, Jung JJ (2023) Deep learning for anomaly detection in multivariate time series: approaches, applications, and challenges. Inf Fusion 91:93–102. https://doi.org/10.1016/j.inffus.2022.10.008

    Article  Google Scholar 

  5. Fan J, Zhang K, Huang Y et al (2021) Parallel spatio-temporal attention-based TCN for multivariate time series prediction. Neural Comput Appl. https://doi.org/10.1007/s00521-021-05958-z

    Article  Google Scholar 

  6. Ben SA, Erradi A, Aly HA, Mohamed A (2021) Predicting COVID-19 cases using bidirectional LSTM on multivariate time series. Environ Sci Pollut Res 28:56043–56052. https://doi.org/10.1007/s11356-021-14286-7

    Article  Google Scholar 

  7. Hu J, Zheng W (2020) Multistage attention network for multivariate time series prediction. Neurocomputing 383:122–137. https://doi.org/10.1016/j.neucom.2019.11.060

    Article  Google Scholar 

  8. Tang Y, Song Z, Zhu Y et al (2022) A survey on machine learning models for financial time series forecasting. Neurocomputing 512:363–380. https://doi.org/10.1016/j.neucom.2022.09.003

    Article  Google Scholar 

  9. Grillenzoni C, Fornaciari M (2019) On-line peak detection in medical time series with adaptive regression methods. Econom Stat 10:134–150. https://doi.org/10.1016/j.ecosta.2018.07.002

    Article  Google Scholar 

  10. Zhao J, Deng F, Cai Y, Chen J (2019) Long short-term memory—fully connected (LSTM-FC) neural network for PM2.5 concentration prediction. Chemosphere 220:486–492. https://doi.org/10.1016/j.chemosphere.2018.12.128

    Article  Google Scholar 

  11. Khan MA, Etminani-Ghasrodashti R, Kermanshachi S et al (2022) Do ridesharing transportation services alleviate traffic crashes? A time series analysis. Traffic Inj Prev 23:333–338. https://doi.org/10.1080/15389588.2022.2074412

    Article  Google Scholar 

  12. Avendaño-Valencia LD, Chatzi EN (2019) Modelling long-term vibration monitoring data with Gaussian process time-series models. IFAC-PapersOnLine 52:26–31. https://doi.org/10.1016/j.ifacol.2019.12.343

    Article  Google Scholar 

  13. Bashir F, Wei HL (2018) Handling missing data in multivariate time series using a vector autoregressive model-imputation (VAR-IM) algorithm. Neurocomputing 276:23–30. https://doi.org/10.1016/j.neucom.2017.03.097

    Article  Google Scholar 

  14. Prado R, Molina F, Huerta G (2006) Multivariate time series modeling and classification via hierarchical VAR mixtures. Comput Stat Data Anal 51:1445–1462. https://doi.org/10.1016/j.csda.2006.03.002

    Article  MathSciNet  Google Scholar 

  15. Tang WH, Röllin A (2021) Model identification for ARMA time series through convolutional neural networks. Decis Support Syst. https://doi.org/10.1016/j.dss.2021.113544

    Article  Google Scholar 

  16. Chyon FA, Suman MNH, Fahim MRI, Ahmmed MS (2022) Time series analysis and predicting COVID-19 affected patients by ARIMA model using machine learning. J Virol Methods 301:114433. https://doi.org/10.1016/j.jviromet.2021.114433

    Article  Google Scholar 

  17. Qiao J, Wang L, Yang C, Gu K (2018) Adaptive Levenberg–Marquardt algorithm based echo state network for chaotic time series prediction. IEEE Access 6:10720–10732. https://doi.org/10.1109/ACCESS.2018.2810190

    Article  Google Scholar 

  18. Qiao J, Wang L, Yang C (2019) Adaptive lasso echo state network based on modified Bayesian information criterion for nonlinear system modeling. Neural Comput Appl 31:6163–6177. https://doi.org/10.1007/s00521-018-3420-6

    Article  Google Scholar 

  19. Eskandarian P, Mohasefi JB, Pirnejad H, Niazkhani Z (2022) A novel artificial neural network improves multivariate feature extraction in predicting correlated multivariate time series. Appl Soft Comput 128:109460. https://doi.org/10.1016/j.asoc.2022.109460

    Article  Google Scholar 

  20. Audibert J, Michiardi P, Guyard F et al (2022) Do deep neural networks contribute to multivariate time series anomaly detection? Pattern Recognit 132:108945. https://doi.org/10.1016/j.patcog.2022.108945

    Article  Google Scholar 

  21. Shi X, Hao K, Chen L et al (2022) Multivariate time series prediction of complex systems based on graph neural networks with location embedding graph structure learning. Adv Eng Informatics 54:101810. https://doi.org/10.1016/j.aei.2022.101810

    Article  Google Scholar 

  22. Chen Y, Xie Z (2022) Multi-channel fusion graph neural network for multivariate time series forecasting. J Comput Sci 64:101862. https://doi.org/10.1016/j.jocs.2022.101862

    Article  Google Scholar 

  23. Park H, Lee G, Lee K (2022) Dual recurrent neural networks using partial linear dependence for multivariate time series. Expert Syst Appl 208:118205. https://doi.org/10.1016/j.eswa.2022.118205

    Article  Google Scholar 

  24. Wang C, Xu S, Liu J et al (2022) Building an improved artificial neural network model based on deeply optimizing the input variables to enhance rutting prediction. Constr Build Mater 348:128658. https://doi.org/10.1016/j.conbuildmat.2022.128658

    Article  Google Scholar 

  25. Cui Z, Kang L, Li L et al (2022) A hybrid neural network model with improved input for state of charge estimation of lithium-ion battery at low temperatures. Renew Energy 198:1328–1340. https://doi.org/10.1016/j.renene.2022.08.123

    Article  Google Scholar 

  26. Jebli I, Belouadha FZ, Kabbaj MI, Tilioua A (2021) Prediction of solar energy guided by Pearson correlation using machine learning. Energy 224:120109. https://doi.org/10.1016/j.energy.2021.120109

    Article  Google Scholar 

  27. Mu Y, Liu X, Wang L (2018) A Pearson’s correlation coefficient based decision tree and its parallel implementation. Inf Sci 435:40–58. https://doi.org/10.1016/j.ins.2017.12.059

    Article  MathSciNet  Google Scholar 

  28. Ircio J, Lojo A, Mori U, Lozano JA (2020) Mutual information based feature subset selection in multivariate time series classification. Pattern Recognit. https://doi.org/10.1016/j.patcog.2020.107525

    Article  Google Scholar 

  29. Depizzol DB, Montalvão J, de Lima F, O, et al (2018) Feature selection for optical network design via a new mutual information estimator. Expert Syst Appl 107:72–88. https://doi.org/10.1016/j.eswa.2018.04.018

    Article  Google Scholar 

  30. Machado M, Reisen VA, Santos JM et al (2020) Use of multivariate time series techniques to estimate the impact of particulate matter on the perceived annoyance. Atmos Environ. https://doi.org/10.1016/j.atmosenv.2019.117080

    Article  Google Scholar 

  31. Guo L, Wu P, Lou S et al (2020) A multi-feature extraction technique based on principal component analysis for nonlinear dynamic process monitoring. J Process Control 85:159–172. https://doi.org/10.1016/j.jprocont.2019.11.010

    Article  Google Scholar 

  32. Sivertsen E, Thyholt K, Rustad T et al (2022) Analysing multivariate storage data of seafood spreads. A case study based on combining split-plot design, principal component analysis and partial least squares predictions. Food Control 131:108385. https://doi.org/10.1016/j.foodcont.2021.108385

    Article  Google Scholar 

  33. Huang G, Chen X, Li L et al (2020) Domain adaptive partial least squares regression. Chemom Intell Lab Syst 201:103986. https://doi.org/10.1016/j.chemolab.2020.103986

    Article  Google Scholar 

  34. Zhang H, Tian X, Deng X, Cao Y (2018) Batch process fault detection and identification based on discriminant global preserving kernel slow feature analysis. ISA Trans 79:108–126. https://doi.org/10.1016/j.isatra.2018.05.005

    Article  Google Scholar 

  35. Shang C, Yang F, Huang B, Huang D (2018) Recursive slow feature analysis for adaptive monitoring of industrial processes. IEEE Trans Ind Electron 65:8895–8905. https://doi.org/10.1109/TIE.2018.2811358

    Article  Google Scholar 

  36. Yuan X, Huang B, Wang Y et al (2018) Deep learning-based feature representation and its application for soft sensor modeling with variable-wise weighted SAE. IEEE Trans Ind Inform 14:3235–3243. https://doi.org/10.1109/TII.2018.2809730

    Article  Google Scholar 

  37. Wiskott L, Sejnowski TJ (2002) Slow feature analysis: unsupervised learning of invariances. Neural Comput 14:715–770. https://doi.org/10.1162/089976602317318938

    Article  Google Scholar 

  38. Huang J, Sun X, Yang X, Shardt YAW (2022) Active nonstationary variables selection based just-in-time co-integration analysis and slow feature analysis monitoring approach for dynamic processes. J Process Control 117:112–121. https://doi.org/10.1016/j.jprocont.2022.07.008

    Article  Google Scholar 

  39. Wiskott L (2003) Estimating driving forces of nonstationary time series with slow feature analysis. 1–8

  40. Shang C, Yang F, Gao X, Huang D (2015) Extracting latent dynamics from process data for quality prediction and performance assessment via slow feature regression. Proc Am Control Conf. https://doi.org/10.1109/ACC.2015.7170850

    Article  Google Scholar 

  41. Shih SY, Sun FK, Lee H, yi, (2019) Temporal pattern attention for multivariate time series forecasting. Mach Learn 108:1421–1441. https://doi.org/10.1007/s10994-019-05815-0

    Article  MathSciNet  Google Scholar 

  42. Farhi N, Kohen E, Mamane H, Shavitt Y (2021) Prediction of wastewater treatment quality using LSTM neural network. Environ Technol Innov 23:101632. https://doi.org/10.1016/j.eti.2021.101632

    Article  Google Scholar 

  43. Nikravesh AY, Ajila SA, Lung CH (2015) Towards an autonomic auto-scaling prediction system for cloud resource provisioning. In: Proceedings of 10th international symposium software of engineering adapt self-managing systerm SEAMS. https://doi.org/10.1109/SEAMS.2015.22

  44. Zhang Z, Ye L, Qin H et al (2019) Wind speed prediction method using shared weight long short-term memory network and Gaussian process regression. Appl Energy 247:270–284. https://doi.org/10.1016/j.apenergy.2019.04.047

    Article  Google Scholar 

  45. Tschumitschew K, Klawonn F (2017) Effects of drift and noise on the optimal sliding window size for data stream regression models. Commun Stat Theory Methods 46:5109–5132. https://doi.org/10.1080/03610926.2015.1096388

    Article  MathSciNet  Google Scholar 

  46. Yu J (2018) State of health prediction of lithium-ion batteries: multiscale logic regression and Gaussian process regression ensemble. Reliab Eng Syst Saf 174:82–95. https://doi.org/10.1016/j.ress.2018.02.022

    Article  Google Scholar 

  47. Fan L, Kodamana H, Huang B (2018) Identification of robust probabilistic slow feature regression model for process data contaminated with outliers. Chemom Intell Lab Syst 173:1–13. https://doi.org/10.1016/j.chemolab.2017.12.009

    Article  Google Scholar 

  48. Cao D, Chen Y, Chen J et al (2021) An improved algorithm for the maximal information coefficient and its application. R Soc Open Sci. https://doi.org/10.1098/rsos.201424

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China (2021ZD0112301), and National Natural Science Foundation of China (62173008, 62021003, and 61890930-5).

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Shuang Yang, Wenjing Li & Junfei Qiao

  2. Beijing Artificial Intelligence Institute, Beijing, 100124, China

    Shuang Yang, Wenjing Li & Junfei Qiao

  3. Engineering Research Center of Intelligence Perception and Autonomous Control, Ministry of Education, Beijing, 100124, China

    Shuang Yang, Wenjing Li & Junfei Qiao

  4. Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing, 100124, China

    Shuang Yang, Wenjing Li & Junfei Qiao

  5. Beijing Laboratory for Intelligent Environmental Protection, Beijing, 100124, China

    Shuang Yang, Wenjing Li & Junfei Qiao

Authors
  1. Shuang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenjing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junfei Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SY: Conceptualization, Methodology, Formal analysis, Writing—original draft preparation. WL: Writing—review and editing, Supervision, Funding acquisition. JQ: Resources, Supervision, Project administration.

Corresponding author

Correspondence to Junfei Qiao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, S., Li, W. & Qiao, J. A WSFA-based adaptive feature extraction method for multivariate time series prediction. Neural Comput & Applic 36, 1959–1972 (2024). https://doi.org/10.1007/s00521-023-09198-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09198-1

Keywords

Search

Navigation