Skip to main content

Advertisement

Spatio-temporal attention-based hybrid deep network for time series prediction of industrial process

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Industrial time series involves a large amount of production process information, which effectively reflects the production status of the industrial process. To better understand characteristics and patterns of changes in production conditions, it is crucial to analyze and predict industrial time series data. Given the involvement of numerous parameters and complex physical-chemical reactions in industrial processes, attaining precise predictive performance utilizing a single model remains a formidable challenge. In this paper, we propose a novel hybrid deep learning prediction method based on spatio-temporal attention and temporal convolution network. The proposed method aims to handle the multivariate coupling characteristics and dynamic nonlinear features in industrial time series through different model structures for accurate prediction. In this method, historical data are first segmented into multiple consecutive inputs along the temporal dimension, which are then used as inputs to the subsequent attention mechanism module. To realize the mapping from points to series in the temporal dimension, the segmented input is processed using both the adaptive attention mechanism and one-dimensional convolution. Then the spatio-temporal coupling features are further explored through the spatio-temporal attention model. In addition, to extract dynamic nonlinear features from historical data, a parallel temporal convolutional network with temporal pattern attention is utilized. In order to evaluate the prediction performance of the proposed model, we use two different real-world industrial time series datasets for comprehensive evaluation. The experimental results demonstrate the effectiveness and accuracy of the proposed method. Code is available at https://github.com/TensorPulse/MACnet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

This paper involves two types of datasets, which are public datasets and real datasets. The public dataset is available in the industrial big data industry innovation platform of China Information and Communication Research Institute, [https://www.industrial-bigdata.com/Data]. In addition, the real dataset that supports the findings of this study are available from a factory, but restrictions apply to the availability of these data, which were used under licence for the current study, and so are not publicly available.

Notes

  1. https://www.industrial-bigdata.com/Data

References

  1. Zhang X, Lei Y, Chen H, Zhang L, Zhou Y (2020) Multivariate time-series modeling for forecasting sintering temperature in rotary kilns using dcgnet. IEEE Trans Industr Inf 17(7):4635–4645

    Article  MATH  Google Scholar 

  2. Jiang K, Jiang Z, Xie Y, Pan D, Gui W (2022) Prediction of multiple molten iron quality indices in the blast furnace ironmaking process based on attention-wise deep transfer network. IEEE Trans Instrum Meas 71:1–14

    MATH  Google Scholar 

  3. Zhai N, Zhou X (2020) Temperature prediction of heating furnace based on deep transfer learning. Sensors 20(17):4676

    Article  MATH  Google Scholar 

  4. Zhou X, Zhai N, Li S, Shi H (2023) Time series prediction method of industrial process with limited data based on transfer learning. IEEE Trans Ind Inform

  5. Zeng P, Hu G, Zhou X, Li S, Liu P, Liu S (2022) Muformer: A long sequence time-series forecasting model based on modified multi-head attention. Knowl-Based Syst 254:109584

    Article  MATH  Google Scholar 

  6. Mumuni A, Mumuni F (2021) Cnn architectures for geometric transformation-invariant feature representation in computer vision: a review. SN Computer Science 2:1–23

    Article  MATH  Google Scholar 

  7. Petneházi G (2019) Recurrent neural networks for time series forecasting, arXiv preprint arXiv:1901.00069

  8. Lindemann B, Maschler B, Sahlab N, Weyrich M (2021) A survey on anomaly detection for technical systems using lstm networks. Comput Ind 131:103498

    Article  Google Scholar 

  9. Kumar D, Mathur H, Bhanot S, Bansal RC (2021) Forecasting of solar and wind power using lstm rnn for load frequency control in isolated microgrid. Int J Model Simul 41(4):311–323

    Article  MATH  Google Scholar 

  10. Nasiri H, Ebadzadeh MM (2022) Mfrfnn: Multi-functional recurrent fuzzy neural network for chaotic time series prediction. Neurocomputing 507:292–310

    Article  MATH  Google Scholar 

  11. Hu J, Zheng W (2020) Multistage attention network for multivariate time series prediction. Neurocomputing 383:122–137

    Article  MATH  Google Scholar 

  12. Du S, Li T, Yang Y, Horng S-J (2020) Multivariate time series forecasting via attention-based encoder-decoder framework. Neurocomputing 388:269–279

    Article  MATH  Google Scholar 

  13. Zhang Q-L, Yang Y-B (2021) Sa-net: Shuffle attention for deep convolutional neural networks. In: ICASSP 2021-2021 IEEE International conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 2235–2239

  14. Bai S, Kolter JZ, Koltun V (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling, arXiv preprint arXiv:1803.01271

  15. Shih S-Y, Sun F-K, Lee H-Y (2019) Temporal pattern attention for multivariate time series forecasting. Mach Learn 108:1421–1441

    Article  MathSciNet  MATH  Google Scholar 

  16. Dragomiretskiy K, Zosso D (2013) Variational mode decomposition. IEEE Trans Signal Process 62(3):531–544

    Article  MathSciNet  MATH  Google Scholar 

  17. Guo M-H, Xu T-X, Liu J-J, Liu Z-N, Jiang P-T, Mu T-J, Zhang S-H, Martin RR, Cheng M-M, Hu S-M (2022) Attention mechanisms in computer vision: A survey. Computational Visual Media 8(3):331–368

    Article  MATH  Google Scholar 

  18. Chorowski JK, Bahdanau D, Serdyuk D, Cho K, Bengio Y (2015) Attention-based models for speech recognition. Adv Neural Inform Process Syst 28

  19. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  20. Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  21. Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) Eca-net: Efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11534–11542

  22. Zhou J, Qian S, Yan Z, Zhao J, Wen H (2021) Esa-net: A network with efficient spatial attention for smoky vehicle detection. In: 2021 IEEE International instrumentation and measurement technology conference (I2MTC). IEEE, pp 1–6

  23. Wang Y, Chen J, Chen X, Zeng X, Kong Y, Sun S, Guo Y, Liu Y (2020) Short-term load forecasting for industrial customers based on tcn-lightgbm. IEEE Trans Power Syst 36(3):1984–1997

    Article  MATH  Google Scholar 

  24. Cai C, Li Y, Su Z, Zhu T, He Y (2022) Short-term electrical load forecasting based on vmd and gru-tcn hybrid network. Appl Sci 12(13):6647

    Article  Google Scholar 

  25. Sagheer A, Kotb M (2019) Time series forecasting of petroleum production using deep lstm recurrent networks. Neurocomputing 323:203–213

    Article  MATH  Google Scholar 

  26. Tan D, Chen L, Jiang C, Zhong W, Du W, Qian F, Mahalec V (2020) A circular target feature detection framework based on dcnn for industrial applications. IEEE Trans Industr Inf 17(5):3303–3313

  27. Lai G, Chang W-C, Yang Y, Liu H (2018) Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st international ACM SIGIR conference on research & development in information retrieval, pp 95–104

  28. Fu E, Zhang Y, Yang F, Wang S (2022) Temporal self-attention-based conv-lstm network for multivariate time series prediction. Neurocomputing 501:162–173

    Article  MATH  Google Scholar 

Download references

Acknowledgements

We appreciate the data support provided by the industrial big data industry innovation platform of China Information and Communication Research Institute. This work was supported by the Basic Research Program Project of Shenyang Institute of Automation, Chinese Academy of Sciences under Grant 2022000346, the Liaoning Province Applied Basic Research Program Project of China under Grant 2022JH2/101300255, and the Specific Research Assistant Funding Program of the Chinese Academy of Sciences under Grant E329100101.

Author information

Authors and Affiliations

Authors

Contributions

Dong Lu: Conceptualization, Methodology, Experiment, Formal analysis, Investigation, Writing—original draft. Xiaofeng Zhou: Supervision, Writing review & editing. Shuai Li: Validation, Writing review & editing, Supervision

Corresponding author

Correspondence to Xiaofeng Zhou.

Ethics declarations

Competing Interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, D., Zhou, X. & Li, S. Spatio-temporal attention-based hybrid deep network for time series prediction of industrial process. Appl Intell 55, 169 (2025). https://doi.org/10.1007/s10489-024-06033-5

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06033-5

Keywords