Abstract
Industrial time series involves a large amount of production process information, which effectively reflects the production status of the industrial process. To better understand characteristics and patterns of changes in production conditions, it is crucial to analyze and predict industrial time series data. Given the involvement of numerous parameters and complex physical-chemical reactions in industrial processes, attaining precise predictive performance utilizing a single model remains a formidable challenge. In this paper, we propose a novel hybrid deep learning prediction method based on spatio-temporal attention and temporal convolution network. The proposed method aims to handle the multivariate coupling characteristics and dynamic nonlinear features in industrial time series through different model structures for accurate prediction. In this method, historical data are first segmented into multiple consecutive inputs along the temporal dimension, which are then used as inputs to the subsequent attention mechanism module. To realize the mapping from points to series in the temporal dimension, the segmented input is processed using both the adaptive attention mechanism and one-dimensional convolution. Then the spatio-temporal coupling features are further explored through the spatio-temporal attention model. In addition, to extract dynamic nonlinear features from historical data, a parallel temporal convolutional network with temporal pattern attention is utilized. In order to evaluate the prediction performance of the proposed model, we use two different real-world industrial time series datasets for comprehensive evaluation. The experimental results demonstrate the effectiveness and accuracy of the proposed method. Code is available at https://github.com/TensorPulse/MACnet.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
This paper involves two types of datasets, which are public datasets and real datasets. The public dataset is available in the industrial big data industry innovation platform of China Information and Communication Research Institute, [https://www.industrial-bigdata.com/Data]. In addition, the real dataset that supports the findings of this study are available from a factory, but restrictions apply to the availability of these data, which were used under licence for the current study, and so are not publicly available.
References
Zhang X, Lei Y, Chen H, Zhang L, Zhou Y (2020) Multivariate time-series modeling for forecasting sintering temperature in rotary kilns using dcgnet. IEEE Trans Industr Inf 17(7):4635–4645
Jiang K, Jiang Z, Xie Y, Pan D, Gui W (2022) Prediction of multiple molten iron quality indices in the blast furnace ironmaking process based on attention-wise deep transfer network. IEEE Trans Instrum Meas 71:1–14
Zhai N, Zhou X (2020) Temperature prediction of heating furnace based on deep transfer learning. Sensors 20(17):4676
Zhou X, Zhai N, Li S, Shi H (2023) Time series prediction method of industrial process with limited data based on transfer learning. IEEE Trans Ind Inform
Zeng P, Hu G, Zhou X, Li S, Liu P, Liu S (2022) Muformer: A long sequence time-series forecasting model based on modified multi-head attention. Knowl-Based Syst 254:109584
Mumuni A, Mumuni F (2021) Cnn architectures for geometric transformation-invariant feature representation in computer vision: a review. SN Computer Science 2:1–23
Petneházi G (2019) Recurrent neural networks for time series forecasting, arXiv preprint arXiv:1901.00069
Lindemann B, Maschler B, Sahlab N, Weyrich M (2021) A survey on anomaly detection for technical systems using lstm networks. Comput Ind 131:103498
Kumar D, Mathur H, Bhanot S, Bansal RC (2021) Forecasting of solar and wind power using lstm rnn for load frequency control in isolated microgrid. Int J Model Simul 41(4):311–323
Nasiri H, Ebadzadeh MM (2022) Mfrfnn: Multi-functional recurrent fuzzy neural network for chaotic time series prediction. Neurocomputing 507:292–310
Hu J, Zheng W (2020) Multistage attention network for multivariate time series prediction. Neurocomputing 383:122–137
Du S, Li T, Yang Y, Horng S-J (2020) Multivariate time series forecasting via attention-based encoder-decoder framework. Neurocomputing 388:269–279
Zhang Q-L, Yang Y-B (2021) Sa-net: Shuffle attention for deep convolutional neural networks. In: ICASSP 2021-2021 IEEE International conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 2235–2239
Bai S, Kolter JZ, Koltun V (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling, arXiv preprint arXiv:1803.01271
Shih S-Y, Sun F-K, Lee H-Y (2019) Temporal pattern attention for multivariate time series forecasting. Mach Learn 108:1421–1441
Dragomiretskiy K, Zosso D (2013) Variational mode decomposition. IEEE Trans Signal Process 62(3):531–544
Guo M-H, Xu T-X, Liu J-J, Liu Z-N, Jiang P-T, Mu T-J, Zhang S-H, Martin RR, Cheng M-M, Hu S-M (2022) Attention mechanisms in computer vision: A survey. Computational Visual Media 8(3):331–368
Chorowski JK, Bahdanau D, Serdyuk D, Cho K, Bengio Y (2015) Attention-based models for speech recognition. Adv Neural Inform Process Syst 28
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141
Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19
Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) Eca-net: Efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11534–11542
Zhou J, Qian S, Yan Z, Zhao J, Wen H (2021) Esa-net: A network with efficient spatial attention for smoky vehicle detection. In: 2021 IEEE International instrumentation and measurement technology conference (I2MTC). IEEE, pp 1–6
Wang Y, Chen J, Chen X, Zeng X, Kong Y, Sun S, Guo Y, Liu Y (2020) Short-term load forecasting for industrial customers based on tcn-lightgbm. IEEE Trans Power Syst 36(3):1984–1997
Cai C, Li Y, Su Z, Zhu T, He Y (2022) Short-term electrical load forecasting based on vmd and gru-tcn hybrid network. Appl Sci 12(13):6647
Sagheer A, Kotb M (2019) Time series forecasting of petroleum production using deep lstm recurrent networks. Neurocomputing 323:203–213
Tan D, Chen L, Jiang C, Zhong W, Du W, Qian F, Mahalec V (2020) A circular target feature detection framework based on dcnn for industrial applications. IEEE Trans Industr Inf 17(5):3303–3313
Lai G, Chang W-C, Yang Y, Liu H (2018) Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st international ACM SIGIR conference on research & development in information retrieval, pp 95–104
Fu E, Zhang Y, Yang F, Wang S (2022) Temporal self-attention-based conv-lstm network for multivariate time series prediction. Neurocomputing 501:162–173
Acknowledgements
We appreciate the data support provided by the industrial big data industry innovation platform of China Information and Communication Research Institute. This work was supported by the Basic Research Program Project of Shenyang Institute of Automation, Chinese Academy of Sciences under Grant 2022000346, the Liaoning Province Applied Basic Research Program Project of China under Grant 2022JH2/101300255, and the Specific Research Assistant Funding Program of the Chinese Academy of Sciences under Grant E329100101.
Author information
Authors and Affiliations
Contributions
Dong Lu: Conceptualization, Methodology, Experiment, Formal analysis, Investigation, Writing—original draft. Xiaofeng Zhou: Supervision, Writing review & editing. Shuai Li: Validation, Writing review & editing, Supervision
Corresponding author
Ethics declarations
Competing Interest
All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Lu, D., Zhou, X. & Li, S. Spatio-temporal attention-based hybrid deep network for time series prediction of industrial process. Appl Intell 55, 169 (2025). https://doi.org/10.1007/s10489-024-06033-5
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06033-5