Abstract
For many real world applications, such as stock price prediction and video frame synthesis, sequential data prediction is a fundamental and challenging task. Considering the temporal features of sequential data, existing approaches generally adopt recurrent neural network and its variants for the prediction. However, for sequences with complex structure, these approaches cannot guarantee to obtain promising results. In this paper, to address the above issue, we formulate sequential data prediction as a self-supervised generative learning problem. Concretely, we design a generator to learn the distribution of the sequential data and generate the predicted values, as well as a discriminator to judge whether or not the input sequential data are real or fake. Based on this proposed framework and the adversarial learning scheme, we develop the corresponding networks for vector inputs and high-order tensor inputs, respectively, which are respectively named vector generative network (VGN) and high-order tensor generative network (HTGN). Extensive experiments on five stock price prediction datasets and two video frame prediction datasets demonstrate the effectiveness of our framework, and its advantages over the state-of-the-art approaches. Our main code and the used data have been shared at https://github.com/xsavagek/SSGL.













Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Box GE, Pierce DA (1970) Distribution of residual autocorrelations in autoregressive-integrated moving average time series models. J Am Stat Assoc 65(332):1509–1526
Bengio Y (1999) Markovian models for sequential data. Neural Comput Surv 2(199):129–162
Elman JL (1990) Finding structure in time. Cognit Sci 14(2):179–211
Zhang J-S, Xiao X-C (2000) Predicting chaotic time series using recurrent neural network. Chin Phys Lett 17(2):88
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Graves A (2013) Generating sequences with recurrent neural networks. arXiv:1308.0850
Chen L, Ren J, Chen P, Mao X, Zhao Q (2022) Limited text speech synthesis with electroglottograph based on bi-lstm and modified tacotron-2. Appl Intell 52(13):15193–15209
Xiang Z, Yan J, Demir I (2020) A rainfall-runoff model with lstm-based sequence-to-sequence learning. Water Resour Res (1):56
Xu H, Chai L, Luo Z, Li S (2022) Stock movement prediction via gated recurrent unit network based on reinforcement learning with incorporated attention mechanisms. Neurocomputing 467:214–228
Azad R, Asadi-Aghbolaghi M, Fathy M, Escalera S (2019) Bi-directional convlstm u-net with densley connected convolutions. In: Proceedings of the IEEE/CVF international conference on computer vision (ICCV) workshops
Shi X, Chen Z, Wang H, Yeung D, Wong W, Woo W (2015) Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28: annual conference on neural information processing systems 2015, December 7-12, 2015. Montreal, pp 802–810
Smola AJ, Schölkopf B. (2004) A tutorial on support vector regression. Stat Comput 14 (3):199–222
Asif MT, Dauwels J, Goh CY, Oran A, Fathi E, Xu M, Dhanya MM, Mitrovic N, Jaillet P (2014) Spatiotemporal patterns in large-scale traffic speed prediction. IEEE Trans Intell Trans Sys 15(2):794–804
Wang J, Wang J (2015) Forecasting stock market indexes using principle component analysis and stochastic time effective neural networks. Neurocomputing 156:68–78
Stoean C, Paja W, Stoean R, Sandita A (2019) Deep architectures for long-term stock price prediction with a heuristic-based strategy for trading simulations. PloS one 14(10):0223593
Gao L, Li H, Liu Z, Liu Z, Wan L, Feng W (2021) Rnn-transducer based chinese sign language recognition. Neurocomputing 434:45–54
Qu L, Lyu J, Li W, Ma D, Fan H (2021) Features injected recurrent neural networks for short-term traffic speed prediction. Neurocomputing 451:290–304
Xue X, Gao Y, Liu M, Sun X, Feng J (2021) Gru-based capsule network with an improved loss for personnel performance prediction. Appl Intell (5)
Kim T, Kim HY (2019) Forecasting stock prices with a feature fusion lstm-cnn model using different representations of the same data. PloS one 14(2):0212320
Niu H, Xu K, Wang W (2020) A hybrid stock price index forecasting model based on variational mode decomposition and lstm network. Appl Intell 50(12):4296–4309
Srivastava N, Mansimov E, Salakhudinov R (2015) Unsupervised learning of video representations using lstms. In: International conference on machine learning. PMLR, pp 843–852
Mottaghi R, Rastegari M, Gupta A, Farhadi A (2016) “What happens if...” learning to predict the effect of forces in images. In: European conference on computer vision. Springer, pp 269–285
Yuan M, Dai Q (2022) A novel deep pixel restoration video prediction algorithm integrating attention mechanism. Appl Intell 52(5):5015–5033
Xu Z, Du J, Wang J, Jiang C, Ren Y (2019) Satellite image prediction relying on gan and lstm neural networks. In: ICC 2019-2019 IEEE international conference on communications (ICC). IEEE, pp 1–6
Gammulle H, Denman S, Sridharan S, Fookes C (2019) Multi-level sequence gan for group activity recognition. In: Computer vision – ACCV 2018. Springer, pp 331–346
Zhou H, Zhang S, Peng J, Zhang S, Li J, Xiong H, Zhang W (2020) Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 11106–11115
Goodfellow I, Pougetabadie J, Mirza M, Xu B, Wardefarley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Proceedings of the advances in neural information processing systems
Istiake Sunny MA, Maswood MMS, Alharbi AG (2020) Deep learning-based stock price prediction using lstm and bi-directional lstm model. In: 2020 2nd Novel intelligent and leading emerging sciences conference (NILES), pp 87–92. https://doi.org/10.1109/NILES50944.2020.9257950
Stock Datasets on Yahool Finance Website. https://finance.yahoo.com
Li B, Hoi SC (2012) On-line portfolio selection with moving average reversion. In: Proceedings of the international conference on machine learning
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
Sagheer A, Kotb M (2019) Time series forecasting of petroleum production using deep lstm recurrent networks. Neurocomputing 323:203–213
Mootha S, Sridhar S, Seetharaman R, Chitrakala S (2020) Stock price prediction using bi-directional lstm based sequence to sequence modeling and multitask learning. In: 2020 11th IEEE annual ubiquitous computing, electronics mobile communication conference (UEMCON), pp 0078–0086. https://doi.org/10.1109/UEMCON51285.2020.9298066
Chen J, Du J, Xue Z, Kou F (2020) Prediction of financial big data stock trends based on attention mechanism. In: 2020 IEEE international conference on knowledge graph (ICKG), pp 152–156
Chen Y, Fang R, Liang T, Sha Z, Li S, Yi Y, Zhou W, Song H (2021) Stock price forecast based on cnn-bilstm-eca model. Sci Program
An Z, Feng Z (2021) A stock price forecasting method using autoregressive integrated moving average model and gated recurrent unit network. In: 2021 International conference on big data analysis and computer science (BDACS), pp 31–34. https://doi.org/10.1109/BDACS53596.2021.00015
Patil P, Wu C-SM, Potika K, Orang M (2020) Stock market prediction using ensemble of graph theory, machine learning and deep learning models. In: Proceedings of the 3rd international conference on software engineering and information management, pp 85–92
Sutskever I, Hinton GE, Taylor GW (2009) The recurrent temporal restricted boltzmann machine. In: Proceedings of the advances in neural information processing systems. Vancouver, British Columbia, pp 1601–1608
Moving MNIST Dataset (2015) Website. http://www.cs.toronto.edu/nitish/unsupervised_video/
Lotter W, Kreiman G, Cox D (2015) Unsupervised learning of visual structure using predictive generative networks. arXiv:1511.06380
Gan Z, Li C, Henao R, Carlson ED, Carin L (2015) Deep temporal sigmoid belief networks for sequence modeling. In: Proceedings of the advances in neural information processing systems. Montreal, pp 2467–2475
Srivastava N, Mansimov E, Salakhutdinov R (2015) Unsupervised learning of video representations using lstms. In: Proceedings of the international conference on machine learning. Lille, pp 843–852
Acknowledgements
This work was partially supported by the National Key Research and Development Program of China under Grant No. 2018AAA0100400, HY Project under Grant No. LZY2022033004, the Natural Science Foundation of Shandong Province under Grants No. ZR2020MF131 and No. ZR2021ZD19, Project of the Marine Science and Technology cooperative Innovation Center under Grant No. 22-05-CXZX-04-03-17, the Science and Technology Program of Qingdao under Grant No. 21-1-4-ny-19-nsh, and Project of Associative Training of Ocean University of China under Grant No. 202265007.
We want to thank “Qingdao AI Computing Center” and “Eco-Innovation Center” for providing inclusive computing power and technical support of MindSpore during the completion of this paper.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of Interests
The authors declare that they have no conflicts of interest on this work.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xu, K., Zhong, G., Deng, Z. et al. Self-supervised generative learning for sequential data prediction. Appl Intell 53, 20675–20689 (2023). https://doi.org/10.1007/s10489-023-04578-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-023-04578-5