Skip to main content
SpringerLink
Log in

A deep learning integrated framework for predicting stock index price and fluctuation via singular spectrum analysis and particle swarm optimization

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Due to the complexity and volatility of stock market trading, there are still some issues in the existing prediction methods, including the processing of data noise, inexplicable selection of model parameters, and the lag in predicting price fluctuations. Aiming to better predict daily stock prices and fluctuations caused by high noise non-stationary data in actual trading, we propose a hybrid deep learning framework based on Singular Spectrum Analysis (SSA), multiple feature selection, and Long Short Term Memory (LSTM) network optimized by Particle Swarm Optimization (PSO). Based on Pearson correlation coefficients, we select features highly correlated with the closing price as inputs, and further achieve the noise reduction and optimization of those input sources by applying SSA to decompose the stock price time series into independent component signals. Using the stock price data from China and USA, we compared the prediction performance of our method with several well-known methods, and found that it achieved higher price prediction accuracy during periods of high stock volatility. For example, the R-squared and prediction accuracy of Shanghai Composite Index achieved 0.9998 and 99.01%, while the prediction metrics of S &P 500 reached 0.9883 and 94.26%, respectively. Besides, considering transaction costs, our method also achieved the highest profit in trading tests, even compared to long-term holding strategy.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Data Availability

Data will be made available on request.

References

  1. Xian S, Lei H, Chen K, Li Z (2022) A novel fuzzy time series model based on improved sparrow search algorithm and ceemdan. App Intell 1–28

  2. Wang J (2022) A novel metal futures forecasting system based on wavelet packet decomposition and stochastic deep learning model. Appl Intell 52(8):9334–9352

    Article  MathSciNet  Google Scholar 

  3. Liang M, Wu S, Wang X, Chen Q (2022) A stock time series forecasting approach incorporating candlestick patterns and sequence similarity. Expert Syst Appl 205:117595

    Article  Google Scholar 

  4. Niu H, Xu K, Wang W (2020) A hybrid stock price index forecasting model based on variational mode decomposition and LSTM network. Appl Intell 50:4296–4309

    Article  Google Scholar 

  5. Zhao Y, Yang G (2023) Deep learning-based integrated framework for stock price movement prediction. Appl Soft Comput 133:109921

    Article  Google Scholar 

  6. Pang X, Zhou Y, Wang P, Lin W, Chang V (2020) An innovative neural network approach for stock market prediction. J Supercomput 76:2098–2118

    Article  Google Scholar 

  7. Mallat SG (1989) A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans Pattern Anal Mach Intell 11(7):674–693

    Article  ADS  Google Scholar 

  8. Huang NE, Shen Z, Long SR, Wu MC, Shih HH, Zheng Q, Yen N-C, Tung CC, Liu HH (1998) The empirical mode decomposition and the hilbert spectrum for nonlinear and non-stationary time series analysis. Proc Royal Soc London Ser A Math Phys Eng Sci 454(1971):903–995

    Article  MathSciNet  ADS  Google Scholar 

  9. Wang X, Zhao Q, Zhang C, Wang B, Wang L, Liu W (2023) Enhanced dynamic feature representation learning framework by fourier transform for domain generalization. Inf Sci 649:119624

    Article  Google Scholar 

  10. Kim J, Hasanien HM, Tagayi RK (2023) Investigation of noise suppression in experimental multi-cell battery string voltage applying various mother wavelets and decomposition levels in discrete wavelet transform for precise state-of-charge estimation. J Ener Stor 73:109196

    Article  Google Scholar 

  11. Lotfipoor A, Patidar S, Jenkins DP (2024) Deep neural network with empirical mode decomposition and bayesian optimisation for residential load forecasting. Expert Syst Appl 237:121355

    Article  Google Scholar 

  12. Broomhead DS, King GP (1986) Extracting qualitative dynamics from experimental data. Phys D: Nonlin Phenom 20(2–3):217–236

    Article  MathSciNet  ADS  Google Scholar 

  13. Lahmiri S (2018) Minute-ahead stock price forecasting based on singular spectrum analysis and support vector regression. Appl Math Comput 320:444–451

    MathSciNet  Google Scholar 

  14. Rodrigues PC, Mahmoudvand R (2020) A new approach for the vector forecast algorithm in singular spectrum analysis. Commun Stat-Simul Comput 49(3):591–605

    Article  MathSciNet  Google Scholar 

  15. Lin M, Li X, Chen R, Fujita H, Lin J (2022) Picture fuzzy interactional partitioned heronian mean aggregation operators: an application to madm process. Art Intell Rev 55(2):1171–1208

    Article  Google Scholar 

  16. Md AQ, Kapoor S, AV CJ, Sivaraman AK, Tee KF, Sabireen H, Janakiraman N, (2023) Novel optimization approach for stock price forecasting using multi-layered sequential LSTM. Appl Soft Comput 134:109830

    Article  Google Scholar 

  17. Ratchagit M, Xu H (2022) A two-delay combination model for stock price prediction. Math 10(19):3447

    Article  Google Scholar 

  18. Lin S, Feng Y (2022) Research on stock price prediction based on orthogonal gaussian basis function expansion and pearson correlation coefficient weighted LSTM neural network. https://doi.org/10.23977/ACSS.2022.060504.

  19. Wu M-E, Wang C-H, Chung W-H (2017) Using trading mechanisms to investigate large futures data and their implications to market trends. Soft Comput 21:2821–2834

    Article  Google Scholar 

  20. Schmidhuber J, Hochreiter S et al (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  PubMed  Google Scholar 

  21. Wang C-H, Zhao Q, Tian R (2023) Short-term wind power prediction based on a hybrid markov-based PSO-BP neural network. Energies 16(11):4282. https://doi.org/10.3390/en16114282

    Article  Google Scholar 

  22. Guo Y, Guo J, Sun B, Bai J, Chen Y (2022) A new decomposition ensemble model for stock price forecasting based on system clustering and particle swarm optimization. Appl Soft Comput 130:109726

    Article  Google Scholar 

  23. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95-international Conference on Neural Networks, vol. 4, pp. 1942–1948. IEEE

  24. Ding Y, Cheng L, Pedrycz W, Hao K (2015) Global nonlinear kernel prediction for large data set with a particle swarm-optimized interval support vector regression. IEEE Trans Neural Netwo Learn Syst 26(10):2521–2534

    Article  MathSciNet  Google Scholar 

  25. Wang C-H, Chen S, Zhao Q, Suo Y (2023) An efficient end-to-end obstacle avoidance path planning algorithm for intelligent vehicles based on improved whale optimization algorithm. Math 11(8):1800

  26. Ma D, Yuan D, Huang M, Dong L (2024) Vgc-gan: a multi-graph convolution adversarial network for stock price prediction. Expert Syst Appl 236:121204

  27. Wang X, Li X, Li S (2022) A novel stock indices hybrid forecasting system based on features extraction and multi-objective optimizer. Appl Intell 52(10):11784–11807

    Article  Google Scholar 

  28. Park HJ, Kim Y, Kim HY (2022) Stock market forecasting using a multi-task approach integrating long short-term memory and the random forest framework. Appl Soft Comput 114:108106

    Article  Google Scholar 

  29. Lei L (2018) Wavelet neural network prediction method of stock price trend based on rough set attribute reduction. Appl Soft Comput 62:923–932. https://doi.org/10.1016/j.asoc.2017.09.029

    Article  Google Scholar 

  30. Shi Y, Li W, Zhu L, Guo K, Cambria E (2021) Stock trading rule discovery with double deep Q-network. Appl Soft Comput 107:107320. https://doi.org/10.1016/j.asoc.2021.107320

    Article  Google Scholar 

  31. Swathi T, Kasiviswanath N, Rao AA (2022) An optimal deep learning-based lstm for stock price prediction using twitter sentiment analysis. Appl Intell 52(12):13675–13688

    Article  Google Scholar 

  32. Lin Y, Liu S, Yang H, Wu H (2021) Stock trend prediction using candlestick charting and ensemble machine learning techniques with a novelty feature engineering scheme. IEEE Access 9:101433–101446. https://doi.org/10.1109/ACCESS.2021.3096825

    Article  Google Scholar 

  33. Zhu C, Ma X, Ding W, Zhan J (2023) Long-term time series forecasting with multi-linear trend fuzzy information granules for LSTM in a periodic framework. IEEE Trans Fuzzy Syst 1–15. https://doi.org/10.1109/TFUZZ.2023.3298970

  34. Bhandari HN, Rimal B, Pokhrel NR, Rimal R, Dahal KR, Khatri RK (2022) Predicting stock market index using LSTM. Mach Learn Appl 9:100320

    Google Scholar 

  35. Ji Y, Liew AW-C, Yang L (2021) A novel improved particle swarm optimization with long-short term memory hybrid model for stock indices forecast. IEEE Access 9:23660–23671. https://doi.org/10.1109/ACCESS.2021.3056713

    Article  Google Scholar 

  36. Zolfaghari M, Gholami S (2021) A hybrid approach of adaptive wavelet transform, long short-term memory and arima-garch family models for the stock index prediction. Expert Syst Appl 182:115149. https://doi.org/10.1016/j.eswa.2021.115149

    Article  Google Scholar 

  37. Qiao Z, Chai T, Gu J, Zhou X, Dai S, Zhang X (2019) Singular spectrum analysis based long short-term memory for predicting bitcoin price. In: 2019 IEEE 7th international conference on computer science and network technology (ICCSNT), pp 53–57. https://doi.org/10.1109/ICCSNT47585.2019.8962421

  38. Coussin M (2022) Singular spectrum analysis for real-time financial cycles measurement. J Int Money Finance 120:102532. https://doi.org/10.1016/j.jimonfin.2021.102532

    Article  Google Scholar 

  39. Emmanuel KK, Wagala A, Muriithi DK (2022) Singular spectrum analysis: an application to Kenya’s industrial inputs price index. Euron J Math Stat 3(1), 1–13.https://doi.org/10.24018/ejmath.2022.3.1.73

  40. Chen Y, Hao Y (2017) A feature weighted support vector machine and k-nearest neighbor algorithm for stock market indices prediction. Expert Syst Appl 80:340–355. https://doi.org/10.1016/j.eswa.2017.02.044

    Article  Google Scholar 

  41. Peng S, Han W, Jia G (2022) Pearson correlation and transfer entropy in the chinese stock market with time delay. Data Sci Manag 5(3):117–123

    Article  Google Scholar 

  42. Xu X, Lin M, Luo X, Xu Z (2023) Hrst-LR: a hessian regularization spatio-temporal low rank algorithm for traffic data imputation. IEEE Trans Intell Transpor Syst 24(10):11001–11017. https://doi.org/10.1109/TITS.2023.3279321

    Article  Google Scholar 

  43. Wang C-H, Cai J, Ye Q, Suo Y, Lin S, Yuan J (2023) A two-stage convolution network algorithm for predicting traffic speed based on multi-feature attention mechanisms. J Intell Fuzzy Syst 45(3):5181–5196. https://doi.org/10.3233/JIFS-231133

    Article  Google Scholar 

  44. Chen Y, Lin M, He Z, Polat K, Alhudhaif A, Alenezi F (2023) Consistency-and dependence-guided knowledge distillation for object detection in remote sensing images. Expert Syst Appl 229:120519

    Article  Google Scholar 

  45. He Z, Lin M, Xu Z, Yao Z, Chen H, Alhudhaif A, Alenezi F (2022) Deconv-transformer (dect): a histopathological image classification model for breast cancer based on color deconvolution and transformer architecture. Inf Sci 608:1093–1112

    Article  Google Scholar 

  46. Royston P (1992) Approximating the shapiro-wilk w-test for non-normality. Stat Comput 2:117–119. https://doi.org/10.1007/BF01891203

    Article  Google Scholar 

  47. Easton GS, McCulloch RE (1990) A multivariate generalization of quantile-quantile plots. J Am Stat Assoc 85(410):376–386. https://doi.org/10.1080/01621459.1990.10476210

    Article  Google Scholar 

  48. Lin M, Huang C, Chen R, Fujita H, Wang X (2021) Directional correlation coefficient measures for pythagorean fuzzy sets: their applications to medical diagnosis and cluster analysis. Complex Intell Syst 7:1025–1043

    Article  Google Scholar 

  49. Zoumpekas T, Salam M, Puig A (2022) Effective early stopping of point cloud neural networks. In: Modeling decisions for artificial intelligence, pp 156–167. https://doi.org/10.1007/978-3-031-13448-7_13

  50. Huang HH, Huang H-Y, Oxman JJ (2015) Stock liquidity and corporate bond yield spreads: theory and evidence. J Financ Res 38(1):59–91. https://doi.org/10.1111/jfir.12052

    Article  Google Scholar 

Download references

Funding

This work was supported in part by Fujian Provincial Department of Science and Technology under Grant No. 2021J011070, and Fujian University of Technology under Grant No. GY-Z18148.

Author information

Authors and Affiliations

  1. College of Computer Science and Mathematics, Fujian University of Technology, No. 69, Xuefu South Road, Fuzhou City, 350118, Fujian Province, China

    Chia-Hung Wang, Jinchen Yuan, Yingping Zeng & Shengming Lin

  2. Fujian Provincial Key Laboratory of Big Data Mining and Applications, Fujian University of Technology, No. 69, Xuefu South Road, Fuzhou City, 350118, Fujian Province, China

    Chia-Hung Wang

Authors
  1. Chia-Hung Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinchen Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yingping Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shengming Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, C.-H. W.; methodology, C.-H. W. and J. Y.; software, J. Y. and Y. Z.; validation, J. Y. and Y. Z.; formal analysis, J. Y., and C.-H. W.; investigation, C.-H. W. and J. Y.; data curation, J. Y. and S. L.; writing-original draft preparation, J. Y., and S. L.; writing-review and editing, C.-H. W., and J. Y. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Chia-Hung Wang.

Ethics declarations

Competing interests

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Ethical and informed consent for data used

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, CH., Yuan, J., Zeng, Y. et al. A deep learning integrated framework for predicting stock index price and fluctuation via singular spectrum analysis and particle swarm optimization. Appl Intell 54, 1770–1797 (2024). https://doi.org/10.1007/s10489-024-05271-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05271-x

Keywords

Search

Navigation