Skip to main content
SpringerLink
Log in

A multi-layer and multi-ensemble stock trader using deep learning and deep reinforcement learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The adoption of computer-aided stock trading methods is gaining popularity in recent years, mainly because of their ability to process efficiently past information through machine learning to predict future market behavior. Several approaches have been proposed to this task, with the most effective ones using fusion of a pile of classifiers decisions to predict future stock values. However, using prices information in single supervised classifiers has proven to lead to poor results, mainly because market history is not enough to be an indicative of future market behavior. In this paper, we propose to tackle this issue by proposing a multi-layer and multi-ensemble stock trader. Our method starts by pre-processing data with hundreds of deep neural networks. Then, a reward-based classifier acts as a meta-learner to maximize profit and generate stock signals through different iterations. Finally, several metalearner trading decisions are fused in order to get a more robust trading strategy, using several trading agents to take a final decision. We validate the effectiveness of the approach in a real-world trading scenario, by extensively testing it on the Standard & Poor’s 500 future market and the J.P. Morgan and Microsoft stocks. Experimental results show that the proposed method clearly outperforms all the considered baselines (which still performs very well in the analysed period), and even the conventional Buy-and-Hold strategy, which replicates the market behaviour.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://www.multicharts.com/

  2. https://github.com/Artificial-Intelligence-Big-Data-Lab/A-Multi-Layer-and-Multi-Ensembled-Stock-Trader-Using-Deep-Learning-and-Deep-Reinforcement-Learning

References

  1. Ahmadian S, Khanteymoori AR (2015) Training back propagation neural networks using asexual reproduction optimization. In: Conference on information and knowledge technology (IKT), pp. 1–6

  2. An N, Ding H, Yang J, Au R, Ang TF (2020) Deep ensemble learning for alzheimer’s disease classification. J Biomed Inform 105:103411. https://doi.org/10.1016/j.jbi.2020.103411

    Article  Google Scholar 

  3. Asad M (2015) Optimized stock market prediction using ensemble learning. In: 2015 9Th international conference on application of information and communication technologies (AICT), pp 263–268

  4. Barra S, Carta SM, Corriga A, Podda AS, Recupero DR (2020) Deep learning and time series-to-image encoding for financial forecasting. IEEE/CAA J Autom Sin 73:683–692

  5. Calvi GG, Lucic V, Mandic DP (2019) Support tensor machine for financial forecasting. In: ICASSP 2019 - 2019 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp 8152–8156. https://doi.org/10.1109/ICASSP.2019.8683383

  6. Chun SH, Park YJ (2005) Dynamic adaptive ensemble case-based reasoning: application to stock market prediction. Expert Syst Appl 28(3):435–443

    Article  Google Scholar 

  7. Fenghua W, Jihong X, Zhifang H, Xu G (2014) Stock price prediction based on ssa and svm. Procedia Comput Sci 31:625–631. 2nd International Conference on Information Technology and Quantitative Management ITQM

  8. Fu TC, Lee KK, Sze D, Chung FL, Ng CM (2008) Discovering the correlation between stock time series and financial news. In: Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 01, WI-IAT ’08. IEEE Computer Society, pp 880–883. https://doi.org/10.1109/WIIAT.2008.228

  9. Gao X, Hongkong S, Chan L (2000) An algorithm for trading and portfolio management using q-learning and sharpe ratio maximization. In: International conference on neural information processing, pp 832–837

  10. Gyamerah SA, Ngare P, Ikpe D (2019) On stock market movement prediction via stacking ensemble learning method. In: IEEE Conference on computational intelligence for financial engineering economics (CIFEr), pp 1–8

  11. Han J, Kamber M, Pei J (2011) Data Transformation and Data Discretization, chap. 3. Elsevier, pp 111–118

  12. Hasselt H (2010) Double q-learning. In: Lafferty JD, Williams CKI, Shawe-Taylor J, Zemel RS, Culotta A (eds) Advances in neural information processing systems, vol 23, Curran Associates, Inc, pp 2613–2621

  13. Jalali SMJ, Ahmadian S, Kebria PM, Khosravi A, Lim CP, Nahavandi S (2019) Evolving artificial neural networks using butterfly optimization algorithm for data classification. In: Neural information processing. Springer International Publishing, Cham, pp 596–607

  14. Jalali SMJ, Ahmadian S, Khosravi A, Mirjalili S, Mahmoudi MR, Nahavandi S (2020) Neuroevolution-based autonomous robot navigation: a comparative study. Cogn Syst Res 62:35–43

    Article  Google Scholar 

  15. Kamijo KI, Tanigawa T (1990) Stock price pattern recognition-a recurrent neural network approach. In: 1990 IJCNN International joint conference on neural networks. IEEE, pp 215–221

  16. Kang Q, Zhou H, Kang Y (2018) An asynchronous advantage actor-critic reinforcement learning method for stock selection and portfolio management. In: Proceedings of the 2nd International Conference on Big Data Research, ICBDR 2018. Association for Computing Machinery, New York, pp 141–145. https://doi.org/10.1145/3291801.3291831

  17. Khairi TWA, Zaki R.M, Mahmood WA (2019) Stock price prediction using technical, fundamental and news based approach. In: 2019 2Nd scientific conference of computer sciences (SCCS), pp 177–181

  18. Khan W, Ghazanfar MA, Azam MA, Karami A, Alyoubi KH, Alfakeeh AS (2020) Stock market prediction using machine learning classifiers and social media, news. Journal of Ambient Intelligence and Humanized Computing

  19. Kim T, Kim HY (2019) Forecasting stock prices with a feature fusion lstm-cnn model using different representations of the same data. PLOS One 14(2):1–23. https://doi.org/10.1371/journal.pone.0212320

    Google Scholar 

  20. Kimoto T, Asakawa K, Yoda M, Takeoka M (1990) Stock market prediction system with modular neural networks. In: 1990 IJCNN International joint conference on neural networks. IEEE, pp 1–6

  21. Lee CH, Park KC (1992) Prediction of monthly transition of the composition stock price index using recurrent back-propagation. In: Artificial neural networks. Elsevier, pp 1629–1632

  22. Lee J, Park J, Jangmin O, Lee J, Hong E (2007) A multiagent approach to $q$-learning for daily stock trading. IEEE Trans Syst Man Cybern Part A Syst Hum 37:864–877

    Article  Google Scholar 

  23. Lei K, Zhang B, Li Y, Yang M, Shen Y (2020) Time-driven feature-aware jointly deep reinforcement learning for financial signal representation and algorithmic trading. Expert Syst Appl 140:112872. https://doi.org/10.1016/j.eswa.2019.112872, http://www.sciencedirect.com/science/article/pii/S0957417419305822

  24. Lin Y, Huang T, Chung W, Ueng Y (2020) Forecasting fluctuations in the financial index using a recurrent neural network based on price features. IEEE Transactions on Emerging Topics in Computational Intelligence, pp 1–12

  25. Magdon-Ismail M, Atiya AF (2004) Maximum drawdown. Risk Mag 17(10):99–102

    MATH  Google Scholar 

  26. Mihatsch O, Neuneier R (1999) Risk-sensitive reinforcement learning. In: Advances in neural information processing systems. MIT press, pp 1031–1037

  27. Moody J, Wu L, Liao Y, Saffell M (1998) Performance functions and reinforcement learning for trading systems and portfolios. J Forecast 17(5-6):441–470

    Article  Google Scholar 

  28. Patil P, Wu CSM, Potika K, Orang M (2020) Stock market prediction using ensemble of graph theory, machine learning and deep learning models. In: Proceedings of the 3rd International Conference on Software Engineering and Information Management, ICSIM ’20. Association for Computing Machinery, New York, pp 85–92

  29. Plappert M (2016) keras-rl. https://github.com/keras-rl/keras-rl

  30. Puterman M (2014) Markov decision processes: discrete stochastic dynamic programming. Wiley, New York

  31. Sezer OB, Ozbayoglu AM (2018) Algorithmic financial trading with deep convolutional neural networks: Time series to image conversion approach. Appl Soft Comput 70:525–538. https://doi.org/10.1016/j.asoc.2018.04.024. http://www.sciencedirect.com/science/article/pii/S1568494618302151

  32. Si W, Li J, Ding P, Rao R (2017) A multi-objective deep reinforcement learning approach for stock index future’s intraday trading. In: International symposium on computational intelligence and design (ISCID), vol 2, pp 431–436

  33. Sim HS, Kim HI, Ahn JJ (2019) Is deep learning for image recognition applicable to stock market prediction? Complexity

  34. Sun T, Wang J, Ni J, Cao Y, Liu B (2019) Predicting futures market movement using deep neural networks. In: 18Th IEEE international conference on machine learning and applications (ICMLA), pp 118–125

  35. Surton R, Barto A (1998) Reinforcement Learning: an introduction, vol 1. MIT press, Cambridge

  36. Tan TZ, Quek C, Ng GS (2005) Brain-inspired genetic complementary learning for stock market prediction. In: 2005 IEEE Congress on evolutionary computation, vol 3. IEEE, pp 2653–2660

  37. Tan Z, Yan Z, Zhu G (2019) Stock selection with random forest: An exploitation of excess return in the chinese stock market. Heliyon 5(8):e02310. https://doi.org/10.1016/j.heliyon.2019.e02310. http://www.sciencedirect.com/science/article/pii/S2405844019359705

  38. Wang Z, Oates T (2015) Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. In: Workshops at the twenty-ninth AAAI conference on artificial intelligence

  39. Wang Z, Schaul T, Hessel M, Hasselt H, Lanctot M, Freitas N (2016) Dueling network architectures for deep reinforcement learning. In: International Conference on Machine Learning, Proceedings of Machine Learning Research, vol 48. PMLR, New York, pp 1995–2003

  40. Wolpert DH (1992) Stacked generalization. Neural Netw 5:241–259

    Article  Google Scholar 

  41. Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. Trans Evol Comp 1(1):67–82. https://doi.org/10.1109/4235.585893

    Article  Google Scholar 

  42. Wu Y, Mao J, Li W (2018) Predication of futures market by using boosting algorithm. In: International conference on wireless communications, signal processing and networking (wiSPNET), pp 1–4

  43. Ye C, Ma H, Zhang X, Zhang K, You S (2017) Survival-oriented reinforcement learning model: an effcient and robust deep reinforcement learning algorithm for autonomous driving problem. In: Zhao Y, Kong X, Taubman D (eds) Image and graphics. Springer International Publishing, Cham, pp 417–429

  44. Zhang Y, Wu L (2009) Stock market prediction of s&p 500 via combination of improved bco approach and bp neural network. Expert Syst Appl 36(5):8849–8854

    Article  Google Scholar 

  45. Zhou Z, Gao M, Liu Q, Xiao H (2020) Forecasting stock price movements with multiple data sources: Evidence from stock market in china. Physica A Stat Mech Appl 542:123389. https://doi.org/10.1016/j.physa.2019.123389. http://www.sciencedirect.com/science/article/pii/S0378437119318941

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, University of Cagliari, Via Ospedale 72, 09124, Cagliari, Italy

    Salvatore Carta, Andrea Corriga, Anselmo Ferreira, Alessandro Sebastian Podda & Diego Reforgiato Recupero

Authors
  1. Salvatore Carta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Corriga
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anselmo Ferreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alessandro Sebastian Podda
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Diego Reforgiato Recupero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Salvatore Carta.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The research performed in this paper has been supported by the “Bando ”Aiuti per progetti di Ricerca e Sviluppo“-POR FESR 2014-2020—Asse 1, Azione 1.1.3, Strategy 2- Program 3, Project AlmostAnOracle - AI and Big Data Algorithms for Financial Time Series Forecasting”

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Carta, S., Corriga, A., Ferreira, A. et al. A multi-layer and multi-ensemble stock trader using deep learning and deep reinforcement learning. Appl Intell 51, 889–905 (2021). https://doi.org/10.1007/s10489-020-01839-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01839-5

Keywords

Search

Navigation