Abstract
The rapid growth in automated financial trading has highlighted the need for trustworthy agents capable of adapting to the dynamic and ever-changing nature of financial markets. From an algorithmic viewpoint, financial trading is essentially a complex, dynamic time series problem, characterized by unpredictable and noisy data. Deep Reinforcement Learning (DRL) has shown great promise in addressing this challenge. It naturally aligns with the objective of financial trading-maximizing rewards-without relying on unrealistic assumptions that do not hold true in such volatile and noisy time series data. However, the complexity of the problem still presents challenges for conventional DRL algorithms. To overcome these, the implementation of continual learning agents is crucial for their ability to adjust to changing market conditions. Our approach not only adapts continual learning techniques to dynamic time series but also introduces a novel knowledge transfer loss, which enhances the adaptation of our model. In our extensive evaluation, we show that this approach successfully balances the trade-off between maintaining knowledge of past patterns and adapting to new ones, enhancing the model’s trustworthiness and effectiveness in real-world time series problems, like financial trading.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Avramelou, L., Nousi, P., Passalis, N., Tefas, A.: Deep reinforcement learning for financial trading using multi-modal features. Expert Syst. Appl. 238, 121849 (2024). https://doi.org/10.1016/j.eswa.2023.121849, https://www.sciencedirect.com/science/article/pii/S0957417423023515
Deng, Y., Bao, F., Kong, Y., Ren, Z., Dai, Q.: Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 653–664 (2017). https://doi.org/10.1109/TNNLS.2016.2522401
Fama, E.: Efficient capital markets: a review of theory and empirical work. J. Finance 25, 383–417 (1970)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the International Conference on Machine Learning, pp. 1126–1135 (2017)
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)
Isele, D., Cosgun, A.: Selective experience replay for lifelong learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Keneshloo, Y., Shi, T., Ramakrishnan, N., Reddy, C.K.: Deep reinforcement learning for sequence-to-sequence models. IEEE Trans. Neural Netw. Learn. Syst. 31(7), 2469–2489 (2019)
Khetarpal, K., Riemer, M., Rish, I., Precup, D.: Towards continual reinforcement learning: a review and perspectives (2022)
Kiripatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. 114(13), 3521–3526 (2017). https://doi.org/10.1073/pnas.1611835114
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning (2019)
Mahmud, M., Kaiser, M.S., Hussain, A., Vassanelli, S.: Applications of deep learning and reinforcement learning to biological data. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2063–2079 (2018)
Mermillod, M., Bugaiska, A., Bonin, P.: The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects. Front. Psychol. 4 (2013)
Mnih, V., et al.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)
Moody, J.E., Saffell, M.: Reinforcement learning for trading systems and portfolios. In: Knowledge Discovery and Data Mining (1998)
Nguyen, T.H., Shirai, K., Velcin, J.: Sentiment analysis on social media for stock movement prediction. Expert Syst. Appl. 42(24), 9603–9611 (Nov2015). https://doi.org/10.1016/j.eswa.2015.07.052, https://hal.science/hal-01203094
Oliveira, N., Cortez, P., Areal, N.: The impact of microblogging data for stock market prediction: Using twitter to predict returns, volatility, trading volume and survey sentiment indices. Expert Syst. Appl. 73 (12 2016). https://doi.org/10.1016/j.eswa.2016.12.036
Passalis, N., Tefas, A.: Learning deep representations with probabilistic knowledge transfer. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 268–284 (2018)
Passalis, N., Tzelepi, M., Tefas, A.: Probabilistic knowledge transfer for lightweight deep representation learning. IEEE Trans. Neural Netw. Learn. Syst. 32(5), 2030–2039 (2020)
Pénasse, J.: Understanding alpha decay. Manage. Sci. 68(5), 3966–3973 (2022)
Rusu, A.A., et al.: Policy distillation. arXiv preprint arXiv:1511.06295 (2015)
Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)
Schulman, J., Moritz, P., Levine, S., Jordan, M., Abbeel, P.: High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438 (2015)
Tran, D.T., Iosifidis, A., Kanniainen, J., Gabbouj, M.: Temporal attention-augmented bilinear network for financial time-series data analysis. IEEE Trans. Neural Netw. Learn. Syst. 30(5), 1407–1418 (2018)
Tsantekidis, A., Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., Iosifidis, A.: Forecasting stock prices from limit order book using convolutional neural networks. In: Proceedings of the IEEE International Conference on Business Informatics (2017). https://doi.org/10.1109/CBI.2017.23, iNT=sgn,”Tsantekidis, Avraam”; IEEE International Conference on Business Informatics ; Conference date: 01-01-1900
Tsantekidis, A., Passalis, N., Toufa, A.S., Saitas Zarkias, K., Chairistanidis, S., Tefas, A.: Price trailing for financial trading using deep reinforcement learning. IEEE Trans. Neural Netw. Learn. Syst. PP, 1–10 (06 2020). https://doi.org/10.1109/TNNLS.2020.2997523
Xu, J., Zhu, Z.: Reinforced continual learning. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Zhang, Z., Zohren, S., Roberts, S.: DeepLOB: deep convolutional neural networks for limit order books. IEEE Trans. Signal Process. 67(11), 3001–3012 (2019). https://doi.org/10.1109/tsp.2019.2907260
Acknowledgements
The research project “Energy Efficient and Trustworthy Deep Learning - DeepLET” is implemented in the framework of H.F.R.I call “Basic research Financing (Horizontal support of all Sciences)” under the National Recovery and Resilience Plan “Greece 2.0” funded by the European Union -NextGenerationEU (H.F.R.I. Project Number: 016762).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Katsikas, D., Passalis, N., Tefas, A. (2025). Plasticity Driven Knowledge Transfer for Continual Deep Reinforcement Learning in Financial Trading. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15309. Springer, Cham. https://doi.org/10.1007/978-3-031-78189-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-78189-6_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78188-9
Online ISBN: 978-3-031-78189-6
eBook Packages: Computer ScienceComputer Science (R0)