Abstract
This paper presents a reinforcement learning framework for stock trading systems. Trading system parameters are optimized by Qlearning algorithm and neural networks are adopted for value approximation. In this framework, cooperative multiple agents are used to efficiently integrate global trend prediction and local trading strategy for obtaining better trading performance. Agents communicate with others sharing training episodes and learned policies, while keeping the overall scheme of conventional Q-learning. Experimental results on KOSPI 200 show that a trading system based on the proposed framework outperforms the market average and makes appreciable profits. Furthermore, in view of risk management, the system is superior to a system trained by supervised learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kendall, S. M., Ord, K.: Time Series. Oxford, New York. (1997)
Neuneier, R.: Enhancing Q-Learning for Optimal Asset Allocation. Advances in Neural Information Processing Systems 10. MIT Press, Cambridge. (1998) 936–942
Lee, J.: Stock Price Prediction using Reinforcement Learning. In Proceedings of the 6th IEEE International Symposium on Industrial Electronics. (2001)
Sutton, R. S., Barto, A. G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge. (1998)
Baird, L. C.: Residual Algorithms: Reinforcement learning with Function Approximation. In Proceedings of Twelfth International Conference on Machine Learning. Morgan Kaufmann, San Fransisco. (1995) 30–37
Bengio, Y., Simard, P., Frasconi, P.: Learning Long-Term Dependencies with Gradient is Dificult. IEEE Transactions on Neural Networks 5(2). (1994) 157–166
Jaakkola, M., Jordan, M., Singh, S.: On the Convergence of Stochastic Iterative Dynamic Programming Algorithms. Neural Computation, 6(6). (1994) 1185–2201
Xiu, G., Laiwan, C.: Algorithm for Trading and Portfolio Management Using Qlearning and Sharpe Ratio Maximization. In Proceedings of ICONIP 2000, Korea. (2000) 832–837
Moody, J., Wu, Y., Liao, Y., Saffell, M.: Performance Functions and Reinforcement Learning for Trading Systems and Portfolios. Journal of Forecasting, 17(5–6). (1998) 441–470
Moody, J., Saffell, M.: Learning to Trade via Direct Reinforcement. IEEE Transactions on Neural Networks, 12(4). (2001) 875–889
Neuneier, R., Mihatsch., O.: Risk Sensitive Reinforcement Learning. Advances in Neural Information Processing Systems 11. MIT Press, Cambridge. (1999) 1031–1037
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lee, J.W., O, J. (2002). A Multi-agent Q-learning Framework for Optimizing Stock Trading Systems. In: Hameurlain, A., Cicchetti, R., Traunmüller, R. (eds) Database and Expert Systems Applications. DEXA 2002. Lecture Notes in Computer Science, vol 2453. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-46146-9_16
Download citation
DOI: https://doi.org/10.1007/3-540-46146-9_16
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44126-7
Online ISBN: 978-3-540-46146-3
eBook Packages: Springer Book Archive