Óscar Fernández Vicente
ABSTRACT
Market makers play a key role in financial markets by providing liquidity. They usually fill order books with buy and sell limit orders in order to provide traders alternative price levels to operate. This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. In particular, we propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets. This research analyzes how RL market maker agents behaves in non-competitive (only one RL market maker learning at the same time) and competitive scenarios (multiple RL market markers learning at the same time), and how they adapt their strategies in a Sim2Real scope with interesting results. Furthermore, it covers the application of policy transfer between different experiments, describing the impact of competing environments on RL agents performance. RL and deep RL techniques are proven as profitable market maker approaches, leading to a better understanding of their behavior in stock markets.
- Marco Avellaneda and Sasha Stoikov. 2008. High-frequency trading in a limit order book. Quantitative Finance 8 (4 2008), 217--224. Issue 3. Google Scholar
Cross Ref
- David Byrd, Maria Hybinette, and Tucker Hybinette Balch. 2019. ABIDES: Towards High-Fidelity Market Simulation for AI Research. (4 2019). http://arxiv.org/abs/1904.12066Google Scholar
- Tanmoy Chakraborty and Michael Kearns. 2011. Market making and mean reversion. Proceedings of the ACM Conference on Electronic Commerce, 307--313. Google ScholarDigital Library
- Nicholas Tung Chan and Christian Shelton. 2001. An Electronic Market-Maker.Google Scholar
- David Cournapeau. 2007. Scikit-Learn (Machine learning in Python). https://scikit-learn.org/stable/.Google Scholar
- Thomas G Fischer and Thomas Fischer. 2018. Reinforcement learning in financial markets-a survey. 12 (2018). https://www.iwf.rw.fau.de/research/iwf-discussion-paper-series/Google Scholar
- Sumitra Ganesh, Nelson Vadori, Mengda Xu, Hua Zheng, Prashant Reddy, and Manuela Veloso. 2019. Reinforcement Learning for Market Making in a Multi-agent Dealer Market. (11 2019). http://arxiv.org/abs/1911.05892Google Scholar
- Sven Gronauer and Klaus Diepold. 2021. Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review (2021). Google ScholarDigital Library
- Jens Kober, J. Andrew Bagnell, and Jan Peters. 2013. Reinforcement learning in robotics: A survey. International Journal of Robotics Research 32 (9 2013), 1238--1274. Issue 11. Google ScholarDigital Library
- Xiaofei Lu and Frédéric Abergel. 2018. Order-book modelling and market making strategies. (6 2018). http://arxiv.org/abs/1806.05101Google Scholar
- Scott Lundberg. 2018. SHAP (SHapley Additive exPlanations). https://shap.readthedocs.io/en/latest/index.html.Google Scholar
- Tom Mitchell. 1997. Machine Learning. (1997).Google Scholar
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. 2015. Human-level control through deep reinforcement learning. Nature 518 (2 2015), 529--533. Issue 7540. Google ScholarCross Ref
- Yagna Patel. 2018. Optimizing Market Making using Multi-Agent Reinforcement Learning. (12 2018). http://arxiv.org/abs/1812.10252Google Scholar
- Kun Shao, Zhentao Tang, Yuanheng Zhu, Nannan Li, and Dongbin Zhao. 2019. A Survey of Deep Reinforcement Learning in Video Games. (12 2019). http://arxiv.org/abs/1912.10944Google Scholar
- Thomas Spooner, John Fearnley, Rahul Savani, and Andreas Koukorinis. 2018. Market Making via Reinforcement Learning. (4 2018). http://arxiv.org/abs/1804.04216Google Scholar
- Richard S Sutton and Andrew G Barto. 2011. Reinforcement learning: An introduction. (2011).Google Scholar
- Yi Wang, Hongtao Zhou, and Wei Zeng. 2011. The agent-based simulation of inventory-based model's impact on market maker trading mechanism. IWACI 2011, 601--604. Google ScholarCross Ref
- Christopher Watkins. 1989. Learning from Delayed Rewards. Ph. D. Dissertation. King's College, Cambridge, UK.Google Scholar
Recommendations
Stock Price Formation: Precepts from a Multi-Agent Reinforcement Learning Model
AbstractIn the past, the bottom-up study of financial stock markets relied on first-generation multi-agent systems (MAS) , which employed zero-intelligence agents and often required the additional implementation of so-called noise traders to emulate price ...
A tabular sarsa-based stock market agent
ICAIF '20: Proceedings of the First ACM International Conference on AI in FinanceAutomated stock trading is now the de-facto way that investors have chosen to obtain high profits in the stock market while keeping risk under control. One of the approaches is to create agents employing Reinforcement Learning (RL) algorithms to learn ...
Stock Market Intraday Trading Using Reinforcement Learning
Multi-disciplinary Trends in Artificial IntelligenceAbstractIn this study, Reinforcement Learning (RL) techniques are used to develop trading strategies for the stock market. Conventional trading strategies rely on human intuition and the examination of historical data to make forecasts, whereas RL agents ...
Comments