Intelligent trading of seasonal effects: A decision support algorithm based on reinforcement learning
Introduction
Seasonalities and empirical regularities on financial markets are one of the most frequently studied phenomena in the scientific literature. This is due to simple but promising assumptions, which can easily be translated into a trading strategy. Hence, only investigating this phenomenon has lost its appeal. Therefore, it is our motivation, to not simply create a trading system based on seasonalities, but to verify the signals of this strategy with an intelligent filter in order to provide a robust decision support.
The procedure for filtering is the focus of our work. We use the promising approach of reinforcement learning (RL) to realize an effective filtering. This heuristic method is often used in unstructured and complex situations and provides very good results in the field of robotics, but also increasingly in economic decision-making. Our goal is to find a policy with RL in order to filter the output signals of the basic strategy to improve the reward to risk ratios.
A novel approach is used. To link the trading decision with a reward, we use an artificial neural network (ANN). The same ANN (a simple three layer feedforward network) acts as decision support to determine the optimal parameters for future trades. We use a combination of three major research areas (RL, ANN, seasonalities). We only introduce the basics of each topic. For an in-depth insight many suggestions can be found in the corresponding section. This paper aims to demonstrate the strength of a combination of several economic and interdisciplinary methods.
Our paper is divided into the following parts. Section 2 presents the ideas and methods. Section 2.1 describes the results of a brief analysis on seasonalities. It shows the promising approach of this surprisingly simple strategy. We investigate two major indices (DAX and S&P 500) on detectable trading regularities like upward biases at the turn-of-the-month, exchange holidays and the pre-FOMC (Federal Open Market Committee) announcement drift. Section 2.2 deals with RL and the detailed description of our modified version. Section 2.3 offers a small introduction to the world of artificial neural networks. Section 3 shows a merger of the mentioned disciplines in a fully automated and self-learning trading system. In Section 4 the results are presented and discussed. Section 5 discusses possible limitations of the strategy and provides an outlook for further research. Appendix A shows the complete algorithm in pseudocode.
Section snippets
Ideas and methods
In this section we present an overview of the methods and ideas. First, previous studies about seasonalities on financial markets are introduced in Section 2.1. After that we test whether the basic approach used for our subsequent programming is performing significantly better than a random strategy. Furthermore, in Section 2.2 reinforcement learning and our modifications are described. Section 2.3 gives an insight into the field of artificial neural networks.
Trading system
The basic idea is to use the detectable seasonalities and empirical regularities for an automated trading system. But the question is how this assumption of rising prices after a trade event can be transformed into a profitable behavior. Our research shows that it is difficult to develop firm rules. Obeying fixed rules may lead to large drawdowns. Depending on market conditions, e.g. a different leverage or a customized holding period could be useful. Inactivity or even betting on a price
Results and discussion
The results of the RL strategy were generated by using a self-developed simulation program. The software written in Java mimics a market with real prices of the past (2000–2012) and is also able to create and train neural networks. Thus the agent can be tested realistically over several years and with different parameters (e.g. with different probabilities for exploration). For a detailed analysis learning logs were stored, which give information on how the agent works. The benchmark is a
Conclusion, limitations, and future research
In this paper three major research fields are combined in an intelligent, self-learning and fully automated trading system. The basis is the promising strategy of seasonalities and empirical regularities in financial markets which has been described and discussed in the literature for three decades. For the period from 2000 to 2012, the significance of upward biases at the turn-of-the-month, during exchange holidays and the pre-FOMC announcement drift could be confirmed in many cases. Based on
Acknowledgment
We would like to thank the anonymous reviewers for their interesting and valuable comments to improve the quality of this paper.
Dennis Eilers is a Bachelor Student at Leibniz University of Hanover, Germany. His research interests include machine learning algorithms for intelligent trading strategies.
References (37)
A monthly effect in stock returns
Journal of Financial Economics
(March 1987)- et al.
Technical trading rules and calendar anomalies — are they the same phenomena?
Economics Letters
(February 2010) - et al.
Turn-of-month and pre-holiday effects on stock returns: some international evidence
Journal of Banking & Finance
(June 1992) - et al.
An automated FX trading system using adaptive reinforcement learning
Expert Systems with Applications
(April 2006) Stock returns and the weekend effect
Journal of Financial Economics
(March 1980)- et al.
Is there a monthly effect in stock market returns?: evidence from foreign countries
Journal of Banking & Finance
(May 1989) Neural network techniques for financial performance prediction: integrating fundamental and technical analysis
Decision Support Systems
(September 2004)- et al.
Forecasting the NYSE composite index with technical analysis, pattern recognizer, neural network, and genetic algorithm: a case study in romantic decision support
Decision Support Systems
(March 2002) - et al.
Automatic programming of behavior-based robots using reinforcement learning
Artificial Intelligence
(June 1992) - et al.
Capital market seasonality: the case of stock returns
Journal of Financial Economics
(October 1976)
Neural networks for decision support: problems and opportunities
Decision Support Systems
Forecasting and trading the EUR/USD exchange rate with stochastic Neural Network combination and time-varying leverage
Decision Support Systems
Dangers of data mining: the case of calendar effects in stock returns
Journal of Econometrics
Forecasting with artificial neural networks: the state of the art
International Journal of Forecasting
Sell in May and go away just won't go away
Financial Analysts Journal
What size net gives valid generalization?
Neural Computation
Making financial trading by recurrent reinforcement learning
Knowledge-Based Intelligent Information and Engineering Systems
Neural Networks for Pattern Recognition
Cited by (42)
Frequency matching optimization model of ultrasonic scalpel transducer based on neural network and reinforcement learning
2023, Engineering Applications of Artificial IntelligenceCitation Excerpt :Q-learning algorithm has the characteristics of continuous learning and dynamic adjustment depending on the dynamic working environment (instantaneous characteristic variables). It can select new action vectors according to the value table corresponding to the state action set, and the controller can adjust the state action set continuously according to the new state to maintain the matching between the input and output frequencies of neural networks so that the transducer can realize the stable state of resonance at the maximum amplitude (Eilers et al., 2014). In the resonant state, the amplitude of a transducer is significantly increased and the stable vibration and the highest energy conversion efficiency are realized.
Reinforcement learning based framework for COVID-19 resource allocation
2022, Computers and Industrial EngineeringCitation Excerpt :Reinforcement learning is a machine learning technique that understands and automatically processes goal-oriented learning and decision-making problems. In recent years, reinforcement learning has been applied to many fields and achieved remarkable results (Eilers, Dunis, von Mettenheim, & Breitner, 2014; Silver et al., 2017; Brown et al., 2019). There are many literatures on the application of reinforcement learning methods in supply chain management and resource allocation (Shahrabi, Adibi, & Mahootchi, 2017; Chen, Yang, Li, & Wang, 2020).
Deep reinforcement learning based trading agents: Risk curiosity driven learning for financial rules-based policy
2021, Expert Systems with ApplicationsCitation Excerpt :Precisely, RL has shown promising results on the financial and economic problems (Hull, 2014; Li, Zheng, & Zheng, 2019). Eilers et al. (2014) present a decision support algorithm which uses the powerful ideas of RL in order to improve the economic benefits of the basic seasonality strategy. However, these approaches require large amounts of training data, but the resulting policies are not immediately generalized to novel tasks in the same environment.
An intelligent financial portfolio trading strategy using deep Q-learning
2020, Expert Systems with ApplicationsCitation Excerpt :Bertoluzzo and Corazza (2012) suggested a single-asset trading system using Q-learning with linear and kernel function approximations. Eilers, Dunis, Mettenheim, and Breitner (2014) developed a trading rule for an asset with a seasonal price trend using Q-learning. Zhang et al. (2015) derived a trading rule generator using extended classifier systems combined with RL and a genetic algorithm.
High-performance stock index trading via neural networks and trees
2020, Applied Soft Computing Journal
Dennis Eilers is a Bachelor Student at Leibniz University of Hanover, Germany. His research interests include machine learning algorithms for intelligent trading strategies.
Christian Dunis is an emeritus professor of Banking and Finance from Liverpool John Moores University. He currently works as Risk Manager at a Swiss private bank. His research interests include quantitative trading strategies and artificial intelligence.
Hans-Jörg von Mettenheim is a professor for Decision Support Systems at Leibniz University of Hanover, Germany. His research interests include complex systems and forecasting.
Michael H. Breitner is a professor for Information Systems Research at Leibniz University of Hanover, Germany. His research interests include artificial intelligence, especially artificial neural networks.