Elsevier

Decision Support Systems

Volume 64, August 2014, Pages 100-108
Decision Support Systems

Intelligent trading of seasonal effects: A decision support algorithm based on reinforcement learning

https://doi.org/10.1016/j.dss.2014.04.011Get rights and content

Highlights

  • We show the chances of a trading system based on seasonalities in financial markets.

  • We introduce a decision support algorithm to filter trading signals.

  • The algorithm is based on reinforcement learning and neural networks.

  • We improve the reward to risk ratios of the seasonality strategy.

Abstract

Seasonalities and empirical regularities on financial markets have been well documented in the literature for three decades. While one should suppose that documenting an arbitrage opportunity makes it vanish there are several regularities that have persisted over the years. These include, for example, upward biases at the turn-of-the-month, during exchange holidays and the pre-FOMC announcement drift. Trading regularities is already in and of itself an interesting strategy. However, unfiltered trading leads to potential large drawdowns. In the paper we present a decision support algorithm which uses the powerful ideas of reinforcement learning in order to improve the economic benefits of the basic seasonality strategy. We document the performance on two major stock indices.

Introduction

Seasonalities and empirical regularities on financial markets are one of the most frequently studied phenomena in the scientific literature. This is due to simple but promising assumptions, which can easily be translated into a trading strategy. Hence, only investigating this phenomenon has lost its appeal. Therefore, it is our motivation, to not simply create a trading system based on seasonalities, but to verify the signals of this strategy with an intelligent filter in order to provide a robust decision support.

The procedure for filtering is the focus of our work. We use the promising approach of reinforcement learning (RL) to realize an effective filtering. This heuristic method is often used in unstructured and complex situations and provides very good results in the field of robotics, but also increasingly in economic decision-making. Our goal is to find a policy with RL in order to filter the output signals of the basic strategy to improve the reward to risk ratios.

A novel approach is used. To link the trading decision with a reward, we use an artificial neural network (ANN). The same ANN (a simple three layer feedforward network) acts as decision support to determine the optimal parameters for future trades. We use a combination of three major research areas (RL, ANN, seasonalities). We only introduce the basics of each topic. For an in-depth insight many suggestions can be found in the corresponding section. This paper aims to demonstrate the strength of a combination of several economic and interdisciplinary methods.

Our paper is divided into the following parts. Section 2 presents the ideas and methods. Section 2.1 describes the results of a brief analysis on seasonalities. It shows the promising approach of this surprisingly simple strategy. We investigate two major indices (DAX and S&P 500) on detectable trading regularities like upward biases at the turn-of-the-month, exchange holidays and the pre-FOMC (Federal Open Market Committee) announcement drift. Section 2.2 deals with RL and the detailed description of our modified version. Section 2.3 offers a small introduction to the world of artificial neural networks. Section 3 shows a merger of the mentioned disciplines in a fully automated and self-learning trading system. In Section 4 the results are presented and discussed. Section 5 discusses possible limitations of the strategy and provides an outlook for further research. Appendix A shows the complete algorithm in pseudocode.

Section snippets

Ideas and methods

In this section we present an overview of the methods and ideas. First, previous studies about seasonalities on financial markets are introduced in Section 2.1. After that we test whether the basic approach used for our subsequent programming is performing significantly better than a random strategy. Furthermore, in Section 2.2 reinforcement learning and our modifications are described. Section 2.3 gives an insight into the field of artificial neural networks.

Trading system

The basic idea is to use the detectable seasonalities and empirical regularities for an automated trading system. But the question is how this assumption of rising prices after a trade event can be transformed into a profitable behavior. Our research shows that it is difficult to develop firm rules. Obeying fixed rules may lead to large drawdowns. Depending on market conditions, e.g. a different leverage or a customized holding period could be useful. Inactivity or even betting on a price

Results and discussion

The results of the RL strategy were generated by using a self-developed simulation program. The software written in Java mimics a market with real prices of the past (2000–2012) and is also able to create and train neural networks. Thus the agent can be tested realistically over several years and with different parameters (e.g. with different probabilities for exploration). For a detailed analysis learning logs were stored, which give information on how the agent works. The benchmark is a

Conclusion, limitations, and future research

In this paper three major research fields are combined in an intelligent, self-learning and fully automated trading system. The basis is the promising strategy of seasonalities and empirical regularities in financial markets which has been described and discussed in the literature for three decades. For the period from 2000 to 2012, the significance of upward biases at the turn-of-the-month, during exchange holidays and the pre-FOMC announcement drift could be confirmed in many cases. Based on

Acknowledgment

We would like to thank the anonymous reviewers for their interesting and valuable comments to improve the quality of this paper.

Dennis Eilers is a Bachelor Student at Leibniz University of Hanover, Germany. His research interests include machine learning algorithms for intelligent trading strategies.

References (37)

  • Shimon Schocken et al.

    Neural networks for decision support: problems and opportunities

    Decision Support Systems

    (June 1994)
  • Georgios Sermpinis et al.

    Forecasting and trading the EUR/USD exchange rate with stochastic Neural Network combination and time-varying leverage

    Decision Support Systems

    (December 2012)
  • Ryan Sullivan et al.

    Dangers of data mining: the case of calendar effects in stock returns

    Journal of Econometrics

    (November 2001)
  • Guoqiang Zhang et al.

    Forecasting with artificial neural networks: the state of the art

    International Journal of Forecasting

    (March 1998)
  • Sandro C. Andrade et al.

    Sell in May and go away just won't go away

    Financial Analysts Journal

    (July/August 2013)
  • Eric B. Baum et al.

    What size net gives valid generalization?

    Neural Computation

    (Spring 1989)
  • Francesco Bertoluzzo et al.

    Making financial trading by recurrent reinforcement learning

    Knowledge-Based Intelligent Information and Engineering Systems

    (September 12–14 2007)
  • Christopher M. Bishop

    Neural Networks for Pattern Recognition

    (1995)
  • Cited by (42)

    • Frequency matching optimization model of ultrasonic scalpel transducer based on neural network and reinforcement learning

      2023, Engineering Applications of Artificial Intelligence
      Citation Excerpt :

      Q-learning algorithm has the characteristics of continuous learning and dynamic adjustment depending on the dynamic working environment (instantaneous characteristic variables). It can select new action vectors according to the value table corresponding to the state action set, and the controller can adjust the state action set continuously according to the new state to maintain the matching between the input and output frequencies of neural networks so that the transducer can realize the stable state of resonance at the maximum amplitude (Eilers et al., 2014). In the resonant state, the amplitude of a transducer is significantly increased and the stable vibration and the highest energy conversion efficiency are realized.

    • Reinforcement learning based framework for COVID-19 resource allocation

      2022, Computers and Industrial Engineering
      Citation Excerpt :

      Reinforcement learning is a machine learning technique that understands and automatically processes goal-oriented learning and decision-making problems. In recent years, reinforcement learning has been applied to many fields and achieved remarkable results (Eilers, Dunis, von Mettenheim, & Breitner, 2014; Silver et al., 2017; Brown et al., 2019). There are many literatures on the application of reinforcement learning methods in supply chain management and resource allocation (Shahrabi, Adibi, & Mahootchi, 2017; Chen, Yang, Li, & Wang, 2020).

    • Deep reinforcement learning based trading agents: Risk curiosity driven learning for financial rules-based policy

      2021, Expert Systems with Applications
      Citation Excerpt :

      Precisely, RL has shown promising results on the financial and economic problems (Hull, 2014; Li, Zheng, & Zheng, 2019). Eilers et al. (2014) present a decision support algorithm which uses the powerful ideas of RL in order to improve the economic benefits of the basic seasonality strategy. However, these approaches require large amounts of training data, but the resulting policies are not immediately generalized to novel tasks in the same environment.

    • An intelligent financial portfolio trading strategy using deep Q-learning

      2020, Expert Systems with Applications
      Citation Excerpt :

      Bertoluzzo and Corazza (2012) suggested a single-asset trading system using Q-learning with linear and kernel function approximations. Eilers, Dunis, Mettenheim, and Breitner (2014) developed a trading rule for an asset with a seasonal price trend using Q-learning. Zhang et al. (2015) derived a trading rule generator using extended classifier systems combined with RL and a genetic algorithm.

    View all citing articles on Scopus

    Dennis Eilers is a Bachelor Student at Leibniz University of Hanover, Germany. His research interests include machine learning algorithms for intelligent trading strategies.

    Christian Dunis is an emeritus professor of Banking and Finance from Liverpool John Moores University. He currently works as Risk Manager at a Swiss private bank. His research interests include quantitative trading strategies and artificial intelligence.

    Hans-Jörg von Mettenheim is a professor for Decision Support Systems at Leibniz University of Hanover, Germany. His research interests include complex systems and forecasting.

    Michael H. Breitner is a professor for Information Systems Research at Leibniz University of Hanover, Germany. His research interests include artificial intelligence, especially artificial neural networks.

    View full text