Elsevier

Applied Soft Computing

Volume 52, March 2017, Pages 1143-1153
Applied Soft Computing

Incorporating Markov decision process on genetic algorithms to formulate trading strategies for stock markets

https://doi.org/10.1016/j.asoc.2016.09.016Get rights and content

Highlights

  • The paper proposed a novel application for incorporating Markov decision process on genetic algorithms to develop stock trading strategies.

  • This predicts the results of applying the Markov decision process with real-time computational power to help investors formulate correct timing (portfolio adjustment) and trading strategies (buy or sell).

  • This study thus uses the excellent genetic algorithm parallel space searching ability to provide investors with the optimal stock selection strategy and capital allocation, and combines them with both constructs to solve the portfolio problem and improve return on investment for investors.

  • This research can solve stock selection, market timing and capital allocation at the same time for investors when investing in the stock market.

Abstract

With the arrival of low interest rates, investors entered the stock market to seek higher returns. However, the stock market proved volatile, and only rarely could investors gain excess returns when trading in real time. Most investors use technical indicators to time the market. However the use of technical indicators is associated with problems, such as indicator selection, use of conflicting versus similar indicators. Investors thus have difficulty relying on technical indicators to make stock market investment decisions.

This research combines Markov decision process and genetic algorithms to propose a new analytical framework and develop a decision support system for devising stock trading strategies. This investigation uses the prediction characteristics and real-time analysis capabilities of the Markov decision process to make timing decisions. The stock selection and capital allocation employ string encoding to express different investment strategies for genetic algorithms. The parallel search capabilities of genetic algorithms are applied to identify the best investment strategy. Additionally, when investors lack sufficient money and stock, the architecture of this study can complete the transaction via credit transactions. The experiments confirm that the model presented in this research can yield higher rewards than other benchmarks.

Introduction

Increased globalization has led the government to open numerous investment channels. However, as opportunities have increased, so too has risk. How to grasp opportunity and avoid risk thus has become a major issue in industry and academia.

Stock investment is what most individuals choose for their personal investment. Television broadcasts increasingly numerous equity investment analysis programs. Regardless of the state of the economy, provided investors make good decisions equity analysis can still be profitable.

Correct timing and risk-aversion have always been important in investment. Previous studies mostly use fundamental or technical analysis to solve these key issues. The stock market is an open market, and numerous external factors can affect stock prices. Internal messages, government industrial policy or the financial conditions of individual firms can all influence corporate stock prices. Thus it is important to build a profitable and accurate investment model.

With the recent rapid development of science and technology, artificial intelligence has achieved significant progress. Scholars began to apply artificial intelligence financial management research. Artificial intelligence theories have been effectively applied to investment and become human decision-making tools. Famous techniques include neural networks [30], genetic algorithms [32], genetic programming [16], fuzzy theories [17] and decision trees [39].

Two forms of stock investment analysis exist, one is a mathematical theory-based approach that tries to represent stock trading behavior by creating a stock market model to identify an interpretable profit. However, traditional mathematical theories are inadequate to explain the problem of non-linear, complex, and multi-targeted models and thus successfully profit from the stock market. Another method is based on artificial intelligence studies that forecast stock values via machine learning to construct a model and help investors make investment decisions, such as artificial neural networks [48]. However, input factors of artificial neural networks are difficult to define and select.

The implied parallel processing space search capabilities offer fast speed, high reliability, and flexibility due to the space search speed of genetic algorithms, and thus offer entirely new computational methods [26]. Based on their powerful search skills and combinatorial optimization solving ability, genetic algorithms can help solve complex problems such as portfolio selection and capital allocation in relation to stock investments. The Markov decision process involves the use of mathematical theory to achieve a forecast by sequential derivation function in the Markov chain. Markov decision process, which is capable of prediction, is suitable to solve stock investment problems [19].

Previously, most studies on stock timing focused on technical analysis, but the use of technical indicators frequently is often problematic. For example, investors can encounter difficulty in choosing the type or number of technical indicators and how to resolve similarity or contradiction between technical indicators. Most previous literature on the stock market considers only reactions to general transactions while ignoring credit transactions, and cannot take into account whether trading on margin is permitted or ownership is insufficient. This investigation thus combines Markov decision-making and genetic algorithms to help investors make the best decisions, such as those regarding selection, timing and capital allocation, when facing stock market uncertainty. This paper also considers credit transactions.

This study combines the Markov decision process and genetic algorithms available to investors in the stock investment strategy and uses them to develop a decision support system. This predicts the results of applying the Markov decision process with real-time computational power to help investors devise correct timing and trading strategies. This study thus uses the excellent genetic algorithm parallel space searching ability to provide investors with the optimal stock selection strategy and capital allocation, and combines them with both constructs to solve the portfolio problem and improve return on investment for investors.

Section snippets

Literature review

The stock market requires investors to consider risk and return. To diversify their investment risk, investors may allocate their funds among different targets. A complete investment strategy should incorporate stock selection, timing and capital allocation.

Timing aims to identify the best opportunity to buy or sell a stock. Korczak et al. [35], Allen et al. [1], Chang chien et al. [7], Jiang et al. [29] Huang et al. [27], Bao [3] and Badawy et al. [4] combined the genetic algorithms and other

Research framework

This research proposed a new analytical framework for the general portfolio stock trading. Calculation using the expected profit can yield timing signals for portfolio adjustment and develop a daily strategy regarding stocks to buy, sell or hold. When combined with genetic algorithms, this transaction system can perform stock selection and capital allocation. That is this study encoded the capital allocation (0%–100%) for each stock for portfolio into the chromosome of the genetic algorithm. By

Experimental analysis

This study apply the new analytical framework on Taiwan's stock market to prove the validity of the new research framework. In the experiments, the eight categories listed in the index can serve as investment targets to simulate the stock trading process. Markov decision process can identify the timing of the adaptive portfolio and determine the trading strategies. The Markov decision process combined with genetic algorithm can solve capital allocation problems. The experimental results were

Conclusions and future research directions

This investigation combines the Markov decision process and genetic algorithms to provide a stock investment decision support system for stock market investment. The Markov decision process can help investors solve timing problems. This method differs from the technical indicators traditionally used to forecast market timing. Markov chain can predict whether investors must adjust their portfolios and the optimal trading strategy. Genetic algorithms have powerful spatial parallel search

References (53)

  • S. Gelman et al.

    Markov-switching in target stocks during takeover bids

    J. Empir. Finance

    (2009)
  • L. Ghezzi et al.

    Stock valuation along a Markov chain

    Appl. Math. Comput.

    (2003)
  • S. Gold et al.

    Computerized stock screening rules for portfolio selection

    Financial Serv. Rev.

    (1999)
  • A. Gorgulho et al.

    Applying a GA kernel on optimizing technical analysis rules for stock picking and portfolio composition

    Expert Syst. Appl.

    (2011)
  • M.R. Hassan

    A combination of hidden Markov model and fuzzy model for stock market forecasting

    Neurocomputing

    (2009)
  • Y.T. Hsu et al.

    Forecasting the turning time of stock market based on Markov-fourier grey model

    Expert Syst. Appl.

    (2009)
  • C.F. Huang

    A hybrid stock selection model using genetic algorithms and support vector regression

    Appl. Soft Comput.

    (2012)
  • S.C. Huang et al.

    Integrating GA-based time-scale feature extractions with SVMs for stock index forecasting

    Expert Syst. Appl.

    (2008)
  • Y. Kara et al.

    Predicting direction of stock price index movement using artificial neural networks and support vector machines: the sample of the Istanbul stock exchange

    Expert Syst. Appl.

    (2011)
  • H.J. Kim et al.

    A hybrid approach based on neural networks and genetic algorithms for detecing temporal patterns in stock markets

    Appl. Soft Comput.

    (2007)
  • S. Papadamou et al.

    Improving technical trading systems by using a new MATLAB-based genetic algorithm procedure

    Math. Comput. Modell.

    (2007)
  • H. Soleimani et al.

    Markowitz-based portfolio selection with minimum transaction lots, cardinality constraints and regarding sector capitalization using genetic algorithm

    Expert Syst. Appl.

    (2009)
  • J.Z. Wang et al.

    Forecasting stock indices with back propagation neural network

    Expert Syst. Appl.

    (2011)
  • Y.F. Wang et al.

    Incorporating the Markov chain concept into fuzzy stochastic prediction of stock indexes

    Appl. Soft Comput.

    (2010)
  • M. Versace et al.

    Predicting the exchange traded fund DIA with a combination of genetic algorithms and neural networks

    Expert Syst. Appl.

    (2004)
  • Y. Bao

    Stock market prediction model based on genetic algorithm and support vector regression

  • Cited by (41)

    • Multi-agent model based proactive risk management for equity investment

      2023, Engineering Applications of Artificial Intelligence
    • On multi-scale event-based optimization

      2023, Results in Control and Optimization
      Citation Excerpt :

      Thus, modeling, simulation and optimization of this kind of multi-scale practical systems has outstanding practical significance, and has been paid attention to by the academic and industry. For a long time, Markov decision process (MDP) has been widely used as a general tool for stochastic optimization of large-scale systems and applied in areas such as Internet of Things [1], autopilot [2], supply and demand matching [3], financial markets [4]. However, it still has some difficulties in practical application.

    • StockNet—GRU based stock index prediction

      2022, Expert Systems with Applications
      Citation Excerpt :

      Real stock market data does not satisfy the stationary and linearity assumption. To overcome the drawbacks of traditional models i.e., to model the non-linearity, machine learning approaches have been developed such as support vector machine (SVM) (Patel, Shah, Thakkar, & Kotecha, 2015a; Qian & Gao, 2017), genetic algorithm (Chang & Lee, 2017), fuzzy logic (Dutta, Biswal, Acharya, & Mishra, 2018) and hybrid models (Patel, Shah, Thakkar, & Kotecha, 2015b). In the past few years, enhanced machine learning approaches have been replaced by deep learning models in stock market prediction.

    • The flying sidekick traveling salesman problem with stochastic travel time: A reinforcement learning approach

      2022, Transportation Research Part E: Logistics and Transportation Review
    • Forecasting the overnight return direction of stock market index combining global market indices: A multiple-branch deep learning approach

      2022, Expert Systems with Applications
      Citation Excerpt :

      In machine learning techniques, hyperparameters have significant impact on the predictive performance. Therefore, various evolutionary algorithms, such as particle swarm optimization (PSO) (Hung, 2011; Liu et al., 2018), firefly algorithm (FFA) (Kazem et al., 2013), and genetic algorithm (GA) (Chang & Lee, 2017; Pan et al., 2017), have been used to optimize these parameters. Among them, the GA inspired by the Darwin’s theory is an effective optimization method because of its parallel search mechanism.

    • Share price prediction of aerospace relevant companies with recurrent neural networks based on PCA

      2021, Expert Systems with Applications
      Citation Excerpt :

      Evolutionary optimization with wrapper techniques is often used to optimize machine learning models or select features for stock price (trend) prediction. For example, Harmony Search and Genetic Algorithm was used to find the best structure of neural network (Göçken et al., 2016), neural network with greedy algorithm based on the feature reduction with wrapper techniques for the prediction of stock price trend (Lertyingyod & Benjamas, 2017), simulated annealing algorithm was used to optimise feature space and model parameters (Torun & Tohumolu, 2011), and a Markov decision process was incorporated on genetic algorithms to develop stock trading strategies (Chang & Lee, 2017). Many other techniques were used to improve the performance of computational finance.

    View all citing articles on Scopus
    View full text