Skip to main content
Springer Nature Link
Log in

Improving Portfolio Performance Using a Novel Method for Predicting Financial Regimes

  • Conference paper
  • First Online:
Machine Learning, Optimization, and Data Science (LOD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14506))

  • 568 Accesses

Abstract

This work extends a previous work in regime detection, which allowed trading positions to be profitably adjusted when a new regime was detected, to ex ante prediction of regimes, leading to substantial performance improvements over the earlier model, over all three asset classes considered (equities, commodities, and foreign exchange), over a test period of four years. The proposed new model is also benchmarked over this same period against a hidden Markov model, the most popular current model for financial regime prediction, and against an appropriate index benchmark for each asset class, in the case of the commodities model having a test period cost-adjusted cumulative return over four times higher than that expected from the index. Notably, the proposed model makes use of a contrarian trading strategy, not uncommon in the financial industry but relatively unexplored in machine learning models. The model also makes use of frequent short positions, something not always desirable to investors due to issues of both financial risk and ethics; however, it is discussed how further work could remove this reliance on shorting and allow the construction of a long-only version of the model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/bukosabino/ta. Last accessed 29 March 2023.

  2. 2.

    https://scikit-learn.org/stable/. Last accessed 6 September 2022.

  3. 3.

    https://www.kaggle.com/code/marketneutral/purged-time-series-cv-xgboost-optuna. Last accessed 12 July 2023.

  4. 4.

    https://github.com/Ekeany/Boruta-Shap. Last accessed 5 April 2023.

References

  1. Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2623–2631 (2019). https://doi.org/10.1145/3292500.3330701

  2. Aldrich, J.H., Nelson, F.D.: Linear Probability, Logit, and Probit Models. Sage (1984)

    Google Scholar 

  3. Ballings, M., Van den Poel, D., Hespeels, N., Gryp, R.: Evaluating multiple classifiers for stock price direction prediction. Expert Syst. Appl. 42(20), 7046–7056 (2015). https://doi.org/10.1016/j.eswa.2015.05.013

    Article  Google Scholar 

  4. Baum, L.E., Petrie, T.: Statistical inference for probabilistic functions of finite state Markov chains. Ann. Math. Stat. 37(6), 1554–1563 (1966). https://www.jstor.org/stable/2238772

  5. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). https://doi.org/10.1023/A:1010933404324

    Article  Google Scholar 

  6. Chicco, D., Jurman, G.: The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genom. 21(1), 1–13 (2020). https://doi.org/10.1186/s12864-019-6413-7

    Article  Google Scholar 

  7. Christ, M., Kempa-Liehr, A.W., Feindt, M.: Distributed and parallel time series feature extraction for industrial big data applications. arXiv preprint arXiv:1610.07717 (2016). https://doi.org/10.48550/arXiv.1610.07717

  8. De Prado, M.L.: Advances in Financial Machine Learning. Wiley, Hoboken (2018)

    Google Scholar 

  9. Giudici, P., Abu Hashish, I.: A hidden Markov model to detect regime changes in cryptoasset markets. Qual. Reliab. Eng. Int. 36(6), 2057–2065 (2020). https://doi.org/10.1002/qre.2673

    Article  Google Scholar 

  10. Hosking, J.: Fractional differencing. Biometrika 68, 165–175 (1981)

    Article  MathSciNet  Google Scholar 

  11. Israelsen, C.: A refinement to the Sharpe ratio and information ratio. J. Asset Manag. 5(6), 423–427 (2005). https://doi.org/10.1057/palgrave.jam.2240158

    Article  Google Scholar 

  12. Kaufman, P.: Smarter Trading: Improving Performance in Changing Markets. McGraw-Hill, New York (1995)

    Google Scholar 

  13. Krolzig, H.: Markov-Switching Vector Autoregressions: Modelling, Statistical Inference, and Application to Business Cycle Analysis. Springer, New York (1997). https://doi.org/10.1007/978-3-642-51684-9

    Book  Google Scholar 

  14. Kursa, M.B., Rudnicki, W.R.: Feature selection with the Boruta package. J. Stat. Softw. 36, 1–13 (2010). https://doi.org/10.18637/jss.v036.i11

    Article  Google Scholar 

  15. Lee, Y., Ow, L.T.C., Ling, D.N.C.: Hidden Markov models for forex trends prediction. In: 2014 International Conference on Information Science & Applications (ICISA), pp. 1–4. IEEE (2014). https://doi.org/10.1109/ICISA.2014.6847408

  16. Lo, A.W., MacKinlay, A.C.: A Non-random Walk Down Wall Street. Princeton University Press, Princeton (2011)

    Book  Google Scholar 

  17. Matthews, B.W.: Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim. Biophys. Acta (BBA)-Protein Struct. 405(2), 442–451 (1975). https://doi.org/10.1016/0005-2795(75)90109-9

  18. Milosevic, N.: Equity forecast: predicting long term stock price movement using machine learning. arXiv preprint arXiv:1603.00751 (2016). https://doi.org/10.48550/arXiv.1603.00751

  19. Pavlov, V., Hurn, S.: Testing the profitability of moving-average rules as a portfolio selection strategy. Pac. Basin Financ. J. 20(5), 825–842 (2012). https://doi.org/10.1016/j.pacfin.2012.04.003

    Article  Google Scholar 

  20. Piger, J.: Turning points and classification. In: Fuleky, P. (ed.) Macroeconomic Forecasting in the Era of Big Data. ASTAE, vol. 52, pp. 585–624. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-31150-6_18

    Chapter  Google Scholar 

  21. Pomorski, P.: Features and assets used in this work (2023). https://figshare.com/articles/conference_contribution/LOD2023_appendix/23681205

  22. Pomorski, P., Gorse, D.: Improving on the Markov-switching regression model by the use of an adaptive moving average. In: Gartner, W.C. (ed.) New Perspectives and Paradigms in Applied Economics and Business. Springer Proceedings in Business and Economics, pp. 17–30. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-23844-4_2

    Chapter  Google Scholar 

  23. Shapley, L.S., et al.: A Value for N-Person Games (1953)

    Google Scholar 

  24. e Silva, E.G.D.S., Legey, L.F.L., e Silva, E.A.D.S.: Forecasting oil price trends using wavelets and hidden Markov models. Energy Econ. 32(6), 1507–1519 (2010). https://doi.org/10.1016/j.eneco.2010.08.006

  25. Strobl, C., Boulesteix, A.L., Zeileis, A., Hothorn, T.: Bias in random forest variable importance measures: illustrations, sources and a solution. BMC Bioinform. 8(1), 1–21 (2007). https://doi.org/10.1186/1471-2105-8-25

    Article  Google Scholar 

  26. Uysal, A.S., Mulvey, J.M.: A machine learning approach in regime-switching risk parity portfolios. J. Financ. Data Sci. 3(2), 87–108 (2021). https://doi.org/10.3905/jfds.2021.1.057

    Article  Google Scholar 

  27. Vrontos, S.D., Galakis, J., Vrontos, I.D.: Modeling and predicting US recessions using machine learning techniques. Int. J. Forecast. 37(2), 647–671 (2021). https://doi.org/10.1016/j.ijforecast.2020.08.005

    Article  Google Scholar 

  28. Ward, F.: Spotting the danger zone: forecasting financial crises with classification tree ensembles and many predictors. J. Appl. Economet. 32(2), 359–378 (2017). https://doi.org/10.1002/jae.2525

    Article  MathSciNet  Google Scholar 

  29. Yazdani, A.: Machine learning prediction of recessions: an imbalanced classification approach. J. Financ. Data Sci. 2(4), 21–32 (2020). https://doi.org/10.3905/jfds.2020.1.040

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University College London, London, WC1E 6BT, UK

    Piotr Pomorski & Denise Gorse

Authors
  1. Piotr Pomorski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Denise Gorse
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Denise Gorse .

Editor information

Editors and Affiliations

  1. University of Catania, Catania, Catania, Italy

    Giuseppe Nicosia

  2. Newcastle University, Newcastle upon Tyne, UK

    Varun Ojha

  3. University of Oxford, Oxford, UK

    Emanuele La Malfa

  4. University of Cambridge, Cambridge, UK

    Gabriele La Malfa

  5. University of Florida, Gainesville, FL, USA

    Panos M. Pardalos

  6. Dana-Farber Cancer Institute, Boston, MA, USA

    Renato Umeton

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pomorski, P., Gorse, D. (2024). Improving Portfolio Performance Using a Novel Method for Predicting Financial Regimes. In: Nicosia, G., Ojha, V., La Malfa, E., La Malfa, G., Pardalos, P.M., Umeton, R. (eds) Machine Learning, Optimization, and Data Science. LOD 2023. Lecture Notes in Computer Science, vol 14506. Springer, Cham. https://doi.org/10.1007/978-3-031-53966-4_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53966-4_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53965-7

  • Online ISBN: 978-3-031-53966-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics