skip to main content
10.1145/3604237.3627721acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicaifConference Proceedingsconference-collections
research-article

LLMs Analyzing the Analysts: Do BERT and GPT Extract More Value from Financial Analyst Reports?

Published: 25 November 2023 Publication History

Abstract

This paper examines the use of Large Language Models (LLMs), specifically BERT-based models and GPT-3.5, in the sentiment analysis of Korean financial analyst reports. Due to the specialized language in these reports, traditional natural language processing techniques often prove insufficient, making LLMs a better alternative. These models are capable of understanding the complexity and subtlety of the language, allowing for a more nuanced interpretation of the data. We focus our study on the extraction of sentiment scores from these reports, using them to construct and test investment strategies. Given that Korean analyst reports present unique linguistic challenges and a significant ‘buy’ recommendation bias, we employ LLMs fine-tuned for the Korean language and Korean financial texts. The aim of this study is to investigate and compare the effectiveness of LLMs in enhancing the sentiment analysis of financial reports, and subsequently utilize the sentiment scores to construct and test investment strategies, thereby evaluating these models’ potential in extracting valuable insights from the reports. The code is available at https://github.com/msraask3.

References

[1]
AI4Finance-Foundation. 2023. FinGPT Repository. https://github.com/AI4Finance-Foundation/FinGPT.
[2]
Dogu Araci. 2019. Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063 (2019).
[3]
Yalanati Ayyappa, B Vinay Kumar, Sudhabatthula Padma Priya, Siddiboena Akhila, Tamma Priya Vardhan Reddy, and Shaik Mohammad Goush. 2023. Forecasting Equity Prices using LSTM and BERT with Sentiment Analysis. In 2023 International Conference on Inventive Computation Technologies (ICICT). IEEE, 643–648.
[4]
Aysun Bozanta, Sabrina Angco, Mucahit Cevik, and Ayse Basar. 2021. Sentiment Analysis of StockTwits Using Transformer Models. In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE, 1253–1258.
[5]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
[6]
Jeffrey A Busse, T Clifton Green, and Narasimhan Jegadeesh. 2012. Buy-side trades and sell-side recommendations: Interactions and information content. Journal of Financial markets 15, 2 (2012), 207–232.
[7]
Hailiang Chen, Prabuddha De, Yu Jeffrey Hu, and Byoung-Hyoun Hwang. 2014. Wisdom of crowds: The value of stock opinions transmitted through social media. Review of Financial Studies 27, 5 (2014), 1367–1403.
[8]
Poongjin Cho, Ji Hwan Park, and Jae Wook Song. 2021. Equity research report-driven investment strategy in Korea using binary classification on stock price direction. IEEE Access 9 (2021), 46364–46373.
[9]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[10]
Michael Dowling and Brian Lucey. 2023. ChatGPT for (finance) research: The Bananarama conjecture. Finance Research Letters 53 (2023), 103662.
[11]
Jennifer Francis and Leonard Soffer. 1997. The relative informativeness of analysts’ stock recommendations and earnings forecast revisions. Journal of Accounting Research 35, 2 (1997), 193–211.
[12]
Alexander Glodd and Diana Hristova. 2023. Extraction of Forward-looking Financial Information for Stock Price Prediction from Annual Reports Using NLP Techniques. (2023).
[13]
Anne Lundgaard Hansen and Sophia Kazinnik. 2023. Can ChatGPT Decipher Fedspeak?Available at SSRN (2023).
[14]
Frank Hodge and Maarten Pronk. 2006. The impact of expertise and investment familiarity on investors’ use of online financial report information. Journal of Accounting, Auditing & Finance 21, 3 (2006), 267–292.
[15]
Allen H Huang, Amy Y Zang, and Rong Zheng. 2014. Evidence on the information content of text in analyst reports. The Accounting Review 89, 6 (2014), 2151–2180.
[16]
Narasimhan Jegadeesh, Joonghyuk Kim, Susan D Krische, and Charles MC Lee. 2004. Analyzing the analysts: When do recommendations add value?The journal of finance 59, 3 (2004), 1083–1124.
[17]
Narasimhan Jegadeesh and Woojin Kim. 2006. Value of analyst recommendations: International evidence. Journal of Financial Markets 9, 3 (2006), 274–309.
[18]
Jang Ho Kim. 2023. What if ChatGPT were a quant asset manager. Finance Research Letters (2023), 104580.
[19]
KPMG. 2014. A New Vision of Value: Connecting Corporate and Societal Value Creation. Technical Report. KPMG International. https://assets.kpmg.com/content/dam/kpmg/pdf/2014/06/kpmg-survey-business-reporting.pdf Accessed: 21-06-2023.
[20]
Sangah Lee, Hansol Jang, Yunmee Baik, Suzi Park, and Hyopil Shin. 2020. Kr-bert: A small-scale korean-specific language model. arXiv preprint arXiv:2008.03979 (2020).
[21]
Yongjae Lee, John RJ Thompson, Jang Ho Kim, Woo Chang Kim, Francesco A Fabozzi, Frank J Fabozzi, Jim Musumeci, Bruce Feibel, Bradford Cornell, Zoltán Nagy, 2023. An overview of machine learning for asset management. The Journal of Portfolio Management 49, 9 (2023), 31–63.
[22]
Edmund Kwong Wei Leow, Binh P Nguyen, and Matthew Chin Heng Chua. 2021. Robo-advisor using genetic algorithm and BERT sentiments from tweets for hybrid portfolio optimisation. Expert Systems with Applications 179 (2021), 115060.
[23]
Baruch Lev and Feng Gu. 2016. The end of accounting and the path forward for investors and managers. John Wiley & Sons.
[24]
Mei hua Liao and Chia Yun Chang. 2014. Analysts’ Forecasts and Institutional Investors’ Behavior. In Proceedings of the 2014 Eighth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing. 575–579.
[25]
Hsiou-wei Lin and Maureen F McNichols. 1998. Underwriting relationships, analysts’ earnings forecasts and investment recommendations. Journal of accounting and economics 25, 1 (1998), 101–127.
[26]
Yu-Wen Liu, Liang-Chih Liu, Chuan-Ju Wang, and Ming-Feng Tsai. 2018. Riskfinder: A sentence-level risk detector for financial reports. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations. 81–85.
[27]
Andrew W Lo, Manish Singh, Jim Musumeci, Zoltán Nagy, Guido Giese, Xinxin Wang, Andre Mirabelli, Nick Keywork, Avi Turetsky, Barry Griffiths, 2023. From ELIZA to ChatGPT: The Evolution of Natural Language Processing and Financial Applications. The Journal of Portfolio Management (2023).
[28]
Alejandro Lopez-Lira and Yuehua Tang. 2023. Can chatgpt forecast stock price movements. Return Predictability and Large Language Models (2023).
[29]
Tim Loughran and Bill McDonald. 2011. When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks. The Journal of finance 66, 1 (2011), 35–65.
[30]
Tim Loughran and Bill McDonald. 2016. Textual analysis in accounting and finance: A survey. Journal of Accounting Research 54, 4 (2016), 1187–1230.
[31]
Thien Hai Nguyen, Kiyoaki Shirai, and Julien Velcin. 2015. Sentiment analysis on social media for stock movement prediction. In International Conference on Advanced Data Mining and Applications. Springer, 227–240.
[32]
Derya Othan, Zeynep Hilal Kilimci, and Mitat Uysal. 2019. Financial sentiment analysis for predicting direction of stocks using bidirectional encoder representations from transformers (BERT) and deep learning models. In Proc. Int. Conf. Innov. Intell. Technol. 30–35.
[33]
Kyubyong Park, Joohong Lee, Seongbo Jang, and Dawoon Jung. 2020. An empirical study of tokenization strategies for various Korean NLP tasks. arXiv preprint arXiv:2010.02534 (2020).
[34]
Jason D Rennie, Lawrence Shih, Jaime Teevan, and David R Karger. 2003. Tackling the poor assumptions of naive bayes text classifiers. In Proceedings of the 20th international conference on machine learning (ICML-03). 616–623.
[35]
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019).
[36]
Emre Şaşmaz and F Boray Tek. 2021. Tweet sentiment analysis for cryptocurrencies. In 2021 6th International Conference on Computer Science and Engineering (UBMK). IEEE, 613–618.
[37]
Matheus Gomes Sousa, Kenzo Sakiyama, Lucas de Souza Rodrigues, Pedro Henrique Moraes, Eraldo Rezende Fernandes, and Edson Takashi Matsubara. 2019. BERT for stock market sentiment analysis. In 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI). IEEE, 1597–1601.
[38]
Ming-Feng Tsai and Chuan-Ju Wang. 2017. On the risk prediction and analysis of soft information in finance reports. European Journal of Operational Research 257, 1 (2017), 243–250.
[39]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[40]
Antti Virtanen, Jenna Kanerva, Rami Ilo, Jouni Luoma, Juhani Luotolahti, Tapio Salakoski, Filip Ginter, and Sampo Pyysalo. 2019. Multilingual is not enough: BERT for Finnish. arXiv preprint arXiv:1912.07076 (2019).
[41]
Sida I Wang and Christopher D Manning. 2012. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 90–94.
[42]
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564 (2023).
[43]
Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. 2023. FinGPT: Open-Source Financial Large Language Models. arXiv preprint arXiv:2306.06031 (2023).
[44]
Yi Yang, Mark Christopher Siy Uy, and Allen Huang. 2020. Finbert: A pretrained language model for financial communications. arXiv preprint arXiv:2006.08097 (2020).
[45]
Ming Zhang, Jiahao Yang, Meilin Wan, Xuejun Zhang, and Jun Zhou. 2022. Predicting long-term stock movements with fused textual features of Chinese research reports. Expert Systems with Applications 210 (2022), 118312. https://doi.org/10.1016/j.eswa.2022.118312

Cited By

View all
  • (2024)Navigating Complexity: GPT-4's Performance in Predicting Earnings and Stock Returns in China's A-Share MarketHighlights in Business, Economics and Management10.54097/4rwdat9542(189-203)Online publication date: 19-Nov-2024
  • (2024)Approach to a GPT-based Early Detection Tool to Evaluate Heterogeneous Data Sources and Identify Reconfiguration Needs of SMEs in the Production SectorProcedia CIRP10.1016/j.procir.2024.10.140130(631-636)Online publication date: 2024
  • (2024)Multi-attention recommender system for non-fungible tokensEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109179137(109179)Online publication date: Nov-2024
  • Show More Cited By

Index Terms

  1. LLMs Analyzing the Analysts: Do BERT and GPT Extract More Value from Financial Analyst Reports?
            Index terms have been assigned to the content through auto-classification.

            Recommendations

            Comments

            Information & Contributors

            Information

            Published In

            cover image ACM Other conferences
            ICAIF '23: Proceedings of the Fourth ACM International Conference on AI in Finance
            November 2023
            697 pages
            ISBN:9798400702402
            DOI:10.1145/3604237
            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            Published: 25 November 2023

            Permissions

            Request permissions for this article.

            Check for updates

            Qualifiers

            • Research-article
            • Research
            • Refereed limited

            Funding Sources

            Conference

            ICAIF '23

            Contributors

            Other Metrics

            Bibliometrics & Citations

            Bibliometrics

            Article Metrics

            • Downloads (Last 12 months)438
            • Downloads (Last 6 weeks)26
            Reflects downloads up to 17 Feb 2025

            Other Metrics

            Citations

            Cited By

            View all
            • (2024)Navigating Complexity: GPT-4's Performance in Predicting Earnings and Stock Returns in China's A-Share MarketHighlights in Business, Economics and Management10.54097/4rwdat9542(189-203)Online publication date: 19-Nov-2024
            • (2024)Approach to a GPT-based Early Detection Tool to Evaluate Heterogeneous Data Sources and Identify Reconfiguration Needs of SMEs in the Production SectorProcedia CIRP10.1016/j.procir.2024.10.140130(631-636)Online publication date: 2024
            • (2024)Multi-attention recommender system for non-fungible tokensEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109179137(109179)Online publication date: Nov-2024
            • (2024)Language as a Lens: A Hybrid Text Summarization and Sentiment Analysis Approach for Multiclass Stock Return PredictionIntelligent Systems and Applications10.1007/978-3-031-66336-9_31(429-448)Online publication date: 1-Aug-2024

            View Options

            Login options

            View options

            PDF

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader

            HTML Format

            View this article in HTML Format.

            HTML Format

            Figures

            Tables

            Media

            Share

            Share

            Share this Publication link

            Share on social media