Skip to main content
SpringerLink
Log in

Word sense disambiguation application in sentiment analysis of news headlines: an applied approach to FOREX market prediction

  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

Sentiment analysis of textual content has become a popular approach for market prediction. However, lack of a process for word sense disambiguation makes it questionable whether the sentiment expressed by the context is correctly identified. Meanwhile, many studies in natural language processing have focused on word sense disambiguation. However, there has been a weak link between the two logically relevant fields of study. Therefore, with two motivations, we propose a system for FOREX market prediction that exploits word sense disambiguation in sentiment analysis of news headlines and predicts the directional movement of a currency pair. The first motivation is the implementation of a novel word sense disambiguation that can determine the proper senses of all significant words in a news headline. The main contributions of this work that make the first motivation possible, are the introduction of novel approaches termed Relevant Gloss Retrieval, Similarity Threshold, Verb Nominalization, and also optimization measures to decrease execution time. The second motivation is to prove that determination of proper senses of significant words in textual contents can improve the determination of sentiment, conveyed by the context, and consequently any application based on sentiment analysis. Inclusion of the word sense disambiguation into the proposed system proves the achievement of the second motivation. Carried out tests with the same dataset prove that the proposed system outperforms one of the best systems (to our best knowledge) proposed for market prediction and improves accuracy from 83.33% to 91.67%. The detail for reproduction of the system is amply provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Anastasakis, L., & Mort, N. (2009). Exchange rate forecasting using a combined parametric and nonparametric self-organising modelling approach. Expert Systems with Applications, 36(10), 12001–12011.

    Article  Google Scholar 

  • Baccianella, S., Esuli, A., Sebastiani, F. (2010). Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In LREC (Vol. 10 pp. 2200–2204).

  • Banerjee, S., & Pedersen, T. (2002). An adapted lesk algorithm for word sense disambiguation using wordnet. In International conference on intelligent text processing and computational linguistics (pp. 136–145). Berlin: Springer.

  • Bollen, J., Mao, H., Zeng, X. (2011). Twitter mood predicts the stock market. Journal of Computational Science, 2(1), 1–8.

    Article  Google Scholar 

  • Dong, R., O’Mahony, M.P., Schaal, M., McCarthy, K., Smyth, B. (2016). Combining similarity and sentiment in opinion mining for product recommendation. Journal of Intelligent Information Systems, 46(2), 285–312.

    Article  Google Scholar 

  • Evans, M.D., & Lyons, R.K. (2008). How is macro news transmitted to exchange rates? Journal of Financial Economics, 88(1), 26–50.

    Article  Google Scholar 

  • Farooq, U., Dhamala, T.P., Nongaillard, A., Ouzrout, Y., Qadir, M.A. (2015). A word sense disambiguation method for feature level sentiment analysis. In 2015 9th international conference on software, knowledge, information management and applications (SKIMA) (pp. 1–8). IEEE.

  • Fellbaum C. (1998). WordNet. Wiley Online Library.

  • Hagenau, M., Liebmann, M., Neumann, D. (2013). Automated news reading: stock price prediction based on financial news using context-capturing features. Decision Support Systems, 55(3), 685– 697.

    Article  Google Scholar 

  • Howe, D.C. (2009). Rita: creativity support for computational literature. In Proceedings of the seventh ACM conference on creativity and cognition (pp. 205–210). ACM.

  • Huang, C.J., Liao, J.J., Yang, D.X., Chang, T.Y., Luo, Y.C. (2010). Realization of a news dissemination agent based on weighted association rules and text mining techniques. Expert Systems with Applications, 37(9), 6409–6413.

    Article  Google Scholar 

  • Jiang, J.J., & Conrath, D.W. (1997). Semantic similarity based on corpus statistics and lexical taxonomy. arXiv:cmp-lg/9709008.

  • Kehagias, A., Petridis, V., Kaburlasos, V.G., Fragkou, P. (2003). A comparison of word-and sense-based text categorization using several classification algorithms. Journal of Intelligent Information Systems, 21(3), 227–247.

    Article  Google Scholar 

  • Lesk, M. (1986). Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In Proceedings of the 5th annual international conference on systems documentation (pp. 24–26). ACM.

  • Levinson, M. et al. (2014). The economist guide to financial markets: why they exist and how they work. The Economist, 1, 17. 6th edition.

  • Li, X., Szpakowicz, S., Matwin, S. (1995). A wordnet-based algorithm for word sense disambiguation. In IJCAI (Vol. 95 pp. 1368–1374).

  • Li, Q., Wang, T., Li, P., Liu, L., Gong, Q., Chen, Y. (2014). The effect of news and public mood on stock movements. Information Sciences, 278, 826–840.

    Article  Google Scholar 

  • Lin, D. et al. (1998). An information-theoretic definition of similarity. In Icml, (Vol. 98 pp. 296–304).

  • Liu, Y., Scheuermann, P., Li, X., Zhu, X. (2007). Using wordnet to disambiguate word senses for text classification. In Computational Science–ICCS 2007 (pp. 781–789).

  • Luk, A. (1993). Statistical sense disambiguation with relatively small corpora using dictionary definitions. In Proceedings of the 33rd annual meeting of ACL (pp. 181–188).

  • Miller, G., & Fellbaum, C. (1998). Wordnet: an electronic lexical database. MIT Press, Cambridge.

  • Mittermayer, M.A. (2004). Forecasting intraday stock price trends with text mining techniques. In Proceedings of the 37th annual Hawaii international conference on system sciences, 2004 (p. 10). IEEE.

  • Mladenović, M., Mitrović, J., Krstev, C., Vitas, D. (2016). Hybrid sentiment analysis framework for a morphologically rich language. Journal of Intelligent Information Systems, 46(3), 599–620.

    Article  Google Scholar 

  • Mostafa, M.M. (2013). More than words: social networks’ text mining for consumer brand sentiments. Expert Systems with Applications, 40(10), 4241–4251.

    Article  Google Scholar 

  • Nassirtoussi, A.K., Aghabozorgi, S., Wah, T.Y., Ngo, D.C.L. (2014). The text mining homepage. sites.google.com/site/bigdatasetmining/Projects/textmining.

  • Nassirtoussi, A.K., Aghabozorgi, S., Wah, T.Y., Ngo, D.C.L. (2015). Text mining of news-headlines for forex market prediction: a multi-layer dimension reduction algorithm with semantics and sentiment. Expert Systems with Applications, 42(1), 306–324.

    Article  Google Scholar 

  • Nizer, P., & Nievola, J.C. (2012). Predicting published news effect in the Brazilian stock market. Expert Systems with Applications, 39(12), 10674–10680.

    Article  Google Scholar 

  • Patwardhan, S., Banerjee, S., Pedersen, T. (2007). Umnd1: unsupervised word sense disambiguation using contextual semantic relatedness. In Proceedings of the 4th international workshop on semantic evaluations, association for computational linguistics (pp. 390–393).

  • Peramunetilleke, D., & Wong, R.K. (2002). Currency exchange rate forecasting from news headlines. Australian Computer Science Communications, 24(2), 131–139.

    Google Scholar 

  • Rao, T., & Srivastava, S. (2012). Using twitter sentiments and search volumes index to predict oil, gold, forex and markets indices. Tech. rep.

  • Resnik, P. (1995). Using information content to evaluate semantic similarity in a taxonomy. arXiv:cmp-lg/9511007.

  • Schumaker, R.P., Zhang, Y., Huang, C.N., Chen, H. (2012). Evaluating sentiment in financial news articles. Decision Support Systems, 53(3), 458–464.

    Article  Google Scholar 

  • Siganos, A., Vagenas-Nanos, E., Verwijmeren, P. (2014). Facebook’s daily sentiment and international stock markets. Journal of Economic Behavior & Organization, 107, 730–743.

    Article  Google Scholar 

  • Sprenger, T.O., Tumasjan, A., Sandner, P.G., Welpe, I.M. (2014). Tweets and trades: the information content of stock microblogs. European Financial Management, 20(5), 926–957.

    Article  Google Scholar 

  • Sul, H.K., Dennis, A.R., Yuan, L.I. (2014). Trading on Twitter: the financial information content of emotion in social media. In 2014 47th Hawaii international conference on system sciences (HICSS) (pp. 806–815). IEEE.

  • Toutanova, K., Klein, D., Manning, C.D., Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 conference of the North American chapter of the association for computational linguistics on human language technology (Vol. 1 pp. 173–180). Association for Computational Linguistics.

  • Wilson, T., Wiebe, J., Hoffmann, P. (2005). Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the conference on human language technology and empirical methods in natural language processing (pp. 347–354). Association for Computational Linguistics.

  • Yarowsky, D. (1995). Unsupervised word sense disambiguation rivaling supervised methods. In Proceedings of the 33rd annual meeting on association for computational linguistics (pp. 189–196). Association for Computational Linguistics.

  • Zhang, X., Fuehres, H., Gloor, P.A. (2011). Predicting stock market indicators through twitter “hope it is not as bad as i fear”. Procedia-Social and Behavioral Sciences, 26, 55–62.

    Article  Google Scholar 

Download references

Acknowledgements

We want to thank Arman Khadjeh Nassirtoussi (Nassirtoussi et al. 2015) for letting us have access to their data. Without using the same data, we could not compare the results.

Author information

Authors and Affiliations

  1. Department of Computer Engineering and Information Technology, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran

    Saeed Seifollahi & Mehdi Shajari

Authors
  1. Saeed Seifollahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mehdi Shajari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehdi Shajari.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Seifollahi, S., Shajari, M. Word sense disambiguation application in sentiment analysis of news headlines: an applied approach to FOREX market prediction. J Intell Inf Syst 52, 57–83 (2019). https://doi.org/10.1007/s10844-018-0504-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-018-0504-9

Keywords

Search

Navigation