Skip to main content
SpringerLink
Log in
Book cover

Semantic Web Evaluation Challenge

SemWebEval 2018: Semantic Web Challenges pp 140–150Cite as

Fine-Tuning of Word Embeddings for Semantic Sentiment Analysis

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 927))

Abstract

In this paper, we present a state-of-the-art deep-learning approach for sentiment polarity classification. Our approach is based on a 2-layer bidirectional Long Short-Term Memory network, equipped with a neural attention mechanism to detect the most informative words in a natural language text. We test different pre-trained word embeddings, initially keeping these features frozen during the first epochs of the training process. Next, we allow the neural network to perform a fine-tuning of the word embeddings for the sentiment polarity classification task. This allows projecting the pre-trained embeddings in a new space which takes into account information about the polarity of each word, thereby being more suitable for semantic sentiment analysis. Experimental results are promising and show that the fine-tuning of the embeddings with a neural attention mechanism allows boosting the performance of the classifier.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://www.maurodragoni.com/research/opinionmining/events/challenge-2018/.

  2. 2.

    http://www.maurodragoni.com/research/opinionmining/dranziera/embeddings-evaluation.php.

References

  1. Atzeni, Mattia, Dridi, Amna, Reforgiato Recupero, Diego: Fine-grained sentiment analysis on financial microblogs and news headlines. In: Dragoni, Mauro, Solanki, Monika, Blomqvist, Eva (eds.) SemWebEval 2017. CCIS, vol. 769, pp. 124–128. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69146-6_11

    Chapter  Google Scholar 

  2. Atzeni, Mattia, Reforgiato Recupero, Diego: Deep learning and sentiment analysis for human-robot interaction. In: Gangemi, Aldo, Gentile, Anna Lisa, Nuzzolese, Andrea Giovanni, Rudolph, Sebastian, Maleshkova, Maria, Paulheim, Heiko, Pan, Jeff Z., Alam, Mehwish (eds.) ESWC 2018. LNCS, vol. 11155, pp. 14–18. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98192-5_3

    Chapter  Google Scholar 

  3. Baziotis, C., Pelekis, N., Doulkeridis, C.: DataStories at SemEval-2017 Task 4: deep LSTM with attention for message-level and topic-based sentiment analysis. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 747–754. Association for Computational Linguistics (2017)

    Google Scholar 

  4. Cliche, M.: BB\_twtr at SemEval-2017 Task 4: Twitter sentiment analysis with CNNs and LSTMs. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). Association for Computational Linguistics (2017)

    Google Scholar 

  5. Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Cohen, W.W., McCallum, A., Roweis, S.T. (eds.) Proceedings of the Twenty-Fifth International Conference Machine Learning (ICML 2008), Helsinki, Finland, June 5–9, 2008. ACM International Conference Proceeding Series, vol. 307, pp. 160–167. ACM (2008). http://doi.acm.org/10.1145/1390156.1390177

  6. Dridi, A., Atzeni, M., Reforgiato Recupero, D.: Finenews: fine-grained semantic sentiment analysis on financial microblogs and news. Int. J. Mach. Learn. Cybern. (2018). https://doi.org/10.1007/s13042-018-0805-x

  7. Federici, M., Dragoni, M.: A knowledge-based approach for aspect-based opinion mining. In: Sack, H., Dietze, S., Tordai, A., Lange, C. (eds.) Semantic Web Challenges, pp. 141–152. Springer International Publishing, Cham (2016)

    Chapter  Google Scholar 

  8. Gangemi, A., Presutti, V., Reforgiato Recupero, D.: Frame-based detection of opinion holders and topics: a model and a tool. IEEE Comput. Intell. Mag. 9(1), 20–30 (2014). https://www.scopus.com/inward/record.uri?eid=2-s2.0-84893375456&doi=10.1109%2fMCI.2013.2291688&partnerID=40&md5=d4133e755c2d02956702b55a9b8dc5ab

    Article  Google Scholar 

  9. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). Nov

    Article  Google Scholar 

  10. Iguider, Walid, Reforgiato Recupero, Diego: Language independent sentiment analysis of the shukran social network using apache spark. In: Dragoni, Mauro, Solanki, Monika, Blomqvist, Eva (eds.) SemWebEval 2017. CCIS, vol. 769, pp. 129–132. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69146-6_12

    Chapter  Google Scholar 

  11. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE. IEEE Press (1998)

    Google Scholar 

  12. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Burges, C.J.C., Bottou, L., Ghahramani, Z., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5–8, 2013, Lake Tahoe, Nevada, United States, pp. 3111–3119. Curran Associates Inc. (2013). http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality

  13. Palogiannidi, E., et al.: Tweester at semeval-2016 task 4: sentiment analysis in twitter using semantic-affective model adaptation. In: Bethard, S., Cer, D.M., Carpuat, M., Jurgens, D., Nakov, P., Zesch, T. (eds.) Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2016. The Association for Computer Linguistics (2016)

    Google Scholar 

  14. Petrucci, Giulio, Dragoni, Mauro: The IRMUDOSA system at ESWC-2017 challenge on semantic sentiment analysis. In: Dragoni, Mauro, Solanki, Monika, Blomqvist, Eva (eds.) SemWebEval 2017. CCIS, vol. 769, pp. 148–165. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69146-6_14

    Chapter  Google Scholar 

  15. Raffel, C., Ellis, D.P.W.: Feed-forward networks with attention can solve some long-term memory problems. CoRR abs/ arXiv:1512.08756 (2015). http://arxiv.org/abs/1512.08756

  16. Recupero, D., Consoli, S., Gangemi, A., Nuzzolese, A., Spampinato, D.: A semantic web based core engine to efficiently perform sentiment analysis. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) vol. 8798, pp. 245–248 (2014). https://www.scopus.com/inward/record.uri?eid=2-s2.0-84908681970&doi=10.1007

    Google Scholar 

  17. Reforgiato Recupero, D., Cambria, E., Di Rosa, E.: Semantic sentiment analysis challenge at eswc2017. In: Dragoni, M., Solanki, M., Blomqvist, E. (eds.) Semantic Web Challenges, pp. 109–123. Springer International Publishing, Cham (2017)

    Chapter  Google Scholar 

  18. Reforgiato Recupero, D., Presutti, V., Consoli, S., Gangemi, A., Nuzzolese, A.G.: Sentilo: frame-based sentiment analysis. Cognit. Comput. 7(2), 211–225 (2015). https://doi.org/10.1007/s12559-014-9302-z

    Article  Google Scholar 

  19. Rosenthal, S., Farra, N., Nakov, P.: SemEval-2017 task 4: sentiment analysis in Twitter. In: Proceedings of the 11th International Workshop on Semantic Evaluation. SemEval 2017. Association for Computational Linguistics (2017)

    Google Scholar 

  20. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014). Jan

    MathSciNet  MATH  Google Scholar 

  21. Stojanovski, D., Strezoski, G., Madjarov, G., Dimitrovski, I.: Finki at semeval-2016 task 4: deep learning architecture for twitter sentiment analysis. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 149–154. Association for Computational Linguistics (2016)

    Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge Sardinia Regional Government for the financial support (Convenzione triennale tra la Fondazione di Sardegna e gli Atenei Sardi Regione Sardegna L.R. 7/2007 annualità 2016 DGR 28/21 del 17.05.2016, CUP: F72F16003030002). This work has been supported by Sardinia Regional Government (P.O.R. Sardegna F.S.E. Operational Programme of the Autonomous Region of Sardinia, European Social Fund 2014-2020 - Axis IV Human Resources, Objective l.3, Line of Activity l.3.1).

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, University of Cagliari, Via Ospedale 72, 09124, Cagliari, Italy

    Mattia Atzeni & Diego Reforgiato Recupero

Authors
  1. Mattia Atzeni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Diego Reforgiato Recupero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mattia Atzeni .

Editor information

Editors and Affiliations

  1. Laboratoire d'Informatique de Paris-Nord, Paris 13 University, Villetaneuse, France

    Davide Buscaldi

  2. University of Bologna, Bologna, Italy

    Aldo Gangemi

  3. Department of Mathematics and Computer Science, University of Cagliari, Cagliari, Italy

    Diego Reforgiato Recupero

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Atzeni, M., Reforgiato Recupero, D. (2018). Fine-Tuning of Word Embeddings for Semantic Sentiment Analysis. In: Buscaldi, D., Gangemi, A., Reforgiato Recupero, D. (eds) Semantic Web Challenges. SemWebEval 2018. Communications in Computer and Information Science, vol 927. Springer, Cham. https://doi.org/10.1007/978-3-030-00072-1_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-00072-1_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-00071-4

  • Online ISBN: 978-3-030-00072-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation