skip to main content
10.1145/2851613.2851820acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article

On the combination of "off-the-shelf" sentiment analysis methods

Published: 04 April 2016 Publication History

Abstract

Sentiment analysis has become an important topic on the Web, especially in social media, with applications in many domains such as the monitoring of businesses and products as well as the analysis of the repercussion of important events. Several methods and techniques have been independently developed for this purpose in the literature. However, recent work has showed that all of them have varying degrees of coverage and prediction accuracy, with no "silver bullet" for all cases and scenarios. In this paper, we tackle this issue by proposing ensemble combination methods aimed at combining the outputs of several state-of-the-art proposals in order to maximize both goals, which sometimes can be conflicting. We focus on combining "off-the-shelf" methods, increasing enormously the applicability of our strategy. We tested our solutions in a very rich experimentation environment, covering thirteen widely used methods and fourteen labeled datasets from many domains, including messages from social networks, movie and product reviews, opinions and comments in news articles. Our experimental results demonstrate that we can be very successful in our goal, meaning that our proposal can produce a real and important impact in the area of sentiment classification research.

References

[1]
M. Arajo, P. Gonalves, F. Benevenuto, and M. Cha. ifeel: A system that compares and combines sentiment analysis methods. In WWW, 2014.
[2]
M. M. Bradley and P. J. Lang. Affective norms for English words (ANEW): Stimuli, instruction manual, and affective ratings. Technical report, Center for Research in Psychophysiology, University of Florida, 1999.
[3]
F. Bravo-Marquez, M. Mendoza, and B. Poblete. Combining strengths, emotions and polarities for boosting twitter sentiment analysis. In WISDOM, 2013.
[4]
E. Cambria, R. Speer, C. Havasi, and A. Hussain. Senticnet: A publicly available semantic resource for opinion mining. In AAAI Fall Symposium Series, 2010.
[5]
S. Canuto, M. A. Gonçalves, and F. Benevenuto. Exploiting new sentiment-based meta-level features for effective sentiment analysis. In Proc. of WSDM, 2016.
[6]
Y. Dang, Y. Zhang, and H. Chen. A lexicon-enhanced method for sentiment classification: An experiment on online product reviews. IEEE Intelligent Systems, 25, 2010.
[7]
N. Diakopoulos and D. Shamma. Characterizing debate performance via aggregated twitter sentiment. In Proc. CHI, 2010.
[8]
P. S. Dodds and C. M. Danforth. Measuring the happiness of large-scale written expression: songs, blogs, and presidents. J. of Happiness Studies, 11, 2009.
[9]
Esuli and Sebastiani. Sentiwordnet: A publicly available lexical resource for opinion mining. In Proc. LREC, 2006.
[10]
R. Feldman. Techniques and applications for sentiment analysis. Commun. ACM, 56, 2013.
[11]
A. Go, R. Bhayani, and L. Huang. Twitter sentiment classification using distant supervision. Processing, 2009.
[12]
P. Gonçalves, F. Benevenuto, and M. Cha. PANAS-t: A Psychometric Scale for Measuring Sentiments on Twitter. abs/1308.1857v1, 2013.
[13]
P. Gonalves, M. Arajo, F. Benevenuto, and M. Cha. Comparing and combining sentiment analysis methods. In Proc. COSN, 2013.
[14]
M. Hu and B. Liu. Mining and summarizing customer reviews. Proc. KDD'04, pages 168--177, 2004.
[15]
C. J. Hutto and E. Gilbert. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In ICWSM, 2014.
[16]
T. Joachims. Learning to Classify Text using Support Vector Machines. Kluwer, 2002.
[17]
G. A. Miller. Wordnet: a lexical database for english. Communications of the ACM, 38, 1995.
[18]
S. Mohammad. #emotional tweets. In SEM, 2012.
[19]
S. Mohammad and P. D. Turney. Crowdsourcing a word-emotion association lexicon. Computational Intelligence, 29, 2013.
[20]
S. M. Mohammad, S. Kiritchenko, and X. Zhu. Nrc-canada: Building the state-of-the-art in sentiment analysis of tweets. In Proc. SemEval-2013, 2013.
[21]
F. Moraes, M. A. Vasconcelos, P. Prado, D. H. Dalip, J. M. Almeida, and M. A. Gonalves. Polarity detection of foursquare tips. SOCINFO, 2013.
[22]
A. Mudinas, D. Zhang, and M. Levene. Combining lexicon and learning based approaches for concept-level sentiment analysis. In WISDOM, 2012.
[23]
B. Pang and L. Lee. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proc. Annual meeting of ACL Conference, 2004.
[24]
B. Pang and L. Lee. Opinion mining and sentiment analysis. Found. and Trends in IR, 2, 2008.
[25]
B. Pang, L. Lee, and S. Vaithyanathan. Thumbs up?: sentiment classification using machine learning techniques. In EMNLP, 2002.
[26]
R. Plutchik. A general psychoevolutionary theory of emotion, pages 3--33. Academic press, New York, 1980.
[27]
F. Ribeiro, M. Araújo, P. Gonçalves, F. Benevenuto, and M. A. Gonçalves. A benchmark comparison of state-of-the-practice sentiment analysis methods. arXiv preprint arXiv:1512.01818, 2015.
[28]
A. Sharma and S. Dey. A boosted svm based ensemble classifier for sentiment analysis of online reviews. SIGAPP Appl. Comput. Rev., 13, 2013.
[29]
R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Conf. on Empirical Methods in NLP, 2013.
[30]
Y. Su, Y. Zhang, D. Ji, Y. Wang, and H. Wu. Ensemble learning for sentiment classification. In CLSW, 2012.
[31]
Y. R. Tausczik and J. W. Pennebaker. The psychological meaning of words: Liwc and computerized text analysis methods. J. of Lang. and Soc. Psych., 29, 2010.
[32]
M. Thelwall. Heart and soul: Sentiment strength detection in the social web with sentistrength, 2013. http://sentistrength. wlv.ac.uk/documentation/SentiStrengthChapter.pdf.
[33]
H. Wang, D. Can, A. Kazemzadeh, F. Bar, and S. Narayanan. A system for real-time twitter sentiment analysis of 2012 u.s. presidential election cycle. In ACL System Demonstrations, 2012.
[34]
D. Watson and L. Clark. Development and validation of brief measures of positive and negative affect: the panas scales. J. of Pers. and So. Psych., 54, 1985.
[35]
D. Wolpert. Stacked generalization. Neural Networks, 5, 1992.
[36]
D. H. Wolpert and W. G. Macready. No free lunch theorems for optimization. IEEE Trans. On Evol. Comp., 1, 1997.
[37]
Yelp. Yelp dataset challenge. http://www.yelp.com/dataset_challenge, 2014. Accessed April 23, 2014.
[38]
L. Zhang, R. Ghosh, M. Dekhil, M. Hsu, and B. Liu. Combining lexicon-based and learning-based methods for twitter sentiment analysis. Technical report, HP, 2011.

Cited By

View all
  • (2022)Polarities Inconsistency of MOOC Courses Reviews Based on Users and Sentiment Analysis MethodsProceedings of 3rd International Conference on Machine Learning, Advances in Computing, Renewable Energy and Communication10.1007/978-981-19-2828-4_34(361-369)Online publication date: 18-Sep-2022
  • (2019)Modeling Spatiotemporal Factors Associated With Sentiment on Twitter: Synthesis and Suggestions for Improving the Identification of Localized DeviationsJournal of Medical Internet Research10.2196/1288121:5(e12881)Online publication date: 8-May-2019
  • (2019)Extracting Potentially High Profit Product Feature Groups by Using High Utility Pattern Mining and Aspect Based Sentiment AnalysisHigh-Utility Pattern Mining10.1007/978-3-030-04921-8_9(233-260)Online publication date: 19-Jan-2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SAC '16: Proceedings of the 31st Annual ACM Symposium on Applied Computing
April 2016
2360 pages
ISBN:9781450337397
DOI:10.1145/2851613
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 April 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. machine learning
  2. sentiment analysis
  3. social networks

Qualifiers

  • Research-article

Conference

SAC 2016
Sponsor:
SAC 2016: Symposium on Applied Computing
April 4 - 8, 2016
Pisa, Italy

Acceptance Rates

SAC '16 Paper Acceptance Rate 252 of 1,047 submissions, 24%;
Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25
The 40th ACM/SIGAPP Symposium on Applied Computing
March 31 - April 4, 2025
Catania , Italy

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)6
  • Downloads (Last 6 weeks)0
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Polarities Inconsistency of MOOC Courses Reviews Based on Users and Sentiment Analysis MethodsProceedings of 3rd International Conference on Machine Learning, Advances in Computing, Renewable Energy and Communication10.1007/978-981-19-2828-4_34(361-369)Online publication date: 18-Sep-2022
  • (2019)Modeling Spatiotemporal Factors Associated With Sentiment on Twitter: Synthesis and Suggestions for Improving the Identification of Localized DeviationsJournal of Medical Internet Research10.2196/1288121:5(e12881)Online publication date: 8-May-2019
  • (2019)Extracting Potentially High Profit Product Feature Groups by Using High Utility Pattern Mining and Aspect Based Sentiment AnalysisHigh-Utility Pattern Mining10.1007/978-3-030-04921-8_9(233-260)Online publication date: 19-Jan-2019
  • (2019)10SENTJournal of the Association for Information Science and Technology10.1002/asi.2411770:3(242-255)Online publication date: 6-Feb-2019
  • (2016)Exploiting New Sentiment-Based Meta-level Features for Effective Sentiment AnalysisProceedings of the Ninth ACM International Conference on Web Search and Data Mining10.1145/2835776.2835821(53-62)Online publication date: 8-Feb-2016

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media