Skip to main content
SpringerLink
Log in

“Do we like this, or do we like like this?”: Reflections on a Human-Centered Machine Learning Approach to Sentiment Analysis

  • Conference paper
  • First Online:
Artificial Intelligence in HCI (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14050))

Included in the following conference series:

Abstract

Machine Learning is a powerful tool, but it also has a great potential to cause harm if not approached carefully. Designers must be reflexive and aware of their algorithms’ impacts, and one such way of reflection is known as human-centered machine learning. In this paper, we approach a classical problem that has been approached through ML - sentiment analysis - through a Human-Centered Machine Learning lens. Through a case study of trying to differentiate between degrees of positive emotions in reviews of online fanfiction, we offer a set of recommendations for future designers of ML-driven sentiment analysis algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahmad, M., Aftab, S., Ali, I.: Sentiment analysis of tweets using svm. Int. J. Comput. Appl. 177(5), 25–29 (2017)

    Google Scholar 

  2. Al Amrani, Y., Lazaar, M., El Kadiri, K.E.: Random forest and support vector machine based hybrid approach to sentiment analysis. Procedia Comput. Sci. 127, 511–520 (2018)

    Article  Google Scholar 

  3. Altman, M., Wood, A., Vayena, E.: A harm-reduction framework for algorithmic fairness. IEEE Secur. Privacy 16(3), 34–45 (2018)

    Article  Google Scholar 

  4. Andalibi, N., Buss, J.: The human in emotion recognition on social media: attitudes, outcomes, risks. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–16 (2020)

    Google Scholar 

  5. Annett, M., Kondrak, G.: A comparison of sentiment analysis techniques: polarizing movie blogs. In: Bergler, S. (ed.) AI 2008. LNCS (LNAI), vol. 5032, pp. 25–35. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68825-9_3

    Chapter  Google Scholar 

  6. Aragon, C., Guha, S., Kogan, M., Muller, M., Neff, G.: Human-Centered Data Science: An Introduction. MIT Press (2022)

    Google Scholar 

  7. Black, R.W.: Language, culture, and identity in online fanfiction. E-learning Digital Media 3(2), 170–184 (2006)

    Article  Google Scholar 

  8. Bond, F., Foster, R.: Linking and extending an open multilingual wordnet. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1352–1362 (2013)

    Google Scholar 

  9. Boyd, D., Crawford, K.: Critical questions for big data: provocations for a cultural, technological, and scholarly phenomenon. Inf. Commun. Soc. 15(5), 662–679 (2012)

    Article  Google Scholar 

  10. Burnap, P., Rana, O.F., Avis, N., Williams, M., Housley, W., Edwards, A., Morgan, J., Sloan, L.: Detecting tension in online communities with computational twitter analysis. Technol. Forecast. Soc. Chang. 95, 96–108 (2015)

    Article  Google Scholar 

  11. Cambo, S.A., Gergle, D.: Model positionality and computational reflexivity: promoting reflexivity in data science. In: CHI Conference on Human Factors in Computing Systems, pp. 1–19 (2022)

    Google Scholar 

  12. Campbell, J., Aragon, C., Davis, K., Evans, S., Evans, A., Randall, D.: Thousands of positive reviews: distributed mentoring in online fan communities. In: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, pp. 691–704 (2016)

    Google Scholar 

  13. Chancellor, S.: Towards practices for human-centered machine learning. arXiv preprint arXiv:2203.00432 (2022)

  14. Chancellor, S., Baumer, E.P., De Choudhury, M.: Who is the “human” in human-centered machine learning: the case of predicting mental health from social media. In: Proceedings of the ACM on Human-Computer Interaction 3(CSCW), pp. 1–32 (2019)

    Google Scholar 

  15. Chen, N.C., Drouhard, M., Kocielnik, R., Suh, J., Aragon, C.R.: Using machine learning to support qualitative coding in social science: shifting the focus to ambiguity. ACM Trans. Interact. Intell. Syst. (TiiS) 8(2), 1–20 (2018)

    Article  Google Scholar 

  16. Costanza-Chock, S.: Design justice: community-led practices to build the worlds we need. The MIT Press (2020)

    Google Scholar 

  17. Daeli, N.O.F., Adiwijaya, A.: Sentiment analysis on movie reviews using information gain and k-nearest neighbor. J. Data Sci. Appl. 3(1), 1–7 (2020)

    Google Scholar 

  18. Díaz, M., Johnson, I., Lazar, A., Piper, A.M., Gergle, D.: Addressing age-related bias in sentiment analysis. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2018)

    Google Scholar 

  19. Draude, C., Klumbyte, G., Lücking, P., Treusch, P.: Situated algorithms: a sociotechnical systemic approach to bias. Online Information Review (2019)

    Google Scholar 

  20. Dym, B., Brubaker, J.R., Fiesler, C., Semaan, B.: “Coming out okay” community narratives for LGBTQ identity recovery work. In: Proceedings of the ACM on Human-Computer Interaction 3(CSCW), pp. 1–28 (2019). https://doi.org/10.1145/3359256. https://dl.acm.org/doi/10.1145/3359256

  21. Ekman, P.: All emotions are basic. The nature of emotion: Fundamental questions, pp. 15–19 (1994)

    Google Scholar 

  22. Evans, S., Davis, K., Evans, A., Campbell, J.A., Randall, D.P., Yin, K., Aragon, C.: More than peer production: Fanfiction communities as sites of distributed mentoring. In: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, pp. 259–272 (2017)

    Google Scholar 

  23. Fiesler, C., Proferes, N.: “Participant” perceptions of twitter research ethics. Social Media+ Society 4(1), 2056305118763366 (2018)

    Google Scholar 

  24. Figueroa, A., Ghosh, S., Aragon, C.: Generalized cohen’s kappa: a novel inter-rater reliability metric for non-mutually exclusive categories. In: Proceedings of the Human Interface and the Management of Information Thematic Area in the context of the 25th International Conference on Human-Computer Interaction (HCI International). Springer (2023)

    Google Scholar 

  25. Ghosh, S., Figueroa, A.: Establishing tiktok as a platform for informal learning: Evidence from mixed-methods analysis of creators and viewers. In: Proceedings of the 56th Hawaii International Conference on System Sciences, pp. 2431–2440 (2023)

    Google Scholar 

  26. Ghosh, S., Froelich, N., Aragon, C.: “i love you, my dear friend”: analyzing the role of emotions in the building of friendships in online fanfiction communities. In: Proceedings of the 15th International Conference on Social Computing and Social Media in the context of the 25th International Conference on Human-Computer Interaction (HCI International). Springer (2023)

    Google Scholar 

  27. Goel, A., Gautam, J., Kumar, S.: Real time sentiment analysis of tweets using naive bayes. In: 2016 2nd International Conference on Next Generation Computing Technologies (NGCT), pp. 257–261. IEEE (2016)

    Google Scholar 

  28. Gui, X., Chen, Y., Kou, Y., Pine, K., Chen, Y.: Investigating support seeking from peers for pregnancy in online health communities. In: Proceedings of the ACM on Human-Computer Interaction 1(CSCW), pp. 1–19 (2017)

    Google Scholar 

  29. Gupta, A., Lamba, H., Kumaraguru, P., Joshi, A.: Faking sandy: characterizing and identifying fake images on twitter during hurricane sandy. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 729–736 (2013)

    Google Scholar 

  30. Hegde, Y., Padma, S.: Sentiment analysis using random forest ensemble for mobile product reviews in kannada. In: 2017 IEEE 7th International Advance Computing Conference (IACC), pp. 777–782. IEEE (2017)

    Google Scholar 

  31. Hicks, A., Rutherford, M., Fellbaum, C., Bian, J.: An analysis of wordnet’s coverage of gender identity using twitter and the national transgender discrimination survey. In: Proceedings of the 8th Global WordNet Conference (GWC), pp. 123–130 (2016)

    Google Scholar 

  32. Hong, L., Doumith, A.S., Davison, B.D.: Co-factorization machines: modeling user interests and predicting individual decisions in twitter. In: Proceedings of the sixth ACM International Conference on Web Search and Data Mining, pp. 557–566 (2013)

    Google Scholar 

  33. Hu, X., Tang, J., Gao, H., Liu, H.: Unsupervised sentiment analysis with emotional signals. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 607–618 (2013)

    Google Scholar 

  34. Huq, M.R., Ahmad, A., Rahman, A.: Sentiment analysis on twitter data using KNN and svm. Int. J. Adv. Comput. Sci. Appl. 8(6) (2017)

    Google Scholar 

  35. Kassens-Noor, E.: Twitter as a teaching practice to enhance active and informal learning in higher education: the case of sustainable tweets. Act. Learn. High. Educ. 13(1), 9–21 (2012)

    Article  Google Scholar 

  36. Kaya, M., Fidan, G., Toroslu, I.H.: Sentiment analysis of Turkish political news. In: 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, vol. 1, pp. 174–180. IEEE (2012)

    Google Scholar 

  37. Kivran-Swaine, F., Brody, S., Diakopoulos, N., Naaman, M.: Of joy and gender: emotional expression in online social networks. In: The ACM Conference on Computer Supported Cooperative Work Companion, pp. 139–142 (2012)

    Google Scholar 

  38. Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al.: Computational social science. Science 323(5915), 721–723 (2009)

    Article  Google Scholar 

  39. Levonian, Z., Dow, M., Erikson, D., Ghosh, S., Miller Hillberg, H., Narayanan, S., Terveen, L., Yarosh, S.: Patterns of patient and caregiver mutual support connections in an online health community. In: Proceedings of the ACM on Human-Computer Interaction 4(CSCW3), pp. 1–46 (2021)

    Google Scholar 

  40. López-Chau, A., Valle-Cruz, D., Sandoval-Almazán, R.: Sentiment analysis of twitter data through machine learning techniques. In: Ramachandran, M., Mahmood, Z. (eds.) Software Engineering in the Era of Cloud Computing. CCN, pp. 185–209. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-33624-0_8

    Chapter  Google Scholar 

  41. Lulu: The slow dance of the infinite stars (2013)

    Google Scholar 

  42. Lulu: Archive of our own: 2020 statistics, November 2020

    Google Scholar 

  43. Lulu: Archive of our own: Overall gender and sexuality of ao3 users, November 2020

    Google Scholar 

  44. Maynard, D.G., Greenwood, M.A.: Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis. In: Lrec 2014 proceedings. ELRA (2014)

    Google Scholar 

  45. Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)

    Article  Google Scholar 

  46. Mohammad, S.M.: Ethics sheet for automatic emotion recognition and sentiment analysis. Comput. Linguist. 48(2), 239–278 (2022)

    Article  Google Scholar 

  47. Nasukawa, T., Yi, J.: Sentiment analysis: capturing favorability using natural language processing. In: Proceedings of the 2nd International Conference on Knowledge Capture, pp. 70–77 (2003)

    Google Scholar 

  48. Neviarouskaya, A., Prendinger, H., Ishizuka, M.: Sentiful: a lexicon for sentiment analysis. IEEE Trans. Affect. Comput. 2(1), 22–36 (2011)

    Article  Google Scholar 

  49. Noble, S.U.: Algorithms of oppression. In: Algorithms of Oppression. New York University Press (2018)

    Google Scholar 

  50. O’neil, C.: Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway books (2016)

    Google Scholar 

  51. Ortigosa-Hernández, J., Rodríguez, J.D., Alzate, L., Lucania, M., Inza, I., Lozano, J.A.: Approaching sentiment analysis by using semi-supervised learning of multi-dimensional classifiers. Neurocomputing 92, 98–115 (2012)

    Article  Google Scholar 

  52. Pang, B., Lee, L., et al.: Opinion mining and sentiment analysis. Found. Trends Inf. Retrieval 2(1–2), 1–135 (2008)

    Google Scholar 

  53. Poria, S., Gelbukh, A., Cambria, E., Yang, P., Hussain, A., Durrani, T.: Merging senticnet and wordnet-affect emotion lists for sentiment analysis. In: 2012 IEEE 11th International Conference on Signal Processing, vol. 2, pp. 1251–1255. IEEE (2012)

    Google Scholar 

  54. Rana, S., Singh, A.: Comparative analysis of sentiment orientation using SVM and naive bayes techniques. In: 2016 2nd International Conference on Next Generation Computing Technologies (NGCT), pp. 106–111. IEEE (2016)

    Google Scholar 

  55. Rennie, J.D., Shih, L., Teevan, J., Karger, D.R.: Tackling the poor assumptions of naive bayes text classifiers. In: Proceedings of the 20th International Conference on Machine Learning (ICML-03), pp. 616–623 (2003)

    Google Scholar 

  56. Roback, A., Hemphill, L.: “i’d have to vote against you” issue campaigning via Twitter. In: Proceedings of the 2013 Conference on Computer Supported Cooperative Work Companion, pp. 259–262 (2013)

    Google Scholar 

  57. Rudnicka, E., Bond, F., Grabowski, Ł., Piasecki, M., Piotrowski, T.: Lexical perspective on wordnet to wordnet mapping. In: Proceedings of the 9th Global Wordnet Conference, pp. 209–218 (2018)

    Google Scholar 

  58. Saif, H., He, Y., Alani, H.: Semantic Sentiment analysis of Twitter. In: Cudré-Mauroux, P., Heflin, J., Sirin, E., Tudorache, T., Euzenat, J., Hauswirth, M., Parreira, J.X., Hendler, J., Schreiber, G., Bernstein, A., Blomqvist, E. (eds.) ISWC 2012. LNCS, vol. 7649, pp. 508–524. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35176-1_32

    Chapter  Google Scholar 

  59. Scheuerman, M.K., Wade, K., Lustig, C., Brubaker, J.R.: How we’ve taught algorithms to see identity: Constructing race and gender in image databases for facial analysis. Proceedings of the ACM on Human-computer Interaction 4(CSCW1), 1–35 (2020)

    Article  Google Scholar 

  60. Shen, J.H., Fratamico, L., Rahwan, I., Rush, A.M.: Darling or babygirl? investigating stylistic bias in sentiment analysis. In: Proc. of FATML (2018)

    Google Scholar 

  61. Singh, A.K., Shashi, M.: Vectorization of text documents for identifying unifiable news articles. Int. J. Adv. Comput. Sci. Appl. 10(7) (2019)

    Google Scholar 

  62. Stanoevska-Slabeva, K., Schmid, B.F.: A typology of online communities and community supporting platforms. In: Proceedings of the 34th Annual Hawaii International Conference on System Sciences, pp. 10-pp. IEEE (2001)

    Google Scholar 

  63. Sterling, S., Marton, H.: Design justice: an exhibit of emerging design practices, vol. 2. The Allied Media Conference (2016)

    Google Scholar 

  64. Suresh, H., Guttag, J.: A framework for understanding sources of harm throughout the machine learning life cycle. In: Equity and Access in Algorithms, Mechanisms, and Optimization, pp. 1–9 (2021)

    Google Scholar 

  65. Thelwall, M.: Gender bias in sentiment analysis. Online Information Review (2018)

    Google Scholar 

  66. Tosenberger, C.: “Oh my god, the fanfiction!”: Dumbledore’s outing and the online harry potter fandom. Children’s Literature Association Quarterly 33(2), 200–206 (2008)

    Google Scholar 

  67. Venigalla, A.S.M., Chimalakonda, S., Vagavolu, D.: Mood of india during covid-19-an interactive web portal based on emotion analysis of twitter data. In: Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing, pp. 65–68 (2020)

    Google Scholar 

  68. Wang, S.I., Manning, C.D.: Baselines and bigrams: Simple, good sentiment and topic classification. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 90–94 (2012)

    Google Scholar 

  69. Wiens, J., Saria, S., Sendak, M., Ghassemi, M., Liu, V.X., Doshi-Velez, F., Jung, K., Heller, K., Kale, D., Saeed, M., et al.: Do no harm: a roadmap for responsible machine learning for health care. Nat. Med. 25(9), 1337–1340 (2019)

    Article  Google Scholar 

  70. Yang, X., Steck, H., Liu, Y.: Circle-based recommendation in online social networks. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1267–1275 (2012)

    Google Scholar 

  71. Yin, K., Aragon, C., Evans, S., Davis, K.: Where no one has gone before: a meta-dataset of the world’s largest fanfiction repository. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 6106–6110 (2017)

    Google Scholar 

  72. Zadeh, A., Chen, M., Poria, S., Cambria, E., Morency, L.P.: Tensor fusion network for multimodal sentiment analysis. Empirical Methods in Natural Language Processing (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Washington, Seattle, USA

    Sourojit Ghosh, Murtaza Ali, Anna Batra, Cheng Guo, Mohit Jain, Joseph Kang, Julia Kharchenko, Varun Suravajhela, Vincent Zhou & Cecilia Aragon

Authors
  1. Sourojit Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Murtaza Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anna Batra
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cheng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mohit Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joseph Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Julia Kharchenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Varun Suravajhela
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Vincent Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Cecilia Aragon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sourojit Ghosh .

Editor information

Editors and Affiliations

  1. Siemens Corporation, Princeton, NJ, USA

    Helmut Degen

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ghosh, S. et al. (2023). “Do we like this, or do we like like this?”: Reflections on a Human-Centered Machine Learning Approach to Sentiment Analysis. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2023. Lecture Notes in Computer Science(), vol 14050. Springer, Cham. https://doi.org/10.1007/978-3-031-35891-3_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35891-3_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35890-6

  • Online ISBN: 978-3-031-35891-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation