Skip to main content
SpringerLink
Log in

Does sentiment help requirement engineering: exploring sentiments in user comments to discover informative comments

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

User comments are valuable resources for software improvement; however, owing to excessive volume, informative comments need to be selectively analyzed. We attempt to address this problem by sentiment analysis and expect sentiment can be a useful indicator for finding informative comments. In this study, we analyze the informative level according to the sentiment of the comment using sentiment analysis. To understand the sentiment in detail, we divide it into four groups and evaluate the characteristics of each group through experiments. Applying topic modeling, we evaluate the informative level of the extracted topic and evaluate the proportion of sentiments by sentiment analysis of the related comments. Additionally, we manually evaluate the informative score of the sample comments in each sentiment group to verify the tendencies observed in the experiments. We find that the probability of being associated with requirements is very low when positive, or when both positive and negative sentiments are weak. In contrast, it has been shown that informative comments are concentrated in negative or strongly negative and positive comments, which are very few among all comments. In particular, the comments observed as strongly positive and negative are highly informative, which is a characteristic that has been overlooked in previous studies. We propose a sentiment model that specifies the sentiment, and confirm sentiments that are highly related to informative comments through sentiment analysis methods and expert evaluations. From these results, it is expected that analyzing negative comments or strongly negative and positive comments can contribute to effective requirement engineering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. https://www.apple.com/macos/big-sur/.

  2. https://store.steampowered.com/.

  3. https://play.google.com/store.

  4. http://nltk.org/.

  5. http://sentistrength.wlv.ac.uk/.

References

  • Ahmed Abbasi, A. H., Dhar, M.: Benchmarking twitter sentiment analysis tools. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pp.1–7 (2014)

  • AppBrain: Ratings of apps on Google Play. AppBrain. https://www.appbrain.com/stats/android-app-ratings (2021). Accessed 23 May 2021

  • Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  • Bouazizi, M., Ohtsuki, T.: A pattern-based approach for multi-class sentiment analysis in Twitter. IEEE Access. 5, 20617–20639 (2017)

    Article  Google Scholar 

  • Carreño, L. V. G., Winbladh, K.: Analysis of user comments: an approach for software requirements evolution. In: Proceedings of the 35th International Conference on Software Engineering (ICSE), pp. 582–591 (2013)

  • Chen, N., Lin, J., Hoi, S. C., Xiao, X., Zhang, B.: AR-miner: mining informative reviews for developers from mobile app marketplace. In: Proceedings of the 36th International Conference on Software Engineering (ICSE), pp. 767–778 (2014)

  • Ferrari, A., Esuli, A.: An NLP approach for cross-domain ambiguity detection in requirements engineering. Autom. Softw. Eng. 26(3), 559–598 (2019)

    Article  Google Scholar 

  • Goul, M., Marjanovic, O., Baxley, S., Vizecky, K.: Managing the enterprise business intelligence app store: Sentiment analysis supported requirements engineering. In: Proceedings of the 45th Hawaii International Conference on System Sciences (HICSS), pp. 4168–4177 (2012)

  • Gu, X., Kim, S.: What Parts of Your Apps are Loved by Users?. In: Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 760–770 (2015)

  • Guzman, E., Maalej, W.: How do users like this feature? a fine grained sentiment analysis of app reviews. In: Proceedings of the IEEE 22nd International Requirements Engineering Conference (RE), pp. 153–162 (2014)

  • Guzman, E., Azócar, D., Li, Y.: Sentiment analysis of commit comments in GitHub: an empirical study. In: Proceedings of the 11th Working Conference on Mining Software Repositories (MSR), pp. 352–355 (2014)

  • Harman, M., Jia, Y., Zhang, Y.: App store mining and analysis: MSR for app stores. In: Proceedings of the 9th IEEE Working Conference on Mining Software Repositories (MSR), pp. 108–111 (2012)

  • Hu, M. Liu, B.: Mining and summarizing customer reviews. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 168–177 (2004a)

  • Hu, M., Liu, B.: Mining opinion features in customer reviews. In: Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI), pp. 755–760 (2004b)

  • Islam, M. R., Zibran, M. F.: Leveraging automated sentiment analysis in software engineering. In: Proceedings of the 14th International Conference on Mining Software Repositories (MSR), pp. 203–214 (2017)

  • Jo, Y., Oh, A. H.: Aspect and sentiment unification model for online review analysis. In: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, pp. 815–824 (2011)

  • Jongeling, R., Sarkar, P., Datta, S., Serebrenik, A.: On negative results when using sentiment analysis tools for software engineering research. Empir. Softw. Eng. 22(5), 2543–2584 (2017)

    Article  Google Scholar 

  • Khalid, H., Shihab, E., Nagappan, M., Hassan, A.E.: What do mobile app users complain about? IEEE Softw. 32(3), 70–77 (2014)

    Article  Google Scholar 

  • Khalid, H.: On identifying user complaints of iOS apps. In: Proceedings of the 35th International Conference on Software Engineering (ICSE), pp. 1474–1476 (2013)

  • Maalej, W., Pagano, D.: On the socialness of software. In: Proceedings of the IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing, pp. 864–871 (2011)

  • Martin, W., Sarro, F., Jia, Y., Zhang, Y., Harman, M.: A survey of app store analysis for software engineering. IEEE Trans. Softw. Eng. 43(9), 817–847 (2017)

    Article  Google Scholar 

  • Martínez-Cámara, E., Montejo-Ráez, A., Martín-Valdivia, M. T., López, L. A. U.: Sinai: Machine learning and emotion of the crowd for sentiment analysis in microblogs. In: Proceedings of the Second Joint Conference on Lexical and Computational Semantics (* SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), pp. 402–407, (2013)

  • Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)

    Article  Google Scholar 

  • Mukhtar, N., Khan, M.A., Chiragh, N., Nazir, S.: Identification and handling of intensifiers for enhancing accuracy of Urdu sentiment analysis. J. Exp. Syst. 35(6), 1–12 (2018)

    Google Scholar 

  • Mukhtar, N., Khan, M.A., Chiragh, N., Jan, A.U., Nazir, S.: Recognition and effective handling of negations in enhancing the accuracy of Urdu sentiment analyzer. Mehran Univ. Res. J. Eng. Technol. 39(4), 759–770 (2020)

    Article  Google Scholar 

  • Pagano, D., Maalej, W.: User feedback in the appstore: An empirical study. In: Proceedings of the 21st IEEE International Requirements Engineering Conference (RE), pp. 125–134 (2013)

  • Saif, H., Fernández, M., He, Y., Alani, H.: On stopwords, filtering and data sparsity for sentiment analysis of twitter. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC14), pp. 810–817 (2014)

  • Santos, R., Groen E. C., Villeal K.: An overview of user feedback classification approaches. In: 2nd Workshop on Natural Language Processing for Requirements Engineering & NLP Tool Showcase co-located with REFSQ-2019 Workshops, Doctoral Symposium, Live Studies Track, and Poster Track co-located with the 25th International Conference on Requirements Engineering (RE), (2019)

  • Schofield, A., Mimno, D.: Comparing apples to apple: The effects of stemmers on topic models. Trans. Assoc. Comput. Linguistics 4, 287–300 (2016)

    Article  Google Scholar 

  • ThelWall, M., Buckley, K., Paltoglou, G., Cai, D., Kappas, A.: Sentiment Strength Detection in Short Informal Text. J. Am. Soc. Inform. Sci. Technol. 61(12), 2544–2558 (2010)

    Article  Google Scholar 

  • Thelwall, M., Buckley, K., Paltoglou, G.: Sentiment strength detection for the social web. J. Am. Soc. Inform. Sci. Technol. 63(1), 163–173 (2012)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1C1C1014611).

Author information

Authors and Affiliations

  1. Department of Computer Science, Korea Military Academy, Seoul, South Korea

    Jongwook Jeong

  2. Department of Software Convergence Engineering, Kunsan National University, Gunsan, South Korea

    Neunghoe Kim

Authors
  1. Jongwook Jeong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Neunghoe Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: JJ ; Methodology: JJ ; Formal analysis and investigation: JJ; Writing original draft preparation: JJ, NK; Writing review and editing: NK; Funding acquisition: NK; Resources: JJ, NK; Supervision: JJ.

Corresponding author

Correspondence to Neunghoe Kim.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jeong, J., Kim, N. Does sentiment help requirement engineering: exploring sentiments in user comments to discover informative comments. Autom Softw Eng 28, 18 (2021). https://doi.org/10.1007/s10515-021-00295-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10515-021-00295-w

Keywords

Search

Navigation