Skip to main content
Springer Nature Link
Log in

Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool

  • Conference paper
  • First Online:
Product-Focused Software Process Improvement (PROFES 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15452))

  • 313 Accesses

Abstract

User reviews on software platforms offer direct insights into user attitudes toward software packages, serving as invaluable sources for assessing software quality. However, there is a lack of effective methods or tools to analyze the quality attributes that are of most concern in user reviews, as well as the strengths and weaknesses of software packages in terms of quality. This paper introduces ISOftSentiment, a hybrid tool developed to overcome this challenge and facilitate efficient evaluation of software package quality attributes based on user reviews. ISOftSentiment integrates a sentiment analysis model with the ISO/IEC 25010 product quality model and a fuzzy consensus mechanism. This integration processes user reviews to produce a quality sentiment matrix, which not only summarizes the quality analysis results but also highlights the consensus sentiment for relevant quality attributes. The matrix provides insights into both the strengths and weaknesses of software packages. Based on these insights, software engineers gain a clearer understanding of a package’s quality and trustworthiness, thereby making more informed decisions during the software selection or evaluation process and avoiding the use of unreliable components. Consequently, this tool ensures that the final product maintains high standards of quality.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://StackOverflow.com/.

  2. 2.

    https://www.g2.com/.

  3. 3.

    https://www.trustradius.com/.

  4. 4.

    https://www.brentozar.com/archive/2015/10/how-to-download-the-stack-overflow-database-via-bittorrent/.

  5. 5.

    https://figshare.com/s/29e7db728044a62fd26b.

References

  1. Abdalkareem, R., Oda, V., Mujahid, S., Shihab, E.: On the impact of using trivial packages: an empirical case study on npm and PyPi. Empir. Softw. Eng. 25, 1168–1204 (2020)

    Article  Google Scholar 

  2. Ahmad, A., Feng, C., Ge, S., Yousif, A.: A survey on mining stack overflow: question and answering (q &a) community. Data Technol. Appl. 52(2), 190–247 (2018)

    Google Scholar 

  3. Asghar, S., Umar, M.: Requirement engineering challenges in development of software applications and selection of customer-off-the-shelf (cots) components. Int. J. Softw. Eng. 1(1), 32–50 (2010)

    Google Scholar 

  4. Atoum, I.: A novel framework for measuring software quality-in-use based on semantic similarity and sentiment analysis of software reviews. J. King Saud Univ.-Comput. Inf. Sci. 32(1), 113–125 (2020)

    Google Scholar 

  5. Ayala, C., Hauge, Ø., Conradi, R., Franch, X., Li, J.: Selection of third party software in off-the-shelf-based software development-an interview study with industrial practitioners. J. Syst. Softw. 84(4), 620–637 (2011)

    Article  Google Scholar 

  6. Bi, T., Liang, P., Tang, A., Xia, X.: Mining architecture tactics and quality attributes knowledge in stack overflow. J. Syst. Softw. 180, 111005 (2021)

    Article  Google Scholar 

  7. Bu, W., Shu, H., Kang, F., Hu, Q., Zhao, Y.: Software subclassification based on bertopic-bert-bilstm model. Electronics 12(18), 3798 (2023)

    Article  Google Scholar 

  8. Farshidi, S., Jansen, S., van der Werf, J.M.: Capturing software architecture knowledge for pattern-driven design. J. Syst. Softw. 169, 110714 (2020)

    Article  Google Scholar 

  9. Hou, F., Jansen, F., De Vries, A., Jansen, S.: The role of software trust in selection of open-source and closed software. In: 2023 IEEE/ACM 11th International Workshop on Software Engineering for Systems-of-Systems and Software Ecosystems (SESoS), pp. 30–37. IEEE (2023)

    Google Scholar 

  10. Hsu, H.M., Chen, C.T.: Aggregation of fuzzy opinions under group decision making. Fuzzy Sets Syst. 79(3), 279–285 (1996)

    Article  MathSciNet  Google Scholar 

  11. Hussain, M.E., Hussain, R.: Real time Aho-corasick implementation of string matching technique suitable for smart IoT. In: 2021 International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON), pp. 1–6 (2021)

    Google Scholar 

  12. International Organization for Standardization: ISO/IEC 25010:2011 - Systems and software engineering. Technical Report ISO/IEC 25010, ISO (2011)

    Google Scholar 

  13. Jing, N., Wu, Z., Wang, H.: A hybrid model integrating deep learning with investor sentiment analysis for stock price prediction. Expert Syst. Appl. 178, 115019 (2021)

    Article  Google Scholar 

  14. de Jonge, M.: Developing product lines with third-party components. Electron. Notes Theor. Comput. Sci. 238(5), 63–80 (2009)

    Article  Google Scholar 

  15. Kenton, J.D.M.W.C., Toutanova, L.K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of naacL-HLT, vol. 1, p. 2 (2019)

    Google Scholar 

  16. Leopairote, W., Surarerks, A., Prompoon, N.: Evaluating software quality in use using user reviews mining. In: The 2013 10th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 257–262. IEEE (2013)

    Google Scholar 

  17. Li, P.L., Ko, A.J., Zhu, J.: What makes a great software engineer? In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 1, pp. 700–710. IEEE (2015)

    Google Scholar 

  18. Li, Z., Shen, Z.: Deep semantic mining of big multimedia data advertisements based on needs ontology construction. Multimedia Tools Appl. 81(20), 28079–28102 (2022)

    Article  Google Scholar 

  19. Lin, B., Zampetti, F., Bavota, G., Di Penta, M., Lanza, M., Oliveto, R.: Sentiment analysis for software engineering: how far can we go? In: Proceedings of the 40th International Conference on Software Engineering, pp. 94–104 (2018)

    Google Scholar 

  20. Liu, B.: Introduction. In: Liu, B. (ed.) Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data, pp. 1–14. Springer, Berlin, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19460-3_1

    Chapter  Google Scholar 

  21. Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)

  22. Mäntylä, M.V., Novielli, N., Lanubile, F., Claes, M., Kuutila, M.: Bootstrapping a lexicon for emotional arousal in software engineering. In: 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR), pp. 198–202. IEEE (2017)

    Google Scholar 

  23. Medhat, W., Hassan, A., Korashy, H.: Sentiment analysis algorithms and applications: a survey. Ain Shams Eng. J. 5(4), 1093–1113 (2014)

    Article  Google Scholar 

  24. Mihalcea, R., Tarau, P.: TextRank: bringing order into text. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp. 404–411 (2004)

    Google Scholar 

  25. Mutanov, G., Karyukin, V., Mamykova, Z.: Multi-class sentiment analysis of social media data with machine learning algorithms. Comput. Mater. Continua 69(1) (2021)

    Google Scholar 

  26. Obaidi, M., Nagel, L., Specht, A., Klünder, J.: Sentiment analysis tools in software engineering: a systematic mapping study. Inf. Softw. Technol., 107018 (2022)

    Google Scholar 

  27. Ohm, M., Stuke, C.: SoK: practical detection of software supply chain attacks. In: Proceedings of the 18th International Conference on Availability, Reliability and Security, pp. 1–11 (2023)

    Google Scholar 

  28. Oriol, M., Marco, J., Franch, X.: Quality models for web services: a systematic mapping. Inf. Softw. Technol. 56(10), 1167–1182 (2014)

    Article  Google Scholar 

  29. Pang, B., Lee, L., et al.: Opinion mining and sentiment analysis. Found. Trends® Inf. Retrieval 2(1–2), 1–135 (2008)

    Google Scholar 

  30. Pang, G., et al.: Aspect-level sentiment analysis approach via BERT and aspect feature location model. Wirel. Commun. Mob. Comput. 2021, 1–13 (2021)

    Article  Google Scholar 

  31. Pao, D., Lin, W., Liu, B.: A memory-efficient pipelined implementation of the Aho-corasick string-matching algorithm. ACM Trans. Archit. Code Optim. (TACO) 7(2), 1–27 (2010)

    Article  Google Scholar 

  32. Qi, J., Zhang, Z., Jeon, S., Zhou, Y.: Mining customer requirements from online reviews: a product improvement perspective. Inf. Manage. 53(8), 951–963 (2016)

    Article  Google Scholar 

  33. Ruoning, X., Xiaoyan, Z.: Extensions of the analytic hierarchy process in fuzzy environment. Fuzzy Sets Syst. 52(3), 251–257 (1992)

    Article  MathSciNet  Google Scholar 

  34. Shukri, S.E., Yaghi, R.I., Aljarah, I., Alsawalqah, H.: Twitter sentiment analysis: a case study in the automotive industry. In: 2015 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT), pp. 1–5. IEEE (2015)

    Google Scholar 

  35. Van Oordt, S., Guzman, E.: On the role of user feedback in software evolution: a practitioners’ perspective. In: 2021 IEEE 29th International Requirements Engineering Conference (RE), pp. 221–232. IEEE (2021)

    Google Scholar 

  36. Wankhade, M., Rao, A.C.S., Kulkarni, C.: A survey on sentiment analysis methods, applications, and challenges. Artif. Intell. Rev. 55(7), 5731–5780 (2022)

    Article  Google Scholar 

  37. Zou, Y., Liu, C., Jin, Y., Xie, B.: Assessing software quality through web comment search and analysis. In: Favaro, J., Morisio, M. (eds.) Safe and Secure Software Reuse, pp. 208–223. Springer, Berlin, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38977-1_14

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Utrecht University, Utrecht, The Netherlands

    Fang Hou, Liang Feng & Slinger Jansen

  2. University of Amsterdam, Amsterdam, The Netherlands

    Siamak Farshidi

  3. Lappeenranta University of Technology, Lappeenranta, Finland

    Slinger Jansen

Authors
  1. Fang Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liang Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Siamak Farshidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Slinger Jansen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fang Hou .

Editor information

Editors and Affiliations

  1. University of Tartu, Tartu, Estonia

    Dietmar Pfahl

  2. Blekinge Institute of Technology, Karlskrona, Sweden

    Javier Gonzalez Huerta

  3. Leibniz Universität Hannover, Hannover, Germany

    Jil Klünder

  4. University of Tartu, Tartu, Estonia

    Hina Anwar

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hou, F., Feng, L., Farshidi, S., Jansen, S. (2025). Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool. In: Pfahl, D., Gonzalez Huerta, J., Klünder, J., Anwar, H. (eds) Product-Focused Software Process Improvement. PROFES 2024. Lecture Notes in Computer Science, vol 15452. Springer, Cham. https://doi.org/10.1007/978-3-031-78386-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-78386-9_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-78385-2

  • Online ISBN: 978-3-031-78386-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics