An analysis of the factors determining software product quality: A comparative study
Introduction
According to the study conducted by Cap Gemini, Sogeti, and HP, disclosed in the “World Quality Report 2014–15” [1], investments in the area of quality assurance have increased in recent years. The percentage of budget invested in this area increased from 18% in 2012 to 23% in 2013, and reached 26% in 2014. However, even with this obvious growth of investment, only one group (1% to 3%) of executives surveyed in 2012 and 2013 reported that their companies used more than 40% of their IT budget for quality assurance. However, even with the increased investment in the area of quality assurance, there is still no guarantee of the quality of the developed products. It should be noted that much of the success of software projects relates to user satisfaction and, consequently, the quality of the generated products. To address these software quality issues, the ISO/IEC (International Standards Organization/International Electrotechnical Commission) published the 25,000 family of standards known as SQuaRE (Software Product Quality Requirements and Evaluation), which presents the Model of Software Product Quality [2]. This model is based on software product quality characteristics and sub-characteristics that can be used for both specifying software quality requirements as well as for their evaluation. Due to the importance of this topic, several studies have been developed in the area of quality that explore software product quality characteristics and sub-characteristics [3]. However, most of the studies address purely technological aspects such as: Metrics to assess the functional quality of the products generated [4], the quality of the generated code presented [5], [6], and the number of errors found or aspects related to the product's usability [7], [8]. Little research has focused primarily on the analysis of behavioral aspects that could affect the quality of software products, with Hoffman [9] and Acuña et al. [10] being among the few examples.
After identifying this gap, a study was conducted in 2010 by Gorla and Lin [11], with the main objective of identifying the factors beyond the scope of the software product that could influence software quality in organizations. These factors could be organizational, technological, or individual. To conduct this study, the authors sent a survey to some American CIOs (Chief Information Officers). The choice of these respondents was motivated by a prior study [12] that indicated improved quality in information technology as one of the top five concerns of IT executives. At the end of the study, 112 responses were obtained, which were assessed using different methods of analysis such as factor analysis and logistic regression. The result of the first analysis was intended to derive the factors (individual, technological, and organizational) that influence the quality of software products. The result of the second analysis sought to measure the strength of the association between the factors and attributes of software quality. The results of these analyses could help CIOs and CEOs (Chief Executive Officers) in the development of quality improvement programs, enabling a suitable management of resources within an organization.
In the study conducted by Gorla and Lin [11], it was possible to derive the factors determining software quality and to identify which of those identified factors were predominant. The authors identified the “Capacity of the users” as the most representative factor of the first analysis (factor analysis), i.e., a factor classified as individual. However, at the end of the second analysis (logistic regression), factors related to organizational aspects were identified as the most influential for software quality. This result is not surprising, given that the respondents were all CIOs who, in general, have a strategic view of the business, thereby considering organizational factors to be more relevant than technological ones.
Due to the importance of identifying the factors determining software quality in order to ensure a better management of resources within organizations, this study aims to conduct a comparison with the results obtained in the first stage of analysis performed by Gorla and Lin, while using different profiles of respondents involved in the software development cycle. It will thus be possible to complement the prior study, which only explored the perceptions of American CIOs, by adding the perspectives of new respondents.
The inclusion of new respondents when compiling the results is relevant because it will thus be possible to try to generalize the results and reduce the bias generated in the previous study. To this end, the same data collection method (survey) will be used and the same analysis (factor analysis) will be performed, as proposed by Gorla and Lin [11], but from the perspective of new respondents.
This paper is organized as follows: Section 2 presents the literature review and an explanation of the ISO/IEC 9126 and ISO/IEC 25000 standards. Section 3 describes the model used in the research as well as the relationship between the variables. Section 4 presents the research method used and the statistical calculations in detail. Section 5 presents the analysis of the results and Section 6 discusses the results achieved. Section 7 concludes by presenting some limitations and possibilities for future work.
Section snippets
History of software quality models
According to the ISO/IEC 8402 standard [13], which is referenced in the ISO/IEC 9126 [3] and ISO/IEC 25000 [2] standards, software quality is the software product's ability to satisfy explicit and implicit needs under specific conditions. Since software quality is considered multidimensional, it is very important to establish which aspects are important to evaluate.
Various software quality models have previously been proposed such as the McCall model in 1977 [14], followed by the Boehm model in
Basic structure of the research
This section presents the basic structure of the research conducted by Gorla and Lin [11], also used in this study to make possible the comparative study. Fig. 2 illustrates how the relationship of the organizational, technological, and individual factors with software quality was conceived.
In the research conducted by Gorla and Lin [11], the software quality was represented by five characteristics, identified by: (1) reliability, (2) ease of use, (3) maintainability, (4) usefulness, and (5)
Research method
A survey was conducted with three distinct stages: collection of personal identification data, collection of the characteristics of the organization where the respondent works, and the evaluation of 24 items that might or might not influence software product quality in the opinion of each respondent.
The target audience was professionals involved in the software development cycle, but specifically those handling operational activities, with less involvement in management activities. This made it
Results
Since the main objective of this study is to explore the perceptions of different users involved in the software development cycle in order to analyze, through a comparative study, the factors determining software product quality, an exploratory factor analysis was performed, taking into consideration the model of principal components and the normalized varimax rotation. The results from the questionnaires were described in frequencies and percentages, according to the response options (none,
Discussion of the results
According to the data presented in Section 5, this study identified nine factors that determine software product quality: (1) capabilities of users, (2) suitability of the employed technology, (3) technological capacity, (4) size of the organization, (5) influence of the upper management, (6) maturity of the organization, (7) stability of organization, (8) quality of the service provided, and (9) available budget. For each factor, its percentage of variance was calculated, which represents the
Conclusion
The main objective of this study was to explore the perceptions of different users involved in the software development cycle in the assessment of factors determining software product quality. This made it possible to conduct a comparative analysis of this study and the prior study by Gorla and Lin [11], which explored the perception of only one type of respondent. The results of the study by Gorla and Lin were based on 112 responses from American CIOs, related to information technology
Karina Curcio has a Bachelor's degree in Informatics, Master's in Electrical Engineering at Federal University of Technology – Paraná. She is a member of the Software Engineering Research Group at Pontifical Catholic University of Paraná and her research interests are in software quality, software process improvement and innovation.
References (33)
- et al.
How do personality, team processes and task characteristics relate to job satisfaction and software quality?
Inf. Softw. Technol.
(2009) - et al.
Determinants of software quality: a survey of information systems project managers
Inf. Softw. Technol.
(2010) - ...
International organization for standardization/international electrotechnical
Software Engineering – Product Quality – Part 1: Quality Model
(2001)- et al.
Metrics to evaluate functional quality: a systematic review
- et al.
Is software quality visible in the code?
IEEE Softw.
(August 1998) - et al.
Standardized Code Quality Benchmarking for Improving Software Maintainability
(2012) - et al.
Usability Measurement and Metrics: A Consolidated Model
(2006) - et al.
Usability Meanings and Interpretations in ISO Standards
(2003)
Behavioral economics in software quality engineering
Empir. Softw. Eng.
Key issues for IT executives 2007
MIS Q. Exec.
Quality Management and Quality Assurance — Vocabulary
Factors in Software Quality
Quantitative evaluation of software quality
A model of software product quality
IEEE Trans. Softw. Eng.
Cited by (19)
Application of a human-centered design for embedded machine learning model to develop data labeling software with nurses: Human-to-Artificial Intelligence (H2AI)
2024, International Journal of Medical InformaticsEvaluation of bioresource value models: Sustainable development in the agriculture biorefinery sector
2022, Journal of Agriculture and Food ResearchCitation Excerpt :The validation category ensures technical, operational, and data validity. The internal rating of the modeling tool shows efficient use of the model and recommendations [35]. In this study, the criteria selected from McCall's quality factor data include a vast database for the quality criteria to evaluate the modeling tools.
Building a hierarchical structure model of enablers that affect the software process improvement in software SMEs—A mixed method approach
2019, Computer Standards and InterfacesA survey on software smells
2018, Journal of Systems and SoftwareCitation Excerpt :C8: Politics: Organizational politics for control, position, and power influence the software quality (Curcio et al., 2016; Lavallée and Robillard, 2015; Stribrny and Mackin, 2006). C9: Team culture: Many authors (Acuña et al., 2008; Curcio et al., 2016; Tom et al., 2013) have recognized the practices and the culture prevailed within a team or an organization as a cause of software smells. C10: Poor human resource planning: Poor planning of human resources required for a software project may force the present development team to adopt quick fixes to meet the deadlines (Lavallée and Robillard, 2015).
An Experience Report on Assessing Software Engineer's Outputs in Practice
2023, Proceedings - 2023 IEEE/ACM International Conference on Software and System Processes, ICSSP 2023A Heuristic Tool for Measuring Software Quality Using Program Language Standards
2022, International Arab Journal of Information Technology
Karina Curcio has a Bachelor's degree in Informatics, Master's in Electrical Engineering at Federal University of Technology – Paraná. She is a member of the Software Engineering Research Group at Pontifical Catholic University of Paraná and her research interests are in software quality, software process improvement and innovation.
Andreia Malucelli has a Bachelor's degree in Informatics, Master's in Electrical Engineering and Doctorate in Electrical and Computing Engineering. Professor at Pontifical Catholic University of Paraná, a Researcher at the Software Engineering Research Group, with experience in computer science, software engineering, artificial intelligence, organizational learning, ontologies, multiagent systems and healthcare information systems.
Sheila Reinehr has a Bachelor's degree in Mechanical Engineering, Master's in Informatics and Doctorate in Engineering. Professor at Pontifical Catholic University of Paraná, a Researcher at the Software Engineering Research Group, with experience in computer science, software engineering, software process improvement, software quality, project and portfolio management, software product lines, and metrics.
Marco Antônio Paludo has a Bachelor's degree in Computer Science, Master's in Applied Informatics and a PhD candidate at Pontifical Catholic University of Paraná, where he also is a lecturer and Project Manager with experience in computer science, software engineering, software process improvement, software quality and modeling.