Elsevier

Computer Standards & Interfaces

Volume 48, November 2016, Pages 10-18
Computer Standards & Interfaces

An analysis of the factors determining software product quality: A comparative study

https://doi.org/10.1016/j.csi.2016.04.002Get rights and content

Highlights

  • Individual character is the most relevant factor identified as determining of software products' quality.

  • Competence, training, knowledge, and level of user involvement as well as resistance to change are items related to this factor.

  • Technological aspects had the highest ratings compared to organizational aspects due to the strong relationship of the items.

Abstract

There has been a great deal of research on software quality, but few studies have stressed the factors beyond the scope of software products that can influence the final product's quality. These factors can also determine project success.

Objective

In this paper, a comparative study is conducted of the determinants of software quality, based on a prior study that only explored U.S. CIOs' (Chief Information Officers) perceptions of factors that could affect the final quality of software products. The aim of this study is to explore the perceptions of different users involved in the software development cycle and generate results that can be generalized and employed as an aid in the management of software project resources.

Method

The study was conducted through an online survey to various users involved in the software development cycle in Brazil. The respondents analyzed the same 24 items proposed in the previous study, categorized into individual, technological, and organizational factors. Based on the 175 responses obtained, a factor analysis technique was applied, considering the statistical model of the main components in order to identify the factors determining the quality of software products.

Results

After the factor analysis, it was identified that all 24 analyzed items displayed factor loadings greater than 0.5. Nine factors (9 eigenvalues greater than 1.0) were extracted from this analysis, with the value of the total variance equal to 72%.

Conclusion

Based upon the comparison between the studies, it was concluded that the most relevant factor identified in both surveys presented an individual character. This factor related items such as competence, training, knowledge, and level of user involvement as well as resistance to change. It was also identified through factor analysis that technological aspects had the highest ratings due to the strong relationship of the items comprising these factors compared to organizational aspects.

Introduction

According to the study conducted by Cap Gemini, Sogeti, and HP, disclosed in the “World Quality Report 2014–15” [1], investments in the area of quality assurance have increased in recent years. The percentage of budget invested in this area increased from 18% in 2012 to 23% in 2013, and reached 26% in 2014. However, even with this obvious growth of investment, only one group (1% to 3%) of executives surveyed in 2012 and 2013 reported that their companies used more than 40% of their IT budget for quality assurance. However, even with the increased investment in the area of quality assurance, there is still no guarantee of the quality of the developed products. It should be noted that much of the success of software projects relates to user satisfaction and, consequently, the quality of the generated products. To address these software quality issues, the ISO/IEC (International Standards Organization/International Electrotechnical Commission) published the 25,000 family of standards known as SQuaRE (Software Product Quality Requirements and Evaluation), which presents the Model of Software Product Quality [2]. This model is based on software product quality characteristics and sub-characteristics that can be used for both specifying software quality requirements as well as for their evaluation. Due to the importance of this topic, several studies have been developed in the area of quality that explore software product quality characteristics and sub-characteristics [3]. However, most of the studies address purely technological aspects such as: Metrics to assess the functional quality of the products generated [4], the quality of the generated code presented [5], [6], and the number of errors found or aspects related to the product's usability [7], [8]. Little research has focused primarily on the analysis of behavioral aspects that could affect the quality of software products, with Hoffman [9] and Acuña et al. [10] being among the few examples.

After identifying this gap, a study was conducted in 2010 by Gorla and Lin [11], with the main objective of identifying the factors beyond the scope of the software product that could influence software quality in organizations. These factors could be organizational, technological, or individual. To conduct this study, the authors sent a survey to some American CIOs (Chief Information Officers). The choice of these respondents was motivated by a prior study [12] that indicated improved quality in information technology as one of the top five concerns of IT executives. At the end of the study, 112 responses were obtained, which were assessed using different methods of analysis such as factor analysis and logistic regression. The result of the first analysis was intended to derive the factors (individual, technological, and organizational) that influence the quality of software products. The result of the second analysis sought to measure the strength of the association between the factors and attributes of software quality. The results of these analyses could help CIOs and CEOs (Chief Executive Officers) in the development of quality improvement programs, enabling a suitable management of resources within an organization.

In the study conducted by Gorla and Lin [11], it was possible to derive the factors determining software quality and to identify which of those identified factors were predominant. The authors identified the “Capacity of the users” as the most representative factor of the first analysis (factor analysis), i.e., a factor classified as individual. However, at the end of the second analysis (logistic regression), factors related to organizational aspects were identified as the most influential for software quality. This result is not surprising, given that the respondents were all CIOs who, in general, have a strategic view of the business, thereby considering organizational factors to be more relevant than technological ones.

Due to the importance of identifying the factors determining software quality in order to ensure a better management of resources within organizations, this study aims to conduct a comparison with the results obtained in the first stage of analysis performed by Gorla and Lin, while using different profiles of respondents involved in the software development cycle. It will thus be possible to complement the prior study, which only explored the perceptions of American CIOs, by adding the perspectives of new respondents.

The inclusion of new respondents when compiling the results is relevant because it will thus be possible to try to generalize the results and reduce the bias generated in the previous study. To this end, the same data collection method (survey) will be used and the same analysis (factor analysis) will be performed, as proposed by Gorla and Lin [11], but from the perspective of new respondents.

This paper is organized as follows: Section 2 presents the literature review and an explanation of the ISO/IEC 9126 and ISO/IEC 25000 standards. Section 3 describes the model used in the research as well as the relationship between the variables. Section 4 presents the research method used and the statistical calculations in detail. Section 5 presents the analysis of the results and Section 6 discusses the results achieved. Section 7 concludes by presenting some limitations and possibilities for future work.

Section snippets

History of software quality models

According to the ISO/IEC 8402 standard [13], which is referenced in the ISO/IEC 9126 [3] and ISO/IEC 25000 [2] standards, software quality is the software product's ability to satisfy explicit and implicit needs under specific conditions. Since software quality is considered multidimensional, it is very important to establish which aspects are important to evaluate.

Various software quality models have previously been proposed such as the McCall model in 1977 [14], followed by the Boehm model in

Basic structure of the research

This section presents the basic structure of the research conducted by Gorla and Lin [11], also used in this study to make possible the comparative study. Fig. 2 illustrates how the relationship of the organizational, technological, and individual factors with software quality was conceived.

In the research conducted by Gorla and Lin [11], the software quality was represented by five characteristics, identified by: (1) reliability, (2) ease of use, (3) maintainability, (4) usefulness, and (5)

Research method

A survey was conducted with three distinct stages: collection of personal identification data, collection of the characteristics of the organization where the respondent works, and the evaluation of 24 items that might or might not influence software product quality in the opinion of each respondent.

The target audience was professionals involved in the software development cycle, but specifically those handling operational activities, with less involvement in management activities. This made it

Results

Since the main objective of this study is to explore the perceptions of different users involved in the software development cycle in order to analyze, through a comparative study, the factors determining software product quality, an exploratory factor analysis was performed, taking into consideration the model of principal components and the normalized varimax rotation. The results from the questionnaires were described in frequencies and percentages, according to the response options (none,

Discussion of the results

According to the data presented in Section 5, this study identified nine factors that determine software product quality: (1) capabilities of users, (2) suitability of the employed technology, (3) technological capacity, (4) size of the organization, (5) influence of the upper management, (6) maturity of the organization, (7) stability of organization, (8) quality of the service provided, and (9) available budget. For each factor, its percentage of variance was calculated, which represents the

Conclusion

The main objective of this study was to explore the perceptions of different users involved in the software development cycle in the assessment of factors determining software product quality. This made it possible to conduct a comparative analysis of this study and the prior study by Gorla and Lin [11], which explored the perception of only one type of respondent. The results of the study by Gorla and Lin were based on 112 responses from American CIOs, related to information technology

Karina Curcio has a Bachelor's degree in Informatics, Master's in Electrical Engineering at Federal University of Technology – Paraná. She is a member of the Software Engineering Research Group at Pontifical Catholic University of Paraná and her research interests are in software quality, software process improvement and innovation.

References (33)

  • R. Hofman

    Behavioral economics in software quality engineering

    Empir. Softw. Eng.

    (2010)
  • J. Luftman et al.

    Key issues for IT executives 2007

    MIS Q. Exec.

    (2008)
  • AS/NZS ISO 8402

    Quality Management and Quality Assurance — Vocabulary

    (1994)
  • J.A. McCall et al.

    Factors in Software Quality

    (1977)
  • B.W. Boehm et al.

    Quantitative evaluation of software quality

  • R.G. Dromey

    A model of software product quality

    IEEE Trans. Softw. Eng.

    (February 1995)
  • Cited by (19)

    • Evaluation of bioresource value models: Sustainable development in the agriculture biorefinery sector

      2022, Journal of Agriculture and Food Research
      Citation Excerpt :

      The validation category ensures technical, operational, and data validity. The internal rating of the modeling tool shows efficient use of the model and recommendations [35]. In this study, the criteria selected from McCall's quality factor data include a vast database for the quality criteria to evaluate the modeling tools.

    • A survey on software smells

      2018, Journal of Systems and Software
      Citation Excerpt :

      C8: Politics: Organizational politics for control, position, and power influence the software quality (Curcio et al., 2016; Lavallée and Robillard, 2015; Stribrny and Mackin, 2006). C9: Team culture: Many authors (Acuña et al., 2008; Curcio et al., 2016; Tom et al., 2013) have recognized the practices and the culture prevailed within a team or an organization as a cause of software smells. C10: Poor human resource planning: Poor planning of human resources required for a software project may force the present development team to adopt quick fixes to meet the deadlines (Lavallée and Robillard, 2015).

    • An Experience Report on Assessing Software Engineer's Outputs in Practice

      2023, Proceedings - 2023 IEEE/ACM International Conference on Software and System Processes, ICSSP 2023
    • A Heuristic Tool for Measuring Software Quality Using Program Language Standards

      2022, International Arab Journal of Information Technology
    View all citing articles on Scopus

    Karina Curcio has a Bachelor's degree in Informatics, Master's in Electrical Engineering at Federal University of Technology – Paraná. She is a member of the Software Engineering Research Group at Pontifical Catholic University of Paraná and her research interests are in software quality, software process improvement and innovation.

    Andreia Malucelli has a Bachelor's degree in Informatics, Master's in Electrical Engineering and Doctorate in Electrical and Computing Engineering. Professor at Pontifical Catholic University of Paraná, a Researcher at the Software Engineering Research Group, with experience in computer science, software engineering, artificial intelligence, organizational learning, ontologies, multiagent systems and healthcare information systems.

    Sheila Reinehr has a Bachelor's degree in Mechanical Engineering, Master's in Informatics and Doctorate in Engineering. Professor at Pontifical Catholic University of Paraná, a Researcher at the Software Engineering Research Group, with experience in computer science, software engineering, software process improvement, software quality, project and portfolio management, software product lines, and metrics.

    Marco Antônio Paludo has a Bachelor's degree in Computer Science, Master's in Applied Informatics and a PhD candidate at Pontifical Catholic University of Paraná, where he also is a lecturer and Project Manager with experience in computer science, software engineering, software process improvement, software quality and modeling.

    View full text