Skip to main content

Matching terms of quality models and meta-models: toward a unified meta-model of OSS quality

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Context

In the last two decades, open-source software (OSS) has gained increasing attention due to its voluntary supporters, growing community, and ease of accessibility in cloud repositories. Standardization in OSS quality is of vital importance as a communication vehicle for stakeholders in identifying and selecting high-quality products. Thus, meta-models help to define a standardized language and enable to propose quality models that can be used to perform comparable measurements.

Objective

Considering the lack of a comprehensive meta-model of OSS quality in the literature, there appears a need to see a more complete picture of OSS quality and to represent its concepts more formally. Therefore, in this study, it is aimed to develop a solid base for a comprehensive meta-model of OSS quality to create a common understanding among stakeholders.

Method

A systematic way has been followed toward developing a common structure, defining a consistent terminology and, finally, providing a meta-model of OSS quality. In this context, (1) the common structure of the quality models for OSS has been investigated, (2) the terms of the general-purpose meta-models of software quality have been analyzed based on the international standards, and (3) the terms of the quality models for OSS have been mapped with the elements of these meta-models.

Results

An initial meta-model of OSS quality, which employs a unified structure from the OSS quality models and eliminates the inconsistencies determined in the general-purpose meta-models of software quality, has been proposed and an implementation of this meta-model has been demonstrated.

Conclusion

This initial meta-model of OSS quality with a standard terminology can be taken as a guide by researchers who will propose or revise their OSS quality models. It will allow developing multiple OSS quality models with homogenous structure and terms, and also enable comparing the evaluation results obtained by these models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The data that support the findings of this study are openly available in Zenodo at the following URL: definition of the terminologies in the SQMM, Zenodo, https://doi.org/10.5281/zenodo.6367596.

References

  • Adewumi, A., Misra, S., & Omoregbe, N. (2013). A review of models for evaluating quality in open source software. IERI Procedia, 4, 88–92.

    Article  Google Scholar 

  • Adewumi, A., Misra S., Omoragbe, N., Crawford, B., & Soto, R. (2016). A systematic literature review of open source software quality assessment models. SpringerPlus, 5.1, 1936.

  • Adewumi, A., Misra S., & Omoragbe, N. (2019). FOSSES: framework for open‐source software evaluation and selection. Software: Practice and Experience, 49.5, 780–812.

  • Ahmad, N., & Laplante, P. A. (2013). A systematic approach to evaluating open source software. Howard C, ed. Strategic Adoption of Technological Innovations. Hershey, PA: IGI Global: pp 50–69.

  • Al-Badareen, A. B., Selamat, M. H., Jabar, M. A., Din, J., & Turaev, S. (2011). Software quality models: a comparative study. International Conference on Software Engineering and Computer Systems, (pp. 46–55). Springer, Berlin, Heidelberg.

  • Al-Dhaqm, A., Razak, S., Othman, S. H., Ngadi, A., Ahmed, M. N., & Ali Mohammed, A. (2017). Development and validation of a database forensic metamodel (DBFM). PloS one, 12(2), e0170793.

  • Alfonzo, O., Domínguez, K., Rivas, L., Perez, M., Mendoza, L., & Ortega, M. (2008). Quality measurement model for analysis and design tools based on FLOSS. 19th Australian conference on software engineering. Perth, (pp 26–28) Australia.

  • Alvaro, A., Almeida, E. S., & Meira, S. R. L. (2010). A software component quality framework. ACM SIGSOFT Software Engineering Notes, 35(1), 1–4.

    Article  Google Scholar 

  • Aversano, L., & Tortorella, M. (2013). Quality evaluation of floss projects: application to ERP systems. Information and Software Technology55.7, 1260–1276.

  • Barcellos, M.P., de Almeida Falbo, R., & Dal Moro, R. (2010). A well-founded software measurement ontology. FOIS, (pp. 213–226).

  • Barcellos, M. P., & de Almeida Falbo, R. (2013). A software measurement task ontology. Proceedings of the 28th Annual ACM Symposium on Applied Computing, (pp. 311–318).

  • Bertoa, M., & Vallecillo, A. (2002). Quality attributes for COTS components”, I+D Computación, Vol 1. Nro, 2, 128–144.

    Google Scholar 

  • Bertoa, M. F., Vallecillo, A., & García, F. (2006). An ontology for software measurement. Ontologies for Software Engineering and Software Technology, (pp. 175–196). Springer, Berlin, Heidelberg.

  • Beydoun, G., Low, G., Henderson-Sellers, B., Mouratidis, H., Gomez-Sanz, J. J., Pavon, J., & Gonzalez-Perez, C. (2009). FAML: A generic metamodel for MAS development. IEEE Transactions on Software Engineering, 35(6), 841–863.

    Article  Google Scholar 

  • Boehm, B. W., Brown, H., & Lipow, M. (1978). Quantitative evaluation of software quality. Proceedings of the 2nd International Conference on Software Engineering, (pp. 592–605).

  • Briand, L., Morasca, S., & Basili, V. (2002). An operational process for goal driven definition of measures. IEEE Transactions on Software Engineering, 28(12), 1106–1125.

    Article  Google Scholar 

  • Ciolkowski, M., & Soto, M. (2008). Towards a comprehensive approach for assessing open source projects. Software Process and Product Measurement, (pp. 316–330). Springer, Berlin, Heidelberg.

  • Chirinos, L., Losavio, F., & Bøegh, J. (2005). Characterizing a data model for software measurement. Journal of Systems and Software, 74(2), 207–226.

    Article  Google Scholar 

  • Czarnacka, C. B. (2009). The ISO/IEC Standards for the Software Processes and Products Measurement. (pp. 187–200).

  • Del Bianco, V., Lavazza, L., Morasca, S., & Taibi, D. (2009). Quality of open source software: The QualiPSo Trustworthiness Model. IFIP International Conference on Open Source Systems. Springer, Berlin, Heidelberg.

  • Dromey, R. G. (1995). A model for software product quality. IEEE Transactions on Software Engineering, 21(2), 146–162.

    Article  Google Scholar 

  • Duijnhouwer, F. W., & Widdows, C. (2003). Capgemini Expert Letter Open Source Maturity Model, Capgemini, tinyurl.com/yxdbvjk6.

  • Eghan, E. E., Alqahtani, S.S., Forbes, C., & Rilling, J. (2019). API trustworthiness: an ontological approach for software library adoption. Software Quality, Journal, 27.3, 969–1014.

  • Garcia, F., Bertoa, M. F., Calero, C., Vallecillo, A., Ruiz, F., Piattini, M., & Genero, M. (2006). Towards a consistent terminology for software measurement. Information and Software Technology, 48(8), 631–644.

    Article  Google Scholar 

  • Garcia, F., Ruiz, F., Calero, C., Bertoa, M. F., Vallecillo, A., Mora, B., & Piattini, M. (2009). Effective use of ontologies in software measurement. Knowledge Eng. Review, 24(1), 23–40.

    Article  Google Scholar 

  • Garcia, F., Serrano, M., Cruz-Lemus, J., Ruiz, F., Piattini, M., & ALARCOS Research Group. (2007). Managing software process measurement: a metamodel-based approach. Information Sciences, 177(12), pp. 2570–2586.

  • Georgiadoui, E. (2003). GEQUAMO–a generic, multilayered, customizable software quality model. Software Quality Journal, 11(4), 313–323. https://doi.org/10.1023/A:1025817312035

    Article  Google Scholar 

  • Grady, R. B. (1992). Practical software metrics for project management and process improvement. Prentice Hall.

    Google Scholar 

  • Haaland, K., Groven, A.K., Regnesentral, N., Glott, R., Tannenberg, A., & FreeCode, A.S. (2010). Free/libre open source quality models–a comparison between two approaches. 4th FLOS International Workshop on Free/Libre/Open Source Software, (pp. 1–17).

  • Hauge, O., Osterlie, T., & Sorensen, C. F. (2009). An empirical study on selection of open source software–preliminary results. ICSE Workshop on Emerging Trends in Free/Libre/Open Source Software Research and Development. IEEE.

  • Hasnain, S., Ali, M. K., Akhter, J., Ahmed, B., & Abbas, N. (2020). Selection of an industrial boiler for a soda-ash production plant using analytical hierarchy process and TOPSIS approaches. Case Studies in Thermal Engineering, 19, 100636.

    Article  Google Scholar 

  • Henderson-Sellers, B., & Bulthuis, A. (1996). COMMA: Sample metamodels. JOOP, 9(7), 44–48.

    Google Scholar 

  • Hwang, C. L., & Yoon, K. (1981). Multiple attribute decision making: Methods and applications. Springer-Verlag.

    Book  MATH  Google Scholar 

  • Işıklar, G., & Büyüközkan, G. (2007). Using a multi-criteria decision making approach to evaluate mobile phone alternatives. Computer Standards & Interfaces, 29(2), 265–274.

    Article  Google Scholar 

  • IEEE. (1998). Standard for a Software Quality Metrics Methodology. IEEE Standards, (pp. 1061–1998).

  • IEEE 610.12. (1990). IEEE Standard Glossary of Software Engineering Terminology.

  • ISO/IEC 9126–1. (2001). Software Engineering - Product Quality - Part 1: Quality Model, International Organization for Standardization, Geneva, Switzerland.

  • ISO/IEC 15939. (2007). Software engineering – software measurement process, second edition.

  • ISO/IEC 25010. (2008). Software Engineering: Software Product Quality Requirements and Evaluation (SQuaRE) Quality Model and Guide. International Organization for Standardization. Geneva, Switzerland.

  • ISO/IEC 14598. (1999). Information Technology, Software Product Evaluation: Process for Developers. Software Engineering.

  • ISO/IEC 15504–1. (2004). Information technology – Process assessment – Concepts and vocabulary.

  • ISO/IEC 14143–6. (2012). Information technology, Software measurement, Functional size measurement.

  • ISO/IEC. ISO/IEC 12207. (2008). System and software engineering – Software life-cycle processes, second edition.

  • ISO/IEC 19761. (2002). Software Engineering COSMIC-FFP, A functional size measurement method. International Organization for Standardization ISO, Geneva.

  • ISO/IEC 25020. (2019). Systems and software engineering, Systems and software Quality Requirements and Evaluation (SQuaRE), Quality measurement framework.

  • ISO, International Standard ISO VIM. (1993). International Vocabulary of Basic and General Terms in Metrology, International Standards Organization, Geneva, Switzerland, second edition.

  • Jadhav, A. S., & Sonar, R. M. (2011). Framework for evaluation and selection of the software packages: A hybrid knowledge based system approach. Journal of Systems and Software, 84(8), 1394–1407.

    Article  Google Scholar 

  • Jean-Christophe, D., & Alexandre, S. (2008). Comparing assessment methodologies for free/open source software: OpenBRR and QSOS. International Conference on Product Focused Software Process Improvement. Springer, Berlin, Heidelberg.

  • Khatri, S. K., & Singh, I. (2016). Evaluation of open source software and improving its quality. 5th International Conference on Reliability, Infocom Technologies and Optimization (ICRITO). IEEE.

  • Karagiannis, D., & Kühn, H. (2002). Metamodelling platforms. EC-Web, 2455, p. 182.

  • Khondoker, R., Zaalouk, A., Marx, R., & Bayarou, K. (2014). Feature-based comparison and selection of software defined networking (SDN) controllers. World Congress on Computer Applications and Information Systems (WCCAIS). Hammamet, Tunisia.

  • Kim, H. M. (1999). Representing and reasoning about quality using enterprise models. PhD thesis, Dept. Mechanical and Industrial Engineering, University of Toronto, Canad.

  • Kitchenham, B., Hughes, R. T., & Linkman, S. G. (2001). Modeling software measurement data. IEEE Transactions on Software Engineering, 27(9), 788–804.

    Article  Google Scholar 

  • Kläs, M., Lampasona, C., Nunnenmacher, S., Wagner, S., Herrmannsdörfer, M., & Lochmann, K. (2010). How to evaluate meta-models for software quality. Proceedings of the 20th International Workshop on Software Measurement.

  • Kuwata, Y., Takeda, K., & Miura, H. (2014). A study on maturity model of open source software community to estimate the quality of products. Procedia Computer Science, 35, 1711–171.

  • Lenarduzzi, V., Taibi, D., Tosi, D., Lavazza, L., & Morasca, S. (2020). Open source software evaluation, selection, and adoption: a systematic literature review. 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE.

  • Maki-Asiala, P., & Matinlassi, M. (2006). Quality assurance of open source components: integrator point of view. 30th Annual International Computer Software and Applications Conference (COMPSAC'06), (Vol 2). IEEE.

  • Mc Call, J. A., Richards, P. K., & Walters, G. F. (1977). Factors in Software Quality, Volumes I, II, and III. US Rome Air Development Center Reports, US Department of Commerce, USA.

  • Mcgarry, J., Card, D., Jones, C., Layman, B., Clark, E., Dean, J., & Hall, F. (2002). Practical software measurement: objective information for decision makers. Addison Wesley.

  • Mens, T., Doctors, L., Habra, N., Vanderose, B., & Kamseu, F.  (2011). Qualgen: Modeling and analysing the quality of evolving software systems. 15th European Conference on Software Maintenance and Reengineering. IEEE.

  • Miguel, J. P., Mauricio, D., & Rodríguez, G. (2014). A review of software quality models for the evaluation of software products. arXiv preprint arXiv:1412.2977.

  • Mohagheghi, P., & Dehlen, V. (2008). A metamodel for specifying quality models in model-driven engineering. Proceedings of the Nordic Workshop on Model Driven Engineering.

  • Nistala, P., Nori, K. V., & Reddy, R. (2019). Software quality models: a systematic mapping study. 2019 IEEE/ACM International Conference on Software and System Processes (ICSSP), (pp. 125–134). IEEE.

  • Object Management Group (OMG), Meta Object Facility (MOF). (2019). Core Specification Version 2.5.1. https://www.omg.org/spec/MOF/2.5.1/PDF

  • Othman, S. H., & Beydoun, G. (2010). Metamodelling approach to support disaster management knowledge sharing. 21st Australasian Conference on Information Systems.

  • Othman, S. H., Beydoun, G., & Sugumaran, V. (2014). Development and validation of a Disaster Management Metamodel (DMM). Information Processing & Management, 50(2), 235–271.

    Article  Google Scholar 

  • Orijin, A. (2006). Method for qualification and selection of open source software (QSOS) version 2.0, http://www.qsos.org/.

  • Özcan, E. C., Ünlüsoy, S., & Eren, T. (2017). A combined goal programming–AHP approach supported with TOPSIS for maintenance strategy selection in hydroelectric power plants. Renewable and Sustainable Energy Reviews, 78, 1410–1423.

    Article  Google Scholar 

  • Petrinja, E., Sillitti, A., & Succi, G. (2010). Comparing OpenBRR, QSOS, and OMM assessment models. IFIP International Conference on Open Source Systems. Springer, Berlin, Heidelberg.

  • Raffoul, E., Domínguez, K., Perez, M., Mendoza, L. E., & Griman, A. C. (2008). Quality model for the selection of FLOSS-based issue tracking system. Proceedings of the IASTED international conference on software engineering, Innsbruck, Austria.

  • Ramamoorthy, C. V., Prakash, A., Tsai, W. T., & Usuda, Y. (1984). Software engineering: Problems and perspectives. Computer, 17(10), 191–209.

    Article  Google Scholar 

  • Rawashdeh, A., & Matalka, B. (2006). A new software quality model for evaluating COTS components”. Journal of Computer Science, 2(4), 373–381.

    Article  Google Scholar 

  • Raza, A., Capretz, L. F., & Ahmed, F. (2012). An open source usability maturity model (OS-UMM). Computers in Human Behavior, 28.4, 1109–1121.

  • Rossi, B., Russo, B., & Succi, G. (2012). Adoption of free/libre open source software in public organizations: factors of impact. Information Technology & People.

  • Rout, T. P. (1999). Consistency and conflict in terminology in software engineering standards. Proceedings 4th IEEE International Software Engineering Standards Symposium and Forum (ISESS'99).

  • Ruiz, F., Genero, M., García, F., Piattini, M., & Calero, C. (2003). A proposal of a software measurement ontology. Conference on Computer Science and Operational Research, Buenos Aires, Argentina.

  • Saaty, T. L. (1980). The analytic hierarchy process: Planning, priority setting and resource allocation. McGraw-Hill.

    MATH  Google Scholar 

  • Saaty, T. L. (2008). Decision making with the analytic hierarchy process. International Journal of Services Sciences, 1(1), 83–98.

    Article  Google Scholar 

  • Saaty, T. L., & Sagir, M. (2015). Ranking countries more reliably in the Summer Olympics. International Journal of the Analytic Hierarchy Process, 7(3), 589–610.

    Google Scholar 

  • Sadeghzadeh, H. M., & Rashidi, H. (2017). Software quality models: A comprehensive review and analysis. Journal of Electrical and Computer Engineering Innovations (JECEI), 6(1), 59–76.

    Google Scholar 

  • Samarthyam, G., Suryanarayana, G., Sharma, T., & Gupta, S. (2013). MIDAS: a design quality assessment method for industrial software. Software Engineering in Practice, (pp 911–920). San Francisco, CA, USA.

  • Samoladas, I., Goussios G., & Spinellis, D. (2008). The SQO-OSS quality model: measurement based open source software evaluation. IFIP international conference on open source systems. Springer, Boston, MA.

  • Sarrab, M., & Rehman, O. M. H. (2014). Empirical study of open source software selection for adoption, based on software quality characteristics. Advances in Engineering Software, 69, 1–11.

  • Siemens company, https://www.siemens.com/global/en.html

  • Software Engineering Institute. (2010). CMMI for development, version 1.3, Technical Report CMU/SEI-2010-TR-033.

  • Sohn H., Lee M. G., Seong B. M., & Kim J. B. (2015). Quality evaluation criteria based on open source mobile HTML5 UI framework for development of cross-platform. International Journal of Software Engineering and Its Applications, 9.6, 1–12.

  • Soto, M., & Ciolkowski, M. (2009). The QualOSS open source assessment model measuring the performance of open source communities. Proceedings of the 3rd International Symposium On Empirical Software Engineering and Measurement.

  • Stol, K. J., & Babar, M. A. (2010). Challenges in using open source software in product development: a review of the literature. Proceedings of the 3rd international workshop on emerging trends in free/libre/open source software research and development.

  • Suman, M. W., & Rohtak, M. D. U. (2014). A comparative study of software quality models. International Journal of Computer Science and Information Technologies, 5(4), 5634–5638.

    Google Scholar 

  • Sung, W. J., Kim, J. H., & Rhew, S. Y. (2007) A quality model for open source software selection. Sixth International Conference on Advanced Language Processing and Web Information Technology (ALPIT 2007). IEEE.

  • Taibi, D., Lavazza, L., & Morasca, S. (2007). OpenBQR: A framework for the assessment of OSS. In: IFIP International Conference on Open Source Systems. Springer, Boston, MA.

  • Tanrıöver, Ö. Ö., & Bilgen, S. (2011). A framework for reviewing domain specific conceptual models. Computer Standards & Interfaces, 33(5), 448–464.

    Article  Google Scholar 

  • Tassone, J., Xu, S., Wang, C., Chen, J., & Du, W. (2018). Quality assessment of open source software: a review. IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS) (pp. 411–416). IEEE.

  • Thapar, S. S., Singh, P., & Rani, S. (2012). Challenges to development of standard software quality model. International Journal of Computer Applications, 49(10).

  • Upadhyay, N., Despande, B. M., & Agrawal, V. P. (2011). Towards a software component quality model. International Conference on Computer Science and Information Technology, (pp. 398–412). Springer, Berlin, Heidelberg.

  • Van Solingen, R., Basili, V., Caldiera, G., & Rombach, H. D. (2002). Goal question metric (GQM) approach. Encyclopedia of Software Engineering.

  • Wagner, S., Goeb, A., Heinemann, L., Kläs, M., Lampasona, C., Lochmann, K., Mayr, A., Plösch, R., Seidl, A., Streit, J., & Trendowicz, A. (2015). Operationalised product quality models and assessment: The Quamoco approach. Information and Software Technology, 62, 101–123.

    Article  Google Scholar 

  • Wagner, S., Lochmann, K., Heinemann, L., Kläs, M., Trendowicz, A., Plösch, R., & Streit, J. (2012). The Quamoco product quality modelling and assessment approach. 2012 34th International Conference on Software Engineering (ICSE), (pp. 1133–1142). IEEE.

  • Wagner, S. (2008). Cost-optimisation of analytical software quality assurance: models, data, case studies. VDM Verlag.

  • Wang, X. F., Wang, J. Q., & Deng, S. Y. (2013). A method to dynamic stochastic multi-criteria decision making with log-normally distributed random variables. The Scientific World Journal.

  • Wang, Y., Xu, L., & Solangi, Y. A. (2020). Strategic renewable energy resources selection for Pakistan: Based on SWOT-Fuzzy AHP approach. Sustainable Cities and Society, 52, 101861.

    Article  Google Scholar 

  • Wasserman, M. P., Chan, C. (2006). Business Readiness Rating Project, BRR Whitepaper RFC 1, tinyurl.com/y5srd5sq.

  • Wasserman, A. I., Guo, X., McMillian, B., Qian, K., Wei, M. Y., & Xu, Q. (2017). OSSpal: finding and evaluating open source software. IFIP International Conference on Open Source Systems. Springer, Cham.

  • Yılmaz, N., & Tarhan, A.K. (2022). Quality evaluation models or frameworks for open source software: a systematic literature review. Journal of Software: Evolution and Process, 34(6):e2458. https://doi.org/10.1002/smr.2458.

  • Yılmaz, N., & Tarhan, A. K. (2020). Meta-models for software quality and its evaluation: a systematic literature review. International Workshop on Software Measurement and the 15th International Conference on Software Process and Product Measurement, Mexico.

  • Yilmaz, N., & Kolukısa Tarhan, A. (2022). Definition of the terminologies in the SQMM. Zenodo. https://doi.org/10.5281/zenodo.6367596

  • Zahoor, A., Mehboob, K., & Natha, S. (2017). Comparison of open source maturity models. Procedia Computer Science, 111, 348–354.

Download references

Acknowledgements

This study was carried out as part of a PhD study pursued by the first author at the Graduate School of Science and Engineering of Hacettepe University.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nebi Yılmaz.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1 List of questions to obtain feedback from experts

 

Feedback on the applicability of OSS-QMM in practice

Purpose of question

Q1

Suppose that you would match the terms of the quality model used in your own company for software evaluation, with the terms of the OSS-QMM. Which of the meta-model terms would you match?

It is asked to see the usefulness of the concepts in our meta-model and find out if there are any unused concepts

Q2

As a result of the matching you performed in the previous question (Q1), are there any unused concept of our OSS-QMM?

It is asked to see if there is a missing concept in our meta-model and get feedback from experts in this context

Q3

Do you agree that the terms of the example OSS quality model given in Appendix 2 (derived from OSS-QMM) and the terms of the OSS-QMM are compatible with each other?

Experts are asked to derive an OSS quality model using our meta-model. Then, it is asked to get feedback on the compatibility of the model they derived with the model we derived (Appendix 2)

 

Feedback on the structure of OSS-QMM

 

Q1

Do you agree that the mapping process is compatible with the (5-level) structure of the OSS-QMM?

It is asked to get feedback on the compatibility of the matching process with the structure of the meta-model

Q2

Do you agree that the classification of the OSS-QMM terms (under specification, measurement, and evaluation) are useful?

It was asked to get feedback on whether the classification process contributes to a better understanding of our meta-model and the coexistence of compatible concepts

Q3

Do you agree that the OSS quality models to be derived from the OSS-QMM will have homogeneous structure?

In order to benefit from the software quality modeling experiences of the experts, they are asked for their opinions on whether the structure of the models to be developed from our meta-model would be homogeneous

Q4

Do you agree that the OSS-QMM is understandable?

Experts are asked whether the developed meta-model was understandable and their feedback to make it more understandable

 

Feedback on the content of OSS-QMM

 

Q1

Do you agree that the OSS-QMM is sufficiently general to describe any existing OSS quality model that you already know? (e.g., OSMM, SQO-OSS, and QualOSS)?

Experts are asked to derive an existing OSS quality model they know using our meta model. Then, it is asked whether there were deficiencies in our model

Q2

Do you agree that that relationship between concepts is compatible?

It is asked to get feedback on whether the relationships used between concepts were compatible with the concepts and the structure of our meta-model

Q3

Do you agree that the OSS-QMM is complete?

Considering the structure of the meta-model, its concepts, and the relationships among them, it was asked to get feedback on whether quality models derived from our meta-model address all aspects of OSS products (e.g., code-based and community-based aspects)

Appendix 2 The new operationalized quality model derived from OSS-QMM

figure c

Appendix 3 Matching concepts of OSS-QMM and existing OSS quality models (OSMM, OpenBRR, and SQO-OSS)

OSS-QMM concepts

Quality models terms

Quality model

OSMM

OpenBRR

SQO-OSS

Viewpoint

Developer

Developer

Developer

OSS aspect

Community-based

Code-based

Community-based

Code-based

Community-based

Information need

Calculation of developer size to evaluate maintainability

Calculation of fault proneness to evaluate maintainability

Calculation of developer productivity to evaluate maintainability

Calculation of comment frequency to evaluate maintainability

Calculation of documentation quality to evaluate maintainability

Characteristic

Maintainability

Maintainability

Maintainability

Maintainability

Maintainability

Sub-characteristic

Acceptance

Product quality

Product quality

Analyzability

Analyzability

Entity

Developer

Source code

Contributor

Source code

Contributor

Quality requirement

The large size of developer is desirable for maintainability

The low error proneness of the source code is desirable for maintainability

The productive developers are desirable for maintainability

The high comment frequency is desirable for maintainability

The large number of documents is desirable for maintainability

Impact

Positive

Negative

Positive

Positive

Positive

Measurable concepts

The size of developer

The fault proneness of source code

Productivity of contributors

Complexity of source code

Completeness of documentation

Measure

Number of developers

(Base measure)

Defect density

(Derived measure)

Number of releases

(Base measure)

Weighted method per class

(WMC)

(Base measure)

Number of documents

(Base measure)

Unit

Developer

Defects, lines

Release

Methods

Documents

Scale

Integer from zero to five

(The score (1–5) is

assigned w.r.t. rules given

in OSMM)

Integer from zero to three

(The score (1–3) is assigned w.r.t. rules given in OpenBRR)

Integer from zero to three

(The score (1–3) is assigned w.r.t. rules given in OpenBRR)

Integer from zero to infinity

Integer from zero to infinity

Measurement method

Manually

Automatically

(e.g., Understand scitool, CKJM, Intellij IDEA, etc.)

Manually

Automatically

(e.g., Understand scitool, CKJM, Intellij IDEA, etc.)

Manually

Measurement function

There is no measurement function because it is a base measure

Number of defects/LOC

There is no measurement function because it is a base measure

There is no measurement function because it is a base measure

There is no measurement function because it is a base measure

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yılmaz, N., Tarhan, A.K. Matching terms of quality models and meta-models: toward a unified meta-model of OSS quality. Software Qual J 31, 721–773 (2023). https://doi.org/10.1007/s11219-022-09603-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-022-09603-3

Keywords