Skip to main content
SpringerLink
Log in

Analysis of Ontology Quality Dimensions, Criteria and Metrics

  • Conference paper
  • First Online:
Computational Science and Its Applications – ICCSA 2021 (ICCSA 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12951))

Included in the following conference series:

Abstract

Ontology quality assessment needs to be performed across the ontology development life cycle to ensure that the ontology being modeled meets the intended purpose. To this end, a set of quality criteria and metrics provides a basis to assess the quality with respect to the quality requirements. However, the existing criteria and metrics defined in the literature so far are messy and vague. Thus, it is difficult to determine what set of criteria and measures would be applicable to assess the quality of an ontology for the intended purpose. Moreover, there are no well-accepted methodologies for ontology quality assessment as the way it is in the software engineering discipline. Therefore, a comprehensive review was performed to identify the existing contribution on ontology quality criteria and metrics. As a result, it was identified that the existing criteria can be classified under five dimensions namely syntactic, structural, semantic, pragmatic, and social. Moreover, a matrix with ontology levels, approaches, and criteria/metrics was presented to guide the researchers when they perform a quality assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Studer, R., Benjamins, V.R., Fensel, D.: Knowledge engineering: principles and methods. Data Knowl. Eng. 25, 161–197 (1998)

    Article  Google Scholar 

  2. Natalya, F.N., Deborah, L.M.: What is an ontology and why we need it. https://protege.stanford.edu/publications/ontology_development/ontology101-noy-mcguinness.html. Accessed 09 May 2021

  3. Duque-Ramos, A., Fernández-Breis, J.T., Stevens, R., Aussenac-Gilles, N.: OQuaRE: a SQuaRE-based approach for evaluating the quality of ontologies. J. Res. Pract. Inf. Technol. 43, 159–176 (2011)

    Google Scholar 

  4. Staab, S., Gomez-Perez, A., Daelemana, W., Reinberger, M.-L., Noy, N.F.: Why evaluate ontology technologies? Because it works! IEEE Intell. Syst. 19, 74–81 (2004)

    Google Scholar 

  5. Duque-Ramos, A., et al.: Evaluation of the OQuaRE framework for ontology quality. Expert Syst. Appl. 40, 2696–2703 (2013)

    Article  Google Scholar 

  6. Gómez-Pérez, A.: Towards a framework to verify knowledge sharing technology. Expert Syst. Appl. 11, 519–529 (1996)

    Article  Google Scholar 

  7. Brank, J., Grobelnik, M., Mladenić, D.: A survey of ontology evaluation techniques. In: Proceedings of the Conference on Data Mining and Data Warehouses, pp. 166–170 (2005)

    Google Scholar 

  8. Ontology Summit 2013. http://ontolog.cim3.net/OntologySummit/2013/index.html. Accessed 09 May 2021

  9. McDaniel, M., Storey, V.C.: Evaluating domain ontologies: clarification, classification, and challenges. ACM Comput. Surv. 52, 1–44 (2019)

    Article  Google Scholar 

  10. Burton-Jones, A., Storey, V.C., Sugumaran, V., Ahluwalia, P.: A semiotic metrics suite for assessing the quality of ontologies. Data Knowl. Eng. 55, 84–102 (2005)

    Article  Google Scholar 

  11. Gangemi, A., Catenacci, C., Ciaramita, M., Lehmann, J.: Modelling ontology evaluation and validation. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 140–154. Springer, Heidelberg (2006). https://doi.org/10.1007/11762256_13

    Chapter  Google Scholar 

  12. Gangemi, A., et al.: Ontology evaluation and validation: an integrated formal model for the quality diagnostic task, 53 (2005)

    Google Scholar 

  13. Lantow, B.: OntoMetrics: putting metrics into use for ontology evaluation. In: KEOD, pp. 186–191 (2016)

    Google Scholar 

  14. Tartir, S., Arpinar, I.B., Moore, M., Sheth, A.P., Aleman-Meza, B.: OntoQA: metric-based ontology quality analysis (2005)

    Google Scholar 

  15. ISO/IEC 25000:2005. https://www.iso.org/standard/35683.html. Accessed 09 May 2021

  16. Degbelo, A.: A snapshot of ontology evaluation criteria and strategies. In: Proceedings of the 13th International Conference on Semantic Systems, pp. 1–8. Association for Computing Machinery, New York (2017)

    Google Scholar 

  17. Gómez-Pérez, A.: Ontology Evaluation. In: Staab, S., Studer, R. (eds.) Handbook on Ontologies. International Handbooks on Information Systems, pp. 251–273. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24750-0_13

    Chapter  Google Scholar 

  18. Vrandečić, D.: Ontology evaluation. In: Staab, S., Studer, R. (eds.) Handbook on ontologies. IHIS, pp. 293–313. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-540-92673-3_13

    Chapter  Google Scholar 

  19. Reynoso, L., Amaolo, M., Vaucheret, C., Álvarez, M.: Survey of measures for the structural dimension of ontologies. In: 2013 IEEE 12th International Conference on Cognitive Informatics and Cognitive Computing, pp. 83–92 (2013)

    Google Scholar 

  20. Mc Gurk, S., Abela, C., Debattista, J.: Towards ontology quality assessment. In: MEPDaW/LDQ@ ESWC, pp. 94–106 (2017)

    Google Scholar 

  21. Pak, J., Zhou, L.: A framework for ontology evaluation. In: Sharman, R., Rao, H.R., Raghu, T. S. (eds.) WEB 2009. LNBIP, vol. 52, pp. 10–18. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17449-0_2

    Chapter  Google Scholar 

  22. Hooi, Y.K., Hassan, M.F., Shariff, A.M.: Ontology evaluation: a criteria selection framework. In: 2015 International Symposium on Mathematical Sciences and Computing Research (iSMSC), pp. 298–303 (2015)

    Google Scholar 

  23. Raad, J., Cruz, C.: A survey on ontology evaluation methods: In: Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, pp. 179–186. SCITEPRESS - Science and and Technology Publications, Lisbon (2015)

    Google Scholar 

  24. Verma, A.: An abstract framework for ontology evaluation. In: 2016 International Conference on Data Science and Engineering (ICDSE), pp. 1–6 (2016)

    Google Scholar 

  25. Reyes-Pena, C., Tovar-Vidal, M.: Ontology: components and evaluation, a review. Res. Comput. Sci. 148, 257–265 (2019)

    Article  Google Scholar 

  26. Kitchenham, B.: Procedures for undertaking systematic reviews. Joint Technical report, Computer Science Department, Keele University (TR/SE-0401) and National ICT Australia Ltd. (0400011T.1) (2004)

    Google Scholar 

  27. Gavrilova, T.A., Gorovoy, V.A., Bolotnikova, E.S.: Evaluation of the cognitive ergonomics of ontologies on the basis of graph analysis. Sci. Tech. Inf. Process. 37, 398–406 (2010)

    Article  Google Scholar 

  28. Demaidi, M.N., Gaber, M.M.: TONE: a method for terminological ontology evaluation. In: Proceedings of the ArabWIC 6th Annual International Conference Research Track, pp. 1–10. Association for Computing Machinery, New York (2019)

    Google Scholar 

  29. Cross, V., Pal, A.: Metrics for ontologies. In: NAFIPS 2005 - 2005 Annual Meeting of the North American Fuzzy Information Processing Society, pp. 448–453 (2005)

    Google Scholar 

  30. Zhu, H., Liu, D., Bayley, I., Aldea, A., Yang, Y., Chen, Y.: Quality model and metrics of ontology for semantic descriptions of web services. Tsinghua Sci. Technol. 22, 254–272 (2017)

    Article  Google Scholar 

  31. Evermann, J., Fang, J.: Evaluating ontologies: towards a cognitive measure of quality. Inf. Syst. 35, 391–403 (2010)

    Article  Google Scholar 

  32. Ma, Y., Jin, B., Feng, Y.: Semantic oriented ontology cohesion metrics for ontology-based systems. J. Syst. Softw. 83, 143–152 (2010)

    Article  Google Scholar 

  33. Rico, M., Caliusco, M.L., Chiotti, O., Galli, M.R.: OntoQualitas: a framework for ontology quality assessment in information interchanges between heterogeneous systems. Comput. Ind. 65, 1291–1300 (2014)

    Article  Google Scholar 

  34. McDaniel, M., Storey, V.C., Sugumaran, V.: Assessing the quality of domain ontologies: metrics and an automated ranking system. Data Knowl. Eng. 115, 32–47 (2018)

    Article  Google Scholar 

  35. Oh, S., Yeom, H.Y., Ahn, J.: Cohesion and coupling metrics for ontology modules. Inf. Technol. Manag. 12, 81 (2011)

    Article  Google Scholar 

  36. IEEE Standard for a Software Quality Metrics Methodology: In IEEE Std 1061-1992, pp. 1–96 (1993)

    Google Scholar 

  37. Kitchenham, B., Pfleeger, S.L., Fenton, N.: Towards a framework for software measurement validation. IEEE Trans. Softw. Eng. 21, 929–944 (1995)

    Article  Google Scholar 

  38. Basili, V.R., Caldiera, G., Rombach, H.D.: The goal question metric approach. Encyclopedia Softw. Eng. 2, 528–532 (1994)

    Google Scholar 

  39. Porzel, R., Malaka, R.: A task-based approach for ontology evaluation. In: Proceedings of the Workshop on Ontology Learning and Population at the 16th European Conference on Artificial Intelligence (ECAI), (2004)

    Google Scholar 

  40. OQuaRE: A SQUaRE based quality evaluation framework for Ontologies. http://miuras.inf.um.es/evaluation/oquare/. Accessed 09 May 2021

  41. Wang, R.Y., Strong, D.M.: Beyond accuracy: what data quality means to data consumers. J. Manag. Inf. Syst. 12, 5–33 (1996)

    Article  Google Scholar 

  42. Mouzhi Ge., Markus, H.: A review of information quality research. In. Proceedings of the 2007 International Conference on Information Quality, pp. 76–91 (2007)

    Google Scholar 

  43. Uschold, M., Gruninger, M.: Ontologies: principles, methods and applications. Knowl. Eng. Rev. 11, 93–136 (1996)

    Article  Google Scholar 

  44. Nigel, B.: Human-computer interaction standards. In: Anzai, Y., Ogawa, K.(eds.) Proceedings of the 6th International Conference on Human Computer Interaction, vol. 20, pp. 885–890. Elsevier, Amsterdam (1995)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Uva Wellassa University, Badulla, Sri Lanka

    R. S. I. Wilson

  2. University of Colombo School of Computing, Colombo 07, Sri Lanka

    R. S. I. Wilson & J. S. Goonetillake

  3. University of Ruhuna, Matara, Sri Lanka

    W. A. Indika

  4. Western Sydney University, Penrith, NSW, Australia

    Athula Ginige

Authors
  1. R. S. I. Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. S. Goonetillake
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. W. A. Indika
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Athula Ginige
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. S. I. Wilson .

Editor information

Editors and Affiliations

  1. University of Perugia, Perugia, Italy

    Osvaldo Gervasi

  2. University of Basilicata, Potenza, Potenza, Italy

    Beniamino Murgante

  3. Covenant University, Ota, Nigeria

    Sanjay Misra

  4. University of Cagliari, Cagliari, Italy

    Chiara Garau

  5. University of Cagliari, Cagliari, Italy

    Ivan Blečić

  6. Monash University, Clayton, VIC, Australia

    David Taniar

  7. Kyushu Sangyo University, Fukuoka, Japan

    Bernady O. Apduhan

  8. University of Minho, Braga, Portugal

    Ana Maria A. C. Rocha

  9. Polytechnic University of Bari, Bari, Italy

    Eufemia Tarantino

  10. Polytechnic University of Bari, Bari, Italy

    Carmelo Maria Torre

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wilson, R.S.I., Goonetillake, J.S., Indika, W.A., Ginige, A. (2021). Analysis of Ontology Quality Dimensions, Criteria and Metrics. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2021. ICCSA 2021. Lecture Notes in Computer Science(), vol 12951. Springer, Cham. https://doi.org/10.1007/978-3-030-86970-0_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86970-0_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86969-4

  • Online ISBN: 978-3-030-86970-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation