Skip to main content
SpringerLink
Log in

Ontology Quality Evaluation Methodology

  • Conference paper
  • First Online:
Computational Science and Its Applications – ICCSA 2022 (ICCSA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13375))

Included in the following conference series:

Abstract

Lack of methodologies for ontology quality evaluation causes a challenge in producing good quality ontologies. Thus, we developed an iterative quality methodology to address this gap by analyzing the existing quality theories defined in ontology engineering, as well as, the theories in software engineering. Accordingly, this paper presents the developed methodology including how the other ontology quality theories get associated with it. Moreover, a use case in the agriculture domain has been demonstrated in order to provide an understanding of how the methodology can be applied in a real context. In the future, many experiments are expected to be carried out to fine-tune the methodology and to illustrate its usefulness.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Intended needs: business requirements that the intended users (i.e., persons, parties or organizations) expect from the ontology/ ontology driven information system.

  2. 2.

    Quality model consists of a set of characteristics and the relationships between them that provide the basis for specifying quality requirements and evaluating quality [49].

  3. 3.

    Decision criteria may be “numerical threshold that used to determine the level of confidence in a given results. These will often be set with respect to quality requirements and corresponding evaluation criteria” [48].

References

  1. Gal, A.: Ontology engineering. In: Liu, L., Özsu, M.T. (eds.) Encyclopedia of Database Systems, pp. 1972–1973. Springer, Boston (2009). https://doi.org/10.1007/978-1-4614-8265-9

    Chapter  Google Scholar 

  2. Gómez-Pérez, A., Fernández-López, M., Corcho, O.: Theoretical foundations of ontologies. In: Gómez-Pérez, A., Fernández-López, M., Corcho, O. (eds.) Ontological Engineering: With Examples from the Areas of Knowledge Management, e-Commerce and the Semantic Web, pp. 1–45. Springer, London (2004). https://doi.org/10.1007/1-85233-840-7_1

    Chapter  Google Scholar 

  3. Uschold, M., Gruninger, M.: Ontologies: principles, methods and applications. Knowl. Eng. Rev. 11, 93–136 (1996)

    Article  Google Scholar 

  4. Gruninger, M., Fox, M.S.: Methodology for design and evaluation of ontologies. In: Workshop on Basic Ontological Issues in Knowlege Sharing (1995)

    Google Scholar 

  5. Neuhaus, F., et al.: Towards ontology evaluation across the life cycle: the ontology summit 2013. Appl. Ontol. 8, 179–194 (2013)

    Article  Google Scholar 

  6. Ontology Summit 2013. http://ontolog.cim3.net/OntologySummit/2013/. Accessed 26 June 2021

  7. Brank, J., Grobelnik, M., Mladeni, D.: A survey of ontology evaluation techniques. In: Proceedings of the Conference on Data Mining and Data Warehouses, pp. 166–170 (2005)

    Google Scholar 

  8. Gómez-Pérez, A.: Towards a framework to verify knowledge sharing technology. Expert Syst. Appl. 11, 519–529 (1996)

    Article  Google Scholar 

  9. Gruber, T.R.: Toward principles for the design of ontologies used for knowledge sharing. Int. J. Hum. Comput. Stud. 43, 907–928 (1993)

    Article  Google Scholar 

  10. Yu, J., Thom, J.A., Tam, A.: Requirements-oriented methodology for evaluating ontologies. Inf. Syst. 34(8), 766–791 (2009). https://doi.org/10.1016/j.is.2009.04.002. ISSN 0306-4379

  11. Pak, J., Zhou, L.: A framework for ontology evaluation. In: Sharman, R., Rao, H.R., Raghu, T.S. (eds.) WEB 2009. LNBIP, vol. 52, pp. 10–18. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17449-0_2

    Chapter  Google Scholar 

  12. Lantow, B.: OntoMetrics: putting metrics into use for ontology evaluation. In: KEOD, pp. 186–191 (2016)

    Google Scholar 

  13. Corcho, O., Fernández-López, M., Gómez-Pérez, A.: Methodologies, tools and languages for building ontologies. Where is their meeting point? Data Knowl. Eng. 46, 41–64 (2003)

    Google Scholar 

  14. Aruna, T., Saranya, K., Bhandari, C.: A survey on ontology evaluation Tools. In: 2011 International Conference on Process Automation, Control and Computing, pp. 1–5 (2011)

    Google Scholar 

  15. Wilson, S.I., Goonetillake, J.S., Ginige, A., Walisadeera, A.I.: Towards a usable ontology: the identification of quality characteristics for an ontology-driven decision support system. IEEE Access 10, 12889–12912 (2022)

    Article  Google Scholar 

  16. Rashid, M., Torchiano, M., Rizzo, G., Mihindukulasooriya, N., Corcho, O.: A quality assessment approach for evolving knowledge bases. Semant. Web 10, 349–383 (2019)

    Article  Google Scholar 

  17. Guarino, N., Welty, C.A.: An overview of OntoClean. In: Staab, S., Studer, R. (eds.) Handbook on Ontologies, pp. 151–171. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24750-0_8

    Chapter  Google Scholar 

  18. Rico, M., Caliusco, M.L., Chiotti, O., Galli, M.R.: OntoQualitas: a framework for ontology quality assessment in information interchanges between heterogeneous systems. Comput. Ind. 65, 1291–1300 (2014)

    Article  Google Scholar 

  19. Sánchez, D., Batet, M., Martínez, S., Domingo-Ferrer, J.: Semantic variance: an intuitive measure for ontology accuracy evaluation. Eng. Appl. Artif. Intell. 39, 89–99 (2015)

    Article  Google Scholar 

  20. Fernández, M., Overbeeke, C., Sabou, M., Motta, E.: What makes a good ontology? A case-study in fine-grained knowledge reuse. In: Gómez-Pérez, A., Yu, Y., Ding, Y. (eds.) ASWC 2009. LNCS, vol. 5926, pp. 61–75. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-10871-6_5

    Chapter  Google Scholar 

  21. Tartir, S., Arpinar, I.B., Moore, M., Sheth, A.P., Aleman-Meza, B.: OntoQA: metric-based ontology quality analysis. In: The IEEE Workshop on Knowledge Acquisition from Distributed, Autonomous, Semantically Heterogeneous Data and Knowledge Sources (2005)

    Google Scholar 

  22. Lantow, B.: OntoMetrics: application of on-line ontology metric calculation. In: BIR Workshops, pp. 1–12 (2016)

    Google Scholar 

  23. Gangemi, A., Catenacci, C., Ciaramita, M., Lehmann, J.: Modelling ontology evaluation and validation. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 140–154. Springer, Heidelberg (2006). https://doi.org/10.1007/11762256_13

    Chapter  Google Scholar 

  24. Wilson, R.S.I., Goonetillake, J.S., Indika, W.A., Ginige, A.: Analysis of ontology quality dimensions, criteria and metrics. In: Gervasi, O., et al. (eds.) ICCSA 2021. LNCS, vol. 12951, pp. 320–337. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86970-0_23

    Chapter  Google Scholar 

  25. Wilson, R.S.I.: A Conceptual Model for Ontology Quality Assessment. http://semantic-web-journal.net/system/files/swj3003.pdf. Accessed 04 Apr 2022

  26. Burton-Jones, A., Storey, V.C., Sugumaran, V., Ahluwalia, P.: A semiotic metrics suite for assessing the quality of ontologies. Data Knowl. Eng. 55, 84–102 (2005)

    Article  Google Scholar 

  27. Duque-Ramos, A., Fernández-Breis, J.T., Stevens, R., Aussenac-Gilles, N.: OQuaRE: a SQuaRE-based approach for evaluating the quality of ontologies. J. Res. Pract. Inf. Technol. 43, 159–176 (2011)

    Google Scholar 

  28. Zhu, H., Liu, D., Bayley, I., Aldea, A., Yang, Y., Chen, Y.: Quality model and metrics of ontology for semantic descriptions of web services. Tsinghua Sci. Technol. 22, 254–272 (2017)

    Article  Google Scholar 

  29. McDaniel, M., Storey, V.C.: Evaluating domain ontologies: clarification, classification, and challenges. ACM Comput. Surv. 52, 1–44 (2019)

    Article  Google Scholar 

  30. Raad, J., Cruz, C.: A survey on ontology evaluation methods. In: Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, pp. 179–186. SCITEPRESS - Science and and Technology Publications, Lisbon (2015)

    Google Scholar 

  31. Degbelo, A.: A snapshot of ontology evaluation criteria and strategies. In: Proceedings of the 13th International Conference on Semantic Systems, pp. 1–8. Association for Computing Machinery, New York (2017)

    Google Scholar 

  32. W3C RDF Validation Service. https://www.w3.org/RDF/Validator/. Accessed 29 Dec 2021

  33. Horridge, M.: OWL 2 Validator. http://mowl-power.cs.man.ac.uk:8080/validator/. Accessed 29 Dec 2021

  34. Sure, Y., Erdmann, M., Angele, J., Staab, S., Studer, R., Wenke, D.: OntoEdit: collaborative ontology development for the semantic web. In: Horrocks, I., Hendler, J. (eds.) ISWC 2002. LNCS, vol. 2342, pp. 221–235. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-48005-6_18

    Chapter  MATH  Google Scholar 

  35. Dividino, R., Romanelli, M., Sonntag, D.: Semiotic-based ontology evaluation tool S-OntoEval. In: Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC 2008) (2008)

    Google Scholar 

  36. García-Ramos, S., Otero, A., Fernández-López, M.: OntologyTest: a tool to evaluate ontologies through tests defined by the user. In: Omatu, S., et al. (eds.) IWANN 2009. LNCS, vol. 5518, pp. 91–98. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02481-8_13

    Chapter  Google Scholar 

  37. Schober, D., Tudose, I., Svatek, V., et al.: OntoCheck: verifying ontology naming conventions and metadata completeness in Protégé 4. J. Biomed. Semant. 3, S4 (2012). https://doi.org/10.1186/2041-1480-3-S2-S4

  38. Daga, E.: XDTools - NeOn Wiki. http://neon-toolkit.org/wiki/XDTools.html. Accessed 29 Dec 2021

  39. Lambrix, P., Liu, Q.: Debugging the missing is-a structure within taxonomies networked by partial reference alignments. Data Knowl. Eng. 86, 179–205 (2013)

    Article  Google Scholar 

  40. Poveda-Villalón, M., Gómez-Pérez, A., Suárez-Figueroa, M.C.: Oops! (ontology pitfall scanner!): an on-line tool for ontology evaluation. Int. J. Semant. Web. Inf. Syst. IJSWIS. 10, 7–34 (2014)

    Article  Google Scholar 

  41. Lozano-Tello, A., Gomez-Perez, A.: ONTOMETRIC: a method to choose the appropriate ontology. J. Database Manag. JDM 15, 1–18 (2004)

    Article  Google Scholar 

  42. Schekotihin, K., Rodler, P., Schmid, W.: OntoDebug: interactive ontology debugging plug-in for Protégé. In: Ferrarotti, F., Woltran, S. (eds.) FoIKS 2018. LNCS, vol. 10833, pp. 340–359. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-90050-6_19

    Chapter  Google Scholar 

  43. McDaniel, M., Storey, V.C., Sugumaran, V.: Assessing the quality of domain ontologies: metrics and an automated ranking system. Data Knowl. Eng. 115, 32–47 (2018)

    Article  Google Scholar 

  44. Amith, M., et al.: OntoKeeper: semiotic-driven ontology evaluation tool for biomedical ontologists. In: 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1614–1617 (2018)

    Google Scholar 

  45. Kondylakis, H., et al.: Delta: a modular ontology evaluation system. Information 12, 301 (2021)

    Article  Google Scholar 

  46. van Solingen, R., Basili, V., Caldiera, G., Rombach, H.D.: Goal question metric (GQM) approach. In: Encyclopedia of Software Engineering. American Cancer Society (2002)

    Google Scholar 

  47. Carriero, V.A., Gangemi, A., Mancinelli, M.L., Nuzzolese, A.G., Presutti, V., Veninata, C.: Pattern-based design applied to cultural heritage knowledge graphs. Semantic Web. 12, 313–357 (2021)

    Article  Google Scholar 

  48. ISO/IEC 25040:2011(en), Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Evaluation process. https://www.iso.org/obp/ui/#iso:std:iso-iec:25040:ed-1:v1:en. Accessed 03 Jan 2022

  49. ISO/IEC 25010:2011(en), Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models. https://www.iso.org/obp/ui/#iso:std:iso-iec:25010:ed-1:v1:en. Accessed 26 June 2021

  50. Walisadeera, A.I., Ginige, A., Wikramanayake, G.N.: User centered ontology for Sri Lankan farmers. Ecol. Inform. 26, 140–150 (2015). https://doi.org/10.1016/j.ecoinf.2014.07.008

    Article  Google Scholar 

  51. Protégé. https://protege.stanford.edu/. Accessed 14 Nov 2021

  52. Vrandečić, D., Gangemi, A.: Unit tests for ontologies. In: Meersman, R., Tari, Z., Herrero, P. (eds.) OTM 2006. LNCS, vol. 4278, pp. 1012–1020. Springer, Heidelberg (2006). https://doi.org/10.1007/11915072_2

    Chapter  Google Scholar 

  53. Fernández-Izquierdo, A., García-Castro, R.: How to validate ontologies with themis. In: Hitzler, P., et al. (eds.) ESWC 2019. LNCS, vol. 11762, pp. 52–57. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32327-1_11

    Chapter  Google Scholar 

  54. List of Reasoners | OWL research at the University of Manchester. http://owl.cs.manchester.ac.uk/tools/list-of-reasoners/. Accessed 14 Nov 2021

Download references

Acknowledgement

The author acknowledges the support received from the LK Domain Registry (https://www.domains.lk/index.php/outreach/research-grants) in publishing this paper.

Author information

Authors and Affiliations

  1. Uva Wellassa University, Badulla, Sri Lanka

    R. Shyama I. Wilson

  2. University of Colombo School of Computing, Colombo 07, Sri Lanka

    R. Shyama I. Wilson & Jeevani S. Goonetillake

  3. Western Sydney University, Penrith, NSW, Australia

    Athula Ginige

  4. University of Ruhuna, Matara, Sri Lanka

    Walisadeera Anusha Indika

Authors
  1. R. Shyama I. Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeevani S. Goonetillake
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Athula Ginige
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Walisadeera Anusha Indika
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. Shyama I. Wilson .

Editor information

Editors and Affiliations

  1. University of Perugia, Perugia, Italy

    Osvaldo Gervasi

  2. University of Basilicata, Potenza, Potenza, Italy

    Beniamino Murgante

  3. Universidad de Málaga, Malaga, Spain

    Eligius M. T. Hendrix

  4. Monash University, Clayton, VIC, Australia

    David Taniar

  5. Kyushu Sangyo University, Fukuoka, Japan

    Bernady O. Apduhan

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wilson, R.S.I., Goonetillake, J.S., Ginige, A., Indika, W.A. (2022). Ontology Quality Evaluation Methodology. In: Gervasi, O., Murgante, B., Hendrix, E.M.T., Taniar, D., Apduhan, B.O. (eds) Computational Science and Its Applications – ICCSA 2022. ICCSA 2022. Lecture Notes in Computer Science, vol 13375. Springer, Cham. https://doi.org/10.1007/978-3-031-10522-7_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-10522-7_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-10521-0

  • Online ISBN: 978-3-031-10522-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation