Abstract
When ontologies reach a certain size and complexity, faults such as inconsistencies or wrong entailments are hardly avoidable. Locating the faulty axioms that cause these faults is a hard and time-consuming task. Addressing this issue, several techniques for semi-automatic fault localization in ontologies have been proposed. Often, these approaches involve a human expert who provides answers to system-generated questions about the intended (correct) ontology in order to reduce the possible fault locations. To suggest as few and as informative questions as possible, existing methods draw on various algorithmic optimizations as well as heuristics. However, these computations are often based on certain assumptions about the interacting user and the metric to be optimized.
In this work, we critically discuss these optimization criteria and suppositions about the user. As a result, we suggest an alternative, arguably more realistic metric to measure the expert’s effort and show that existing approaches do not achieve optimal efficiency in terms of this metric. Moreover, we detect that significant differences regarding user interaction costs arise if the assumptions made by existing works do not hold. As a remedy, we suggest a new notion of expert question that does not rely on any assumptions about the user’s way of answering. Experiments on faulty real-world ontologies testify that the new querying method minimizes the necessary expert consultations in the majority of cases and reduces the computation time for the best next question by at least 80 % in all scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
All information about OntoDebug can be found at http://isbi.aau.at/ontodebug/.
- 2.
An ontology \(\mathcal {O}\) is coherent iff there do not exist any unsatisfiable classes in \(\mathcal {O}\). A class C is unsatisfiable in \(\mathcal {O}\) iff \(\mathcal {O}\models C \sqsubseteq \bot \). See also [13, Def. 1 and 2].
- 3.
Throughout the presented examples, we use Description Logic notation. For details, see [1].
- 4.
Note, the finally remaining diagnosis does not necessarily contain all faulty axioms in the ontology, as, e.g., some existing faults in the ontology might not yet have surfaced in terms of problems such as wrong entailments or unsatisfiable classes. However, the (faultiness of the) axioms in the final diagnosis do(es) explain all observed problems in the ontology.
- 5.
Note, a positive answer (y) implicitly provides axiom-level information, i.e., the positive classification of all query-axioms. Thus, the discussed experts differ only in their negation behavior.
- 6.
- 7.
Such (singleton) queries consisting of only axioms explicitly included in the ontology are called explicit (singleton) queries [17].
- 8.
The logical expressivity refers to the power of the logical language used in the ontology in terms of how much can be expressed using this language. In general, the higher the expressivity, the higher the cost of reasoning (and thus the cost of computing queries) with the respective logic tends to be. See [1] for more details on the logical expressivity of ontologies.
- 9.
This is owed to the fact that the efficient generation of optimal singleton queries including “implicit” axioms, i.e., where \(Q \not \subseteq \mathcal {O}\) holds, is still an open research topic (cf. Sect. 4).
- 10.
Note, the presented figures do not expose all results. However, the observations were greatly consistent over all studied ontologies. See the extended version [19] of this paper for all plots.
References
Baader, F., Calvanese, D., McGuinness, D., Nardi, D., Patel-Schneider, P. (eds.): The Description Logic Handbook, 1st edn. Cambridge University Press, Cambridge (2003)
Beck, K.: Test-Driven Development: By Example. Addison-Wesley Professional, Boston (2003)
Ceusters, W., Smith, B., Goldberg, L.: A terminological and ontological analysis of the NCI thesaurus. Methods Inf. Med. 44(4), 498 (2005)
Felfernig, A., Friedrich, G., Jannach, D., Stumptner, M.: Consistency-based diagnosis of configuration knowledge bases. Artif. Intell. 152(2), 213–234 (2004)
Golbeck, J., Fragoso, G., Hartel, F., Hendler, J., Oberthaler, J., Parsia, B.: The national cancer institute’s thesaurus and ontology. JWS 1(1) (2003). http://dx.doi.org/10.2139/ssrn.3199007
Grau, B.C., Horrocks, I., Motik, B., Parsia, B., Patel-Schneider, P., Sattler, U.: OWL 2: the next step for OWL. JWS 6(4), 309–322 (2008)
Horridge, M., Bail, S., Parsia, B., Sattler, U.: The cognitive complexity of OWL justifications. In: Aroyo, L., et al. (eds.) ISWC 2011. LNCS, vol. 7031, pp. 241–256. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25073-6_16
Jannach, D., Schmitz, T., Shchekotykhin, K.: Parallel model-based diagnosis on multi-core computers. JAIR 55, 835–887 (2016)
Kalyanpur, A.: Debugging and repair of OWL ontologies. Ph.D. thesis, University of Maryland (2006)
de Kleer, J., Williams, B.C.: Diagnosing multiple faults. Artif. Intell. 32(1), 97–130 (1987)
Meilicke, C.: Alignment incoherence in ontology matching. Ph.D. thesis, University of Mannheim (2011)
Nikitina, N., Rudolph, S., Glimm, B.: Interactive ontology revision. JWS 12(0), 118–130 (2012)
Qi, G., Hunter, A.: Measuring incoherence in description logic-based ontologies. In: Aberer, K., et al. (eds.) ASWC/ISWC -2007. LNCS, vol. 4825, pp. 381–394. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76298-0_28
Rector, A.L., Brandt, S., Schneider, T.: Getting the foot out of the pelvis: modeling problems affecting use of SNOMED CT hierarchies in practical applications. JAMIA 18(4), 432–440 (2011)
Reiter, R.: A theory of diagnosis from first principles. Artif. Intell. 32(1), 57–95 (1987)
Rodler, P.: Interactive Debugging of Knowledge Bases. Ph.D. thesis, University of Klagenfurt (2015)
Rodler, P.: Towards better response times and higher-quality queries in interactive KB debugging. Technical report, University of Klagenfurt (2016). http://arxiv.org/abs/1609.02584v2
Rodler, P.: On active learning strategies for sequential diagnosis. In: DX, pp. 264–283 (2018)
Rodler, P., Eichholzer, M.: A new expert questioning approach to more efficient fault localization in ontologies. Technical report, University of Klagenfurt (2019). http://arxiv.org/abs/1904.00317
Rodler, P., Herold, M.: StaticHS: a variant of Reiter’s hitting set tree for efficient sequential diagnosis. In: SoCS, pp. 72–80 (2018)
Rodler, P., Jannach, D., Schekotihin, K., Fleiss, P.: Are query-based ontology debuggers really helping knowledge engineers? Knowledge-Based Systems (2019). https://doi.org/10.1016/j.knosys.2019.05.006
Rodler, P., Schekotihin, K.: Reducing model-based diagnosis to knowledge base debugging. In: DX, pp. 284–296 (2018)
Rodler, P., Schmid, W.: On the impact and proper use of heuristics in test-driven ontology debugging. In: Benzmüller, C., Ricca, F., Parent, X., Roman, D. (eds.) RuleML+RR 2018. LNCS, vol. 11092, pp. 164–184. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99906-7_11
Rodler, P., Schmid, W., Schekotihin, K.: A generally applicable, highly scalable measurement computation and optimization approach to sequential model-based diagnosis. CoRR abs/1711.05508 http://arxiv.org/abs/1711.05508 (2017)
Rodler, P., Schmid, W., Schekotihin, K.: Inexpensive cost-optimized measurement proposal for sequential model-based diagnosis. In: DX, pp. 200–218 (2018)
Rodler, P., Shchekotykhin, K., Fleiss, P., Friedrich, G.: RIO: minimizing user interaction in ontology debugging. In: Faber, W., Lembo, D. (eds.) RR 2013. LNCS, vol. 7994, pp. 153–167. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39666-3_12
Schekotihin, K., Rodler, P., Schmid, W.: OntoDebug: interactive ontology debugging plug-in for Protégé. In: Ferrarotti, F., Woltran, S. (eds.) FoIKS 2018. LNCS, vol. 10833, pp. 340–359. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-90050-6_19
Schekotihin, K., Rodler, P., Schmid, W., Horridge, M., Tudorache, T.: A Protégé plug-in for test-driven ontology development. In: ICBO (2018)
Schulz, S., Schober, D., Tudose, I., Stenzhorn, H.: The pitfalls of thesaurus ontologization-the case of the NCI thesaurus. In: AMIA Annual Symposium (2010)
Shchekotykhin, K., Friedrich, G., Fleiss, P., Rodler, P.: Interactive ontology debugging: two query strategies for efficient fault localization. JWS 12–13, 88–103 (2012)
Shchekotykhin, K., Jannach, D., Schmitz, T.: Mergexplain: fast computation of multiple conflicts for diagnosis. In: IJCAI, pp. 3221–3228 (2015)
Shchekotykhin, K.M., Friedrich, G., Rodler, P., Fleiss, P.: Sequential diagnosis of high cardinality faults in knowledge-bases by direct diagnosis generation. In: ECAI (2014)
Siddiqi, S.A., Huang, J.: Sequential diagnosis by abstraction. JAIR 41, 329–365 (2011)
Smith, B., Ashburner, M., Rosse, C., et al.: The OBO foundry: coordinated evolution of ontologies to support biomedical data integration. Nature Biotechnol. 25(11), 1251 (2007)
Tudorache, T., Noy, N.F., Tu, S., Musen, M.A.: Supporting collaborative ontology development in Protégé. In: Sheth, A., et al. (eds.) ISWC 2008. LNCS, vol. 5318, pp. 17–32. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88564-1_2
Acknowledgments
This work was in part supported by the Carinthian Science Fund (KWF), contract KWF-3520/26767/38701. Moreover, we thank Wolfgang Schmid for his technical support during the implementation of our experiments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Rodler, P., Eichholzer, M. (2019). On the Usefulness of Different Expert Question Types for Fault Localization in Ontologies. In: Wotawa, F., Friedrich, G., Pill, I., Koitz-Hristov, R., Ali, M. (eds) Advances and Trends in Artificial Intelligence. From Theory to Practice. IEA/AIE 2019. Lecture Notes in Computer Science(), vol 11606. Springer, Cham. https://doi.org/10.1007/978-3-030-22999-3_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-22999-3_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22998-6
Online ISBN: 978-3-030-22999-3
eBook Packages: Computer ScienceComputer Science (R0)