Abstract
We propose an ontology-based approach to interpret hand-drawn sketches, originating from empirical results of experiments with human participants. The approach combines qualitative features of the sequence of sketch strokes with a high-level knowledge, and accentuates the potential effectiveness of interpretation via description logics. The results of an implementation, along with explanations, are presented to show how to extract the semantics of hand-drawn sketches of four object categories.


Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Also, to handwrite or give a preliminary design.
‘Saliency’ here means that the constituent is a required part.
For example, a sketch that is found to consist of exactly one Wheel part, one Body part, and a couple of Window parts will not be recognized as Bus because a bus must have “at least two wheels”.
It is worth reminding the reader here to compare the OWA effect with the ‘third finding’ discussed in Sect. 2, where human reasoning seems not to agree with the OWA.
In fact, more than \(75\%\) of the human participants in the “Sketch Recognition” experiment (cf. Sect. 2) were able to identify a Bus object from a sketch having only those parts.
References
Andrew A, Karlson AK, Brush AB (2009) Investigating the use of voice and ink for mobile micronote capture. INTERACT 2009. Springer, Berlin
Liu Y, Zhang D, Lu G, Ma W-Y (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282
Berners-Lee T, Hendler J, Lassila O (2001) The semantic web. Sci Am 284(5):34–43
Antoniou G, Harmelen Fv (2009) Web ontology language: OWL. Springer, Berlin, pp 91–110
Baader F, Horrocks I, Lutz C, Sattler U (2017) An introduction to description logic. Cambridge University Press, Cambridge
Wang S, Wang G, Gao M, Yu S (2013) Using fuzzy hybrid features to classify strokes in interactive sketches. Adv Mech Eng 5:259152
Davis R (2006) Sketch understanding: toward natural interaction. In: ACM SIGGRAPH 2006 courses, SIGGRAPH ’06 (New York, NY, USA), ACM
Forbus K, Lockwood K, Klenk M, Tomai E, Usher J (2004) Open-domain sketch understanding: the Nusketch approach. In: AAAI fall symposium on making pen-based interaction intelligent and natural, 12, pp 58–63
Forbus K, Usher J, Lovett A, Lockwood K, Wetzel J (2008) CogSketch: open-domain sketch understanding for cognitive science research and for education. In: Alvarado C, Cani M-P (eds) Eurographics workshop on sketch-based interfaces and modeling. The Eurographics Association
Hammond T, Davis R (2005) Ladder, a sketching language for user interface developers. Comput Graph 29(4):518–532
Paulson B, Hammond T (2008) Paleosketch: accurate primitive sketch recognition and beautification. In: Proceedings of 13th international conference on intelligent user interfaces, IUI ’08 (New York, NY, USA), ACM, pp 1–10
McLure M, Friedman S, Lovett A, Forbus K (2011) Edge-cycles: a qualitative sketch representation to support recognition. In: Proceedings of 25th international workshop on QR, pp 108–115
Abdelfattah A, Zakaria W, Abdelghaffar N, Abdelmoneim N, Schneider S, Kühnberger K-U (2016) A preliminary assessment of conceptual salience for automatic sketching. In: Lieto A, Bhatt M, Oltramari A, Vernon D (eds) Proceedings of 4th international workshop on AI and Cog, vol 1895, CEUR-WS, pp 81–92
Abdelfattah A, Zakaria W, Abdelghaffar N, Abdelmoneim N, Elsaadany N, Schwering A, Kühnberger K-U (2015) Towards the modeling of sketch-based understanding: can computers interpret what non-artists draw?. In SHAPES 3.0—the shape of things, Larnaca, Cyprus, 2–6 November 2015
Abdelfattah A, Zakaria W (2017) Employing a restricted set of qualitative relations in recognizing plain sketches. In: Kern-Isberner G, Fürnkranz J, Thimm M (eds) KI 2017: advances in AI, vol 10505 of LNAI, pp 3–14
Neumann B, Möller R (2008) On scene interpretation with description logics. Image Vis Comput 26(1):82–101
Bannour H, Hudelot C (2011) Towards ontologies for image interpretation and annotation. In: 9th international workshop on CBMI, pp 211–216
Donadello I, Serafini L (2015) Mixing low-level and semantic features for image interpretation. In: Agapito L, Bronstein MM, Rother C (eds) Computer vision—ECCV 2014 workshops. Springer, Cham, pp 283–298
Zhang X, Ai T, Stoter JE (2016) Inferring complex geographical concepts with implicit geometries using ontologies: a case of peninsulas. In: Zhang X (ed) AGILE conference. Lecture notes in geoinformation and cartography. Springer, Berlin, pp 21–36
Chen J, Cohn AG, Liu D, Wang S, Ouyang J, Yu Q (2015) A survey of qualitative spatial representations. Knowl Eng Rev 30(1):106–136
Frank AU (1996) Qualitative spatial reasoning: cardinal directions as an example. Geogr Inf Syst 10(3):269–290
Randell DA, Cui Z, Cohn AG (1992) A spatial logic based on regions and connection. In: Proceedings of 3rd international conference on KRR
Batsakis S, Petrakis EGM (2011) Sowl: a framework for handling spatio-temporal information in owl 2.0. In: Bassiliades N, Governatori G, Paschke A (eds) Rule-based reasoning, programming, and applications (Berlin, Heidelberg). Springer, Berlin, pp 242–249
Batsakis S, Antoniou G, Tachmazidis I (2014) Integrated representation of spatial topological and size relations for the semantic web. In: Murty MN, He X, Chillarige RR, Weng P (eds) Multi-disciplinary trends in artificial intelligence. Springer, Berlin, pp 208–219
Hudelot C, Atif J, Bloch I (2014) Alc(f): A new description logic for spatial reasoning in images. In: Computer vision—ECCV 2014 workshops-Zurich, Switzerland, September 6–7 and 12, 2014, Proceedings, Part II, pp 370–384
Sirin E, Parsia B, Grau BC, Kalyanpur A, Katz Y (2007) Pellet: a practical owl-dl reasoner. Web Semant 5(2):51–53
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Abdelghaffar, N.M., Abdelfattah, A.M.H., Taha, A.A. et al. Accentuating Features of Description Logics in High-Level Interpretations of Hand-Drawn Sketches. Künstl Intell 33, 253–265 (2019). https://doi.org/10.1007/s13218-019-00602-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13218-019-00602-4