Elsevier

Pattern Recognition Letters

Volume 28, Issue 6, 15 April 2007, Pages 710-718
Pattern Recognition Letters

Identification of drawing tools by classification of textural and boundary features of strokes

https://doi.org/10.1016/j.patrec.2006.08.003Get rights and content

Abstract

Recent developments in computer vision provide powerful tools for the examination and classification of data of our cultural heritage. It is generally recognized that the cultural heritage we are preserving for future generations will profit considerably from passing over to state of the art technologies. New camera hardware allows new insights into cultural heritage, especially if infrared cameras are concerned, since they allow the study of structures that are visually hidden. In this paper a strategy for the analysis of underdrawing strokes in infrared reflectograms is presented. Underdrawings are the basic concept or “primal sketch” of the artist before the complete painting is created. We focus on infrared reflectograms of medieval panel paintings, since their underdrawings are common and help art historians to study the school of the old masters. The purpose of the stroke analysis is the determination of the drawing tool used to draft the painting. This information allows significant support for a systematic stylistic approach in the analysis of paintings. Stroke segmentation in paintings is related to the extraction and recognition of handwriting, therefore similar techniques to segment the strokes from the background incorporating boundary information are used. Following the segmentation of single strokes, a classification of strokes with respect to the drawing tool used to create the strokes is performed. Two different classification methods, one texture-based and one based on active contour models are combined in order to improve the classification results, which are presented and discussed for strokes on selected test panels.

Introduction

The application of digital image analysis is an important tool for the examination of works of art. While now many imaging techniques for contactless investigations are available that utilize the electromagnetic spectrum of UV, VIS, NIR and X-ray, the analysis of these images is mainly done by visual interpretation (Arney and Arney, 2001, Mairinger, 2003). To support art experts, a computer aided analysis will provide objective, comparable and reproducible data.

This work is motivated by a very specific field of art history, the examination of underdrawings. Underdrawings, the first sketch of an artist when he starts his work, provide important information for art historians about workshops and developments in a certain time period. To reveal the underdrawing, an imaging technique called infrared reflectography is applied (Boer, 1970). In Fig. 1 an example of an underdrawing is given. The left image shows the color image of a detail of an Austrian panel painting from the 15th century, on the right a subimage of the corresponding IR image containing strokes of the underdrawing is depicted. Conservators and art historians are interested in the development of underdrawings, their relation to other drawings and differences between underdrawings and the covering painting. Another aspect concerns the style of the underdrawing, whether the underdrawing is sketchy, freehand or a copy from a template. Finally, an important question is, what kind of materials and drawing tools are used for the creation of the underdrawing (Bomford, 2002). Motivated by this art historical application the goal of the present work lies in the computer-aided analysis of strokes, more specifically to differentiate between the different drawing tools and materials used.

There have been several attempts to apply image analysis techniques for art historical applications to extract information about images of paintings and drawings in order to identify the artist or to differentiate between different types of paintings. Kröner and Lattner (1998) reported a simple histogram-based method for differentiating between two classes of drawings. Keren (2002) used a Bayes classifier with features calculated from the DCT for the problem of painter classification. Li and Wang (2004) applied a mixture of 2-D multiresolution hidden Markov models (MHMMs) in combination with wavelets to capture different styles in Chinese ink paintings and used these models to classify different artists. van den Herik and Postma (2000) demonstrated a feature-based classification approach for an automatic labeling of paintings. They mainly concentrated on the question of what features are relevant in the specific domain and concluded that domain knowledge together with neural-network techniques will allow the recognition of the visual signature of painters in the future. Some early attempts to capture the style of a painting are published in (Kirsch and Kirsch, 1988). A tool for the extraction of compositional information from paintings in terms of sizes, shapes, proportions and locations of figures is presented by Tanaka et al. (2000).

In contrast to these methods, our approach is not a global point of view, i.e., to classify whole paintings, but to identify and classify individual strokes in underdrawings.

One major aspect of our work is to implement the knowledge of experts, by trying to reproduce the way they analyze a drawing. Hence the features used in our algorithms coincide with their visual impressions. The characteristics of different drawing materials will be explored by identifying and extracting low level features of the individual strokes for a classification. Toward the investigation of strokes in underdrawings of real paintings we first analyzed strokes applied on test panels. This allows for a better control and evaluation of the data and algorithms. In addition we worked on simulated underdrawings (IR images from the test panels). Our analysis approach concentrates on two prominent characteristics of drawn strokes: variations in the smoothness of the boundary and variations of texture. The boundary analysis is based on an active contour method (Xu and Prince, 1998) and the texture is analyzed using the discrete wavelet transformation (Porter and Canagarajah, 1996).

The organization of this paper is as follows. Section 2 covers the data material, the types of strokes used, the data set for the experiments and image preprocessing steps. Section 3 describes the methods for the feature extraction module and experimental results are given in Section 4. Finally Section 5 gives a summary and some proposals for future work.

Section snippets

Data sets

The drawing tools used in medieval panel paintings can be categorized globally into two different types, into those that are fluid and into a group that consists of dry drawing materials. The following sections will briefly discuss properties of strokes applied with different drawing tools, the test panels used for our experiments and finally preprocessing steps to extract individual strokes from the image as a starting base for further analysis.

Feature extraction

In preliminary studies (Kammerer et al., 2003a, Lettner et al., 2004) we investigated the boundary characteristics and the texture of painted strokes and found out that a determination between fluid and dry drawing materials is possible. In this section we give an overview of these two features and the methods of obtaining them.

Results

In our experiments we studied the differences of three types of drawing tools – brush, black chalk and graphite. The study includes two experimental setups: one with scanned images and one with IR images. The number of features was selected according to the small number of test samples available. We tested more features and different combinations of features in an exhaustive manner. The chosen combinations provided the best discrimination. In this section we show the results of these two setups.

Conclusion and outlook

In this paper we presented an approach to classify strokes painted by different drawing tools and drawing materials based on two significant characteristics, the smoothness of the boundary and the granularity (texture) of the stroke surface. The characteristics, noticed by the expert when differentiating between different stroke types, have been implemented straightforwardly. A smoothness feature is defined as the deviation of snakes of different elasticity. The texture is investigated with the

Acknowledgements

We would like to thank Georg Langs for help in the field of active contour models and Allan Hanbury for his support in our project. We further thank Prof. Mairinger for providing his experience in the field of IR examination and finally for the comments of the reviewers to improve the quality of this paper. This work was supported by the Austrian Science Foundation (FWF) under grant P15471-MAT and the European NoE MUSCLE (FP6-507752).

References (21)

  • S. Tanaka et al.

    Composition analyser: Support tool for composition analysis on painting masterpieces

    Knowledge-Based Syst.

    (2000)
  • J. Arney et al.

    Encyclopedia of Imaging Science

    Imaging Science in Art Conservation

    (2001)
  • de Boer, J.V.A., 1970. Infrared reflectography – A contribution to the examination of earlier european paintings, Ph.D....
  • D. Doermann et al.

    Recovery of temporal information from static images of handwriting

    Int. J. Comput. Vision

    (1995)
  • R.O. Duda et al.

    Pattern Classification

    (2001)
  • Hanbury, A., Kammerer, P., Zolda, E., 2003. Painting crack elimination using viscous morphological reconstruction. In:...
  • Kammerer, P., Zolda, E., Sablatnig, R., 2003. Computer aided analysis of underdrawings in infrared reflectograms. In:...
  • Kammerer, P., Langs, G., Sablatnig, R., Zolda, E., 2003. Stroke segmentation in infrared reflectograms. In: Proc. of...
  • M. Kass et al.

    Snakes: Active contour models

    Int. J. Comput. Vision

    (1988)
There are more references available in the full text version of this article.

Cited by (27)

  • New insight on the underdrawing of 16th Flemish-Portuguese easel paintings by combined surface analysis and microanalytical techniques

    2016, Micron
    Citation Excerpt :

    Underdrawings are the preliminary drawings on the panels (or canvas) and much of interest in this study lies in what they reveal about the creative processes of the artist, providing important information about workshops and developments in a certain time period (Bomford et al., 2002; Kammerer et al., 2007).

  • DeepArtist: A Dual-Stream Network for Painter Classification of Highly-Varying Image Resolutions

    2022, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
  • Distinguishing Hand Drawing Style Based on Multilevel Analytics Framework

    2020, Wireless Communications and Mobile Computing
  • Classification of abstract images using machine learning

    2017, ACM International Conference Proceeding Series
View all citing articles on Scopus
View full text