Elsevier

Pattern Recognition

Volume 42, Issue 6, June 2009, Pages 1052-1057
Pattern Recognition

Pattern analysis of dermoscopic images based on Markov random fields

https://doi.org/10.1016/j.patcog.2008.07.011Get rights and content

Abstract

In this paper a method for detecting different patterns in dermoscopic images is presented. In order to diagnose a possible skin cancer, physicians assess the lesion based on different rules. While the most famous one is the ABCD rule (asymmetry, border, colour, diameter), the new tendency in dermatology is to classify the lesion performing a pattern analysis. Due to the colour textured appearance of these patterns, this paper presents a novel method based on Markov random field (MRF) extended for colour images that classifies images representing different dermatologic patterns. First, each image plane in L*a*b* colour space is modelled as a MRF following a finite symmetric conditional model (FSCM). Coupling of colour components is taken into account by supposing that features of the MRF in the three colour planes follow a multivariate Normal distribution. Performance is analysed in different colour spaces. The best classification rate is 86% on average.

Introduction

In the last two decades, a rising incident of malignant melanoma has been observed. Because of the lack of adequate therapies for metastatic melanoma, the best treatment is still early diagnosis and prompt surgical excision of the primary cancer [1]. Dermoscopy (also known as epiluminescence microscopy) is an in vivo method that has been reported to be a useful tool for the early recognition of malignant melanoma [2]. Its use increases diagnostic accuracy between 5% and 30% over clinical visual inspection [3].

In order to give a diagnosis, physicians follow a two-step algorithm: (1) classify the lesion into melanocytic and non-melanocytic type and (2) for the melanocytic ones, classify into benign and malignant lesions. In order to perform the second step, four different approaches are the most commonly used: the ABCD rule of dermoscopy, the 7-point checklist, the Menzies method, and pattern analysis [4].

The currently available digital dermoscopic systems offer the possibility of computer storage and retrieval of dermoscopic images and patient. Some systems even offer the potential of computer assisted diagnosis (CAD) [5], [6]. As diagnostic accuracy with dermoscopy has been shown to depend on the experience of the dermatologist, CAD systems will help less-experienced dermatologists.

Most of the technical papers developing methods to classify automatically dermatologic images are based on the ABCD rule (asymmetry, border irregularity, colour variegation, diameter greater than 6 mm or growing). Normally, the papers present one approach to cover one or some of the “letters” of the rule, that is, some are based on detecting asymmetry [7], [8], borders [9], [10], [11], [12], colour [13], [14], [15] or diameter [14]. There are some papers that cover the whole ABCD criterion. Tomatis et al. detect features for the ABCD rule, but they need a telespectrophotometric system [16]. Larabi et al. [17] extract some parameters to cover the ABCD rule, but they do not use it to classify the lesion but for retrieval. Maglogiannis et al. use a support vector machine to classify border features, colour features and texture features [18].

In any case, all the methods present in the literature, to the best of our knowledge, consist always of a feature extraction step (colour, texture and/or shape characteristics), an optional feature selection step and a final feature classification step. In general, the contribution of the papers is the election of new features to classify the lesion.

One of the novelties of this paper is that it is not based on detecting specific features in the images to cover the four letters of the ABCD rule, but it follows the new tendency in dermatology: to look for specific patterns in the lesions which will lead physicians to an assessment. Looking at the clinical references in this subject, we can see that the procedure can be summarized as a pattern recognition system. Physicians, in order to classify between benign and malign lesions, take into account the overall general appearance of colour, architectural order, symmetry of pattern and homogeneity (CASH). Benign melanocytic lesions tend to have few colours, architectural order, symmetry of pattern or homogeneity. Malignant melanoma often has many colours and much architectural disorder, asymmetry of pattern and heterogeneity [4].

In this sense, the classification of the lesions can be summarized as follows:

  • (a)

    Reticular pattern or network pattern. It is the most common global feature in melanocytic lesions. It represents the junctional component of a melanocytic nevus.

  • (b)

    Globular pattern. It presents numerous “aggregated globules”. It is commonly seen in a congenital nevus, superficial type.

  • (c)

    Cobblestone pattern. Very similar to the globular pattern but is composed of closer aggregated globules, angulated, resembling cobblestones.

  • (d)

    Homogeneous pattern. It appears as diffuse pigmentation, which might be brown, grey-blue, grey-black, or reddish black. No pigmented network is found. It is seen in the homogeneous blue nevi.

  • (e)

    Starburst pattern. It is characterized by the presence of streaks in a radial arrangement. It is commonly seen in Reed nevi or Spitz nevi.

  • (f)

    Parallel pattern. It is exclusively found on the palms and soles due to the particular anatomy of these areas.

  • (g)

    Multicomponent pattern. The combination of three or more distinctive dermoscopic structures within a given lesion is highly suggestive of melanoma.

An illustration of the patterns listed above is presented in Fig. 1.

In order to perform the pattern analysis procedure to classify the dermoscopic images we follow a model-based technique. In these methods, image classification is treated as an incomplete data problem, where the value of each pixel is known and the label, which designates the texture pattern the pixel belongs to, is missing. In such techniques, the image regions are modelled as random fields and the segmentation/classification problem is posed as a statistical optimization problem. It often provides more precise characterization of the image regions [19]. Most of the existing techniques use the spatial interaction models like Markov random field (MRF) or Gibbs random field (GRF) to model digital images. Stochastic model-based image segmentation/classification methods can be either supervised (model parameters obtained from training set) or un-supervised (model parameters have to be estimated from the observed image).

In our case, one important characteristic of texture patterns is colour. Panjwani et al. develop MRF models for unsupervised segmentation of textured colour images [20]. Each model is defined within each colour plane, taking into account interactions between the three planes. They work in the RGB colour space and perform a region splitting phase and an agglomerative clustering phase. Kato et al. [21] use a combination grey-level based texture features and colour instead of direct modelling of colour textures. The advantage is that most of the classical texture features can be used in their model. The colour features are calculated in the L*u*v* colour space. Tab et al. present a multiresolution colour image segmentation algorithm [22]. Regarding the MRF model for coloured textures, they assume conditional independence of the channels and they use the YUV colour space.

Regarding the use of MRF in segmentation of dermoscopic images not much appears in the literature. To the best of our knowledge, only Gao et al. [23] present something related. In their paper a comparison between different segmentation techniques is presented (principal component transform PCT/median cut algorithm, adaptive thresholding in PCT, k-means, MRF technique, etc.). The MRF technique is poorly explained, and they use the first plane of the PCT of the dermoscopic images, that is, is a grey-level version of MRF modelling. But, in any case, their purpose is to separate the lesion from the healthy skin, which is not our aim (we perform a classification of the different patterns that a lesion can present). In [14] the authors present a melanoma recognition system, but the use of MRF is not for the segmentation or characterization of the coloured patterns, but as a classifier (spin glass-MRF). Their inputs are features characterizing either the C letter of the ABCD rule or the D letter of the same rule.

In this paper a pattern recognition algorithm to detect different colour textured patterns is presented. New tendencies in dermatology, instead of applying rules such as the ABCD rule or the Menzies test, detect benign or malign lesions performing a pattern recognition step. In our case, instead of detecting features such as border irregularity, colours or some texture descriptors, we analyse the different coloured patterns that a skin lesion can present, i.e., globular pattern, reticular pattern, cobblestone pattern, homogeneous pattern and parallel pattern. To this purpose, an MRF model-based classification in the L*a*b* colour space is performed.

We devote Section 2 to explain the classification algorithm, Section 3 to present the results and Section 4 to expose the conclusions.

Section snippets

Model-based classification algorithm

For textured images, the theory based on MRF is an important field, which has been developed extensively in the last decades. MRF theory provides a convenient and consistent way for modelling context dependent entities such as image pixels and correlated features [24]. This is achieved through characterizing mutual influences among such entities using conditional MRF distributions.

In this paper we present a supervised method of classification. We are going to classify the following patterns:

Experimental results

The proposed algorithm has been tested on a database containing 100 40×40 image samples of the following patterns: reticular, globular, cobblestone, homogeneous and parallel. For each type of pattern 20 images has been used. These images were provided by the Dermatology Unit at Virgen del Rocío Hospital (Sevilla, Spain). All of them were taken with a dermoscope Fotofinder, Schuco International London Limited. We have performed 10-fold cross-validation [30]: 90% of the total set of images has

Conclusions and future work

In this paper, a novel classification algorithm based on MRF modelling has been presented. The first novelty of the paper is the methodology followed to analyse the dermoscopic images. To the best of our knowledge, references in the literature apply the ABCD rule in order to classify the lesion, extracting features that measures characteristic such as asymmetry of border, colour information of the lesion, diameter of the lesion, etc. In this paper, we present a new methodology to classify the

Acknowledgements

The authors thank the physician Amalia Serrano for providing and classifying the dermoscopic images used in this paper. This work is financed by Project FIS05-2028.

About the Author—CARMEN SERRANO received the M.S. degree in Telecommunication Engineering from the University of Seville, Spain, in 1996 and the Ph.D. degree in January 2002. In 1996, she joined the Signal Processing and Communication Department at the same university, where she is currently an Associate Professor. Her research interests concern image processing and, in particular, colour image segmentation, classification and compression, mainly with biomedical applications. She is author of

References (30)

  • M. Binder et al.

    Epiluminescence microscopy-based classification of pigmented skin lesions using computerized image analysis and an artificial neural network

    Melanoma Res.

    (1998)
  • P. Schmidt

    Segmentation of digitized dermatoscopic images by two-dimensional color clustering

    IEEE Trans. Med. Imaging

    (1998)
  • P. Schmid-Saugeon et al.

    Towards a computer-aided diagnosis system for pigmented skin lesions

    Computerized Med. Imaging Graphics

    (2003)
  • B. Erkol et al.

    Automatic lesion boundary detection in dermoscopy images using gradient vector flow snakes

    Skin Res. Technol.

    (2005)
  • C. Grana et al.

    A new algorithm for border description of polarized light surface microscopic images of pigmented skin lesions

    IEEE Trans. Med. Imaging

    (2003)
  • Cited by (70)

    • Hyper-fusion network for semi-automatic segmentation of skin lesions

      2022, Medical Image Analysis
      Citation Excerpt :

      This is attributed to the challenges in interpreting skin lesion images where there can be diverse visual characteristics such as variations in size, shape boundaries (e.g., ‘fuzzy’), artifacts and has hairs (Fig. 1) (Barata et al., 2015). Therefore, automated image analysis is a valuable aid for clinical decision support (CDS) systems and for the image-based diagnosis of skin lesions (Serrano and Acha 2009; Esteva et al., 2017). Skin lesion segmentation is the fundamental for these CDS systems and has motivated the development of numerous segmentation methods.

    • Deep neural networks and advanced computer vision algorithms in the early diagnosis of skin diseases

      2022, State of the Art in Neural Networks and their Applications: Volume 2
    • Step-wise integration of deep class-specific learning for dermoscopic image segmentation

      2019, Pattern Recognition
      Citation Excerpt :

      These limitations can be attributed to the skin lesions having various sizes and shapes, fuzzy boundaries, low contrast when compared to the skin, heterogeneity and skin hair (Fig. 1) [7]. These problems have motivated the development of computer aided diagnosis (CAD) systems that can assist dermatologists’ clinical diagnoses [8,9]. Skin lesion segmentation is a fundamental requirement for these CAD systems.

    View all citing articles on Scopus

    About the Author—CARMEN SERRANO received the M.S. degree in Telecommunication Engineering from the University of Seville, Spain, in 1996 and the Ph.D. degree in January 2002. In 1996, she joined the Signal Processing and Communication Department at the same university, where she is currently an Associate Professor. Her research interests concern image processing and, in particular, colour image segmentation, classification and compression, mainly with biomedical applications. She is author of numerous papers both in journals and international conferences.

    About the Author—BEGOÑA ACHA received the Ph.D. degree in Telecommunication Engineering in July 2002. She has been working as Associate Professor since 1996 in the Signal Processing and Communications Department of the University of Seville. Her current research activities include works in the field of colour image processing and its medical applications, in particular in segmentation, classification and compression. She is author of numerous papers both in journals and international conferences.

    View full text