Elsevier

Pattern Recognition

Volume 46, Issue 1, January 2013, Pages 86-97
Pattern Recognition

Pattern classification of dermoscopy images: A perceptually uniform model

https://doi.org/10.1016/j.patcog.2012.07.027Get rights and content

Abstract

Pattern classification of dermoscopy images is a challenging task of differentiating between benign melanocytic lesions and melanomas. In this paper, a novel pattern classification method based on color symmetry and multiscale texture analysis is developed to assist dermatologists' diagnosis. Our method aims to classify various tumor patterns using color–texture properties extracted in a perceptually uniform color space. In order to design an optimal classifier and to address the problem of multicomponent patterns, an adaptive boosting multi-label learning algorithm (AdaBoost.MC) is developed. Finally, the class label set of the test pattern is determined by fusing the results produced by boosting based on the maximum a posteriori (MAP) and robust ranking principles. The proposed discrimination model for multi-label learning algorithm is fully automatic and obtains higher accuracy compared to existing multi-label classification methods. Our classification model obtains a sensitivity (SE) of 89.28%, specificity (SP) of 93.75% and an area under the curve (AUC) of 0.986. The results demonstrate that our pattern classifier based on color–texture features agrees with dermatologists' perception.

Highlights

► A novel AdaBoost.MC classifier was developed to solve the problem of multicomponent patterns. ► This classifier focused more on color and texture properties of lesions in a uniform color space. ► The overall system has shown good performance to model CASH rule for dermoscopy images.

Introduction

Non-invasive malignant melanoma (MM) is widely diagnosed using digital dermoscopy. In fact, dermoscopy is one of the most cost-effective methods for the detection and analysis of pigmented (PSLs) and non-pigmented skin lesions (Non-PSLs). Many dermatologists [1], [2] use clinical ABCD (A: Asymmetry, B: Border, C: Color, D: Differential structures); Menzies' method; 7-point checklist and pattern analysis methods to diagnose and classify the lesions. In particular, it is very difficult to distinguish among lesions and even experienced dermatologists [3] have a diagnostic accuracy below 85%. Therefore, recently computer-assisted diagnosis systems (CADs) have been developed. For CADs to decide if a lesion is benign, melanoma or suspect [4], it would be desirable to have automated systems that can provide assistance to less experienced dermatologists. Automated systems for dermoscopy images [5], [6], [7] usually have four stages: (1) artifact removal, (2) lesion segmentation, (3) ABCD and texture related feature extraction and optimization, and finally (4) classification. In practice, the classification algorithms, which utilizes low-level feature extraction such as color, texture and shape, are quite sophisticated and have achieved considerable success. The extraction of low-level image features that correlate with high-level image semantics, however, remains a challenging task.

A few discrimination approaches have been proposed in the literature that attempt to bridge this semantic gap between low-level features and high-level semantics. However, many of them use non-uniform color spaces e.g., RGB, HSV and some of them use approximately uniform color space such as CIELab [8]. These techniques do not correlate well with the physician's perception due to the use of a non-uniform color space or limited gray scale image properties. Moreover, color appearance models are not utilized in these classification techniques. These appearance models correlate to well with human perception and are capable to predict a wide variety of visual phenomena, which is not possible in color spaces.

Various lesion classification systems have been proposed in the literature. In [9], Ganster et al. proposed an automated melanoma recognition by the nonparametric “KNN classifier”. A machine learning algorithm was developed in [10] to characterize melanoma by a feature vector that contained shape, color and texture information, as well as local and global parameters. Burroni et al. [11] used the K-NN classifier for classification of melanoma. Automated melanoma classification systems [12], [13] were developed too. Tanaka et al. [14] developed a method for pattern classification of nevus with texture analysis. Their research was devoted to categorize three texture patterns: globular, reticular, and homogeneous patterns with 94% accuracy. Iyatomi et al. [15] developed a classification system based on acral volar skin with three detectors such as parallel ridge, parallel furrow, and fibrillar patterns. They used more than 46 texture features with maximal value of between-class variance, and Mahalanobis distance. In [16], a pattern analysis based on clinical color, architectural order, symmetry of pattern and homogeneity (CASH) technique was modeled by Markov random field (MRF). In that study, mean and variance of each plane of CIELab color space was used to extract the color related features. For seven pattern classes, the authors reported 86% classification accuracy. A different approach was developed in [17] to detect and visualize only pigment network structures based on cyclic graphs. Recently in [18], a pattern classification system (PCS) based on the CASH rule was presented to recognize among six classes instead of multicomponent pattern.

The primarily aim of this paper is to measure the color, architectural order, symmetry of pattern and homogeneity (CASH) of lesions instead of clinical ABCD rule [7], [15]. By using CASH method, physicians can classify between benign and malignant lesions. Benign melanocytic lesions tend to have few colors, an architectural order, symmetry of pattern and are homogeneous. Malignant melanomas often have many colors, architectural disorder, asymmetry of pattern and are heterogeneous. To automate CASH model, some effective methods [14], [15], [16], [17], [18] were proposed. Particularly in [16], CIELab color space [16] model was utilized to identify seven patterns using maximum likelihood (ML) criteria. Melanoma or Clark nevus lesions often contain multicomponent patterns meaning that a lesion contain 2, 3 or more pattern classes. As a result, if a classification decision rule is used based on ML [16] then it cannot provide multiple decisions to match with multiple pattern classes simultaneously because it was based on the single state maximum probability concept. However, if single input pattern is provided to match a single class output then it may provide better classification results. Therefore, this study is focused on providing multi-patterns as an input to match with multi-class outputs, concurrently as shown in Fig. 1. In this example, an input lesion is classified into three patterns.

Physicians are more capable of interpreting color–texture information than any automated method. This is because; the human visual system (HVS) plays an important role in the categorization and recognition of objects. Similarly, it has been proven that HVS used model based approach to take decisions based on the fuzzy logical model of perception and signal detection theory. Humans are using these two models to discriminate among patterns and to make decisions. By combining these two models, the conclusion is that we should focus on developing a pattern analysis model based on human perception. As a result, the first aim of this study is to develop an effective pattern classification model based on CASH, which is closer to physician perception. Secondly, to develop an efficient and optimized pattern classification method in a CIECAM02 (JCh) perceptually uniform color space. Thirdly, an adaptive boosting (AdaBoost.MH) multi-label input machine learning algorithm is used to develop (AdaBoost.MC) multi-label output method for effective patterns detector and to solve the multicomponent pattern problem in dermoscopy images. The AdaBoost.MC algorithm is developed by integrating maximum a posterior probability (MAP) along with ranking concept. The MAP probability technique is utilized since it provides most popular statistical criteria to get optimality. In this multi-label boosting algorithm, the classes are ranked according to their level of similarity. The class is ranked first with the highest probability; second best probability is ranked second and so on based on label-weighted score.

In this study, the classifications of different pattern classes in the lesion diagnosis process are summarized as follows:

  • (a)

    Reticular pattern or pigmented network: It is the most common global feature present in a junctional nevus, compound nevus, lentigo or melanosis.

  • (b)

    Globular pattern: It presents itself as small aggregated globules and may have different colors, which has high specificity for diagnosis of compound and intradermal nevi.

  • (c)

    Cobblestone pattern: It is similar to Globular pattern but they are large, closely aggregated and angulated.

  • (d)

    Homogeneous pattern: diffuse and homogeneous blue-grayish pigmentation is present and absence of pigmented network, which characterizes the blue nevi.

  • (e)

    Parallel ridge pattern (PRP): The specific type of pattern found in palm or sole, which may be benign melanocytic nevi and acral melanomas if it has parallel ridge pattern.

  • (f)

    Starburst pattern: It is characterized by the presence of pigmented streaks in a radial arrangement. It is commonly seen in Red nevi or pigmented Spitz nevi.

  • (g)

    Multicomponent pattern: This pattern has high specificity for diagnosis of melanoma and consists of presence of three or more dermoscopic features in a single lesion.

The multicomponent pattern is shown in Fig. 1, while the rest of the abovementioned dermoscopic patterns are shown in Fig. 2.

Section snippets

Outline of the proposed pattern classification model

To analyze the above mentioned patterns, a computerized CASH model is proposed as illustrated in Fig. 3. From each dermoscopy image, Region-of-interest (ROI) is selected first, which is then transformed it into the CIECAM02 (JCh) uniform color space. Afterwards color attributes such as the number of colors, percentage of occurrence and their similarities are measured. For texture feature analysis, the local and global statistical properties are extracted using the multiscale steerable pyramid

Region-of-interest (ROI) extraction

In order to obtain effective pattern extraction and classification, a region of size (450×450) is automatically selected from the center of each dermoscopy image having 768×512 pixels. This step is called ROI extraction. Seven pattern groups of total 350 are selected from a data set of 1039 dermoscopic images. A detailed description of this selected data set is presented in Section 6.1.

Color space transform

The proposed perceptually adapted pattern classification is intended to make the early diagnosis of skin

Pattern analysis and feature vector construction

Pattern or texture analysis plays an important role in many image processing tasks such as in remote sensing, medical, natural scenes and content based image retrieval (CBIR) systems. The main benefit of extracting the effective texture features is that they provide better classification results. A number of researchers have proposed algorithms for texture analysis, but they are limited to gray scale or have used non-uniform color space. In this paper, optimized color–texture features are

Pattern classification model by multi-label learning

After extracting the set of features (fi), the next step is to devise a suitable machine learning algorithm to assess the features' discriminative power. Melanomas often exhibit multicomponent patterns so any learning algorithm must take multi-label input and provide multi-label output at the same time. Multi-label learning [36] refers to the classification problem where each input pattern can be assigned to multiple class labels, simultaneously. It has found applications in many domains, such

Dermoscopy data set

Skin lesions of dermoscopy images are used in the experiment, which has been collected as a CD resource from the two European university hospitals as part of the EDRA-CDROM, 2002 [44]. This data set contained 1039 color images in total with spatial resolution of 768×512 pixels. All these images were captured during routine clinical assessments to imitate the a priori probabilities of the clinical diagnosis. In total 350 dermoscopic images are selected from this data set as Reticular (50),

Results and discussion

The average results of the proposed model on this data set are shown in Table 1. Table 1 demonstrates each pattern classifier by using boosting, maximum a posterior (MAP) and robust ranking principles in terms of sensitivity: SE, specificity: SP and average error(E) along with area under the ROC curve (AUC) analysis. Fig. 6 shows the corresponding receiving operating characteristic curve (ROC) of each multi-label machine learning algorithm. In this figure, each classifier performance is shown

Conclusion

In this paper, a novel pattern classification model related to dermatologist's perception is proposed. The purpose of this study was to develop an effective pattern classification (AdaBoost.MC) model based on extraction of color symmetry and multiscale-texture feature in uniform CIECAM02 (JCh) color space. To the best of our knowledge, many studies for lesions classification of dermoscopy have been devoted towards clinical ABCD rule but few studies on pattern analysis have been developed in the

Conflict of interest

All authors in this paper have no potential conflict of interest.

Acknowledgments

This research was supported by the National 863 High Technology Research and Development Program of China (Grant no. 2006AA02Z347) and the Chinese Scholarship Council (CSC) (Grant no. 2008GXZ143). We would like to thank anonymous reviewers for their valuable comments and suggestions.

Qaisar Abbas received his BSc degree in Computer Science from the Bahauddin Zakaryia University (BZU) (Multan, Pakistan) in 2001. He received his MSc in Computer Science from BZU and Doctor of engineering degree (PhD) from the University of HUST at (Wuhan, China) in 2004 and 2011, respectively. He was working as a software manager under Business Craft company (Spanish), which provided IT solutions from 2004–2009 and also as a lecture from 2006–2008 in the same University, where he did masters.

References (45)

  • S. Li et al.

    Texture classification using the support vector machines

    Pattern Recognition

    (2003)
  • A. Sengur

    Wavelets transform and adaptive neuro-fuzzy inference system for color texture classification

    Expert Systems with Applications

    (2008)
  • H. Zhou et al.

    A novel extended local binary pattern operator for texture analysis

    Information Sciences

    (2008)
  • M.R. Boutell et al.

    Learning multi-label scene classification

    Pattern Recognition

    (2004)
  • M.-L. Zhang et al.

    ML-kNN: a lazy learning approach to multi-label learning

    Pattern Recognition

    (2007)
  • E. Song et al.

    Semi-supervised multi-class Adaboost by exploiting unlabeled data

    Expert Systems with Applications

    (2011)
  • M.-L. Zhang et al.

    Feature selection for multi-label naive bayes classification

    Information Sciences

    (2009)
  • A.P. Bradley

    The use of the area under the ROC curve in the evaluation of machine learning algorithms

    Pattern Recognition

    (1997)
  • H. Blum et al.

    Digital image analysis for diagnosis of cutaneous melanoma. Development of a highly effective computer algorithm based on analysis of 837 melanocytic lesions

    British Journal of Dermatology

    (2004)
  • A.G. Isasi et al.

    Melanomas non-invasive diagnosis application based on the ABCD rule and pattern recognition image processing algorithms

    Computers in Biology and Medicine

    (2011)
  • A. Mojsilovic et al.

    Matching and retrieval based on the vocabulary and grammar of color patterns

    IEEE Transactions on Image Processing

    (2000)
  • H. Ganster et al.

    Automated melanoma recognition

    IEEE Transactions on Medical Imaging

    (2001)
  • Cited by (109)

    • Hyper-fusion network for semi-automatic segmentation of skin lesions

      2022, Medical Image Analysis
      Citation Excerpt :

      Skin lesion images such as dermoscopy are commonly acquired as a non-invasive imaging technique for the in-vivo evaluation of pigmented skin lesions and play an important role in early diagnosis (Celebi et al., 2007). The identification of melanoma from skin lesion images using human vision alone, can be subjective, inaccurate and poorly reproducible, even amongst experienced dermatologists (Celebi et al., 2008; Abbas et al., 2013). This is attributed to the challenges in interpreting skin lesion images where there can be diverse visual characteristics such as variations in size, shape boundaries (e.g., ‘fuzzy’), artifacts and has hairs (Fig. 1) (Barata et al., 2015).

    View all citing articles on Scopus

    Qaisar Abbas received his BSc degree in Computer Science from the Bahauddin Zakaryia University (BZU) (Multan, Pakistan) in 2001. He received his MSc in Computer Science from BZU and Doctor of engineering degree (PhD) from the University of HUST at (Wuhan, China) in 2004 and 2011, respectively. He was working as a software manager under Business Craft company (Spanish), which provided IT solutions from 2004–2009 and also as a lecture from 2006–2008 in the same University, where he did masters. He is currently working as an Assistant Professor and the founding director of the Image Processing and Analysis Laboratory in the Department of Computer Science at the National Textile University in Faisalabad. His research interests include: image processing, medical image analysis, Genetic programming and pattern classification.

    M. Emre Celebi received his BSc degree in computer engineering from Middle East Technical University (Ankara, Turkey) in 2002. He received his MSc and PhD degrees in computer science and engineering from the University of Texas at Arlington (Arlington, TX, USA) in 2003 and 2006, respectively. He is currently an associate professor in the Department of Computer Science at Louisiana State University, Shreveport, LA, USA). His research interests include medical image analysis, color image processing, content-based image retrieval, and open-source software development.

    Irene Fondón received the M.S. degree in Telecommunication Engineering from the University of Seville, Spain, in 2004. She has completed her PhD degree in the year of 2011. She has been working as an associated professor since 2005 in the Signal Processing and Communications Department of the University of Seville. Her current research activities include works in the field of image processing and its medical applications. She is author of papers both in journals and international conferences.

    Carmen Serrano received the MS degree in Telecommunication Engineering from the University of Seville, Spain, in 1996 and the PhD degree in January 2002. In 1996, she joined the Signal Processing and Communication Department at the same university, where she is currently an Associate Professor. Her research interests concern image processing and, in particular, color image segmentation, classification and compression, mainly with biomedical applications. She is author of numerous papers both in journals and international conferences. Dr. Guangzhi Ma is an Associate Professor at Huazhong University of Science and Technology. His research interests concern about Data mining and Knowledge Discovery. He is author of numerous papers both in journals and international conferences.

    Guangzhi Ma is an Associate Professor at Huazhong University of Science and Technology. His research interests concern about Data mining and Knowledge Discovery. He is author of numerous papers both in journals and international conferences.

    View full text