Joint sparse coding based spatial pyramid matching for classification of color medical image

Abstract

Although color medical images are important in clinical practice, they are usually converted to grayscale for further processing in pattern recognition, resulting in loss of rich color information. The sparse coding based linear spatial pyramid matching (ScSPM) and its variants are popular for grayscale image classification, but cannot extract color information. In this paper, we propose a joint sparse coding based SPM (JScSPM) method for the classification of color medical images. A joint dictionary can represent both the color information in each color channel and the correlation between channels. Consequently, the joint sparse codes calculated from a joint dictionary can carry color information, and therefore this method can easily transform a feature descriptor originally designed for grayscale images to a color descriptor. A color hepatocellular carcinoma histological image dataset was used to evaluate the performance of the proposed JScSPM algorithm. Experimental results show that JScSPM provides significant improvements as compared with the majority voting based ScSPM and the original ScSPM for color medical image classification.

Introduction

Medical imaging plays an important role in clinical practices. With the rapid development of modern medical imaging techniques, various medical images both in grayscale and color have been generated.

While color images such as microscopic images, endoscopic images, and photographic images have important applications in practice, analysis of color medical images is still a relatively unexplored area as compared with grayscale images. For example, in most computer-aided medical image detection, analysis, and classification systems, color images are usually converted to grayscale for further processing to make use of available algorithms and reduce computational complexity [1], [2]. As a result, plenty of useful color information is discarded resulting in reduced performance. Furthermore, specially developed color feature descriptors are rare. Although some algorithms can extract features from individual color channels, most of them only treat each channel as a grayscale image, and ignores inherent correlation among different channels [3].

In recent years, the sparse coding (SC) technique has been successfully used in various applications [4], [5], [6]. In pattern recognition, SC alone can work as a classifier [5], [7], [8], and even further be embedded in a classification framework [5], [9]. The sparse coding based linear spatial pyramid matching (ScSPM) is a popular SC-embedded classification method [9]. It computes a spatial pyramid image representation with SC of local descriptors instead of the K-means vector quantization (VQ) in traditional SPM [9], [10], and thus significantly improves the feature generation performance. ScSPM has been widely used in image classification, and various improved algorithms have been proposed. For example, Zhang et al. applied the non-negative SC to ScSPM to reduce information loss during the encoding process for image representation [11]; Gao et al. proposed the Laplacian SC and hypergraph Laplacian SC based ScSPM, which preserves the locality and similarity information among the instances to be encoded and alleviate instability of SC [12]. However, ScSPM and its variants are usually applied to grayscale images. The existing SC methods used in ScSPM fail to consider either the color information, or the inherent correlation among different color channels in a color image.

Recently, the joint sparsity model (JSM) for SC has achieved great success in image processing and analysis, e.g., image fusion [13], [14], denoising [15], restoration [16], annotation [17], and pattern recognition [18], [19], [20], [21], [22]. Generally, the joint sparsity models (JSM) can be classified into three categories [23]: JSM-1 (sparse common component + innovations), JSM-2 (common sparse supports) and JSM-3 (nonsparse common + sparse innovations). In JSM-1, all signals share a common sparse component, and meanwhile each individual signal has a sparse innovations component [23]. It is suitable to represent color images, because different color channels share the same scenes with common information and also have individual color information. This way, the inter-correlation among different color channels can be represented by a common sparse component, while the unique portion of each color channel is then characterized by the sparse innovation component. JSM-1 has been applied to image fusion [13], [14], denoising [15], and restoration [16]. However, to the knowledge of the authors, applications of JSM-1 to classification have not been reported.

Since SC in ScSPM can be regarded as one step in generating features for a classifier, JSM-1 has the potential to be used in ScSPM to generate features with color information from color images. In this work, we propose a joint sparse coding based SPM (JScSPM) method for the classification of color medical images. The joint dictionary construction and joint SC are used to combine the inherently correlated contents and the individual color information in different color channels, and generate a color descriptor in a much easier way as compared to specially designed color features.