Joint sparse coding based spatial pyramid matching for classification of color medical image
Introduction
Medical imaging plays an important role in clinical practices. With the rapid development of modern medical imaging techniques, various medical images both in grayscale and color have been generated.
While color images such as microscopic images, endoscopic images, and photographic images have important applications in practice, analysis of color medical images is still a relatively unexplored area as compared with grayscale images. For example, in most computer-aided medical image detection, analysis, and classification systems, color images are usually converted to grayscale for further processing to make use of available algorithms and reduce computational complexity [1], [2]. As a result, plenty of useful color information is discarded resulting in reduced performance. Furthermore, specially developed color feature descriptors are rare. Although some algorithms can extract features from individual color channels, most of them only treat each channel as a grayscale image, and ignores inherent correlation among different channels [3].
In recent years, the sparse coding (SC) technique has been successfully used in various applications [4], [5], [6]. In pattern recognition, SC alone can work as a classifier [5], [7], [8], and even further be embedded in a classification framework [5], [9]. The sparse coding based linear spatial pyramid matching (ScSPM) is a popular SC-embedded classification method [9]. It computes a spatial pyramid image representation with SC of local descriptors instead of the K-means vector quantization (VQ) in traditional SPM [9], [10], and thus significantly improves the feature generation performance. ScSPM has been widely used in image classification, and various improved algorithms have been proposed. For example, Zhang et al. applied the non-negative SC to ScSPM to reduce information loss during the encoding process for image representation [11]; Gao et al. proposed the Laplacian SC and hypergraph Laplacian SC based ScSPM, which preserves the locality and similarity information among the instances to be encoded and alleviate instability of SC [12]. However, ScSPM and its variants are usually applied to grayscale images. The existing SC methods used in ScSPM fail to consider either the color information, or the inherent correlation among different color channels in a color image.
Recently, the joint sparsity model (JSM) for SC has achieved great success in image processing and analysis, e.g., image fusion [13], [14], denoising [15], restoration [16], annotation [17], and pattern recognition [18], [19], [20], [21], [22]. Generally, the joint sparsity models (JSM) can be classified into three categories [23]: JSM-1 (sparse common component + innovations), JSM-2 (common sparse supports) and JSM-3 (nonsparse common + sparse innovations). In JSM-1, all signals share a common sparse component, and meanwhile each individual signal has a sparse innovations component [23]. It is suitable to represent color images, because different color channels share the same scenes with common information and also have individual color information. This way, the inter-correlation among different color channels can be represented by a common sparse component, while the unique portion of each color channel is then characterized by the sparse innovation component. JSM-1 has been applied to image fusion [13], [14], denoising [15], and restoration [16]. However, to the knowledge of the authors, applications of JSM-1 to classification have not been reported.
Since SC in ScSPM can be regarded as one step in generating features for a classifier, JSM-1 has the potential to be used in ScSPM to generate features with color information from color images. In this work, we propose a joint sparse coding based SPM (JScSPM) method for the classification of color medical images. The joint dictionary construction and joint SC are used to combine the inherently correlated contents and the individual color information in different color channels, and generate a color descriptor in a much easier way as compared to specially designed color features.
Section snippets
Sparse coding in original ScSPM algorithm
The flowchart of the original ScSPM is shown in Fig. 1(a). For SC in ScSPM, let X be a set of D-dimensional local descriptors extracted from a gray image, i.e. . SC in ScSPM is used to solve the following optimization problem [9]:where is a set of sparse codes, and is an over-complete dictionary trained with the local descriptors of a gray image. Here, a unit L2-norm constraint on dk
Dataset
Hepatocellular carcinoma (HCC) is the most frequent primary liver malignancy, which is generally graded as being well differentiated, moderately differentiated, poorly differentiated and undifferentiated [25]. Recent studies on computer-aided grading of HCC propose to use multifractal feature description for grayscale images [26]. Therefore, we use the HCC histological image dataset to evaluate performance of the proposed JScSPM.
There are 66 HCC images all sized 1024 × 76, including 21 well
Results and discussion
Table 1 gives results of ScSPM, VScSPM and JScSPM for multi-class task with HIK-SVM. It is observed that JScSPM has the best performance with mean classification accuracy, sensitivity and specificity being 91.95 ± 1.29%, 91.70 ± 1.34% and 95.89 ± 0.68%, respectively. The results of CSK-SVM in Table 2 are consistent to those of HIK-SVM. The mean classification accuracy, sensitivity, and specificity results of JScSPM are 91.52 ± 1.36%, 91.39 ± 1.34% and 95.73 ± 0.67%, respectively. JScSPM significantly
Conclusions
In conclusion, we propose a joint SC based ScSPM algorithm for the classification of color medical images. The results indicate that JScSPM outperforms the original ScSPM and VScSPM algorithms. The joint SC used in JScSPM can easily transform a grayscale feature descriptor to a color one without special design. Therefore, it has the potential for more applications not only in color medical images but also in other multi-view or vector-valued images.
The joint SC with JSM-1 in JScSPM can jointly
Acknowledgements
This work is partly supported by the Shanghai Municipal Natural Science Foundation (12ZR1410800) and the Innovation Program of Shanghai Municipal Education Commission (13YZ016). The authors are grateful to Professor Shuozhong Wang for his assistance in improving the language usage.
References (28)
- et al.
A high-throughput active contour scheme for segmentation of histopathologcial imagery
Med Image Anal
(2011) - et al.
Sparse representation and learning in visual recognition: theory and applications
Signal Process
(2013) - et al.
Sparse coding based visual tracking: review and experimental comparison
Patt Recog
(2013) Histopathology of liver cancers
Best Pract Res Clin Gastroent
(2005)- et al.
Computational grading of hepatocellular carcinoma using multifractal feature description
Comput Med Imaging Graph
(2013) - et al.
Integrative multimodal analysis of glioblastoma using TCGA molecular data pathology images and clinical outcomes
IEEE Trans Biomed Eng
(2011) - et al.
Histopathological image analysis: a review
IEEE Rev Biomed Eng
(2009) - et al.
Sparse representation for computer vision and pattern recognition
Proc IEEE
(2010) - et al.
Robust face recognition via sparse representation
IEEE Trans Patt Anal Mach Int
(2009) - et al.
Sparse representation approaches for the classification of high dimensional biological data
BMC Syst Biol
(2013)
Linear spatial pyramid matching using sparse coding for image classification
CVPR
Beyond bags of features: spatial pyramid matching for recognizing natural scene categories
CVPR
Image classification by non-negative sparse coding, low-rank and sparse decomposition
CVPR
Laplacian sparse coding, hypergraph Laplacian sparse coding, and applications
IEEE Trans Patt Anal Mach Int
Cited by (20)
Multiscale superpixel method for segmentation of breast ultrasound
2020, Computers in Biology and MedicineCitation Excerpt :The multiscale process solves the parameter setting problem and reduces the computation time. In the literature, several multiscale algorithms are available, one of which is the Laplacian Pyramid (LP) [24]. Although the LP generates good multiscale results, it does not preserve edges.
Histopathological image classification through discriminative feature learning and mutual information-based multi-channel joint sparse representation
2020, Journal of Visual Communication and Image RepresentationCitation Excerpt :However, these have been neglected in the above-mentioned methods. To address this problem, Shi et al. [42] adopted the joint sparsity model (JSM) [41] to construct a joint dictionary that can encode the common component and particular components. However, the sub-dictionaries of common and particular components in a joint dictionary was same, which results in the high similarity between sparse features [42].
Quaternion Grassmann average network for learning representation of histopathological image
2019, Pattern RecognitionCitation Excerpt :We then evaluate GANet and QGANet respectively on the following three histopathological image datasets, whose typical example images are shown in Fig. 4. Hepatocellular carcinoma (HCC) image dataset [59]. This HCC dataset was acquired by the Olympus BX51 at the Medical College of Nantong University.
Multi-channel EEG-based sleep stage classification with joint collaborative representation and multiple kernel learning
2015, Journal of Neuroscience MethodsCitation Excerpt :Joint SR (JSR) codes that contain multi-view data information can be generated with JSM. JSM has also been successfully applied to multi-view or multi-modality image processing and analysis (Yuan et al. 2010; Zhang et al., 2012a,b; Yu et al., 2013; Shekhar et al., 2014; Shi et al., 2015). In contrast to most SR algorithms that emphasize the role of L1-norm sparsity constraint, collaborative representation (CR) uses the whole training samples to represent the query sample with non-sparse L2-norm (Zhang et al., 2011).
Machine learning in medical imaging
2015, Computerized Medical Imaging and Graphics