Quaternion Grassmann average network for learning representation of histopathological image

Abstract

Histopathological image analysis works as ‘gold standard’ for cancer diagnosis. Its computer-aided approach has attracted considerable attention in the field of digital pathology, which highly depends on the feature representation for histopathological images. The principal component analysis network (PCANet) is a novel unsupervised deep learning framework that has shown its effectiveness for feature representation learning. However, PCA is susceptible to noise and outliers to affect the performance of PCANet. The Grassmann average (GA) is superior to PCA on robustness. In this work, a GA network (GANet) algorithm is proposed by embedding GA algorithm into the PCANet framework. Moreover, since quaternion algebra is an excellent tool to represent color images, a quaternion-based GANet (QGANet) algorithm is further developed to learn effective feature representations containing color information for histopathological images. The experimental results based on three histopathological image datasets indicate that the proposed QGANet achieves the best performance on the classification of color histopathological images among all the compared algorithms.

Introduction

Cancers seriously threaten human health, and therefore it is always highly demanded for the most accurate diagnosis of cancers. It is well known that the high-resolution histopathological image is a ‘gold standard’ for the diagnosis of almost all types of cancer in clinical practice [1], [2]. However, this kind of pathology diagnosis is highly dependent on the subjective decisions of pathologists [2], [3]. Therefore, it is necessary to develop a computer-aided diagnosis (CAD) or quantitative analysis system for histopathological image analysis, which can help the pathologists reduce the workload, and more importantly, the possibility of errors, as a second opinion tool [1], [2], [3], [4], [5], [6], [7]. Moreover, it also can provide an effective decision support tool for the training of young radiologists [1], [2], [3], [4], [5], [6], [7].

In a CAD system, feature representation is one of the most critical steps. For example, the commonly used features in histopathological images mainly represent the local cell-level information (e.g., size and shape) or the holistic architecture of tissue (e.g., topology and layout of all cells) [1], [5]. For more details on the hand-crafted features based histopathological image classification, we refer to [1], [3], [5], [8], [9], [10]. Although a remarkable progress of various feature extraction methods has been achieved, the hand-crafted features mostly have poor generalization ability among the histopathological images of different cancers [11], [12]. Therefore, feature representation has become a fairly decisive factor in histopathological image based CAD, while it is still very challenging [12].

In the past several years, deep learning (DL) has demonstrated superior performance to the hand-crafted features in various applications [13], [14], [15]. It has also gained a good reputation in the feature representation of histopathological images [6], [7], [12]. Ciresan et al. first used convolutional neural networks (CNN) to detect mitosis in breast histopathological images and won the mitosis detection competition in ICPR 2012 contest [16]. Then, various CNN-based DL algorithms have been developed for the purpose of cell detection and counting, cell segmentation and tissue classification for histopathological images [3], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32]. Except for the CNN framework, other DL algorithms, including the stacked auto-encoder (SAE), restricted Boltzmann machine (RBM), deep belief networks (DBN) and their variants, have also been successfully applied to histopathological images, especially for cell detection [8], [33], [34], [35], [36], [37]. All these DL-based methods show superior or even state-of-the-art performance for histopathological images.

More recently, Chan et al. proposed a principal component analysis network (PCANet) algorithm [38]. PCANet is essentially an unsupervised DL framework, which only has three simple basic components: the cascaded PCA as a deep network, binary hashing as a nonlinear layer, and block-wise histograms for feature pooling layer. PCANet has shown its effectiveness for feature representation learning. Moreover, compared with the commonly used DL algorithms, PCANet has the advantages of simpler network architecture and fewer parameters. Therefore, PCANet and its variants, such as DLANet [39], SRDANet [40], SPCANet [41], 2DPCANet [42], MPCANet [43], and R-VCANet [44], have been widely used for various image classification tasks.

The effectiveness of PCANet for image representation makes it feasible for histopathological image analysis. Moreover, since PCANet is an unsupervised learning algorithm, it is more suitable for medical image processing in the small dataset case. On the other hand, in histopathological images, the noises and outliers are inevitable, and they can result from artifact, non-standardized staining protocol, focusing inaccuracy, diversity of imaging devices, diversity of resolution, etc. [1], [9]. Therefore, some small regions in a histopathological image may be affected by noise and outliers. PCA is sensitive to noise and outliers, resulting in the degeneration of robustness [45], and then further affects the ability of the feature representation in PCANet. Therefore, it is critical to improve the robustness of the PCA in the PACNet framework. Fortunately, Grassmann averages (GA) can address this issue by averaging all subspaces generated by the data on the Grassmann manifold to realize dimensionality reduction [46]. It is worth noting that GA is consistent with PCA in the Gaussian data case, while it is more robust than PCA [46]. The experimental results have shown that GA outperforms PCA on several tasks in computer vision, such as dimensionality reduction, background modeling, image restoration and shadow removal [46]. Since histopathological images always carry noise and outliers during the imaging procedure, GA algorithm would have superior performance over PCA for the dimensionality reduction of histopathological images. Moreover, inspired by the PCANet framework, we will construct a GA based network (GANet) with the similar network framework in this work.

Hematoxylin and Eosin (H&E) staining is commonly used to reveal cellular components and enhance the visibility of spatial structures of histological components in clinical practice, which makes the color information particularly important and helpful for diagnosis. As a result, many color descriptors have been widely used to represent histopathological images [1], [2]. However, the existing GA algorithm can only handle grayscale images. When a color histopathological image is converted into the grayscale one, rich color information will be lost, which leads to deteriorated feature representation. Although we can conduct the GANet algorithm on each color channel image and then concatenate them to form the fused color features, the intrinsic correlation among the color channels is discarded by this color information fusion strategy [1]. Therefore, it is essential to integrate color information into GANet such that more effective representation of histopathological images can be learned.

Quaternion algebra was first proposed by Hamilton in 1866 [48]. It has been demonstrated that quaternion algebra is excellent at color image representation mathematically. It comprises one real and three imaginary parts, and these three imaginary parts are used to represent the three different color channels, respectively. Various quaternion-based feature extraction algorithms for color images have been proposed in recent years [49], [50], [51], [52], [53], especially the quaternion PCA (QPCA) [54]. More recently, Zeng et al. proposed the quaternion PCANet that outperforms PCANet for the feature representation of color images [55]. A novel algorithm, named quaternion GA (QGA), is therefore considered by integrating the quaternion algebra and the GA algorithm. We will further develop a novel QGA network (QGANet) algorithm according to the PCANet framework [46].

In this work, we develop a QGANet algorithm that can effectively learn the feature representation of the color histopathological images for cancer diagnosis based on our previous work [56]. The main contributions of this work are threefold: (1) A GANet algorithm is first proposed to improve the robustness of the learned representative features for gray histopathological images; (2) The QGA algorithm is then developed by incorporating the quaternion algebra into GA algorithm for the dimensionality reduction of color images; (3) The QGANet algorithm, which can learn more effective feature representations from color histopathological images, is finally proposed.