Feature extraction from histopathological images based on nucleus-guided convolutional neural network for breast lesion classification
Introduction
Breast cancer is the second most commonly occurring cancer for females. According to an estimation from the American Cancer Society in 2017 [1], around 30% of the new cancer cases in American women consist of breast cancer. New technologies [2], [3], [4] for breast cancer diagnosis have been developed in recent years. However, final diagnosis still currently relies on biopsies [5]. Owing to the development in digital pathology, the whole slide image (WSI) of a histopathological section can be captured within a few minutes by a micro scanner, and stored in a data server. These cancer images provide valuable data sources for researches in computer science and related areas, such as image processing, pattern recognition, and data mining. As a result, many approaches to histopathological image analysis [6], [7], [8] have been developed and applied to practical clinical diagnoses, relieving the workload of pathologists and assisting pathologists in making more reliable and consistent diagnoses.
Across various applications of histopathological image analysis, image classification and content-based image retrieval (CBIR) are always important challenges. Histopathological image classification aims to directly predict the types of lesions (for example, the classification of benign and malignant tumors). A reliable classification system can provide a verification of the diagnosis of the doctor. Meanwhile, CBIR can search and return cases with similar content to the query image. Using information regarding diagnoses of similar cases for reference, doctors can reach a more reliable diagnosis for the query image. For these applications, feature extraction plays a crucial role. However, histopathological images may contain hundreds of structures [9] and the appearance within the same lesion is varied [10], which makes feature extraction a challenging task.
In this paper, a novel nuclei-guided feature extraction method based on convolutional neural network is proposed for histopathological images stained by hematoxylin and eosin (HE). The nuclei in the image are first detected. Considering the location information of the nuclei, a fine-designed neural network is trained to extract features regarding the pattern and distribution of the nuclei. Through classification experiments on a breast-lesion database, the proposed features are validated to be effective for histopathological images.
The remainder of this paper is organized as follows. Section 2 reviews relevant work regarding histopathological image analysis. Section 3 introduces the proposed feature extraction method. The experiment and discussion are presented in Sections 4 and 5. Finally, Section 6 summarizes the present contributions and suggests directions for future work.
Section snippets
Related work
Many feature extraction methods have been proposed for histopathological images. They can be broadly classified into two categories: statistics based method and learning based method.
Inspired by the success of natural image analysis, some researchers [11], [12], [13], [14] have employed classical feature descriptors, such as color histogram, scale-invariant feature transform (SIFT) [15], histogram of oriented gradient (HOG) [16], and local binary pattern (LBP) [17], to depict histopathological
Method
According to pathologists [5], [43], the appearance of the cell nucleus and its surrounding cytoplasm, as well as the distribution of nuclei, are important indicators for cancer diagnosis. Therefore, both the appearance and distribution of nuclei are considered. The proposed framework consists of an offline training stage and an online encoding stage, where the former can be divided into three steps: nuclei detection, pre-training, and fine-tuning.
Fig. 1 presents a flowchart of the training
Experiment
In this paper, a novel nucleus-guided feature extraction framework based on CNN is proposed for histopathological images. The proposed algorithm is implemented in Matlab 2013a on the PC with a 12 Intel Core Processor (2.10 GHz) and a GPU of Nvidia Tesla k40. The implementation of the whole network is based on the UFLDL tutorial.1
The performance of the proposed method is evaluated using a fine-annotated histopathological image
Discussion
In this paper, the nucleus features including the pattern and spatial distribution are extracted from histopathological images using a designed CNN combined with the guide of nuclei, which has good classification performance for breast lesions. Regarding the proposed method, we have the following discussion:
Conclusion
The appearance and spatial distribution of cell nuclei are significant indicators for cancer diagnosis, according to which a nucleus-guided feature extraction framework based on a convolutional neural network has been proposed for histopathological image representation and classification in this paper. The nuclei are first detected from the histopathological image, and then used to guide the training of the convolutional neural network. With the nucleus-guided strategy, the network paid more
Acknowledgment
This work was supported by the National Natural Science Foundation of China (No. 61371134 and 61471016) and project of Motic-BUAA Image Technology Research and Development Center.
Yushan Zheng received the B.E. and M.S. degree from Beihang University in 2012 and 2015 respectively. He is currently a Ph.D. candidate majored in Pattern Recognition and Intelligent System at Beihang University. His research interests include medical image processing, analysis, classification and retrieval.
References (50)
- et al.
Large-scale medical image analytics: recent methodologies, applications and future directions
Med. Image Anal.
(2016) - et al.
Computer-aided prognosis of neuroblastoma on whole-slide images: classification of stromal development
Pattern Recognit.
(2009) - et al.
Weakly supervised histopathology cancer image segmentation and classification
Med. Image Anal.
(2014) - et al.
A comparative study of texture measures with classification based on feature distributions
Pattern Recognit.
(1996) - et al.
Computer-aided diagnosis of breast cancer based on fine needle biopsy microscopic images
Comput. Biol. Med.
(2013) Texture measures combination for improved meningioma classification of histopathological images
Pattern Recognit.
(2010)- et al.
Computer-aided diagnosis from weak supervision: a benchmarking study
Comput. Med. Imaging Graph.
(2015) - et al.
An unsupervised feature learning framework for basal cell carcinoma image analysis
Artif. Intell. Med.
(2015) - et al.
A deep convolutional neural network for segmenting and classifying epithelial and stromal regions in histopathological images
Neurocomputing
(2016) - et al.
High-throughput histopathological image analysis via robust cell segmentation and hashing
Med. Image Anal.
(2015)
Cancer statistics, 2017
CA
Identifying relations between imaging phenotypes and molecular subtypes of breast cancer: model discovery and external validation
J. Magn. Reson. Imaging
Unsupervised clustering of quantitative image phenotypes reveals breast cancer subtypes with distinct prognoses and molecular pathways
Clin. Cancer Res.
Robbins and Cotran Pathologic Basis of Disease
Computer-aided prostate cancer diagnosis from digitized histopathology: a review on texture-based systems
IEEE Rev. Biomed. Eng.
Medical image analysis: progress over two decades and the challenges ahead
IEEE Trans. Pattern Anal. Mach. Intell.
Histopathological image analysis: a review
IEEE Rev. Biomed. Eng.
Automated detection of dcis in whole-slide h&e stained breast histopathology images
IEEE Trans. Med. Imaging
WHO Classification of Tumours of the Breast
Histopathology image classification using bag of features and kernel functions
Artificial Intelligence in Medicine in Europe
Localization of diagnostically relevant regions of interest in whole slide images
International Conference on Pattern Recognition
Distinctive image features from scale-invariant keypoints
Int. J. Comput. Vision
Histograms of oriented gradients for human detection
IEEE Conference on Computer Vision and Pattern Recognition
Computerized classification of intraductal breast lesions using histopathological images
IEEE Trans. Biomed. Eng.
Automated analysis and diagnosis of skin melanoma on whole slide histopathological images
Pattern Recognit.
Cited by (106)
Interpretable pap-smear image retrieval for cervical cancer detection with rotation invariance mask generation deep hashing
2023, Computers in Biology and MedicineAn advanced W-shaped network with adaptive multi-scale supervision for osteosarcoma segmentation
2023, Biomedical Signal Processing and ControlA review: The detection of cancer cells in histopathology based on machine vision
2022, Computers in Biology and MedicineCitation Excerpt :Stanitsas et al. [145] used the AlexNet to classify pathological images, which greatly reduced the requirements for graphics processing unit memory, thus avoiding the need for complex hardware. Zheng et al. [146] proposed a kernel-guided feature extraction framework based on CNN for the representation and classification of breast histopathology images with an average classification accuracy of 96.4%. Han et al. [147] adopted GoogleNet to significantly improve the classification performance through 22 layers deep network and novel initial module.
Iterative brain tumor retrieval for MR images based on user's intention model
2022, Pattern RecognitionCitation Excerpt :This implies a need to design a feature extraction framework to encode/combine both low-level and high-level features. One way to design such a framework is to use the deep learning that can extract powerful feature representations fully describing both the low-level and high-level information and embed the feature extraction phase into self-learning [20]. Nowadays, deep learning has been applied in MR image analysis such as segmentation [21,22], brain tumor classification [23,24], computer aided diagnosis and prediction [25], brain image organization [26].
Dermoscopic image retrieval based on rotation-invariance deep hashing
2022, Medical Image Analysis
Yushan Zheng received the B.E. and M.S. degree from Beihang University in 2012 and 2015 respectively. He is currently a Ph.D. candidate majored in Pattern Recognition and Intelligent System at Beihang University. His research interests include medical image processing, analysis, classification and retrieval.
Zhiguo Jiang received the B.E., M.S., and Ph.D. degrees from the Beihang University in 1987, 1990, and 2005, respectively. He is currently a professor in Image Processing Center, School of Astronautics, Beihang University. His research interests include medical image processing and classification, remotely sensed image processing, target detection and recognition.
Fengying Xie received the Ph.D. Degree in Pattern Recognition and Intelligent System from Beihang University in 2009. She is now a professor in Image Processing Center, School of Astronautics, Beihang University. Her research interests include biomedical image and remote sensing image processing, image quality assessment, image segmentation and classification.
Haopeng Zhang received his B.S. and Ph.D. degrees from Beihang University in 2008 and 2014, respectively, and is currently with the Image Processing Center, School of Astronautics, Beihang University. His main research interests are multi-view object recognition, 3D object recognition and pose estimation, etc.
Yibing Ma received the B.E. degree from Beihang University in 2011. He is currently a Ph.D. candidate majored in Pattern Recognition and Intelligent System at Beihang University. His research interests include medical image processing, analysis, classification and retrieval.
Huaqiang Shi is a deputy chief physician of the General Hospital of the Air Force of PLA. He is also the medical advisor of Motic (Xiamen) Medical Diagnostic Systems Co. Ltd. His research interests are diagnosis of breast cancer and the clinical application of image analysis technology.
Yu Zhao is the Software R&D Manager of Motic (Xiamen) Medical Diagnostic Systems Co. Ltd. His research interests are clinical application of image analysis technology, telemedicine system and content-based medical image retrieval.