Weakly supervised histopathology cancer image segmentation and classification

Highlights

• We propose a new learning method, multiple clustered instance learning (MCIL), along the line of weakly supervised learning.

• The proposed MCIL simultaneously performs image classification (cancer vs. non-cancer image), segmentation, and clustering.

• We embed clustering into MIL and derive a principled solution to performing the three tasks in an integrated framework.

• We introduce contextual constraints as a prior for MCIL, which significantly reduces the ambiguity in multiple instance learning.

Abstract

Labeling a histopathology image as having cancerous regions or not is a critical task in cancer diagnosis; it is also clinically important to segment the cancer tissues and cluster them into various classes. Existing supervised approaches for image classification and segmentation require detailed manual annotations for the cancer pixels, which are time-consuming to obtain. In this paper, we propose a new learning method, multiple clustered instance learning (MCIL) (along the line of weakly supervised learning) for histopathology image segmentation. The proposed MCIL method simultaneously performs image-level classification (cancer vs. non-cancer image), medical image segmentation (cancer vs. non-cancer tissue), and patch-level clustering (different classes). We embed the clustering concept into the multiple instance learning (MIL) setting and derive a principled solution to performing the above three tasks in an integrated framework. In addition, we introduce contextual constraints as a prior for MCIL, which further reduces the ambiguity in MIL. Experimental results on histopathology colon cancer images and cytology images demonstrate the great advantage of MCIL over the competing methods.

Introduction

Histopathology image analysis is a vital technology for cancer recognition and diagnosis (Tabesh et al., 2007, Park et al., 2011, Esgiar et al., 2002, Madabhushi, 2009). High resolution histopathology images provide reliable information differentiating abnormal tissues from the normal ones. In this paper, we use tissue microarrays (TMAs) which are referred to histopathology images here. Fig. 1 shows a typical histopathology colon cancer image, together with a non-cancer image. Recent developments in specialized digital microscope scanners make digitization of histopathology readily accessible. Automatic cancer recognition from histopathology images thus has become an increasingly important task in the medical imaging field (Esgiar et al., 2002, Madabhushi, 2009). Some clinical tasks (Yang et al., 2008) for histopathology image analysis include: (1) detecting the presence of cancer (image classification); (2) segmenting images into cancer and non-cancer region (medical image segmentation); (3) clustering the tissue region into various classes. In this paper, we aim to develop an integrated framework to perform classification, segmentation, and clustering altogether.

Several practical systems for classifying and grading cancer histopathology images have been recently developed. These methods are mostly focused on the feature design including fractal features (Huang and Lee, 2009), texture features (Kong et al., 2009), object-level features (Boucheron, 2008), and color graphs features (Altunbay et al., 2010, Ta et al., 2009). Various classifiers (Bayesian, KNN and SVM) are also investigated for pathological prostate cancer image analysis (Huang and Lee, 2009).

From a different angle, there is a rich body of literature on supervised approaches for image detection and segmentation (Viola and Jones, 2004, Shotton et al., 2008, Felzenszwalb et al., 2010, Tu and Bai, 2010). However, supervised approaches require a large amount of high quality annotated data, which are labor-intensive and time-consuming to obtain. In addition, there is intrinsic ambiguity in the data delineation process. In practice, obtaining the very detailed annotation of cancerous regions from a histopathology image could be a challenging task, even for expert pathologists.

Unsupervised learning methods (Duda et al., 2001, Loeff et al., 2005, Tuytelaars et al., 2009), on the other hand, ease the burden of having manual annotations, but often at the cost of inferior results.

In the middle of the spectrum is the weakly supervised learning scenario. The idea is to use coarsely-grained annotations to aid automatic exploration of fine-grained information. The weakly supervised learning direction is closely related to semi-supervised learning in machine learning (Zhu, 2008). One particular form of weakly supervised learning is multiple instance learning (MIL) (Dietterich et al., 1997) in which a training set consists of a number of bags; each bag includes many instances; the goal is to learn to predict both bag-level and instance-level labels while only bag-level labels are given in training. In our case, we aim at automatically learning image models to recognize cancers from weakly supervised histopathology images. In this scenario, only image-level annotations are required. It is relatively easier for a pathologist to label a histopathology image than to delineate detailed cancer regions in each image.

In this paper, we develop an integrated framework to classify histopathology images as having cancerous regions or not, segment cancer tissues from a cancer image, and cluster them into different types. This system automatically learns the models from weakly supervised histopathology images using multiple clustered instance learning (MCIL), derived from MIL. Many previous MIL-based approaches have achieved encouraging results in the medical domain such as major adverse cardiac event (MACE) prediction (Liu et al., 2010), polyp detection (Dundar et al., 2008, Fung et al., 2006, Lu et al., 2011), pulmonary emboli validation (Raykar et al., 2008), and pathology slide classification (Dundar et al., 2010). However, none of the above methods aim to perform medical image segmentation. They also have not provided an integrated framework for the task of simultaneous classification, segmentation, and clustering.

We propose to embed the clustering concept into the MIL setting. The current literature in MIL assumes single cluster/model/classifier for the target of interest (Viola et al., 2005), single cluster within each bag (Babenko et al., 2008, Zhang and Zhou, 2009, Zhang et al., 2009), or multiple components of one object (Dollár et al., 2008). Since cancer tissue clustering is not always available, it is desirable to discover/identify the classes of various cancer tissue types; this results in patch-level clustering of cancer tissues. The incorporation of clustering concept leads to an integrated system that is able to simultaneously perform image segmentation, image-level classification, and patch-level clustering.

In addition, we introduce contextual constraints as a prior for cMCIL, which reduces the ambiguity in MIL. Most of the previous MIL methods make the assumption that instances are distributed independently, without considering the correlations among instances. Explicitly modeling the instance interdependencies (structures) can effectively improve the quality of segmentation. In our experiment, we show that while obtaining comparable results in classification, cMCIL improves the segmentation significantly (over 20%) compared MCIL. Thus, it is beneficial to explore the structural information in the histopathology images.