Multi-shape graph cuts with neighbor prior constraints and its application to lung segmentation from a chest CT volume

Abstract

This paper presents a novel graph cut algorithm that can take into account multi-shape constraints with neighbor prior constraints, and reports on a lung segmentation process from a three-dimensional computed tomography (CT) image based on this algorithm. The major contribution of this paper is the proposal of a novel segmentation algorithm that improves lung segmentation for cases in which the lung has a unique shape and pathologies such as pleural effusion by incorporating multiple shapes and prior information on neighbor structures in a graph cut framework. We demonstrate the efficacy of the proposed algorithm by comparing it to conventional one using a synthetic image and clinical thoracic CT volumes.

Introduction

Accurate segmentation is a prerequisite for quantitative lung computed tomography (CT) image analysis and also for computer-aided diagnoses. The segmentation method described in this paper will be built in the development of a computer aided diagnosis (CAD) system for a thoracic CT volume. Many CAD systems that detect and/or quantify pathologies require accurate segmentation of the target organ as its initial step. In this paper, we propose a method that aims to accurately extract lungs that contain pathologies such as tumors, emphysema, and diffuse pulmonary disease, as well as pleural effusion. In this study, we included pleural effusion as a segmentation target for the following two reasons: First, the segmentation of pleural effusion is beneficial in diagnosis because the mere existence of pleural effusion is usually symptomatic of pathologies such as tumors, tuberculosis, thoracic empyema, and pneumonia. Its presence, location, and volume are important factors that can assist a physician during diagnosis (Light, 2008). Second, it facilitates the usage of a statistical shape model (SSM) in the segmentation process (Cremers et al., 2007, Heimann and Meinzer, 2009). A pleural effusion is fluid that builds up in a pleural cavity (Fig. 1b), which is space that is usually occupied by the lungs in healthy subjects. Therefore, by considering the combination of a lung and a pleural effusion as one region, the shape of the region is identical to that of the lungs of a subject not afflicted with pleural effusion. The shape in Fig. 1c, reconstructed from a SSM that learned from subjects without pleural effusion, is beneficial in the segmentation of the region comprising a lung and a pleural effusion.

Many methods for automatically extracting the lung regions from three-dimensional (3D) CT volumes have been proposed (Sluimer et al., 2006). Since a normal lung appears dark in a CT image and is surrounded by denser regions, most methods focus on this contrast information. Although such methods are known to be simple and effective (Hu et al., 2001), they often fail to extract regions affected by pathologies, especially when the pathologies are attached to pleura of the lungs. This is a challenging problem because many pathologies exhibit different properties when compared with normal tissues, and we often find that the region of pathology cannot be recovered by post-processing such as morphological operations or holes filling. In addition, contrast-based methods also fail to extract pleural effusion (Fig. 1d) because it is denser than normal lung tissue.

Other approaches use different features of lungs, such as shape, rather than contrasts to extract lungs with pathologies. Sluimer et al. (2008) proposed a registration-based approach in which a shape template is registered to an input CT volume. They achieved significant improvements in the segmentation of lungs with pathologies, but the algorithm is time-consuming because of the combination of the registration and classification processes. In addition, it suffers from low accuracy in segmentation due to errors in registration and classification. Kido and Tsunomori (2009) proposed another registration based method using a template obtained from a normal case. Two step matching improved the performance of the case with severe plural effusion but still suffered from error in the registration. Hua et al. (2011) proposed a method that combines the classification process with a graph-search algorithm. The method has been shown to be effective in cases containing pathologies. However, the construction of the graph is limited to a narrow band around the pre-segmented lung surface obtained from the gray-value statistics-based method, which might fail in cases with large pathologies. One of the workshops held in conjunction with the Medical Image Computing and Computer Assisted Intervention (MICCAI) conference in 2011 hosted a segmentation challenge called Lobe and Lung Analysis (LOLA). Its goal was to compare methods for (semi-)automatic segmentation of the lungs from chest CT scans. The results of the contest showed that methods based on contrast information got high marks and proved effective in segmentation of the lungs. Although their dataset covered various pathologies of the lungs, the target region to be segmented did not include pleural effusion. Therefore, our target is different from that of LOLA 2011.

With regards to the segmentation of pleural effusion, a few of methods have been proposed in the past. Yao et al. (2009) proposed a stepwise method to extract pleural effusion using the pre-segmented lung surface obtained from the gray-value statistics-based process followed by visceral and parietal pleura detection based on active contour models. Their process could not extract both lung and pleural effusion simultaneously, which is different from our approach. Furthermore, the segmentation process fails when the lung is not extracted correctly. In addition, the continuity of the pleura surface between slices was not maintained due to the use of 2D active contour models. Moreover, their experiment was limited to cases with pleural effusion.

In this paper, we propose an s–t graph cut-based (Boykov and Funka-Lea, 2006) segmentation algorithm, with multiple-shape and neighbor priors (i.e., the aorta and the body cavity), which can optimize the energy function defined in an entire CT volume. A graph cut algorithm is a noteworthy development in which the global minimum for a sub-modular function of a binary label problem is guaranteed. Furthermore, a number of researchers have introduced a shape prior into the graph cut approach in order to obtain accurate segmentation (Freedman and Zhang, 2005, Slabaugh and Unal, 2005, Funka-Lea et al., 2006). These papers propose several ways in which a general shape constraint such as an ellipse, a star, or an arbitrary shape defined by the user can be incorporated. Shimizu et al. (2010) combined a patient-specific shape estimated by a statistical shape model (SSM) with graph cuts. However, all of the above methods consider single-shape information only, which might be different from the true shape, thus resulting in inadequate performance. Combining multiple-shape information helps to reduce such differences. Linguraru et al. (2012) proposed a shape-based energy computed by the Parzen window method which is a population statistic based method but the combinatorial problem of multiple shape information remains unsolved. In addition to such shape a priori, neighbor constrained energy was also introduced in a graph cut approach by Shimizu et al. (2010) to boost segmentation performance. It incorporated a prior knowledge about the region to be segmented based on neighbor structures (i.e. the dorsal ribs) and was shown to be effective in segmentation. However, the energy still suffered from low accuracy in lung segmentation as a result of the various patterns in shape, especially in the area close to the aorta.

The major contribution of this paper is the proposal of a graph cut-based segmentation algorithm that improves lung segmentation by incorporating multiple shapes and prior information on neighbor structures of the lungs (i.e., the aorta and the body cavity). In this paper, we extend the term “lungs” to refer not only to the region of normal lung tissues, but also to the regions of pathological abnormality, including pleural effusion. By permitting a graph cut algorithm to consider multiple shape priors, our method improves lung segmentation without relying too strongly on a single shape prior. Moreover, we introduce novel neighbor constrained energy terms to extract the lungs with various shapes and appearances caused by differences in shape and pathologies, including pleural effusion.

In the remainder of this paper, we will explain the details of our proposed method and demonstrate its efficacy using the experimental results of a synthetic image and 97 thoracic CT volumes taken in daily clinical routine. These CT volumes contain pulmonary diseases such as a lung tumor, emphysema, diffuse pulmonary disease, and a pleural effusion; and include both non-contrast and contrast CT volumes. The quantitative evaluation is performed using an average of the Jaccard Index of the lungs and the sensitivity of a lesion attached to the chest wall. This is followed by a discussion of the effectiveness of our proposed method.