Lens structure segmentation from AS-OCT images via shape-based learning

Highlights

• Transform segmentation task from classification to regression by making models learn a level set function.

• Design a shape-based loss with a convexity constraint term.

• Improve segmentation performance at the boundary with weak contrast.

Abstract

Background and objectives

The lens is one of the important refractive media in the eyeball. Abnormality of the nucleus or cortex in the lens can lead to ocular disorders such as cataracts and presbyopia. To achieve an accurate diagnosis, segmentation of these ocular structures from anterior segment optical coherence tomography (AS-OCT) is essential. However, weak-contrast boundaries of the object in the images present a challenge for accurate segmentation. The state-of-the-art (SOTA) methods, such as U-Net, treat segmentation as a binary classification of pixels, which cannot handle pixels on weak-contrast boundaries well.

Methods

In this paper, we propose to incorporate shape prior into a deep learning framework for accurate nucleus and cortex segmentation. Specifically, we propose to learn a level set function, whose zero-level set represents the object boundary, through a convolutional neural network. Moreover, we design a novel shape-based loss function, where the shape prior knowledge can be naturally embedded into the learning procedure, leading to improvement in performance. We collect a high-quality AS-OCT image dataset with precise annotations to train our model.

Results

Abundant experiments are conducted to verify the effectiveness of the proposed framework and the novel shape-based loss. The mean Intersection over Unions (MIoUs) of the proposed method for lens nucleus and cortex segmentation are 0.946 and 0.957, and the mean Euclidean Distance (MED) measure, which can reflect the accuracy of the segmentation boundary, are 6.746 and 2.045 pixels. In addition, the proposed shape-based loss improves the SOTA models on the nucleus and cortex segmentation tasks by an average of 0.0156 and 0.0078 in the MIoU metric and 1.394 and 0.134 pixels in the MED metric.

Conclusion

We transform the segmentation from a classification task to a regression task by making the model learn the level set function, and embed shape information in deep learning by designing loss functions. This allows the proposed method to be more efficient in the segmentation of the object with weak-contrast boundaries.

Concise abstract

We propose to incorporate shape priors into a deep learning framework for accurate nucleus and cortex segmentation from AS-OCT images. Specifically, we propose to learn a level set function, where the zero-level set represents the boundary of the target. Meanwhile, we design a novel shape-based loss function in which additional convex shape prior can be embedded in the learning process, leading to an improvement in performance. The IOUs for nucleus and cortex segmentation are 0.946 and 0.957, while the MED that reflects the accuracy of the boundary are 6.746 and 2.045 pixels. The proposed shape-based loss improves the SOTA model for nucleus and cortex segmentation by an average of 0.0156 and 0.0078 in IOU, and 1.394 and 0.134 pixels in MED. We transform segmentation from classification to regression by making the model learn a level set function, resulting in improved performance at the boundary with weak contrast.

Introduction

The crystalline lens is one of the important refractive media in the eyeball, which is a biconvex lens with elasticity. The shape of the lens can be adjusted by the ciliary muscle to focus on nearby or far objects. With age, the nucleus of the lens progressively condenses, expands, and loses its elasticity. Therefore, the adjusting ability of the lens becomes poor, and presbyopia occurs [1]. In addition to presbyopia, cataract is also caused by abnormalities in the lens. Cataract is the largest contributor to global blindness in adults aged 50 years and older in 2020 and is approximately 45% of the cases of global blindness [2]. The primary reason for cataracts is lens opacity caused by the degeneration of lens proteins, which prevents light from being projected onto the retina, resulting in blurred vision. In clinical practice, a standard indicator of cataract grading is the Lens Opacities Classification System III (LOCS III) [3]. However, LOCS III is relatively subjective and the outcomes are affected by the experience of the ophthalmologists. Recently, many studies have focused on the correlation between LOCS III and quantitative nuclear densities calculated from anterior segment optical coherence tomography (AS-OCT) [4,5]. Moreover, nuclear density is proposed to be an objective metric for cataract grading [6,7].

Accurate nucleus segmentation is very important to calculate the nuclear density from AS-OCT. Meanwhile, the segmentations of the nucleus and cortex are also meaningful for the automatic diagnosis of presbyopia and other diseases. However, it is challenging to segment the accurate nucleus because the boundaries between the cortex and the nucleus are fuzzy (see Fig. 1(B)). Zhang et al. [8] proposed a guide-based M-shape convolutional network (G-MNet) to segment the capsule, cortex, and nucleus from AS-OCT images. The G-MNet could segment the high-resolution images guided by the multi-scale and low-resolution segmentation outputs. Cao et al. [9] first extracted the lens area and then segmented the nuclear structure by using a ShuffleSeg network. Finally, they adopted a curve fitting processing to improve the nucleus segmentation. Although these methods can produce nucleus segmentation, low accuracy results are still produced due to the low contrast between the target and background in the image and the lack of object shape constraints in the methods. With the help of the shape information, the nucleus segmentation results can be improved. Thus, Yin et al. [10] proposed a two-step framework by using a deep learning network followed by a post-processing step with a shape template. However, due to individual differences, the segmentation results obtained by the shape template method could not fit the object boundary of each sample well.

The level set is a non-parametric shape representation. In the segmentation task, the segmented boundary can be represented as the zero-level set of the high-dimensional surface corresponding to the level set function [11]. Recently, researchers focus on combining the level set method and deep learning frameworks. The common combination method is considering the output of the deep frameworks as an initial shape, and it can be evolved by level set function to finetune the segmentation result. In addition, the level set methods can be considered recurrent processing, so some algorithms [12], [13], [14] incorporated the level set with the recurrent neural network. The first combination is not an end-to-end solution, and the second combination makes the model complex.

In this paper, we propose an algorithm to incorporate the shape priors with the deep learning network and design an end-to-end framework. A level set function corresponding to the object shape is learned, so the pixel classification strategy commonly used in image segmentation is replaced by a regression strategy, which is similar to object segmentation methods by predicting the signed distance map [15,16]. The main contributions of our algorithm are summarized as follows:

• A loss function considering shape prior is developed to supervise the deep neural network to learn a level set function, which allows for the end-to-end training and inference processes.

• A convexity constraint term is designed in the shape-based loss function to ensure the boundaries of the nucleus and cortex segmentation results are convex, which accords with the physiology.

• A level set normalization method is proposed to alleviate the problem of the weak-contrast boundary between the nucleus and the cortex.