Change detection based on tensor RPCA for longitudinal retinal fundus images

Abstract

Change detection of longitudinal fundus images is an important problem in computer aided diagnosis system (CAD). Detecting regions of change in multiple fundus images from the same eye is seldom developed in the literature due to the complication and interpretability. This paper presents a longitudinal change detection framework based on tensor robust principal component analysis (RPCA) for a long retinal fundus image serial. The proposed method chooses an image of the best condition in serial as the background, then models each image as a slice of tensor, utilizes total variation to constraint the temporal continuity of change regions, finally obtains the change regions by Tucker decomposition and alternating direction method of multipliers (ADMM). Comparing with the method based on matrix RPCA, tensor RPCA preserves the original spatial structure of each image, imposes the temporal continuity on change regions and models the background by patches to avoid the little disturbance of blood vessels. Results on a real fundus image serial are presented and show the effectiveness of the proposed algorithm.

Introduction

Detecting regions of change in images of the same scene at different times is very popular in video surveillance [1], [2], [3], remote sensing [4], [5], [6], [7], [8] and medical field [9], [10], [11], [12], [13]. Differing from sufficiently sampled frames for video surveillance, change detection for a longitudinal medical serial often meets more challenges because of the less frames often including only two or a few images and the long temporal span of the continuous images sometimes lasting weeks or months. Hence many change detection methods in medical field mainly focus on a pair of images [9], [11], [14].

Change detection of a longitudinal fundus serial with lots of images is discussed seldom in the previous literature and it is a challenging task. The long medical serial usually is captured under the condition that the status of a patient changes over a long time and the clinical symptom presents a complex appearance. Hence the long serial often indicates the complexity of the state of illness. Quantifying the change in the long serial can help ophthalmologists to provide more informative analysis and effective treatment and the medicine usage [15], [16], [17]. Automatic change detection for the longitudinal fundus images can significantly reduce the burden on the ophthalmologists and may alleviate the inter and intra observer variability.

Traditional change detection methods for a pair of images are based on difference images [14], [18], [19], [20]. After preprocessing the image pair is compared pixel-pair-wisely by the subtraction operator to produce a difference image and the change features are segmented from the difference image [9], [18], [19]. The difference image based on intensity subtraction is sensitive to illumination variations and calibration errors. Sometimes the difference image is generated based on the ratio operator which compares the image pair pixel-to-pixel on ratio [21]. It is robust to the multiplicative noise but fails on detecting small lesions. Gong et al combine a mean-ratio operation with a log-ratio operation to generate the difference image [21]. Such a fusion method takes the dominant advantage in big lesions but the small lesions are filtered as noises. Change detection based on these operators is straightforward and easy to implement, nevertheless it is difficult to detect both big and small lesions at the same time.

Change detection methods for the long medical serial rather than the image pair are seldom discussed in literature [14]. Robust principle component analysis (RPCA) based on an image matrix is first used to detect the change regions of the long serial in [22] by low-rank decomposition and the basic idea is the similarity between the medical image serial and the video serial. Matrix RPCA can detect the lesions with any size and provides a good solution to deal with the illumination variations in the serial but the spatial relevance inside the images is broken by vectorizing the images into vectors and it still cannot remove the random noise’s disturbance such as random light spots.

Approaches based on background modeling and subtraction were well-studied on video surveillance [1]. The advantage of background subtraction is simultaneously separating video background and extracting the foreground objects from a video stream. Subspace learning projects the background into a low-rank subspace fitting the majority of the data [23]. Statistic background modeling provides an effective method to estimate the background with dynamic noises [24]. Principal component analysis (PCA) uses an eigen-background to model each background pixel. Waters et al. [25] observed that frames in video background possess strong temporal correlation and moving objects often occupy a small region in video foreground and proposed a robust principal component analysis (RPCA) model to deal with this task. RPCA vectorizes the video frames and turns the whole sequence into an image matrix, then decompose the image matrix to get the change component. However the spatial structure of frames are broken after the images are vectorized [26], [27], [28]. A tensor framework of RPCA is introduced to model the background and the foreground objects are obtained by background subtraction [29].

Change detection on the long fundus serial differs from video serials in many factors. In surveillance applications, continuous frames are spaced apart by seconds rather than days or months, and a lot of frames are used to model the background. The background model in video surveillance is learned and maintained by a large number of video frames [1], [30], [31]. For the medical image serial, the background is modeled only by a single image named reference image [22]. Images are insufficiently sampled and the span of continuous frames usually lasts several weeks or months, which makes the background model hard to estimate. Furthermore, the background model in video surveillance is estimated from the frames without any foreground objects, and the detection results are easy to understand. Images in the fundus serial involve lesions and the normal fundus image without lesions are hard to capture, which leads to the result of change detection unexplainable [14].

Illumination variation has a critical impact on the change detection [32], [33], [34]. Many researchers put great efforts to deal with the illumination variations in change detection [9], [12], [13]. The iterative robust homomorphic surface fitting (IRHSF) is specially conceived to model the illumination for the fundus image by calculating the curvature of the retinal surface [9]. Change detection based on matrix RPCA combines intra-image correction with the inter-image normalization and extends the image pair into a serial with many frames in order to disperse the illumination variations [22]. Illumination normalization can alleviate the distraction of illumination to a certain extent, but the local intensity still distracts the detection of lesions.

In clinic, ophthalmologists attend to monitor the evolution of diseases and find out how diseases vary. They are more interested in where the regions of change are rather than locating and segmenting the lesions in the serial. In order to monitor the evolution, ophthalmologists compare each status in the serial with the best condition to generate a change serial to quantify their therapeutic schedules. Hence the relative regions of change in the medical serial can be detected based on some existing status rather than an ideal status. One can choose the best status as the baseline status, which includes fewer lesions than other images and is the closest stage to the normal status, and regard it as the background model. Once the background is chosen, the disease evolution can be obtained by background modeling and subtraction. Under this circumstance, the change result is easy to understand and explain for ophthalmologists.

In this paper, a change detection method for the long fundus serial based on background subtraction and tensor RPCA is proposed. The best status in the serial is chosen and its intensity is normalized with other frames at the beginning. Then the serial with normalized illumination is regarded as a tensor, and the change detection is achieved by Turcker decomposition of the tensor. Change regions are calculated and the quantitative analysis is given at the end. The contribution of the proposed method in this paper is summarized in several aspects. First, tensor RPCA model is used to detect the serial change which extends matrix decomposition to a tensor-based Turcker decomposition and preserves the domain-specific prior knowledge of the longitudinal retinal fundus images. Second, the proposed method imposes the spatial-temporal continuity on the change regions by total variation whereas the spatial continuity of the background model and change regions is broken and the temporal factor is ignored for matrix RPCA. Third, longitudinal images are reconstructed by patches which avoids the random noise spot and produces the clear change regions.

This paper is organized as follows. Dataset is presented and some preprocessing approaches are introduced in Section 2. The tensor model is presented and the detailed algorithm is described in Section 3. In Section 4, extensive experiments on clinic data are shown to substantiate the superiority of the proposed models over the other existing ones and the discussion is given. The conclusion is made and the future work is mentioned in Section 5.