An enhancement method for color retinal images based on image formation model

doi:10.1016/j.cmpb.2017.02.026

Computer Methods and Programs in Biomedicine

Volume 143, May 2017, Pages 137-150

https://doi.org/10.1016/j.cmpb.2017.02.026 Get rights and content

Highlights

•
A new enhancement method based on image formation model is proposed to enhance the color retinal images.
•
This enhancement method can deal with illumination problems, contrast enhancement, and color preservation in color retinal image simultaneously.
•
This enhancement method can improve the poor quality of retinal image caused by different factors, which facilitates the reliable clinic diagnosis.
•
Experimental results indicated that better enhancement image can be produced compared with state-of-art algorithms, especially for blurry retinal images.

Abstract

Background and objective

The good quality of color retinal image is essential for doctors to make a reliable diagnose in clinics. Due to major reasons like acquisition process and retinal diseases, most retinal images can show poor illuminance, blur and low contrast, further impeding the process of identifying the underlying retinal condition.

Methods

Image formation model of scattering is proposed to enhance color retinal images in this paper. Two parameters of this model, background illuminance and transmission map, are estimated based on extracted background and foreground. The complex nature of the foreground of a retinal image, involving pixels with both low and high intensity, posed a challenge to the proper extraction of these pixels. Therefore, a new method combining Mahalanobis distance discrimination and global spatial entropy-based contrast enhancement is proposed to extract foreground pixels. It extracts background and foreground in high intensity region and low intensity region respectively and it can perform well in blurry image with tiny intensity range.

Results

The proposed method is evaluated using 319 color retinal images from three different databases. Experimental results indicated that the proposed method can perform well on illumination problems, contrast enhancement and color preservation.

Conclusion

This study proposes a new method of enhancing overall retinal image and produces better enhancement images than several state-of-the-art algorithms, especially for blurry retinal images. This method can facilitate analysis and reliable diagnosis for both ophthalmologists and computer-aided analysis.

Introduction

Retinal imaging is widely used in medical diagnosis of ocular diseases such as glaucoma, cataract, age-related macular degeneration and diabetic retinopathy in recent years [1]. As a standard image modality, fundus camera is usually used to acquire retinal images, showing structures like optic disc, retinal vessels and several other. The changes detected in these structures can be indicative of a pathological condition associated with diseases such as glaucoma and diabetic retinopathy, which can further be confirmed by performing detailed analysis of these retinal images. Therefore, the analysis of retinal images is a useful and helpful diagnostic tool. In fact, the analysis of retinal images can also render beneficial to the classification of the disease stages, on identifying the underlying problem.

Due to retinal pathology and imaging configuration, retinal images acquired via fundus camera often have low intensity contrast. Some of the major contributing factors may include, curved surface of the retina, degree of dilation, unexpected movements of the patient's eye and other diseases such as cataract [2]. Low contrast in non-uniform illumination is often caused by the first three factors as shown in Fig. 1(b). On the other hand, blurry retina shown in Fig. 1(c) is commonly found due to cataract, which prevents the light from reaching the retina. In general, different types of pathologies may emerge in a retina simultaneously, and a poor quality retinal image may cause hindrance to their clear identification. On comparing the retinal images of varying quality in Fig. 1, it can be noted that the Fig. 1(a) clearly shows all retinal structures such as small vessels whereas both figures Fig. 1(b) and (c) can only barely show the larger retinal structures. This further proves the very necessity of augmenting the retinal image quality in helping with the diagnostics of the various retinal disorders.

Contrast enhancement algorithms have been investigated for many years, and they can be categorized into two major groups namely, data domain method and restored model method. Methods of enhancement that are based on data domain mainly consist of transform-domain algorithm and image-domain algorithm [3]. Transform-domain algorithms decompose an image into several sub-bands to improve the frequency components of the image with associated parameters globally or locally [4], [5], [6]. Although these algorithms show promising results, this also adds to complexity of computation which is why image-domain contrast enhancement algorithms are more widely used. Image-domain contrast enhancement algorithms change the gray-level of individual pixels according to histogram. The commonly used method called histogram equalization (HE) [7] helps transforming these gray-levels in accordance to the cumulative distribution function of input image histogram. With the on-going development and research, a lot more efficient methods such as weighted thresholded histogram equalization [8], histogram modification framework [9], and adaptive gamma correction with weighting distribution [10] have been proposed based on the HE. On the other hand, enhancement based on restored model is usually performed by reversing the process that worsens the image quality, and emphasizes features of the image by imaging a point source. Recently this method is being widely applied in enhancing underwater image [11], [12] and dehazing [13], [14]. Enhancement based on restored model utilizes the priori knowledge to model the image degradation that comes in several forms like noise, camera mis-focus and camera exposure time [15]. This distinguishing feature makes this method quite unique compared with the former two, as it uses the realistic data from the prior knowledge of imaging as opposed to mere accentuation of the existing image features used by former two.

For retinal images, algorithms of enhancing retinal image are mostly used as pre-processing in the detection of retinal structures and lesions. Equalization of illumination technique [16], [17] is utilized to correct non-illumination of retinal image for vessel segmentation. However, this method may not be able to heighten the overall retinal image. Another technique called histogram matching between the red and green planes are applied in the pre-process of vessel segmentation [18] as it can enhance the large and prominent dark features but it may not be efficient in identifying conditions such as microaneurysms due to the reduced contrast of bright and tiny dark objects. Enhancement using matched filter [19], [20], [21] may render beneficial in improving local contrast but similar to other methods, it may affect other structures which do not fit to this filter. Local contrast enhancement depending on mean and variance of the enhanced image is applied to enhance and extract bright lesions [22].

Since all the above mentioned methods are applied to enhance only certain parts of the retinal image and they cannot enhance all the structures and lesions, the algorithms of enhancing the overall contrast are studied [2], [23], [24], [25]. A contourlet transform-based enhancement [23] is proposed to enhance the green channel of retinal image. This method enhances the green channel of the retinal image but has also been found to be limited to the grayscale image. Besides, this method has also deemed an unreliable one due to its lack in producing an adequate amount of enhancement in the poor contrast regions. Thus, contrast limited adaptive histogram equalization (CLAHE) is employed to aid the detection of retinal changes in diabetic retinopathy imagery [24], which works well in enhancing green channel of retinal image. Since the color of object can also play a substantial part in the diagnostic procedures, a color remapping scheme (CRS) [25] based on luminosity and contrast is raised to enhance and normalize retinal image on each color plane of RGB color space. Being inspired by this method, a solution for color retinal image enhancement based on domain knowledge of the retina geometry and imaging conditions is presented [2], which achieved desired enhancement on vascular topography, dark and bright pathology. Although these methods [2], [25] are able to improve the overall contrast and correct for non-uniform illumination successfully, they seem incapable of maintaining the quality of the retinal images in the presence of a blur or noise. A new enhancement method based on fusing basic information of background with original retinal image is proposed, which can handle blurry images but will cause information loss when processing retinal images with uneven illuminance [26].

Poor quality of retinal images can be caused by several factors. Poor illuminance and low contrast of retinal images are two common problems in clinic. Original data preservation is also important for clinical diagnosis and retinal analysis. Since, existing methods proposed in paper [24] and paper [25] that are based on data domain mostly tend to focus on enhancing the contrast of retinal images. These methods are always performed on each channel of an RGB image separately without prior knowledge of observed image and are therefore insufficient in preserving original illuminance distribution. CRS [25] is a method based on the restored model that normalizes and enhances the retinal image according to local mean variance of illuminance. Although it may successfully handle the problem of poor illuminance and color preservation, it happens to fail in the blurry images with very low contrast.

Due to the issues of poor illuminance, low contrast and color distortion, a new method of enhancing retinal image based on restored model is proposed. The main contribution of this work can be summarized as follows: (1) It is the first time that a scattering model of imaging is employed to model the degeneration of retina imaging and enhance the foreground of retinal image which contains retinal structures and lesions. Unlike other methods based on restored model, our method emphasizes on both accentuation of image features and realistic data preservation. (2) Two important parameters (background illuminance and transmission map) of this model are estimated according to the intensity information of separated foreground and background, and an improved method is proposed to separate foreground and background well, which combines gray-scale global spatial entropy-based method [27] and Mahalanobis distance discriminant based on local mean and variance of luminance [25]. Our method can perform much better in extracting the foreground pixels of blur image with low intensity. (3) Compared with previous methods of enhancing retinal images, the proposed method can perform well on poor illumination problems, contrast enhancement and color preservation simultaneously. Both brighter and darker objects relative to background can be enhanced successfully in retinal images. Totally, 319 retinal images from three different databases are tested, and desired results are achieved.

Section snippets

Image formation model

In general, process of imaging is closely related with scattering of light by physical media, which is highly complex and depends on types, orientations, size, and distributions of particles constituting the media, as well as wavelengths, polarization states, and directions of the incident light [28]. In this paper, to model the degeneration of retina imaging, we assume that the process of retina imaging confirms to the common dichromatic scattering process, as illustrated in Fig. 2. Fig. 2

Enhancement on color fundus image

As seen from the dichromatic scattering model (see Eq. (3)), the quality of an image is determined by two components: background light and direct transmission, which corresponds to the parameters of background illuminance B and transmission map t(x, y) respectively. This model provides potential to enhance poor illumination image by adjusting parameter B. It also has the advantage to enhance low contrast image by adjusting parameter t(x, y) which is related to medium. The flow chart of the

Experimental results

Our proposed method can deal with illumination problems, contrast enhancement and color preservation simultaneously. In our experiments, 100 blurry retinal images (1396 × 1124 pixels, 45° field of view) from Beijing Institute of Ophthalmology, Tongren Hospital, which are diagnosed as cataract by ophthalmologists, 130 retinal images from standard diabetic retinopathy database “Calibration Level 0'' (DIARETDB0, 1500 × 1152 pixels, 50° field of view) [33], and 89 retinal images from standard

Discussion

The method proposed in this paper relies on prior knowledge of original image. In our model, parameter t decides the extent of contrast. To further emphasize and preserve the original illumination distribution, t is estimated based on separated background and foreground and the combined information of three channels. Therefore, our method has demonstrated its ability to both enhance the image quality as well as preserve the original color distribution.

Correct estimation of foreground and

Conclusion

Retinal image enhancement is a challenge as the foreground includes pixels with both lower and higher intensity compared to background. In this paper, a new enhancement model based on image formation model of scattering is proposed to deal with illumination problems, contrast enhancement, and color preservation in color retinal image. Two important parameters of this restored model, which contain transmission map t and background illuminance B, are estimated. Due to non-uniform of background

Reference (35)

T. Celik
Two-dimensional histogram equalization and contrast enhancement
Pattern Recognit.
(2012)
N.M. Salem et al.
Novel and adaptive contribution of the red channel in pre-processing of colour fundus images
J. Franklin Inst.
(2007)
R. Su et al.
A new method for linear feature and junction enhancement in 2D images based on morphological operation, oriented anisotropic Gaussian function and Hessian information
Pattern Recognit.
(2014)
P. Feng et al.
Enhancing retinal image by the Contourlet transform
Pattern Recognit. Lett.
(2007)
M. Foracchia et al.
Luminosity and contrast normalization in retinal images
Med. Image Anal.
(2005)
M.D. Abràmoff et al.
Retinal imaging and image analysis
IEEE Rev. Biomed. Eng.
(2010)
G.D. Joshi et al.
Colour retinal image enhancement based on domain knowledge
D.J. Jobson et al.
A multiscale retinex for bridging the gap between color images and the human observation of scenes
IEEE Trans. Image Process.
(1997)
J. Mukherjee et al.
Enhancement of color images by scaling the DCT coefficients
IEEE Trans. Image Process.
(2008)
R. Fattal
Edge-avoiding wavelets and their applications
ACM Trans. Graphics (TOG)
(2009)

J.B. Zimmerman et al.

An evaluation of the effectiveness of adaptive histogram equalization for contrast enhancement

IEEE Trans. Med. Imaging

(1988)

Q. Wang et al.

Fast image/video contrast enhancement based on weighted thresholded histogram equalization

IEEE Trans. Consum. Electron.

(2007)

T. Arici et al.

A histogram modification framework and its application for image contrast enhancement

IEEE Trans. Image Process.

(2009)

S.-C. Huang et al.

Efficient contrast enhancement using adaptive gamma correction with weighting distribution

IEEE Trans. Image Process.

(2013)

E. Nascimento et al.

Stereo based structure recovery of underwater scenes from automatically restored images

J.Y. Chiang et al.

Underwater image enhancement by wavelength compensation and dehazing

IEEE Trans. Image Process.

(2012)

S.G. Narasimhan et al.

Contrast restoration of weather degraded images

IEEE Trans. Pattern Anal. Mach. Intell.

(2003)

Cited by (53)

Fundus image enhancement using visual transformation and maximum a posterior estimation
2023, Biomedical Signal Processing and Control
Fundus imaging is used to diagnose many common diseases in human vision. Because of uneven brightness, blur effect, and low contrast, fundus images are not helpful for diagnosis. The proposed technique provides an effective enhancement technique for fundus images. The method performs the appropriate illumination improvement, details correction, and denoising operations. A maximum a posterior (MAP) estimator is used to calculate the illumination and reflection components, which is motivated by the retinex method. The estimated illumination part is put through gamma correction to smooth its unevenness. A straightforward visual transformation algorithm is applied to address the unequal luminosity at the structural layer; it is used to enhance the details of the structure layer. The suggested methods were examined on public datasets (STARE, CHASEDB1, and DRIVE). To further confirm the recommended strategy's benefit in aiding the diagnosis, ophthalmologists' quality assessments were also taken. The objective evaluation used performance parameters such as GMSD, NIQE, HPSI, VSI, MCSD, and PSNR. According to experimental results, the suggested method may simultaneously manage the duties of illumination enhancement, details improvement, noise, and artifact suppression, outperforming state-of-the-art methods. According to the qualitative, quantitative, and statistical examination, the suggested algorithm reliably improves low-contrast retinal photos with intact color and a natural appearance.
MUTE: A multilevel-stimulated denoising strategy for single cataractous retinal image dehazing
2023, Medical Image Analysis
In this research, we studied the duality between cataractous retinal image dehazing and image denoising and proposed that the dehazing task for cataractous retinal images can be achieved with the combination of image denoising and sigmoid function. To do so, we introduce the double-pass fundus reflection model in the YPbPr color space and developed a multilevel stimulated denoising strategy termed MUTE. The transmission matrix of the cataract layer is expressed as the superposition of denoised raw images of different levels weighted by pixel-wise sigmoid functions. We further designed an intensity-based cost function that can guide the updating of the model parameters. They are updated by gradient descent with adaptive momentum estimation, which gives us the final refined transmission matrix of the cataract layer. We tested our methods on cataract retinal images from both public and proprietary databases, and we compared the performance of our method with other state-of-the-art enhancement methods. Both visual assessments and objective assessments show the superiority of the proposed method. We further demonstrated three potential applications including blood vessel segmentation, retinal image registrations, and diagnosing with enhanced images that may largely benefit from our proposed methods.
A fundus image enhancer based on illumination-guided attention and optic disc perception GAN
2023, Optik
High-quality fundus images are essential for ophthalmologists in clinical diagnosis of eye diseases. For reasons such as the acquisition process and the retinal disease, most fundus images may suffer poor illumination, blur, and low contrast. In order to obtain high-quality fundus images, a novel network model for fundus image enhancement is constructed in this paper. The model introduces a generated adversarial network (GAN) with illumination-guided attention and optic disc perception. A U-Net with illumination-guided attention is used as the generator, and a global discriminator and a local discriminator with optic disc segmentation are used in the learning process. The model can improve the exposure level of overexposed and underexposed fundus images and also stretch the contrast. Experiments are performed on multiple fundus image datasets taken by the handheld device and high-end device to verify the robustness of the proposed illumination-guided attention and optic disc perception GAN (IGAODPGAN). The retinal structure and pathological characteristics are highlighted significantly after enhancement. Our method is superior to other state-of-the-art algorithms in terms of both subjective and objective assessment, which can provide an efficient solution for fundus image enhancement, especially for the non-uniform illuminated fundus images.
Retinal image enhancement with artifact reduction and structure retention
2023, Pattern Recognition
Enhancement of low-quality retinal fundus images is beneficial to clinical diagnosis of ophthalmic diseases and computer-aided analysis. Enhancement accuracy is a challenge for image generation models, especially when there is no supervision by paired images. To reduce artifacts and retain structural consistency for accuracy improvement, we develop an unpaired image generation method for fundus image enhancement with the proposed high-frequency extractor and feature descriptor. Specifically, we summarize three causes of tiny vessel-like artifacts which always appear in other image generation methods. A high frequency prior is incorporated into our model to reduce artifacts by the proposed high-frequency extractor. In addition, the feature descriptor is trained alternately with the generator using segmentation datasets and generated image pairs to ensure the fidelity of the image structure. Pseudo-label loss is proposed to improve the performance of the feature descriptor. Experimental results show that the proposed method performs better than other methods both qualitatively and quantitatively. The enhancement can improve the performance of segmentation and classification in retinal images.
A GAN-based deep enhancer for quality enhancement of retinal images photographed by a handheld fundus camera
2022, Advances in Ophthalmology Practice and Research
Due to limited imaging conditions, the quality of fundus images is often unsatisfactory, especially for images photographed by handheld fundus cameras. Here, we have developed an automated method based on combining two mirror-symmetric generative adversarial networks (GANs) for image enhancement.
A total of 1047 retinal images were included. The raw images were enhanced by a GAN-based deep enhancer and another methods based on luminosity and contrast adjustment. All raw images and enhanced images were anonymously assessed and classified into 6 levels of quality classification by three experienced ophthalmologists. The quality classification and quality change of images were compared. In addition, image-detailed reading results for the number of dubiously pathological fundi were also compared.
After GAN enhancement, 42.9% of images increased their quality, 37.5% remained stable, and 19.6% decreased. After excluding the images at the highest level (level 0) before enhancement, a large number (75.6%) of images showed an increase in quality classification, and only a minority (9.3%) showed a decrease. The GAN-enhanced method was superior for quality improvement over a luminosity and contrast adjustment method (P＜0.001). In terms of image reading results, the consistency rate fluctuated from 86.6% to 95.6%, and for the specific disease subtypes, both discrepancy number and discrepancy rate were less than 15 and 15%, for two ophthalmologists.
Learning the style of high-quality retinal images based on the proposed deep enhancer may be an effective way to improve the quality of retinal images photographed by handheld fundus cameras.
Single-shot retinal image enhancement using untrained and pretrained neural networks priors integrated with analytical image priors
2022, Computers in Biology and Medicine
Retinal images acquired using fundus cameras are often visually blurred due to imperfect imaging conditions, refractive medium turbidity, and motion blur. In addition, ocular diseases such as the presence of cataracts also result in blurred retinal images. The presence of blur in retinal fundus images reduces the effectiveness of the diagnosis process of an expert ophthalmologist or a computer-aided detection/diagnosis system. In this paper, we put forward a single-shot deep image prior (DIP)-based approach for retinal image enhancement. Unlike typical deep learning-based approaches, our method does not require any training data. Instead, our DIP-based method can learn the underlying image prior while using a single degraded image. To perform retinal image enhancement, we frame it as a layer decomposition problem and investigate the use of two well-known analytical priors, i.e., dark channel prior (DCP) and bright channel prior (BCP) for atmospheric light estimation. We show that both the untrained neural networks and the pretrained neural networks can be used to generate an enhanced image while using only a single degraded image. The proposed approach is time and memory-efficient, which makes the solution feasible for real-world resource-constrained environments. We evaluate our proposed framework quantitatively on five datasets using three widely used metrics and complement that with a subjective qualitative assessment of the enhancement by two expert ophthalmologists. For instance, our method has achieved significant performance for untrained CDIPs coupled with DCP in terms of average PSNR, SSIM, and BRISQUE values of 40.41, 0.97, and 34.2, respectively, and for untrained CDIPs coupled with BCP, it achieved average PSNR, SSIM, and BRISQUE values of 40.22, 0.98, and 36.38, respectively. Our extensive experimental comparison with several competitive baselines on public and non-public proprietary datasets validates the proposed ideas and framework.

View all citing articles on Scopus

View full text

An enhancement method for color retinal images based on image formation model

Highlights

Abstract

Background and objective

Methods

Results

Conclusion

Introduction

Section snippets

Image formation model

Enhancement on color fundus image

Experimental results

Discussion

Conclusion

Pattern Recognit.

J. Franklin Inst.

Pattern Recognit.

Pattern Recognit. Lett.

Med. Image Anal.

Retinal imaging and image analysis

IEEE Rev. Biomed. Eng.

Colour retinal image enhancement based on domain knowledge

A multiscale retinex for bridging the gap between color images and the human observation of scenes

IEEE Trans. Image Process.

Enhancement of color images by scaling the DCT coefficients

IEEE Trans. Image Process.

Edge-avoiding wavelets and their applications

ACM Trans. Graphics (TOG)

An evaluation of the effectiveness of adaptive histogram equalization for contrast enhancement

IEEE Trans. Med. Imaging

Fast image/video contrast enhancement based on weighted thresholded histogram equalization

IEEE Trans. Consum. Electron.

A histogram modification framework and its application for image contrast enhancement

IEEE Trans. Image Process.

Efficient contrast enhancement using adaptive gamma correction with weighting distribution

IEEE Trans. Image Process.

Stereo based structure recovery of underwater scenes from automatically restored images

Underwater image enhancement by wavelength compensation and dehazing

IEEE Trans. Image Process.

Contrast restoration of weather degraded images

IEEE Trans. Pattern Anal. Mach. Intell.