Elsevier

Pervasive and Mobile Computing

Volume 41, October 2017, Pages 490-503
Pervasive and Mobile Computing

Automatic diabetic retinopathy diagnosis using adjustable ophthalmoscope and multi-scale line operator

https://doi.org/10.1016/j.pmcj.2017.04.003Get rights and content

Abstract

Diabetic Retinopathy (DR), the most common one of diabetic eye diseases that cause loss of vision and blindness, has become one of major health problems today. However, DR can be eased through timely treatment and periodical screening. In this paper, we proposes an automatic diabetic retinopathy diagnostic system to help patients know about their retinal conditions. We design a portable ophthalmoscope, which is composed of a retinal lens, a smartphone and a frame between them to help patients take fundus images anywhere and anytime. Then the images are transmitted to be analyzed, including localization of optic disk and macular, vessel segmentation, detection of lesions, and grading of DR. We use a multi-scale line operator to improve accuracy in segmenting small-scale vessels, a binary mask and image restoration to reduce the effect of the existence of the vessels on optic disk localization. After the analysis, the fundus image are then graded as normal, mild Non-Proliferative Diabetic Retinopathy (NPDR), moderate NPDR or severe NPDR. The grading process uses region segmentation to improve the efficiency. The final grading results are tested based on the fundus images provided by the hospitals. We evaluate our system through comparing our grading result with those graded by experts, which comes out with an overall accuracy of up to 85%.

Introduction

With the wide use of portable devices and mobile apps, Internet of Things enabled healthcare is becoming a cost-effective way to monitor our physical activity  [1], health condition or diagnose diseases in an easier way. Diabetic retinopathy, also known as one of the most common diabetic eye diseases, often causes vision loss and blindness. According to the global meta-analysis of research data of a total of 12,620 patients with diabetes, the prevalence of DR has now reached 35.36%, while the vision of 11.72% patients are seriously affected  [2]. Regular screening has become the most effective method to detect lesions of DR and grasp the timing of treatment. Since large numbers of manual detections mean high workload for limited qualified doctors, there is an increasing demand for an automatic DR diagnostic system which can help diabetic patients get treated in time. In this paper, we propose a system, including the technique of acquiring images, analyzing images and grading according to the images.

For these techniques, there have been many researches, which have space for improvement. The grading systems are inefficient, because they usually process the whole image to detect the lesions  [3], [4], [5], which is complex and time consuming. In terms of the detections of retinal features and lesions, the main challenges are the low accuracy in small-scale vessel segmentation  [2], and the large computation cost in detecting lesions  [6], [7], [8], [9], [10]. Also, the detection of optical disk based on its characteristics  [11], [12], [13], [14], [15], [16] loses accuracy in special conditions. For the acquisition of the fundus image  [17], [18], [19], the researches nowadays mainly focus on the improvement of the image quality of smartphone’s camera with a simple lens. It is not convenient for the patients hold the lens aimed at their eyes and take a photo at the same time.

Motivated by these former researches, our approach has solved two questions: (a) Is there a portable device of the internet of things that helps patients take fundus images without going to the experts? (b) Can the analysis of the fundus image be more accurate and the grading of DR be more efficient?

To acquire images, we make use of the built-in camera and flashlight of the smartphone, and design an ophthalmoscope which is composed of a 22D lens, a smartphone and an adjustable frame between them. Users can attach the frame to their phones, adjust the bracket themselves, and simply observe the fundus image by the built-in camera. Through selecting the lens with appropriate focal length and choosing the appropriate length of the frame, we can cover 1/4 of the fundus, which ensures the high quality of the image for further analysis.

After acquiring the fundus images, inspired by Ricci’s work  [17], we improve it by using a multi-scale line operator. It can rotate to match vessels of different directions, which is able to improve small-scale vessel segmentation. In addition, we use a binary mask and image restoration, which is based on the fast marching method, to reduce the effect of the existence of the vessels on optic disk localization. We also pave the way for lesion detection with the preprocessing of the images, such as morphological operations and mean filtering.

For the grading of DR, according to The Early Treatment Diabetic Retinopathy Study (ETDRS), Non-Proliferative Diabetic Retinopathy (NPDR) is divided into four grades: normal, mild, moderate and severe. Patients graded as severe NPDR are suggested to go to physicians for further diagnosis. We divide the whole fundus image into three regions which are not overlapped. Each time we only need to analyze one region to get the final grading result or come on to detect the next region, according to the lesion detection in that region. Through region segmentation, the efficiency can be greatly improved since the area of the region that needs to be analyzed is largely reduced.

In summary, the main contributions of this work include:

  • We design a portable ophthalmoscope that can be attached to a smartphone to shoot the fundus image that can cover 1/4 of the fundus, which is enough for lesion detection. The frame between the lens and the smartphone can be adjustable for users to change the incident angle of light into the eye to take images of higher quality.

  • We use a multi-scale line operator to segment small-scale vessels to achieve higher accuracy. And based on that technique, the accuracy of the detection of hemorrhage can reach up to 98.4% compared with the former 80%, the localization of optic disk can achieve accuracy of 97.65%, which is higher than most of the other methods.

  • We use a binary mask and image restoration algorithm based on the fast marching method to erase the vessels in optic disk localization, which can compute in less than 1 s compared with about 3.8 s of other methods.

The rest of this paper is organized as follows. Section  2 describes the related work in aspect of image acquisition, detections of retinal lesions and grading of DR, while Section  3 describes the entire structure of the whole system and then describes each part respectively. Section  4 describes the experimental results and comparisons with previous ones, followed by concluding remarks and future work in Section  5.

Section snippets

Related work

Physical activity (PA) measurement is a crucial task in healthcare technology aimed at monitoring the progression and treatment of many chronic diseases. To improve the feasibility of PA tracking datasets from commercial wearable/mobile devices, Jun Qi et al.  [18] proposes a lifelogging PA intensity pattern decision making approach for lifelong PA measures, including removing some irregular uncertainties (IU) via an Ellipse fitting model  [19]. Certainly, a validation model  [20] is demanded

System overview

The diagnostic system includes a smartphone ophthalmoscope, the analytic computation, and the diagnostic aid. The structure is shown in Fig. 1, in which the smartphone ophthalmoscope and analytical computation are the main components discussed in this article.

First, we design an optical system to build a smartphone ophthalmoscope with satisfactory image quality. The patient can use the device on his or her own, without professional help, which is convenient. The captured image is then uploaded

Image acquisition

The ophthalmoscope contains an indirect retinal lens, a smartphone, and a frame between them. The user, who intends to monitor his or her personal health condition, can use his own smartphone to perform the test. It is also convenient for the user to get familiar with the usage of the ophthalmoscope. The user may first align the center line of the smartphone camera and the center line of the retinal lens on the same line. Then, taking a simplified model eye as a sample, the user may adjust the

Conclusion and future work

In this paper, we present a fully automated diabetic retinopathy diagnosis with the ability of taking a fundus image, analyzing it and grading DR. We typically design a portable ophthalmoscope and improve computational methods in detections of features and lesions. The study has shown encouraging results and indicates that the technique of image acquisition, analyzing, and DR grading is successful in diagnosing diabetic retinopathy. Hence, the system can be used for patients to check the

Acknowledgments

The work is supported by the National Natural Science Foundation of China (No. 61572316, 61671290), National High-tech R&D Program of China (863 Program) (No. 2015AA015904), the Key Program for International S&T Cooperation Project (No. 2016YFE0129500) of China, the Science and Technology Commission of Shanghai Municipality Program (No. 13511505000), the Science and Technology Commission of Shanghai Municipality (No. 16DZ0501100), the interdisciplinary Program of Shanghai Jiao Tong University

References (44)

  • A. Hoover et al.

    Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response

    IEEE Trans. Med. Imagining

    (2000)
  • M. Foracchia et al.

    Detection of optic disc in retinal images by means of a geometrical model of vessel structure

    IEEE Trans. Med. Imaging

    (2004)
  • P. Siddalingaswamy et al.

    Automatic grading of diabetic maculopathy severity levels

  • S. Sekhar et al.

    Automated localisation of optic disk and fovea in retinal fundus images

  • D. Usher et al.

    Automated detection of diabetic retinopathy in digital retinal images: a tool for diabetic retinopathy screening

    Diabetic Med.

    (2004)
  • L.J. Haddock, D.Y. Kim, S. Mukai, Simple, inexpensive technique for high-quality smartphone fundus photography in human...
  • A. Russo et al.

    Comparison of smartphone ophthalmoscopy with slit-lamp biomicroscopy for grading diabetic retinopathy

    Amer. J. Ophthalmol.

    (2015)
  • J. Staal et al.

    Ridge-based vessel segmentation in color images of the retina

    IEEE Trans. Med. Imaging

    (2004)
  • S. Chaudhuri et al.

    Detection of blood vessels in retinal images using two-dimensional matched filters

    IEEE Trans. Med. Imaging

    (1989)
  • Z. Liu et al.

    Automatic image analysis of fundus photograph

  • R. Chrástek et al.

    Optic disc segmentation in retinal images

  • E. Ricci et al.

    Retinal blood vessel segmentation using line operators and support vector classification

    IEEE Trans. Med. Imaging

    (2007)
  • Cited by (0)

    1

    The first three authors contributed equally to this work.

    View full text