Automatic ocular version evaluation in images using random forest

doi:10.1016/j.eswa.2021.114847

Expert Systems with Applications

Volume 176, 15 August 2021, 114847

https://doi.org/10.1016/j.eswa.2021.114847 Get rights and content

Highlights

•
This work proposes an innovative automatic method for ocular version evaluation.
•
Location of the eye steps used Histograms of Oriented Gradients and Random Forest.
•
The sclera was segmented using statistical characteristics and the Random Forest.
•
Version measure uses markers circles for each of the nine eye gaze positions exams.
•
The method achieved a 92.41% accuracy in the dataset with 147 images of 21 patients.

Abstract

The version exam is a test performed by specialists in ophthalmology to detect restrictions, paralysis, and disproportionate actions on the eye muscles in binocular movements, which is the simultaneous movement of the eyes. This test is commonly used in the detection, follow-up, and surgical planning of patients with strabismus. Strabismus is a condition in which the eyes do not have adequate alignment, generating several issues not only related to vision but also in social relations. This condition affects approximately 4% of the world’s population. In practice, the version exam is a subjective test, and the creation of a method that aims to automate the examination can help obtain a more objective result. Thus, this work presents an innovative computational method to perform the version examination automatically through face images. The proposed method is organized in seven main steps: (1) image acquisition; (2) preprocessing through a mean filter and Color Badger; (3) eye localization, based on skin segmentation and using Histograms of Oriented Gradients and Random Forest; (4) sclera segmentation, using statistical characteristics and the Random Forest classifier; (5) limbus localization, using as aid the segmented sclera; (6) eye corner localization, based on eye shape, analyzing the inner and outer eyes corners; and (7) version exam, using circle markers for each of the nine eye positions exams. The automatic measurement presented by the method was evaluated through the mean of the difference between the results provided by the method and the original versions measured by the specialist. When considering version, the proposed method obtained a mean accuracy and error of, respectively, 85.18% and 0.29 for the Medial Rectus muscle, 100% and 0 for the Lateral Rectus muscle, 86.9% and 0.47 for the Inferior Oblique muscle, 87.5% and 0.16 for the Superior Rectus muscle, 100% and 0 for the Superior Oblique muscle, and 95.23% and 0.28 for the Inferior Rectus muscle.

Introduction

Ocular version is the name given to the joint movement of the eyes in the same direction, with both eyes open and without occlusion (Singh et al., 1973). They can be denominated supraversion, infraversion, levoversion and dextroversion when the movement is up, down, left and right, respectively.

Version examination detects and measures restrictions, paralysis and disproportionate actions of the eye muscles during the simultaneous movements of the eyes executed in the version (Wright & Spiegel, 2013). The muscles analyzed are responsible for the performance of eye movements. The examination is often used in the detection, follow-up and surgical planning of patients with strabismus.

Strabismus is a condition in which the visual axes of the eyes are not aligned (Ferri, 2014). One eye may look straight ahead, while the other eye turns inward, outward, upward, or downward. Causing the brain to receive two images at different focus, making the joining process performed by it difficult or impossible. According to Kaplan (2005) strabismus affects approximately 4% of the world’s population. Affecting mainly premature and underweight children (Rosenbaum & Santiago, 1999).

Early detection of ocular misalignment increases treatment success. Detected as a child, it allows proper visual development, avoiding problems such as low vision. An efficient treatment can also improve the visual fields and the depth perspective in adults, providing improved living conditions by opening up more significant employment and socialization (Shimauti, Pesci, Sousa, Padovani, & Schellini, 2012). A study by Hashemi et al. (2019) revealed that 1 in every 50 people had strabismus, severely affecting their quality of life.

Binocular vision disorders affect about 1–5% of the population (Carlton and Kaltenthaler, 2011, Weber et al., 2017, Hashemi et al., 2019), with comitant strabismus, the most common (about 74% of cases). Without proper treatment, patients could suffer irreversible vision loss that could be prevented (Birch, 2013), entirely preventable consequences. Thus, to decrease the effect of strabismus in adults, non-surgical solutions can be adopted, such as prism lenses and orthoptic exercises. However, in the vast majority of cases, surgery is the only solution to reduce the effect of strabismus (Ophthalmologists, 2017). In surgical treatment cases, the surgical plan is defined with the help of some exams, including the version exam.

A version test is useful for detecting “A” and “V” patterns¹ associated with horizontal strabismus, mainly when it is not possible to obtain measurements and is also used for comparing the relative excursions of both eyes, for example, evaluating the patient with thyroid eye disease² and unequal restriction of the inferior rectus muscles (Traboulsi & Utz, 2016). The version exam is used as one of the variables considered by the ophthalmic surgeon in the diagnosis and planning for strabismus surgeries. Since the version exam is, in practice, a subjective test, the creation of a method that aims to automate the examination can help in obtaining a more objective result, since there were no works in literature that propose an automatic method to quantify ocular versions.

Version test is useful for detecting “A” and “V” patterns³ associated with horizontal strabismus, mainly when it is impossible to obtain measurements. It is also used for comparing the relative excursions of both eyes, for example, evaluating the patient with thyroid eye disease⁴ and unequal restriction of the inferior rectus muscles (Traboulsi & Utz, 2016). The version exam is used as one of the ophthalmic surgeon’s variables in the diagnosis and planning for strabismus surgeries. Since the version exam is a subjective test in practice, creating a method that aims to automate the examination can help obtain a more objective result since no works in the literature propose quantifying the ocular versions automatically.

This paper presents a computational method to perform the automatic evaluation of ocular versions based on images, using image processing techniques, pattern recognition and computational intelligence. This method contributes the following: (1) Image acquisition protocol for automatic evaluation of the ocular version; (2) image database of version exam; (3) new method for location of the eyes in color images; (4) new sclera segmentation method; (5) new limbus segmentation method; and (6) building an automatic solution to measure the ocular version.

Regarding the following sections of this paper, Section 2 presents the related works. Section 3 describes the method steps that is used to evaluation the ocular version. In Section 4, we present and discuss the results that are achieved at each stage of the proposed methodology. Finally, Section 5 presents the conclusions, analyzing the efficiency of the techniques used and presented future works.

Section snippets

Related works

In the literature, there are works that propose the use of technology to conduct eye exams or tests, but none of them proposes using images in automatic ocular version evaluation. However, several researches have topics related to the steps of the proposed method as to how achieve a fully automated system: location of the eyes (Section 3.2.2), sclera segmentation (Section 3.2.3) and eye corner localization (Section 3.2.5).

Materials and method

This section first presents the materials used for the automatic ocular versions measurement. Then, we describe a sequence of the stages developed in order to achieve the goals of the proposed method.

Results and discussions

This section presents the results achieved in the tests performed with the proposed method for measuring eye versions using images. First, the accuracy of the method is evaluated for these steps: eye localization (Section 3.2.2), sclera segmentation (Section 3.2.3), limbus location (Section 3.2.4), eye corner location (Section 3.2.5) and automatic version measurement (Section 3.2.6). Making use of every image in the dataset for each step of the proposed method.

Finally, we performed a test of

Conclusion

This work presented an automatic method to perform version examination using image processing and pattern recognition techniques via an image of a face. The proposed method performs the version examination using eye localization, sclera segmentation, limbus localization and eye corner localization. Where locating the eyes showed solid results, achieving an accuracy of 91.49%.

The proposed method was able to segment most of the sclera in the images tested. For eye corner localization, the

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

Our research group acknowledges financial support from CNPQ (Grant No.: 423493/2016-7 and 307210/2018-9) and FAPEMA (Grant No.: UNIVERSAL-00974/18).

References (50)

S. Bazrafkan et al.
An end to end deep neural network for iris segmentation in unconstrained scenarios
Neural Networks
(2018)
E.E. Birch
Amblyopia and binocular vision
Progress in Retinal and Eye Research
(2013)
B. Huang et al.
Eye landmarks detection via weakly supervised learning
Pattern Recognition
(2020)
S.T. Kim et al.
A novel iris center localization based on circle fitting using radially sampled features
P.J. Phillips et al.
The feret database and evaluation procedure for face-recognition algorithms
Image and Vision Computing
(1998)
S.A. Sahmoud et al.
Efficient iris segmentation method in unconstrained environments
Pattern Recognition
(2013)
K.P. Weber et al.
Strabismus measurements with novel video goggles
Ophthalmology
(2017)
Z. Zheng et al.
A robust method for eye features extraction on color image
Pattern Recognition Letters
(2005)
S. Alkassar et al.
Efficient eye corner and gaze detection for sclera recognition under relaxed imaging constraints
S. Alkassar et al.
A novel method for sclera recognition with images captured on-the-move and at-a-distance

S. Alkassar et al.

Robust sclera recognition system with novel sclera segmentation and validation techniques

IEEE Transactions on Systems, Man, and Cybernetics: Systems

(2017)

J.D.S. Almeida et al.

Computer-aided methodology for syndromic strabismus diagnosis

Journal of Digital Imaging

(2015)

V. Badrinarayanan et al.

Segnet: A deep convolutional encoder-decoder architecture for image segmentation

IEEE Transactions on Pattern Analysis and Machine Intelligence

(2017)

Banić, N. and Lončarić, S. (2014). Color badger: a novel retinex-based local tone mapping operator. In International...

J. Carlton et al.

Amblyopia and quality of life: A systematic review

Eye

(2011)

K.C. Chern et al.

Ophthalmology review manual

(2000)

T.H. Cormen et al.

Introduction to algorithms

(2009)

Dalal, N. & Triggs, B. (2005). Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society...

A. Das et al.

A new efficient and adaptive sclera recognition system

A. Das et al.

Ssrbc 2016: Sclera segmentation and recognition benchmarking competition

Ferri, F. F. (2014). Ferri’s Clinical Advisor 2015 E-Book: 5 Books in 1. Elsevier Health...

H. Hashemi et al.

Global and regional prevalence of strabismus: A comprehensive systematic review and meta-analysis

Strabismus

(2019)

M. Kaplan

Seeing Through New Eyes: Changing the Lives of Children with Autism, Asperger Syndrome and other Developmental Disabilities Through Vision Therapy

(2005)

Kumar, N., Berg, A. C., Belhumeur, P. N., & Nayar, S. K. (2009). Attribute and simile classifiers for face...

Lee, K. I., Jeon, J. H., & Song, B. C. (2020). Deep learning-based pupil center detection for fast and accurate eye...

Cited by (3)

Machine learning-based model for recognizing the failure modes of FRP-strengthened RC beams in flexure
2023, Case Studies in Construction Materials
Plate end (PE) debonding and intermediate crack (IC) debonding are the main failure modes of fiber reinforced polymer (FRP) strengthened reinforced concrete (RC) beams in flexure. Different failure modes exhibit different failure characteristics. Therefore, accurately identifying the failure mode is of great significance in selecting methods to prevent the debonding of the strengthened beams. This study first established a primary indicator system through literature research. Then one hundred and eighty-eight FRP-strengthened RC beams containing PE and IC debonding were collected from the published literature of forty-eight researchers. After that, correlation analysis and grey correlation analysis were used to study the data of two failure modes. Finally, the indicator system for the prediction of failure modes was established. After the indicator system was established, six machine learning algorithms, including K-nearest neighbor algorithm (KNN), decision tree (DT), random forest (RF), back propagation neural network (BPNN), logistic regression (LR), and support vector machine (SVM), were used to build the prediction model of failure modes. The evaluation of the models shows that the coefficient of variation of the accuracy of the decision tree is merely 5.4%, which has the best robustness; the average accuracy of the random forest for the two failure modes reaches 93% and 98%, which has the highest precision.
Multiple attentional aggregation network for handwritten Dongba character recognition
2023, Expert Systems with Applications
Citation Excerpt :
For HDCR, the statistical features such as coarse grid (Da et al., 2011) and histogram of oriented gradient (Luo et al., 2020) are more distinguishable than certain structural features that primarily study the strokes or parts structure of Dongba characters. The most commonly used classifiers involve support vector machines (SVM) (Baldomero-Naranjo, Martinez-Merino, & Rodriguez-Chia, 2021), random forests (Pinheiro et al., 2021) and k-nearest neighbor (KNN) (Zhang, Xiao, Gao, Zhang, & Wang, 2022). Nevertheless, these algorithms are basically on the basis of a combination of independent elements and depend considerably on hyperparameters that have different optimum values in their own different datasets.
Handwritten Dongba character recognition (HDCR) is a challenging task due to different recognition granularity and it plays an important role in history and philology. Recently, deep learning-based methods are adopted and outperform traditional approaches. However, they do not well explore the multi-granularity information, leading to many highly similar Dongba characters cannot be recognized accurately. In this paper, a Multiple Attentional Aggregation Network (MAAN) is proposed for HDCR. Specifically, a novel Hybrid Attentional Mapping Unit (HAMU) is designed for multi-scale and richer feature expression. The hybrid attention mechanism in HAMU can enhance the discrimination between important and redundant features. Besides, a novel Spatial Attentional Aggregation Unit (SAAU) is adopted to aggregate multi-scale features of different depths. In this way, more details from large scale features and global semantic information from small scale features will be adequately aggregated. Experiments are conducted on our constructed novel Dongba character image dataset and validate the effectiveness and superiority of our methods and the extensive experimental results demonstrate that our proposed MAAN outperforms state-of-the-art algorithms.
Computational method for aid in the diagnosis of sixth optic nerve palsy through digital videos
2022, Computers in Biology and Medicine
Citation Excerpt :
However, when much of the iris is inside the inner or outer corners, partially occluded, YOLOv3 has difficulty centering the result on the pupil. This may be because, in the training image dataset, there are not many examples of patients looking at the corners of the eye so sharply, which usually occurs in patients with strabismus, where there are version deviations and the eyes are not correctly aligned [35]. Therefore, it was decided to combine the results of eye center detection done by Isophotes and YOLOv3, forming the third tested approach for eye center detection.
The sixth cranial nerve, also known as the abducens nerve, is responsible for controlling the movements of the lateral rectus muscle. Palsies on the sixth nerve prevent some muscles that control eye movements from proper functioning, causing headaches, migraines, blurred vision, vertigo, and double vision. Hence, such palsy should be diagnosed in the early stages to treat it without leaving any sequela. The usual methods for diagnosing the sixth nerve palsy are invasive or depend on expensive equipment, and computer-based methods designed specifically to diagnose the aforementioned palsy were not found until the publication of this work. Therefore, a low-cost, non-invasive method can support or guide the ophthalmologist’s diagnosis. In this context, this work presents a computational methodology to aid in diagnosing the sixth nerve palsy using videos to assist ophthalmologists in the diagnostic process, serving as a second opinion. The proposed method uses convolutional neural networks and image processing techniques to track both eyes’ movement trajectory during the video. With this trajectory, it is possible to calculate the average velocity (AV) in which each eye moves. Since it is known that paretic eyes move slower than healthy eyes, comparing the AV of both eyes can determine if the eye is healthy or paretic. The results obtained with the proposed method showed that paretic eyes move at least 19.65% slower than healthy ones. This threshold, along with the AV of the movement of the eyes, can help ophthalmologists in their analysis. The proposed method reached 92.64% accuracy in diagnosing the sixth optic nerve palsy (SONP), with a $K a p p a$ index of 0.925, which highlights the reliability of the results and gives favorable perspectives for further clinical application.

View full text