Abstract
The recognition of the eye region from images is a challenging task, particularly when dealing with dark or thick sunglasses that cause reflections and interfere with accurate identification. To address this issue, a novel system called AVOA-MRCNN-OLSTM has been proposed. This system combines Optimization-driven Long Short-Term Memory (LSTM) with Mask RCNN to achieve precise eye recognition even in the presence of eyeglass frame interference. A mean histogram equalization approach is used in the system's first stage to eliminate noise, which improves the image quality. The system then uses Mask RCNN for segmentation and localization. A potent deep learning model called Mask RCNN can precisely recognize and isolate particular items inside an image. It is used in this instance to identify and divide the eye region. The AVOA-MRCNN-OLSTM framework makes use of LSTM, a recurrent neural network variety that can retain patterns for longer periods. It can efficiently acquire and use temporal information to increase eye recognition accuracy by integrating LSTM into the system. The proposed AVOA-MRCNN-OLSTM system's effectiveness is shown by experimental findings. It outperforms the performance of existing algorithms, achieving a remarkable accuracy of 99% in just 0.02 seconds of computing time. The potential uses of this development include biometric identity, surveillance systems, and human-computer interfaces, all of which need precise eye recognition.








Similar content being viewed by others
References
Yu M, Zheng H, Peng Z, Dong J, Du H (2020) Facial expression recognition based on a multi-task global-local network. Pattern Recog Lett 131:166–171
Turan C, Lam KM (2018) Histogram-based local descriptors for facial expression recognition (FER): A comprehensive study. J Vis Commun Image Represent 55:331–341
Gou C, Wu Y, Wang K, Wang K, Wang FY, Ji Q (2017) A joint cascaded framework for simultaneous eye detection and eye state estimation. Pattern Recog 67:23–31
Yiu YH, Aboulatta M, Raiser T, Ophey L, Flanagin VL, Zu Eulenburg P, Ahmadi SA (2019) DeepVOG: Open-source pupil segmentation and gaze estimation in neuroscience using deep learning. J Neurosci Methods 324:108307
Shi Y, Zhang Z, Huang K, Ma W, Tu S (2020) Human-computer interaction based on face feature localization. J Vis Commun Image Represent 70:102740
Fuhl W, Tonsen M, Bulling A, Kasneci E (2016) Pupil detection for head-mounted eye tracking in the wild: an evaluation of the state of the art. Machine Vis Appl 27(8):1275–1288
Akhtar Z, Rattani A (2017) A face in any form: new challenges and opportunities for face recognition technology. Computer 50(4):80–90
Pires R, Avila S, Wainer J, Valle E, Abramoff MD, Rocha A (2019) A data-driven approach to referable diabetic retinopathy detection. Artificial intell Med 96:93–106
Funes-Mora KA, Odobez JM (2016) Gaze estimation in the 3d space using rgb-d sensors. Int J Comput Vis 118(2):194–216
Lu F, Sugano Y, Okabe T, Sato Y (2015) Gaze estimation from eye appearance: A head pose-free method via eye image synthesis. IEEE Trans Image Process 24(11):3680–3693
Cristina S, Camilleri KP (2018) Unobtrusive and pervasive video-based eye-gaze tracking. Image Vision Comput 74:21–40
Skodras E, Fakotakis N (2015) Precise localization of eye centers in low resolution color images. Image Vis Comput 36:51–60
Yang S, Tan J, Chen B (2022) Robust spike-based continual meta-learning improved by restricted minimum error entropy criterion. Entropy 24(4):455
Yang S, Linares-Barranco B, Chen B (2022) Heterogeneous ensemble-based spike-driven few-shot online learning. Front Neurosci, 16
Yang S, Gao T, Wang J, Deng B, Azghadi MR, Lei T, Linares-Barranco B (2022) SAM: a unified self-adaptive multicompartmental spiking neuron model for learning with working memory. Front Neurosci, 16
Vu HN, Nguyen MH, Pham C (2022) Masked face recognition with convolutional neural networks and local binary patterns. Appl Intell 52(5):5497–5512
Skodras E, Kanas VG, Fakotakis N (2015) On visual gaze tracking based on a single low cost camera. Signal Process: Image Commun 36:29–42
Chellappa R, Chen JC, Ranjan R, Sankaranarayanan S, Kumar A, Patel VM, Castillo CD (2016) Towards the design of an end-to-end automated system for image and video-based recognition. In 2016 Information Theory and Applications Workshop (ITA) (pp. 1-7). IEEE
Jamshidi S, Azmi R, Sharghi M, Soryani M (2021) Hierarchical deep neural networks to detect driver drowsiness. Multimed Tools Appl 80(10):16045–16058
Priya GN, Wahida Banu RSD (2014) Occlusion invariant face recognition using mean based weight matrix and support vector machine. Sadhana 39(2):303–315
Savaş BK, Becerikli Y (2020) Real time driver fatigue detection system based on multi-task ConNN. Ieee Access 8:12491–12498
Kuo HF, Deng BS, Fang JY (2016) Fabrication of pn Junction With an n-Type Silicon Nanoparticle Layer by Using Infrared Fiber Laser Illumination. IEEE Access 4:6225–6230
Egger B, Schönborn S, Schneider A, Kortylewski A, Morel-Forster A, Blumer C, Vetter T (2018) Occlusion-aware 3d morphable models and an illumination prior for face image analysis. Int J Comput Vis 126(12):1269–1287
Hernandez-Matamoros A, Bonarini A, Escamilla-Hernandez E, Nakano-Miyatake M, Perez-Meana H (2016) Facial expression recognition with automatic segmentation of face regions using a fuzzy based classification approach. Knowledge-Based Syst 110:1–14
Cyganek B, Gruszczyński S (2014) Hybrid computer vision system for drivers' eye recognition and fatigue monitoring. Neurocomputing 126:78–94
Anitha C (2019) Detection and analysis of drowsiness in human beings using multimodal signals. In: Digital Business. Springer, Cham, pp 157–174
Myllyneva A, Hietanen JK (2015) There is more to eye contact than meets the eye. Cognition 134:100–109
Ibrahim LF, Abulkhair M, AlShomrani AD, Al-Garni M, Al-Mutiry A, Al-Gamdi F, Kalenen R (2014) Using Haar classifiers to detect driver fatigue and provide alerts. Multimed Tools Appl 71(3):1857–1877
Data availability statement
All the data is collected from the simulation reports of the software and tools used by the authors. Authors are working on implementing the same using real world data with appropriate permissions.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that we have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
K, D.A., Keshaveni N An AVOA-LSTM with MRCNN for segmenting and classifying the sunglass image-based eye region identification. Multimed Tools Appl 83, 35073–35095 (2024). https://doi.org/10.1007/s11042-023-16800-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-16800-0