Abstract
Eyeglasses are common occlusions on face images. The detection of eyeglasses attributes and eyeglasses removal are key factors for correct automatic face recognition. A method for eyeglasses removal based on attributes detection and improved Total Variation (TV) restoration model is proposed in this paper. First, existence of eyeglasses frames is determined based on the width-length ratio after location of the eyeglasses; second, color coefficient and skin likelihood ratio are defined and color information is determined; third, bright index is defined and the reflective areas are detected based on luminance information. Finally, for rimmed, colorless and non-reflective eyeglasses, influence function based on gray difference ratio is defined to improve TV restoration model for eyeglasses removal. Experimental results show that our proposed method can not only discriminate the existence of the frame, but also detect color information and reflective areas accurately. In addition, the eyeglasses removal effect is superior to the traditional methods.














Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Bai CX, Jiang GY, Yu M, et al. (2013) A micro-image fusion algorithm based on region growing. J Electron 30:91–96
Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. Transactions on Intelligent Systems and Technology 2:389–396
Chen WQ, Wang BL (2016) Methods of eyeglasses detection and frame removal for face image. Computer Engineering and Applications 52:178–182
Chen WQ, Wang BL (2016) Methods of eyeglasses detection and frame removal for face image. Computer Engineering and Applications 52:178–182
Chen WQ, Wang BL (2016) Methods of eyeglasses detection and frame removal for face image. Computer Engineering and Applications 52:178–182
Criminisi A, Prez P, Toyama K (2003) Object removal by exemplar-based inpainting. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2, pp 721–728
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. Computer Vision and Pattern Recognition 1, pp 886–893
Du C, Su G (2005) Eyeglasses removal from facial images. Pattern Recogn Lett 26:2215–2220
Fan HQ, Cao ZM, Jiang YN, Yin Q, Dou CC (2014) Learning deep face representation. IEEE Conference on Computer Vision and Pattern Recognition, pp 1–10
Fang F, Zhou F, Wang RM, et al. (2014) A fast and efficient mesh segmentation method based on improved region growing. Applied Mathematics: A Journal of Chinese Universities(Series B) 29:468–480
Guo P, Su F (2014) Enhanced PCA reconstruction method for eyeglass frame auto-removal. IEEE international conference on network infrastructure and digital content, pp 359–363
Jia Y, Liu P, Niu S (2012) Image processing and programming based onpartial differential equations Science Press
Jiang X, Binkert M, Achermann B, Bunke H (2000) Towards detection of glasses on facial images. Pattern Anal Applic 2:13–29
Jing Z, Mariani R, Wu JK (2000) Glasses detection for face recognition using Bayes rules. Third International Conference on advances in Multimodal Interfaces, pp 127–134
Kang J, Li J (2017) Eye location algorithm based on integral projection and template matching. Journal of Shaanxi University of Science & Technology 35:174–177
Lanitis A, Taylor CJ, Cootes TF (2002) Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis & Machine Intelligence 19(7):743–756
Li K, Wei Y, Yang Z, et al. (2016) Image inpainting algorithm based on TV model and evolutionary algorithm. Soft Comput 20:885–893
Li ZM, Zhao Y, Hu WJ (2014) A method for removing eyeglasses from frontal facial images. Industrial instrumentation and automation 3:51–53
Liu ZM, Li ZM, Wang YY, et al. (2014) The research of glasses removal based on PCA/ICA and error compensation. Optical Technique 40:429–433
Liu ZM, Li ZM, Wang YY, et al. (2014) Research on removing glasses based on PCA/ICA and error compensation algorithm. Optical technology 40:429–433
Mei YP (2012) Research on glasses detection in the face image based on rectangular form filter east China normal university
Moreno-Cadenas JA, Gomez-Castaneda F, Anzueto-Rios A, et al. (2016) Fuzzy inference system for region segmentation using the YCbcr color model, 13th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE), IEEE Computer Society
Namrata S, Utkarsh A, Soumava KR et al (2015) Human identification using Linear Multiclass SVM and Eye Movement biometrics, 8th International Conference on Contemporary Computing. IEEE Computer Society 1, pp 365–369
Pan J, Wang XS, Cheng YH (2016) Ingle-sample face recognition based on LPP feature transfer. IEEE Access 4:2873–2883
Ren MG, Chen YL, Cai XD, Yang C (2014) Glasses detection based on the edge information of face image. Software Guide 13:141–143
SMirkamali S, Nagabhushan P (2015) Object removal by depth-wise image inpainting, Signal. Image and Video Processing 1:1785–1794
Shen H, Li SM, Mao JG, et al. (2018) Summary of digital image restoration technology. Journal of Image and Graphics 1:1764–1775
Sun Y, Wang XG, Tang XO (2014) Deep learning face representation from predicting 10,000 classes. IEEE Conference on Computer Vision and Pattern Recognition, pp 1891–1898
Taigman Y, Yang M, Ranzato MA, Wolf L (2014) Deepface:closing the gap to human-level performance in face verification. IEEE Conference on Computer Vision and Pattern Recognition, pp 1701–1708
Tony C, Marquina A, Mulet P (2000) High-Order Total Variation-Based image restoration. SIAM J Sci Comput 22:503–516
Wu CY, Liu C, Shum HY, et al. (2004) Automatic eyeglasses removal from face images. IEEE Transactions on Pattern Analysis and Machine Intelligence 26:322–336
Wu D, Wen Q (2010) PDE-Based enhancement algorithm for dim-target images. International Conference on Artificial Intelligence and Computational Intelligence, pp 74–77
Yin Y, Li D, Hu L (2013) Adaptive image inpainting algorithm based on CDD model. Journal of Chongqing University 36:80–86
Zhang XL, Zhan K, Zhou HL, Shen YJ (2015) A fast eye state computing algorithm. In: 2015 6Th IEEE international conference on software engineering and service science, beijing, china, pp. 587–590
Zhao Y, Li X (2008) A rapid image inpainting algorithm based on CDD model Computer Simulation 2
Zhao MH, Zhang X, Shi ZH (2018) Eyeglasses detection, location and frame discriminant based on edge information projection. Multimedia tools and applications 77:14931–14949
Zhou X, Zhu Y, Zhang Y (2018) The algorithm of face recognition with occlusion based on partition. Computer Applications and Software 35:183–186
Acknowledgements
This research was funded by National Key Technology R&D Program of China, grant number 2017YFB1402103-3, National Natural Science Foundation of China, grant number 61602373, Natural Science Foundation of Shaanxi province, China, grant number 2019JM-381, and Key Laboratory Foundation of Shaanxi Education Department(grant number 20JS086).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhao, M., Zhang, Z., Zhang, X. et al. Eyeglasses removal based on attributes detection and improved TV restoration model. Multimed Tools Appl 80, 2691–2712 (2021). https://doi.org/10.1007/s11042-020-09715-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09715-7