Skip to main content
SpringerLink
Log in

Eyeglasses removal based on attributes detection and improved TV restoration model

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Eyeglasses are common occlusions on face images. The detection of eyeglasses attributes and eyeglasses removal are key factors for correct automatic face recognition. A method for eyeglasses removal based on attributes detection and improved Total Variation (TV) restoration model is proposed in this paper. First, existence of eyeglasses frames is determined based on the width-length ratio after location of the eyeglasses; second, color coefficient and skin likelihood ratio are defined and color information is determined; third, bright index is defined and the reflective areas are detected based on luminance information. Finally, for rimmed, colorless and non-reflective eyeglasses, influence function based on gray difference ratio is defined to improve TV restoration model for eyeglasses removal. Experimental results show that our proposed method can not only discriminate the existence of the frame, but also detect color information and reflective areas accurately. In addition, the eyeglasses removal effect is superior to the traditional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Bai CX, Jiang GY, Yu M, et al. (2013) A micro-image fusion algorithm based on region growing. J Electron 30:91–96

    Google Scholar 

  2. Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. Transactions on Intelligent Systems and Technology 2:389–396

    Google Scholar 

  3. Chen WQ, Wang BL (2016) Methods of eyeglasses detection and frame removal for face image. Computer Engineering and Applications 52:178–182

    Google Scholar 

  4. Chen WQ, Wang BL (2016) Methods of eyeglasses detection and frame removal for face image. Computer Engineering and Applications 52:178–182

    Google Scholar 

  5. Chen WQ, Wang BL (2016) Methods of eyeglasses detection and frame removal for face image. Computer Engineering and Applications 52:178–182

    Google Scholar 

  6. Criminisi A, Prez P, Toyama K (2003) Object removal by exemplar-based inpainting. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2, pp 721–728

  7. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. Computer Vision and Pattern Recognition 1, pp 886–893

  8. Du C, Su G (2005) Eyeglasses removal from facial images. Pattern Recogn Lett 26:2215–2220

    Article  Google Scholar 

  9. Fan HQ, Cao ZM, Jiang YN, Yin Q, Dou CC (2014) Learning deep face representation. IEEE Conference on Computer Vision and Pattern Recognition, pp 1–10

  10. Fang F, Zhou F, Wang RM, et al. (2014) A fast and efficient mesh segmentation method based on improved region growing. Applied Mathematics: A Journal of Chinese Universities(Series B) 29:468–480

    Article  MathSciNet  Google Scholar 

  11. Guo P, Su F (2014) Enhanced PCA reconstruction method for eyeglass frame auto-removal. IEEE international conference on network infrastructure and digital content, pp 359–363

  12. Jia Y, Liu P, Niu S (2012) Image processing and programming based onpartial differential equations Science Press

  13. Jiang X, Binkert M, Achermann B, Bunke H (2000) Towards detection of glasses on facial images. Pattern Anal Applic 2:13–29

    Google Scholar 

  14. Jing Z, Mariani R, Wu JK (2000) Glasses detection for face recognition using Bayes rules. Third International Conference on advances in Multimodal Interfaces, pp 127–134

  15. Kang J, Li J (2017) Eye location algorithm based on integral projection and template matching. Journal of Shaanxi University of Science & Technology 35:174–177

    Google Scholar 

  16. Lanitis A, Taylor CJ, Cootes TF (2002) Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis & Machine Intelligence 19(7):743–756

    Article  Google Scholar 

  17. Li K, Wei Y, Yang Z, et al. (2016) Image inpainting algorithm based on TV model and evolutionary algorithm. Soft Comput 20:885–893

    Article  Google Scholar 

  18. Li ZM, Zhao Y, Hu WJ (2014) A method for removing eyeglasses from frontal facial images. Industrial instrumentation and automation 3:51–53

    Google Scholar 

  19. Liu ZM, Li ZM, Wang YY, et al. (2014) The research of glasses removal based on PCA/ICA and error compensation. Optical Technique 40:429–433

    Article  Google Scholar 

  20. Liu ZM, Li ZM, Wang YY, et al. (2014) Research on removing glasses based on PCA/ICA and error compensation algorithm. Optical technology 40:429–433

    Article  Google Scholar 

  21. Mei YP (2012) Research on glasses detection in the face image based on rectangular form filter east China normal university

  22. Moreno-Cadenas JA, Gomez-Castaneda F, Anzueto-Rios A, et al. (2016) Fuzzy inference system for region segmentation using the YCbcr color model, 13th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE), IEEE Computer Society

  23. Namrata S, Utkarsh A, Soumava KR et al (2015) Human identification using Linear Multiclass SVM and Eye Movement biometrics, 8th International Conference on Contemporary Computing. IEEE Computer Society 1, pp 365–369

  24. Pan J, Wang XS, Cheng YH (2016) Ingle-sample face recognition based on LPP feature transfer. IEEE Access 4:2873–2883

    Article  Google Scholar 

  25. Ren MG, Chen YL, Cai XD, Yang C (2014) Glasses detection based on the edge information of face image. Software Guide 13:141–143

    Google Scholar 

  26. SMirkamali S, Nagabhushan P (2015) Object removal by depth-wise image inpainting, Signal. Image and Video Processing 1:1785–1794

    Article  Google Scholar 

  27. Shen H, Li SM, Mao JG, et al. (2018) Summary of digital image restoration technology. Journal of Image and Graphics 1:1764–1775

    Google Scholar 

  28. Sun Y, Wang XG, Tang XO (2014) Deep learning face representation from predicting 10,000 classes. IEEE Conference on Computer Vision and Pattern Recognition, pp 1891–1898

  29. Taigman Y, Yang M, Ranzato MA, Wolf L (2014) Deepface:closing the gap to human-level performance in face verification. IEEE Conference on Computer Vision and Pattern Recognition, pp 1701–1708

  30. Tony C, Marquina A, Mulet P (2000) High-Order Total Variation-Based image restoration. SIAM J Sci Comput 22:503–516

    Article  MathSciNet  Google Scholar 

  31. Wu CY, Liu C, Shum HY, et al. (2004) Automatic eyeglasses removal from face images. IEEE Transactions on Pattern Analysis and Machine Intelligence 26:322–336

    Article  Google Scholar 

  32. Wu D, Wen Q (2010) PDE-Based enhancement algorithm for dim-target images. International Conference on Artificial Intelligence and Computational Intelligence, pp 74–77

  33. Yin Y, Li D, Hu L (2013) Adaptive image inpainting algorithm based on CDD model. Journal of Chongqing University 36:80–86

    Google Scholar 

  34. Zhang XL, Zhan K, Zhou HL, Shen YJ (2015) A fast eye state computing algorithm. In: 2015 6Th IEEE international conference on software engineering and service science, beijing, china, pp. 587–590

  35. Zhao Y, Li X (2008) A rapid image inpainting algorithm based on CDD model Computer Simulation 2

  36. Zhao MH, Zhang X, Shi ZH (2018) Eyeglasses detection, location and frame discriminant based on edge information projection. Multimedia tools and applications 77:14931–14949

    Article  Google Scholar 

  37. Zhou X, Zhu Y, Zhang Y (2018) The algorithm of face recognition with occlusion based on partition. Computer Applications and Software 35:183–186

    Google Scholar 

Download references

Acknowledgements

This research was funded by National Key Technology R&D Program of China, grant number 2017YFB1402103-3, National Natural Science Foundation of China, grant number 61602373, Natural Science Foundation of Shaanxi province, China, grant number 2019JM-381, and Key Laboratory Foundation of Shaanxi Education Department(grant number 20JS086).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Xi’an University of Technology, Xi’an, 710048, China

    Minghua Zhao, Zhe Zhang, Xin Zhang, Lili Zhang & Bing Li

  2. Shaanxi Key Laboratory of Network Computing and Security Technology, Xi’an, Shannxi, China

    Minghua Zhao

Authors
  1. Minghua Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhe Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lili Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Minghua Zhao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, M., Zhang, Z., Zhang, X. et al. Eyeglasses removal based on attributes detection and improved TV restoration model. Multimed Tools Appl 80, 2691–2712 (2021). https://doi.org/10.1007/s11042-020-09715-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09715-7

Keywords

Search

Navigation