Skip to main content
SpringerLink
Log in

Face illumination normalization based on generative adversarial network

  • Published:
Natural Computing Aims and scope Submit manuscript

Abstract

Face recognition technology has been widely used in the field of artificial intelligence. The technology needs to be carried out normally under the appropriate light, however, there is not ideal light, even poor-lighted for the face recognition device, and with the head in deflect angle. The poor-lighted under various head poses will influence the face recognition significantly. To address the issue, we present a novel and practical architecture based on deep fully convolutional neural network and generative adversarial networks for illumination normalization of facial images. The proposed method is termed as illumination normalization generative adversarial network. Compared with previous methods based on deep learning, our approach does not require identity and illumination label as input. We preserve identities of faces by an elaborately-designed generator together with content loss. Moreover, the framework of our scheme is simpler than previous methods based on deep learning. It can address the illumination of frontal and non-frontal face. In order to fairly evaluate the proposed method against state-of-the-art models, the peak signal to noise ratio is employed to estimate the performance of illumination normalization algorithm. Experimental results show that the proposed method achieves favorable normalization results against previous models under various head poses and illumination challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Adini Y, Moses Y, Ullman S (1997) Face recognition: the problem of compensating for changes in illumination direction. IEEE Trans Pattern Anal Mach Intell 19:721–732

    Article  Google Scholar 

  • Ahmad F, Khan A, Islam IU, Uzair M, Ullah H (2017) Illumination normalization using independent component analysis and filtering. Imaging Sci J 65(5):308–315

    Article  Google Scholar 

  • Al-Osaimi FR, Bennamoun M, Mian AS (2006) Illumination normalization for color face images. In: International Symposium on Visual Computing, Advances in Visual Computing. pp 90–101.

  • An G, Wu J, Ruan Q (2010) An illumination normalization model for face recognition under varied lighting conditions. Pattern Recogn Lett 31:1056–1067

    Article  Google Scholar 

  • BimaSenaBayu D, Miura J (2013) Fuzzy-based illumination normalization for face recognition. In: 2013 IEEE Workshop on Advanced Robotics and its Social Impacts, pp 131–136

  • Cao Q, Shen L, Xie W, Parkhi OM, Zisserman A (2018) Vggface2: a dataset for recognizing faces across pose and age. In: 13th IEEE International Conference on Automatic Face and Gesture Recognition, pp 67–74

  • Chen T, Yin W, Zhou XS, Comaniciu D, Huang TS (2005) Illumination normalization for face recognition and uneven background correction using total variation-based image models. IEEE Comput Soc Conf Comput vis Pattern Recogni 2:532–539

    Google Scholar 

  • Chen T, Yin W, Zhou XS, Comaniciu D, Huang TS (2006a) Total variation models for variable lighting face recognition. IEEE Trans Pattern Anal Mach Intell 28(9):1519–1524

    Article  Google Scholar 

  • Chen W, Er MJ, Wu S (2006b) Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain. IEEE Trans Syst Man Cybern B Cybern 36(2):458–466

    Article  Google Scholar 

  • Demir U, Lnal GB (2018) Patch-based image in painting with generative adversarial networks. Accessed from https://arxiv.org/abs/1803.07422

  • Du S, Ward RK (2005) Wavelet-based illumination normalization for face recognition. IEEE Int Conf Image Process 2:II–954

    Google Scholar 

  • Fan C-N, Zhang F-Y (2011) Homomorphic filtering-based illumination normalization method for face recognition. Pattern Recogn Lett 32:1468–1479

    Article  Google Scholar 

  • Fitzgibbon AW, Zisserman A (2002) On affine invariant clustering and automatic cast listing in movies. Computer vision. Springer, Berelin, pp 304–320

    Google Scholar 

  • Goel T, Nehra V, Vishwakarma VP (2013) Illumination normalization using down-scaling of low-frequency dct coefficients in dwt domain for face recognition. In: 2013 Sixth International Conference on Contemporary Computing, pp 295–300

  • González-Prieto Á, Mozo A, Talavera E, Gómez-Canaval S (2021) Dynamics of Fourier modes in torus generative adversarial networks. Mathematics 9:325. https://doi.org/10.3390/math9040325

    Article  Google Scholar 

  • Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B (2014) Generative adversarial nets. Adv Neural Inf Process Syst 27:2672–2680

    Google Scholar 

  • Gross R, Matthews I, Cohn J, Kanade T, Baker S (2010) Multi-pie. Image vis Comput 28(5):807–813

    Article  Google Scholar 

  • Guo DQ, Yang Q, Zhang Y, Jiang T, Yan H (2021a) Classification of domestic refuse in medical institutions based on transfer learning and convolutional neural network. Comput Model Eng Sci 127(2):599–620

    Google Scholar 

  • Guo DQ, Yang Q, Zhang Y, Zhang G, Zhu M, Yuan J (2021b) Adaptive object tracking discriminate model for multi-camera panorama surveillance in airport apron. Comput Model Eng Sci 129(1):191–205

    Google Scholar 

  • Guo DQ, Zhang GX, Neri F, Peng S, Yang Q, Liu P (2022) An adaptive kernelized correlation filters via multiple features in the tracking application. J vis Commun Image Represent 84:1–14

    Article  Google Scholar 

  • Han H, Chen SSX, Gao W (2013) A comparative study on illumination preprocessing in face recognition. Pattern Recogn 46:1691–1699

    Article  Google Scholar 

  • Han X, Yang H, Xing G, Liu Y (2020) Asymmetric joint gans for normalizing face illumination from a single image. IEEE Trans Multimed 22(6):1619–1633. https://doi.org/10.1109/TMM.2019.2945197

    Article  Google Scholar 

  • He K, Zhang X, Ren S and Sun, J (2016) Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 770–778

  • Huang R, Zhang S, Li T, He R (2017) Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp 2458–2467

  • Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 5967–5976

  • Jobson D, Rahman Z, Woodell G (1997) Properties and performance of a center/surround retinex. IEEE Trans Image Process 6(3):451–462

    Article  Google Scholar 

  • Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization, international conference on learning representations. Computer science. Accessed from https://arxiv.org/abs/1412.6980

  • Lee PH, Wu SW, Hung YP (2012) Illumination compensation using oriented local histogram equalization and its application to face recognition. IEEE Trans Image Process 21:4280–4289

    Article  MathSciNet  MATH  Google Scholar 

  • Li Y, Meng L, Feng J (2012) Lighting coefficients transfer based face illumination normalization. Chinese Conference on Pattern Recognition, pp 268–275

  • Lin J, Xia Y, Qin T, Chen Z, Liu T Y (2018) Conditional image-to-image translation. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, pp 5524–5532

  • Ma W, Xie X, Yin C, Lai JH (2018) Face image illumination processing based on generative adversarial nets. 2018 24th International Conference on Pattern Recognition (ICPR), pp 2558–2563

  • McLaughlin N, Ming J, Crookes D (2017) Largest matching areas for illumination and occlusion robust face recognition. IEEE Trans Cybern 47(3):796–808. https://doi.org/10.1109/TCYB.2016.2529300

    Article  Google Scholar 

  • Mirza M, Osindero S (2014) Conditional generative adversarial nets. Accessed from https://arxiv.org/abs/1411.1784

  • Moret-Tatay C, Baixauli-Fortea I, Grau Sevilla MD, Irigaray TQ (2020) Can you identify these celebrities? A network analysis on differences between word and face recognition. Mathematics 8(5):699. https://doi.org/10.3390/math8050699

    Article  Google Scholar 

  • Pareja-Corcho J, Betancur-Acosta O, Posada J, Tammaro A, Ruiz-Salguero O, Cadavid C (2020) Reconfigurable 3D CAD feature recognition supporting confluent n-dimensional topologies and geometric filters for prismatic and curved models. Mathematics 8(8):1356. https://doi.org/10.3390/math8081356

    Article  Google Scholar 

  • Phillips P J, Flynn P J, Scruggs T, Bowyer K W, and Worek W (2005) Overview of the face recognition grand challenge. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Vol 1, pp 947–954

  • Pizer SM, Amburn EP, Austin JD, Cromartie R, Zuiderveld K (1987) Adaptive histogram equalization and its variations. Comput vis Graphics Image Process 39(3):355–368

    Article  Google Scholar 

  • Pumarola A, Agudo A, Martinez AM, Sanfeliu A, Moreno-Noguer F (2018) Ganimation: anatomically-aware facial animation from a single image. Eur Conf Comput vis 8:835–851

    Google Scholar 

  • Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. International conference on learning representations. Accessed from https://arxiv.org/abs/1511.06434

  • Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Medical image computing and computer assisted intervention. Springer, Cham, pp 234–241

    Google Scholar 

  • Shan, S, Gao, W, Cao B, Zhao D (2003) Illumination normalization for robust face recognition against varying lighting conditions. In: 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443), pp 157–164

  • Shashua A, Riklin-Raviv T (1999) The quotient image: class-based rerendering and recognition with varying illuminations. IEEE Trans Pattern Anal Mach Intell 23:129–139

    Article  Google Scholar 

  • Shrivastava A, Pfister T, Tuzel O, Susskind J, Webb R (2017) Learning from simulated and unsupervised images through adversarial training. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2242–2251

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Accessed from https://arxiv.org/abs/1409.1556

  • Srisuk S, Petpon A (2008) A gabor quotient image for face recognition under varying illumination. ISVC advances in visual computing. Springer, Berlin, pp 511–520

    Chapter  Google Scholar 

  • Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19:1635–1650

    Article  MathSciNet  MATH  Google Scholar 

  • Tran L, Yin X, Liu X (2017) Disentangled representation learning gan for pose-invariant face recognition. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1283–1292

  • Tsai Y, Hsu L, Hsieh Y, Lin S (2020) The real-time depth estimation for an occluded person based on a single image and openpose method. Mathematics 8(8):1333. https://doi.org/10.3390/math8081333

    Article  Google Scholar 

  • Tu X, Yang F, Xie M, Ma Z (2017) Illumination normalization for face recognition using energy minimization framework. IEICE Trans 100-D:1376–1379

    Article  Google Scholar 

  • Ulyanov D, Vedaldi A, Lempitsky VS (2016) Instance normalization: The missing ingredient for fast stylization. Accessed from https://arxiv.org/abs/1607.08022

  • Vishwakarma VP (2015) Illumination normalization using fuzzy filter in dct domain for face recognition. Int J Mach Learn Cybern 6:17–34

    Article  Google Scholar 

  • Wang H, Li SZ, Wang Y (2004) Generalized quotient image. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2, II

  • Wang B, Li W, Yang W, Liao Q (2011) Illumination normalization based on weber’s law with application to face recognition. IEEE Signal Process Lett 18:462–465

    Article  Google Scholar 

  • Wang X, Yu K, Wu S, Gu J, Liu Y (2018) Esrgan: enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 63–79

  • Wang SH, Sun JD, Mehmood I, Pan CC, Chen Y, Zhang YD (2020a) Cerebral micro-bleeding identification based on a nine-layer convolutional neural network with stochastic pooling. Concurrency Comput 32:e5130.1-e5130.16

    Article  Google Scholar 

  • Wang SH, Muhammad K, Hong J, Sangaiah AK, Zhang YD (2020b) Alcoholism identification via convolutional neural network based on parametric ReLU, dropout, and batch normalization. Neural Comput Appl 32:665–680

    Article  Google Scholar 

  • Xie X, Lam K-M (2005) Face recognition under varying illumination based on a 2d face shape model. Pattern Recogn 38:221–230

    Article  Google Scholar 

  • Xie X, Lam K-M (2006) An efficient illumination normalization method for face recognition. Pattern Recognit Lett 27:609–617

    Article  Google Scholar 

  • Yang FW, Lin HJ, Yen S-H, Wang C-H (2019) A study on the convolutional neural algorithm of image style transfer. Int J Pattern Recognit Artif Intell 33(5):1954020

    Article  Google Scholar 

  • Zhang S, Zhu X, Lei Z, Shi H, Wang X, and Li S Z (2017) S3fd: Single shot scale-invariant face detector. 2017 IEEE International Conference on Computer Vision (ICCV), pp 192–201

  • Zhang Y, Wang L, Guan X, Wei H (2018) Illumination normalization for face recognition via jointly optimized dictionary-learning and sparse representation. IEEE Access 6:66632–66640

    Article  Google Scholar 

  • Zhang W, Zhao X, Morvan JM, Chen L (2019) Improving shadow suppression for illumination robust face recognition. IEEE Trans Pattern Anal Mach Intell 41(3):611–624

    Article  Google Scholar 

  • Zhang Y, Tsang I, Luo Y, Hu C, Lu X, Yu X (2021) Recursive copy and paste GAN: face hallucination from shaded thumbnails. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2021.3061312

    Article  Google Scholar 

  • Zhao X, Evangelopoulos G, Chu D, Shah SK, Kakadiaris IA (2014) Minimizing illumination differences for 3d to 2d face recognition using lighting maps. IEEE Trans Cybern 44:725–736

    Article  Google Scholar 

  • Zhao X, Shah S K and Kakadiaris IA (2012) Illumination normalization using self-lighting ratios for 3d2d face recognition. In: ECCV Workshops. European Conference on Computer Vision, Workshops and Demonstrations, pp 220–229

  • Zheng C, Wu S, Xu W, Xie S (2019) Illumination normalization via merging locally enhanced textures for robust face recognition. Accessed from https://arxiv.org/abs/1905.03904

  • Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp 2242–2251

Download references

Acknowledgements

The authors would like to thank the editor and anonymous reviewers for their insight and suggestions. We also thank Dr. Ferrante Neri (University of Surrey), Prof. DongC. Liu and Dr. Paul Liu (Stork Healthcare) for value suggestions. This work was supported in part by the National Natural Science Foundation of China under Grant 61806028, Grant 61672437, Grant 62103064 and Grant 61702428, Sichuan Science and Technology Program under Grant 21ZDYF2484, Grant 2021YFN0104, Grant 21GJHZ0061, Grant 21ZDYF3629, Grant 21ZDYF0418, Grant 21YYJC1827, Grant 2021YJ0086, Grant 2021YFG0295, Grant 21ZDYF3537, Grant 21ZDYF3598, Grant 2020YFG0177, Grant 2022YFN0020, Chinese Scholarship Council under Grant 202008510036, Opening Project of International Joint Research Center for Robotics and Intelligence System of Sichuan Province under Grant JQZN2021-003, Department of Science and Technology of Sichuan Province under Grant 2019YFSY0043, AECC Sichuan Gas Turbine Establishment, Key Laboratory on Aero-engine Altitude Simulation Technology, and Intelligent Control Education Reform Project of Chengdu University of Information Technology under Grant JYJG2021044. Program of Chengdu Technological University under Grant 2019ZR005, Program Name: Intelligent Sensing of Complex Power Environments Key Technologies and Applications.

Author information

Authors and Affiliations

  1. School of Automation, Chengdu University of Information Technology, Chengdu, 610225, Sichuan, China

    Dequan Guo, Lingrui Zhu, Gexiang Zhang, Qiang Yang, Shiqi Jiang, Sidong Wu & Junbao Liu

  2. Neijiang Vocational and Technical College, Neijiang, 641000, Sichuan, China

    Shenggui Ling

  3. National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu, 610065, Sichuan, China

    Shenggui Ling

  4. Chengdu Industry and Trade College (Chengdu Technician College), Chengdu, 611731, Sichuan, China

    Tianxiang Li

  5. Communication Engineering School, Chengdu Technological University, Chengdu, 611730, Sichuan, China

    Ping Wang

Authors
  1. Dequan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lingrui Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shenggui Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianxiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gexiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qiang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Shiqi Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Sidong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Junbao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shenggui Ling.

Ethics declarations

Conflict of interest

We promise that this manuscript is the authors’ original work and has not been published nor has it been submitted simultaneously elsewhere. All authors have checked the manuscript and have agreed to the submission.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, D., Zhu, L., Ling, S. et al. Face illumination normalization based on generative adversarial network. Nat Comput 22, 105–117 (2023). https://doi.org/10.1007/s11047-022-09892-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11047-022-09892-4

Keywords

Search

Navigation