research-article

Knowledge Guided Representation Disentanglement for Face Recognition from Low Illumination Images

Authors:

Shangfei WangAuthors Info & Claims

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Pages 6655 - 6663

https://doi.org/10.1145/3503161.3548174

Published: 10 October 2022 Publication History

Abstract

Low illumination face recognition is challenging as details are lacking due to lighting conditions. Retinex theory points out that images can be divided into reflectance with color constancy and ambient illumination. Inspired by this, we propose a knowledge-guided representation disentanglement method to disentangle facial images into face-related and illumination-related features, and then leverage the disentangled face-related features for face recognition. Specifically, the proposed method consists of two components: feature disentanglement and face classifier. Following Retinex, high-dimensional face-related features and ambient illumination-related features are extracted from facial images. Reconstruction and crossreconstruction methods are used to make sure the integrity and accuracy of the disentangled features. Furthermore, we find that the influence of illumination changes on illumination-related features should be invariant for faces of different identities, so we design an illumination offset loss to satisfy the prior invariance for better disentanglement. Finally high-dimensional face-related features are mapped to low-dimensional features through the face classifier for use in face recognition task. Experimental results on low illumination and NIR-VIS datasets demonstrate the superiority and effectiveness of our proposed method.

Supplementary Material

MP4 File (MM22-fp1745.mp4)

Presentation video

Download
4.01 MB

References

[1]

Jonas Adler and Sebastian Lunz. 2018. Banach wasserstein gan. Advances in Neural Information Processing Systems, Vol. 31 (2018).

[2]

Debotosh Bhattacharjee and Hiranmoy Roy. 2019. Pattern of local gravitational force (PLGF): A novel local image descriptor. IEEE transactions on pattern analysis and machine intelligence, Vol. 43, 2 (2019), 595--607.

[3]

David H Brainard and Brian A Wandell. 1986. Analysis of the retinex theory of color vision. JOSA A, Vol. 3, 10 (1986), 1651--1661.

[4]

Feng Chen, Nian Wang, Jun Tang, and Fan Zhu. 2021. A feature disentangling approach for person re-identification via self-supervised data augmentation. Applied Soft Computing, Vol. 100 (2021), 106939.

[5]

Jie Chen, Dong Yi, Jimei Yang, Guoying Zhao, Stan Z Li, and Matti Pietikainen. 2009. Learning mappings for face synthesis from near infrared to visual light images. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 156--163.

[6]

Seokeon Choi, Sumin Lee, Youngeun Kim, Taekyung Kim, and Changick Kim. 2020. Hi-CMD: Hierarchical cross-modality disentanglement for visible-infrared person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10257--10266.

[7]

Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4690--4699.

[8]

Chaoyou Fu, Xiang Wu, Yibo Hu, Huaibo Huang, and Ran He. 2019. Dual variational generation for low shot heterogeneous face recognition. Advances in Neural Information Processing Systems, Vol. 32 (2019).

[9]

Wen Gao, Bo Cao, Shiguang Shan, Xilin Chen, Delong Zhou, Xiaohua Zhang, and Debin Zhao. 2007. The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, Vol. 38, 1 (2007), 149--161.

[10]

Xuyang Guo, Meina Kan, Zhenliang He, Xingguang Song, and Shiguang Shan. 2021. Image style disentangling for instance-level facial attribute transfer. Computer Vision and Image Understanding, Vol. 207 (2021), 103205.

[11]

Xianjun Han, Hongyu Yang, Guanyu Xing, and Yanli Liu. 2020. Asymmetric Joint GANs for Normalizing Face Illumination From a Single Image. IEEE Transactions on Multimedia, Vol. 22, 6 (2020), 1619--1633. https://doi.org/10.1109/TMM.2019.2945197

[12]

Ran He, Jie Cao, Lingxiao Song, Zhenan Sun, and Tieniu Tan. 2019. Adversarial cross-spectral face completion for NIR-VIS face recognition. IEEE transactions on pattern analysis and machine intelligence, Vol. 42, 5 (2019), 1025--1037.

[13]

Ran He, Xiang Wu, Zhenan Sun, and Tieniu Tan. 2017. Learning invariant deep representation for nir-vis face recognition. In Thirty-First AAAI Conference on Artificial Intelligence.

Digital Library

[14]

Ran He, Xiang Wu, Zhenan Sun, and Tieniu Tan. 2018. Wasserstein cnn: Learning invariant features for nir-vis face recognition. IEEE transactions on pattern analysis and machine intelligence, Vol. 41, 7 (2018), 1761--1773.

[15]

Weipeng Hu and Haifeng Hu. 2019. Discriminant deep feature learning based on joint supervision loss and multi-layer feature fusion for heterogeneous face recognition. Computer Vision and Image Understanding, Vol. 184 (2019), 9--21.

Digital Library

[16]

Weipeng Hu and Haifeng Hu. 2020. Adversarial disentanglement spectrum variations and cross-modality attention networks for NIR-VIS face recognition. IEEE Transactions on Multimedia, Vol. 23 (2020), 145--160.

[17]

Yaocong Hu, Mingqi Lu, Chao Xie, and Xiaobo Lu. 2021. FIN-GAN: Face illumination normalization via retinex-based self-supervised learning and conditional generative adversarial network. Neurocomputing, Vol. 456 (2021), 109--125.

Digital Library

[18]

Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. 2007. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Technical Report 07--49. University of Massachusetts, Amherst.

[19]

Xun Huang and Serge Belongie. 2017. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE international conference on computer vision. 1501--1510.

[20]

Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. 2018. Multimodal unsupervised image-to-image translation. In Proceedings of the European conference on computer vision (ECCV). 172--189.

Digital Library

[21]

Subhadeep Koley, Hiranmoy Roy, Soumyadip Dhar, and Debotosh Bhattacharjee. 2022. Illumination invariant face recognition using Fused Cross Lattice Pattern of Phase Congruency (FCLPPC). Information Sciences, Vol. 584 (2022), 633--648.

Digital Library

[22]

K.C. Lee, J. Ho, and D. Kriegman. 2005. Acquiring Linear Subspaces for Face Recognition under Variable Lighting. IEEE Trans. Pattern Anal. Mach. Intelligence, Vol. 27, 5 (2005), 684--698.

Digital Library

[23]

Stan Li, Dong Yi, Zhen Lei, and Shengcai Liao. 2013. The casia nir-vis 2.0 face database. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 348--353.

Digital Library

[24]

Hongtao Liang, Jie Gao, and Ning Qiang. 2021. A novel framework based on wavelet transform and principal component for face recognition under varying illumination. Applied Intelligence, Vol. 51, 3 (2021), 1762--1783.

Digital Library

[25]

Fenglin Liu, Xian Wu, Shen Ge, Xuancheng Ren, Wei Fan, Xu Sun, and Yuexian Zou. 2021. DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attention. ACM Transactions on Knowledge Discovery from Data (TKDD), Vol. 16, 1 (2021), 1--19.

[26]

Yan Lu, Yue Wu, Bin Liu, Tianzhu Zhang, Baopu Li, Qi Chu, and Nenghai Yu. 2020. Cross-modality person re-identification with shared-specific feature transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13379--13389.

[27]

Takaya Miyamoto, Hiroshi Hashimoto, Akihiro Hayasaka, Akinori F Ebihara, and Hitoshi Imaoka. 2021. Joint Feature Distribution Alignment Learning for NIR-VIS and VIS-VIS Face Recognition. In 2021 IEEE International Joint Conference on Biometrics (IJCB). IEEE, 1--8.

[28]

Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition. 815--823.

[29]

Lingxiao Song, Man Zhang, Xiang Wu, and Ran He. 2018. Adversarial discriminative heterogeneous face recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.

[30]

Virendra P Vishwakarma and Sahil Dalal. 2020. A novel non-linear modifier for adaptive illumination normalization for robust face recognition. Multimedia Tools and Applications, Vol. 79, 17 (2020), 11503--11529.

Digital Library

[31]

Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. 2018. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5265--5274.

[32]

Huijiao Wang, Haijian Zhang, Lei Yu, Li Wang, and Xulei Yang. 2020. Facial Feature Embedded CycleGAN for VIS-NIR Translation. In ICASSP 2020--2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1903--1907.

[33]

Wenjing Wang, Wenhan Yang, and Jiaying Liu. 2021. Hla-face: Joint high-low adaptation for low light face detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16195--16204.

[34]

Xiang Wu, Huaibo Huang, Vishal M Patel, Ran He, and Zhenan Sun. 2019. Disentangled variational representation for heterogeneous face recognition. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 9005--9012.

Digital Library

[35]

Liping Yang, Bin Yang, and Xiaohua Gu. 2021. Adversarial reconstruction CNN for illumination-robust frontal face image recovery and recognition. International Journal of Cognitive Informatics and Natural Intelligence (IJCINI), Vol. 15, 2 (2021), 18--33.

[36]

Wenhan Yang, Ye Yuan, Wenqi Ren, Jiaying Liu, Walter J Scheirer, Zhangyang Wang, Taiheng Zhang, Qiaoyong Zhong, Di Xie, Shiliang Pu, et al. 2020. Advancing image understanding in poor visibility environments: A collective benchmark study. IEEE Transactions on Image Processing, Vol. 29 (2020), 5737--5752.

[37]

Dong Yi, Zhen Lei, and Stan Z Li. 2015. Shared representation learning for heterogenous face recognition. In 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), Vol. 1. IEEE, 1--7.

[38]

Yinghui Zhang, Chun Liu, Bo Sun, Jun He, and Lejun Yu. 2021. NIR-VIS Heterogeneous Face Synthesis via Enhanced Asymmetric CycleGAN. In 2021 International Joint Conference on Neural Networks (IJCNN). IEEE, 1--8.

[39]

Yutong Zheng, Dipan K Pal, and Marios Savvides. 2018. Ring loss: Convex feature normalization for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5089--5097.

[40]

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In 2017 IEEE International Conference on Computer Vision (ICCV). 2242--2251.

[41]

Yan Zhuang, Shiying Li, Xuwang Yin, Abu Hasnat Mohammad Rubaiyat, Gustavo K Rohde, et al. 2022. Local Sliced-Wasserstein Feature Sets for Illumination-invariant Face Recognition. arXiv preprint arXiv:2202.10642 (2022).

Cited By

Zhuang YLi SShifat-E-Rabbi MYin XRubaiyat ARohde G(2025)Local sliced Wasserstein feature sets for illumination invariant face recognitionPattern Recognition10.1016/j.patcog.2025.111381162(111381)Online publication date: Jun-2025
https://doi.org/10.1016/j.patcog.2025.111381
Zhou XGao ZGong HLi S(2024)DeFFace: Deep Face Recognition Unlocked by Illumination AttributesElectronics10.3390/electronics1322456613:22(4566)Online publication date: 20-Nov-2024
https://doi.org/10.3390/electronics13224566

Index Terms

Knowledge Guided Representation Disentanglement for Face Recognition from Low Illumination Images
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object recognition

Recommendations

Face recognition under varying illumination using gradientfaces

In this correspondence, we propose a novel method to extract illumination insensitive features for face recognition under varying lighting called the Gradientfaces. Theoretical analysis shows Gradientfaces is an illumination insensitive measure, and ...
Illumination-robust face recognition based on Gabor feature face intrinsic identity PCA model
CIMMACS'08: Proceedings of the 7th WSEAS international conference on Computational intelligence, man-machine systems and cybernetics

Robust face recognition under various illumination environments is essential for successful commercialization. Feature-based face recognition relies on a good choice of feature vectors. However, there is no feature vector invariant under illumination ...
Illumination Invariant Face Recognition Using Near-Infrared Images

Most current face recognition systems are designed for indoor, cooperative-user applications. However, even in thus-constrained applications, most existing systems, academic and commercial, are compromised in accuracy by changes in environmental ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

7537 pages

ISBN:9781450392037

DOI:10.1145/3503161

General Chairs:
João Magalhães
NOVA University of Lisbon, Portugal
,
Alberto del Bimbo
University of Florence, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Xavier Alameda-Pineda
Inria, Grenoble, France
,
Qin Jin
Renmin University of China, China
,
Vincent Oria
New Jersey Institute of Technology, USA
,
Laura Toni
University College London, UK

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Anhui Science and Technology Agency

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
182
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhuang YLi SShifat-E-Rabbi MYin XRubaiyat ARohde G(2025)Local sliced Wasserstein feature sets for illumination invariant face recognitionPattern Recognition10.1016/j.patcog.2025.111381162(111381)Online publication date: Jun-2025
https://doi.org/10.1016/j.patcog.2025.111381
Zhou XGao ZGong HLi S(2024)DeFFace: Deep Face Recognition Unlocked by Illumination AttributesElectronics10.3390/electronics1322456613:22(4566)Online publication date: 20-Nov-2024
https://doi.org/10.3390/electronics13224566

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten