Abstract
Partially occluded faces are very common in automatic face recognition (FR) in the real world. We explore the problem of FR with occlusion by embedding Image Gradient Orientations (IGO) into robust error coding. The existing works usually put stress on the error distribution in the non-occluded region but neglect the one in the occluded region due to its unpredictability incurred by irregular occlusion. However, in the IGO domain, the error distribution in the occluded region can be built simply and elegantly by a uniform distribution in the interval \(\left[ -\pi ,\pi \right)\), and the one in the occluded region can be well built by a weight-conditional Gaussian distribution. By incorporating the two error distributions and a Markov random field for the priori distribution of the occlusion support, we propose a joint probabilistic generative model for a novel IGO-embedded Structural Error Coding (IGO-SEC) model. Two methods, a new reconstruction method and a new robust structural error metric, are further presented to boost the performance of IGO-SEC. Extensive experiments on 8 popular robust FR methods and 4 benchmark face databases demonstrate the effectiveness and robustness of IGO-SEC in dealing with facial occlusion and occlusion-like variations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Note that it makes nonsense to impose a robust error metric on each pixel.
The VGG and LCNN model can be downloaded from http://www.robots.ox.ac.uk/~vgg/software/vgg_face/ and https://github.com/AlfredXiangWu/face_verification_experiment, respectively.
Note that the lastest version (Sun et al. 2016) of DeepID was totally based on VGG.
References
Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28:2037–2041
Chan TH, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2015) PCANet: a simple deep learning baseline for image classification? IEEE Trans Image Process 24:5017–5032
Chen SS, Donoho DL, Saunders MA (2001) Atomic decomposition by basis pursuit. SIAM Rev 43:129–159
Colombo A, Cusano C, Schettini R (2011) UMB-DB: a database of partially occluded 3d faces. In: IEEE International Conference on Computer Vision Workshops, Barcelona, Spain, pp 2113–2119
Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23:643–660
Ghazi MM, Ekenel HK (2016) A comprehensive analysis of deep learning based representation for face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops. Las Vegas, USA, pp 34–41
He R, Zheng W, Hu B (2011) Maximum correntropy criterion for robust face recognition. IEEE Trans Pattern Anal Mach Intell 33:1561–1576
He R, Zheng WS, Tan T, Sun Z (2014) Half-quadratic-based iterative minimization for robust sparse representation. IEEE Trans Pattern Anal Mach Intell 36:261–275
Hua G, Yang MH, Learned-Miller E, Ma Y, Turk M, Kriegman DJ, Huang TS (2011) Introduction to the special section on real-world face recognition. IEEE Trans Pattern Anal Mach Intell 33:1921–1924
Huang GB, Mattar M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07–49, University of Massachusetts
Huang GB, Mattar M, Lee H, Learned-Miller E (2012) Learning to align from scratch. Advances in neural information processing systems. Lake Tahoe, Nevada, USA, pp 764–772
Kolmogorov V, Zabih R (2004) What energy functions can be minimized via graph cuts? IEEE Trans Pattern Anal Mach Intell 26:147–159
Lee K, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27:684–698
Li XX, Dai DQ, Zhang XF, Ren CX (2013) Structured sparse error coding for face recognition with occlusion. IEEE Trans Image Process 22:1889–1900
Li XX, He L, Hao P, Liu Z, Li J (2017) Adaptive weberfaces for occlusion-robust face representation and recognition. IET Image Process 11:964–975
Liang R, Li XX (2015) Mixed error coding for face recognition with mixed occlusions. In: International Joint Conference on Artificial Intelligence. Buenos Aires, Argentina, pp 3657–3663
Lin D, Tang X (2007) Quality-driven face occlusion detection and recovery. In: IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, USA, pp 1–7
Liu W, Pokharel PP, Principe JC (2007) Correntropy: properties and applications in non-gaussian signal processing. IEEE Trans Signal Process 55:5286–5298
Luo L, Yang J, Qian J, Tai Y (2015) Nuclear-l1 norm joint regression for face reconstruction and recognition with mixed noise. Pattern Recog 48:3811–3824
Martinez AM (1998) The AR face database. Technical Report 24, Computer Visual Center, Ohio State University
Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: British Machine Vision Conference, London: BMVA Press, Swansea, UK, pp 41.1–41.12
Portugal L, Judice J, Vicente L (1994) A comparison of block pivoting and interior-point algorithms for linear least squares problems with nonnegative variables. Math Comput 63:625–644
Qian J, Yang J, Xu Y (2013) Local structure-based image decomposition for feature extraction with applications to face recognition. IEEE Trans Image Process 22:3591–3603
Qian J, Luo L, Yang J, Zhang F, Lin Z (2015) Robust nuclear norm regularized regression for face recognition with occlusion. Pattern Recog 48:3145–3159
Sun Y, Wang X, Tang X (2015) Deeply learned face representations are sparse, selective, and robust. In: IEEE Conference on Computer Vision and Pattern Recognition. MA, USA, Boston, pp 2892–2900
Sun Y, Wang X, Tang X (2016) Sparsifying neural network connections for face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, pp 4856–4864
Taigman Y, Yang M, Ranzato M, Wolf L (2015) Web-scale training for face identification. In: IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, pp 2746–2754
Tzimiropoulos G, Zafeiriou S, Pantic M (2012) Subspace learning from image gradient orientations. IEEE Trans Pattern Anal Mach Intell 34:2454–66
Wang JJY, Wang (2013) Non-negative matrix factorization by maximizing correntropy for cancer clustering. BMC Bioinform 14:1–11
Wei X, Li CT, Hu Y (2012) Robust face recognition under varying illumination and occlusion considering structured sparsity. In: International conference on digital image computing techniques and applications. Fremantle, Australia, pp 1–7
Wright J, Ma Y (2010) Dense error correction via \(\ell ^1\)-minimization. IEEE Trans Inf Theory 56:3540–3560
Wright J, Yang A, Ganesh A, Sastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31:210–227
Wu X, He R, Sun Z (2015) A lightened CNN for deep face representation. CoRR abs/1511.02683. arXiv:1511.02683
Yang J, Luo L, Qian J, Tai Y, Zhang F, Xu Y (2017) Nuclear norm based matrix regression with applications to face recognition with occlusion and illumination changes. IEEE Trans Pattern Anal Mach Intell 39:156–171
Yang M, Zhang L, Yang J, Zhang D (2011) Robust sparse coding for face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA, pp 625–632
Yang M, Zhang L, Shiu SC, Zhang D (2013a) Gabor feature based robust representation and classification for face recognition with gabor occlusion dictionary. Pattern Recog. 46:1865–1878
Yang M, Zhang L, Shiu SK, Zhang D (2013b) Robust kernel representation with statistical local features for face recognition. IEEE Trans Neural Netw Learn Syst 24:900–912
Yang M, Zhang L, Yang J, Zhang D (2013c) Regularized robust coding for face recognition. IEEE Trans Image Process 22:1753–1766
Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: Which helps face recognition? In: IEEE International Conference on Computer Vision, Barcelona, Spain, pp 471–478
Zhang T, Tang YY, Fang B, Shang Z, Liu X (2009) Face recognition under varying illumination using gradientfaces. IEEE Trans Image Process 18:2599–2606
Zhao F, Feng J, Zhao J, Yang W, Yan S (2018) Robust lstm-autoencoders for face de-occlusion in the wild. IEEE Trans Image Process 27:778–790
Zhou Z, Wagner A, Mobahi H, Wright J, Ma Y (2009) Face recognition with contiguous occlusion using markov random fields. In: IEEE International Conference on Computer Vision. Kyoto, Japan, pp 1050–1057
Acknowledgements
This work is partially supported by Natural Science Foundation of Zhejiang Province (LY18F020031), National Natural Science Foundation of China (61379020, 61402411, 61802347).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, XX., Hao, P., He, L. et al. Image gradient orientations embedded structural error coding for face recognition with occlusion. J Ambient Intell Human Comput 11, 2349–2367 (2020). https://doi.org/10.1007/s12652-019-01257-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-019-01257-7