skip to main content
research-article

Mutual Component Analysis for Heterogeneous Face Recognition

Published: 08 March 2016 Publication History

Abstract

Heterogeneous face recognition, also known as cross-modality face recognition or intermodality face recognition, refers to matching two face images from alternative image modalities. Since face images from different image modalities of the same person are associated with the same face object, there should be mutual components that reflect those intrinsic face characteristics that are invariant to the image modalities. Motivated by this rationality, we propose a novel approach called Mutual Component Analysis (MCA) to infer the mutual components for robust heterogeneous face recognition. In the MCA approach, a generative model is first proposed to model the process of generating face images in different modalities, and then an Expectation Maximization (EM) algorithm is designed to iteratively learn the model parameters. The learned generative model is able to infer the mutual components (which we call the hidden factor, where hidden means the factor is unreachable and invisible, and can only be inferred from observations) that are associated with the person’s identity, thus enabling fast and effective matching for cross-modality face recognition. To enhance recognition performance, we propose an MCA-based multiclassifier framework using multiple local features. Experimental results show that our new approach significantly outperforms the state-of-the-art results on two typical application scenarios: sketch-to-photo and infrared-to-visible face recognition.

References

[1]
Peter N. Belhumeur, João P. Hespanha, and David Kriegman. 1997. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19, 7 (1997), 711--720.
[2]
Stefano Berretti, Alberto Del Bimbo, and Pietro Pala. 2012. Distinguishing facial features for ethnicity-based 3D face recognition. ACM Trans. Intell. Syst. Technol. 3, 3, Article 45 (May 2012), 20 pages.
[3]
Rama Chellappa, Charles L. Wilson, and Saad Sirohey. 1995. Human and machine recognition of faces: A survey. Proc. IEEE 83, 5 (1995), 705--741.
[4]
Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Vol. 1. IEEE, 886--893.
[5]
Changxing Ding, Chang Xu, and Dacheng Tao. 2015. Multi-task pose-invariant face recognition. IEEE Trans. Image Process. 24, 3 (2015), 980--993.
[6]
Zhengming Ding, Shao Ming, and Yun Fu. 2014. Latent low-rank transfer subspace learning for missing modality recognition. In AAAI Conference on Artificial Intelligence. AAAI.
[7]
Ralph Ewerth, Markus Mühling, and Bernd Freisleben. 2012. Robust video content analysis via transductive learning. ACM Trans. Intell. Syst. Technol. 3, 3, Article 41 (May 2012), 26 pages.
[8]
Cong Geng and Xudong Jiang. 2011. Face recognition based on the multi-scale local image structures. Pattern Recognit. 44, 10 (2011), 2565--2575.
[9]
Xiangsheng Huang, Zhen Lei, Mingyu Fan, Xiao Wang, and Stan Z. Li. 2013. Regularized discriminative spectral regression method for heterogeneous face matching. IEEE Trans. Image Process. 22, 1 (2013), 353--362.
[10]
Kui Jia and Shaogang Gong. 2008. Generalized face super-resolution. IEEE Trans. Image Process. 17, 6 (2008), 873--886.
[11]
Thorsten Joachims. 1998. Text categorization with Support Vector Machines: Learning with many relevant features. In Proc. Eur. Conf. Mach. Learn., Claire Ndellec and Cline Rouveirol (Eds.). Lecture Notes in Computer Science, Vol. 1398. Springer, Berlin, 137--142.
[12]
Brendan Klare and Anil K. Jain. 2010. Heterogeneous face recognition: Matching NIR to visible light images. In Proc. Int. Conf. Pattern Recognit. IEEE, 1513--1516.
[13]
Brendan F. Klare and Anil K. Jain. 2013. Heterogeneous face recognition using kernel prototype similarities. IEEE Trans. Pattern Anal. Mach. Intell. 35, 6 (2013), 1410--1422.
[14]
Brendan F. Klare, Zhifeng Li, and Anil K. Jain. 2011. Matching forensic sketches to mug shot photos. IEEE Trans. Pattern Anal. Mach. Intell. 33, 3 (2011), 639--646.
[15]
Zhen Lei and Stan Z. Li. 2009. Coupled spectral regression for matching heterogeneous faces. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. IEEE, 1123--1128.
[16]
Annan Li, Shiguang Shan, Xilin Chen, and Wen Gao. 2009. Maximizing intra-individual correlations for face recognition across pose differences. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. IEEE, 605--611.
[17]
Stan Z. Li. 2009. Heterogeneous face biometrics. In Encyclopedia of Biometrics. Springer, 700--702.
[18]
Yuelong Li, Li Meng, Jufu Feng, and Jigang Wu. 2014. Downsampling sparse representation and discriminant information aided occluded face recognition. Sci. China Inf. Sci. 57 (2014).
[19]
Zhifeng Li, Dihong Gong, Xuelong Li, and Dacheng Tao. 2015. Learning compact feature descriptor and adaptive matching framework for face recognition. IEEE Trans. Image Process. 24, 9 (2015), 2736--2745.
[20]
Zhifeng Li, Dihong Gong, Yu Qiao, and Dacheng Tao. 2014. Common feature discriminant analysis for matching infrared face images to optical face images. IEEE Trans. Image Process. 23, 6 (2014), 2436--2445.
[21]
Zhifeng Li, Dahua Lin, and Xiaoou Tang. 2009. Nonparametric discriminant analysis for face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 4 (2009), 755--761.
[22]
Zhifeng Li, Unsang Park, and Anil K. Jain. 2011. A discriminative model for age invariant face recognition. IEEE Trans. Inf. Forensics Secur. 6, 3 (2011), 1028--1037.
[23]
Zhifeng Li and Xiaoou Tang. 2007. Using support vector machines to enhance the performance of Bayesian face recognition. IEEE Trans. Inf. Forensics Secur. 2, 2 (2007), 174--180.
[24]
Shengcai Liao, Dong Yi, Zhen Lei, Rui Qin, and Stan Z. Li. 2009. Heterogeneous face recognition from local structures of normalized appearance. In Advances in Biometrics. Springer, 209--218.
[25]
Dahua Lin and Xiaoou Tang. 2006. Inter-modality face recognition. In Proc. Eur. Conf. Comput. Vis. Springer, 13--26.
[26]
Qingshan Liu, Xiaoou Tang, Hongliang Jin, Hanqing Lu, and Songde Ma. 2005. A nonlinear approach for face sketch synthesis and recognition. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Vol. 1. IEEE, 1005--1010.
[27]
David G. Lowe. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 2 (2004), 91--110.
[28]
Topi Mäenpää and Matti Pietikäinen. 2003. Multi-scale binary patterns for texture analysis. In Image Analysis. Springer, 885--892.
[29]
Krystian Mikolajczyk and Cordelia Schmid. 2005. A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27, 10 (2005), 1615--1630.
[30]
Timo Ojala, Matti Pietikainen, and Topi Maenpaa. 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 7 (2002), 971--987.
[31]
Karl Ricanek and Tamirat Tesafaye. 2006. Morph: A longitudinal image database of normal adult age-progression. In International Conference on Automatic Face and Gesture Recognition. IEEE, 341--345.
[32]
Ling Shao, Li Liu, and Xuelong Li. 2014a. Feature learning for image classification via multiobjective genetic programming. IEEE Trans. Neural Netw. Learn. Syst. 25, 7 (2014), 1359--1371.
[33]
Ling Shao, Di Wu, and Xuelong Li. 2014b. Learning deep and wide: A spectral method for learning deep networks. IEEE Trans. Neural Netw. Learn. Syst. 25, 12 (2014), 2303--2308.
[34]
Ming Shao, Zhengming Ding, and Yun Fu. 2015. Sparse low-rank fusion based deep features for missing modality face recognition. In IEEE International Conference and Workshops on Automatic Face and Gesture Recognition. IEEE.
[35]
Ming Shao and Yun Fu. 2013. Hierarchical hyperlingual-words for multi-modality face classification. IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (2013).
[36]
Abhishek Sharma and David W. Jacobs. 2011. Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. IEEE, 593--600.
[37]
Xiaoou Tang and Zhifeng Li. 2009. Audio-guided video based face recognition. IEEE Trans. Circ. Syst. Video Technol. 19, 7 (2009), 955--964.
[38]
Xiaoou Tang and Xiaogang Wang. 2004. Face sketch recognition. IEEE Trans. Circuits Syst. Video Technol. 14, 1 (2004), 50--57.
[39]
Joshua B. Tenenbaum and William T. Freeman. 2000. Separating style and content with bilinear models. Neural Comput. 12, 6 (2000), 1247--1283.
[40]
Matthew A. Turk and Alex P. Pentland. 1991. Face recognition using eigenfaces. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. IEEE, 586--591.
[41]
Xiaogang Wang and Xiaoou Tang. 2004. A unified framework for subspace face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 26, 9 (2004), 1222--1228.
[42]
Xiaogang Wang and Xiaoou Tang. 2009. Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 11 (2009), 1955--1967.
[43]
Lior Wolf, Tal Hassner, Yaniv Taigman, and others. 2008. Descriptor based methods in the wild. In Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment, and Recognition.
[44]
John Wright and Gang Hua. 2009. Implicit elastic matching with random projections for pose-variant face recognition. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. IEEE, 1502--1509.
[45]
Ruomei Yan, Ling Shao, and Yan Liu. 2013. Nonlocal hierarchical dictionary learning using wavelets for image denoising. IEEE Trans. Image Process. 22, 12 (2013), 4689--4698.
[46]
Dong Yi, Rong Liu, RuFeng Chu, Zhen Lei, and Stan Z. Li. 2007. Face matching between near infrared and visible light images. In Advances in Biometrics. Springer, 523--530.
[47]
Lei Zhang, Xiantong Zhen, and Ling Shao. 2014. Learning object-to-class kernels for scene classification. IEEE Trans. Image Process. 23, 8 (2014), 3241--3253.
[48]
Wei Zhang, Xiaogang Wang, and Xiaoou Tang. 2011a. Coupled information-theoretic encoding for face photo-sketch recognition. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. IEEE, 513--520.
[49]
Zeda Zhang, Yunhong Wang, and Zhaoxiang Zhang. 2011b. Face synthesis from near-infrared to visual light via sparse representation. International Joint Conference on Biometrics (2011).
[50]
W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld. 2003. Face recognition: A literature survey. ACM Comput. Surv. 35, 4 (Dec. 2003), 399--458.
[51]
Xiantong Zhen, Ling Shao, and Xuelong Li. 2014. Action recognition by spatio-temporal oriented energies. Information Sciences 281 (2014), 295--309.

Cited By

View all
  • (2024)Backdoor Attack With Sparse and Invisible TriggerIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.341193619(6364-6376)Online publication date: 2024
  • (2024)A semantic modular framework for events topic modeling in social mediaMultimedia Tools and Applications10.1007/s11042-023-15745-883:4(10755-10778)Online publication date: 1-Jan-2024
  • (2023)Temporal Dynamic Concept Modeling Network for Explainable Video Event RecognitionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/356831219:6(1-22)Online publication date: 12-Jul-2023
  • Show More Cited By

Index Terms

  1. Mutual Component Analysis for Heterogeneous Face Recognition

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Intelligent Systems and Technology
    ACM Transactions on Intelligent Systems and Technology  Volume 7, Issue 3
    Regular Papers, Survey Papers and Special Issue on Recommender System Benchmarks
    April 2016
    472 pages
    ISSN:2157-6904
    EISSN:2157-6912
    DOI:10.1145/2885506
    • Editor:
    • Yu Zheng
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 March 2016
    Accepted: 01 July 2015
    Revised: 01 July 2015
    Received: 01 October 2014
    Published in TIST Volume 7, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Face recognition
    2. heterogeneous face recognition
    3. mutual component analysis (MCA)

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    • Natural Science Foundation of Guangdong Province
    • National Natural Science Foundation of China
    • Shenzhen Institutes of Advanced Technology
    • Chinese Academy of Sciences
    • Key Laboratory of Human-Machine Intelligence-Synergy Systems
    • Guangdong Innovative Research Team Program
    • Key Research Program of the Chinese Academy of Sciences
    • The Chinese University of Hong Kong
    • Australian Research Council Projects
    • Shun Hing Institute of Advanced Engineering

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)8
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 17 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Backdoor Attack With Sparse and Invisible TriggerIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.341193619(6364-6376)Online publication date: 2024
    • (2024)A semantic modular framework for events topic modeling in social mediaMultimedia Tools and Applications10.1007/s11042-023-15745-883:4(10755-10778)Online publication date: 1-Jan-2024
    • (2023)Temporal Dynamic Concept Modeling Network for Explainable Video Event RecognitionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/356831219:6(1-22)Online publication date: 12-Jul-2023
    • (2023)LARNeXt: End-to-End Lie Algebra Residual Network for Face RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.3279378(1-16)Online publication date: 2023
    • (2023)Towards Robust Model Watermark via Reducing Parametric Vulnerability2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00438(4728-4738)Online publication date: 1-Oct-2023
    • (2023)Backdoor Defense via Adaptively Splitting Poisoned Dataset2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00390(4005-4014)Online publication date: Jun-2023
    • (2022)Towards Age-Invariant Face RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2020.301142644:1(474-487)Online publication date: 1-Jan-2022
    • (2022)Memory-Modulated Transformer Network for Heterogeneous Face RecognitionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2022.317796017(2095-2109)Online publication date: 2022
    • (2021)Multispectral Face Recognition Using Transfer Learning with Adaptation of Domain Specific UnitsSensors10.3390/s2113452021:13(4520)Online publication date: 1-Jul-2021
    • (2021)Kernelized Heterogeneity-Aware Cross-View Face RecognitionFrontiers in Artificial Intelligence10.3389/frai.2021.6705384Online publication date: 20-Jul-2021
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media