Improving face recognition in surveillance video with judicious selection and fusion of representative frames
Article No.: 6, Pages 1 - 7
Abstract
Face recognition in unconstrained surveillance videos is challenging due to the different acquisition settings and face variations. We propose to utilize the complementary correlation between multi-frames to improve face recognition performance. We design an algorithm to build a representative frame set from the video sequence, selecting faces with high quality and large appearance diversity. We also devise a refined Deep Residual Equivariant Mapping (DREAM) block to improve the discriminative power of the extracted deep features. Extensive experiments on two relevant face recognition benchmarks, YouTube Face and IJB-A, show the effectiveness of the proposed method. Our work is also lightweight, and can be easily embedded into existing CNN based face recognition systems.
References
[1]
Xiang An, Xuhan Zhu, Yang Xiao, Lan Wu, Ming Zhang, Yuan Gao, Bin Qin, Debing Zhang, and Ying Fu. 2020. Partial FC: Training 10 Million Identities on a Single Machine. arXiv preprint arXiv:2010.05222 (2020).
[2]
Kaidi Cao, Rong Yu, Li Cheng, Xiaoou Tang, and Change Loy Chen. 2018. Pose-Robust Face Recognition via Deep Residual Equivariant Mapping. In CVPR.
[3]
Qingxing Cao, Liang Lin, Yukai Shi, Xiaodan Liang, and Guanbin Li. 2017. Attention-aware face hallucination via deep reinforcement learning. In CVPR.
[4]
Sheng Chen, Yang Liu, Xiang Gao, and Zhen Han. 2018. Mobile-facenets: Efficient cnns for accurate real-time face verification on mobile devices. In Chinese Conference on Biometric Recognition.
[5]
Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. 2015. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv preprint arXiv:1512.01274 (2015).
[6]
Yu Chen, Ying Tai, Xiaoming Liu, Chunhua Shen, and Jian Yang. 2018. Fsrnet: End-to-end learning face super-resolution with facial priors. In CVPR.
[7]
Zhiyi Cheng, Xiatian Zhu, and Shaogang Gong. 2018. Surveillance face recognition challenge. arXiv preprint arXiv:1804.09691 (2018).
[8]
Jiankang Deng, Guo Jia, and Stefanos Zafeiriou. 2018. ArcFace: Additive Angular Margin Loss for Deep Face Recognition. In CVPR.
[9]
Changxing Ding and Dacheng Tao. 2017. Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE transactions on pattern analysis and machine intelligence 40, 4 (2017), 1002--1014.
[10]
Sixue Gong, Yichu Shi, Nathan D Kalka, and Anil K Jain. 2019. Video face recognition: Component-wise feature aggregation network (C-FAN). In International Conference on Biometrics.
[11]
Yandong Guo, Zhang Lei, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. 2016. MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. In ECCV.
[12]
Tal Hassner, Shai Harel, Eran Paz, and Roee Enbar. 2015. Effective Face Frontalization in Unconstrained Images. In CVPR.
[13]
Tal Hassner, Iacopo Masi, Jungyeon Kim, Jongmoo Choi, Shai Harel, Prem Natarajan, and Gerard Medioni. 2016. Pooling faces: Template based face recognition with pooled face images. In CVPR.
[14]
Brendan F. Klare, Ben Klein, Emma Taborsky, Austin Blanton, Jordan Cheney, Kristen Allen, Patrick Grother, Alan Mah, and Anil K. Jain. 2015. Pushing the Frontiers of Unconstrained Face Detection and Recognition:IARPA Janus Benchmark A. In CVPR.
[15]
Pei Li, Loreto Prieto, Domingo Mery, and Patrick Flynn. 2018. Face recognition in low quality images: a survey. arXiv preprint arXiv:1805.11519 (2018).
[16]
Yu Liu, Junjie Yan, and Wanli Ouyang. 2017. Quality aware network for set to set recognition. In CVPR.
[17]
Tran Luan, Yin Xi, and Xiaoming Liu. 2017. Disentangled Representation Learning GAN for Pose-Invariant Face Recognition. In CVPR.
[18]
Iacopo Masi, Anh Tun Trn, Tal Hassner, Jatuporn Toy Leksut, and Grard Medioni. 2016. Do We Really Need to Collect Millions of Faces for Effective Face Recognition?. In ECCV.
[19]
Connor J Parde, Carlos Castillo, Matthew Q Hill, Y Ivette Colon, Swami Sankaranarayanan, Jun-Cheng Chen, and Alice J O'Toole. 2016. Deep convolutional neural network features and the original image. arXiv preprint arXiv:1611.01751 (2016).
[20]
Omkar M Parkhi, Andrea Vedaldi, and Andrew Zisserman. 2015. Deep face recognition. British Machine Vision Association (2015).
[21]
Yongming Rao, Ji Lin, Jiwen Lu, and Jie Zhou. 2017. Learning discriminative aggregation network for video-based face recognition. In ICCV.
[22]
Swami Sankaranarayanan, Azadeh Alavi, Carlos D Castillo, and Rama Chellappa. 2016. Triplet probabilistic embedding for face verification and clustering. In 2016 IEEE 8th international conference on biometrics theory, applications and systems (BTAS).
[23]
Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In CVPR.
[24]
Ziyi Shen, Wei-Sheng Lai, Tingfa Xu, Jan Kautz, and Ming-Hsuan Yang. 2018. Deep semantic face deblurring. In CVPR.
[25]
Yaniv Taigman, Ming Yang, Marc'Aurelio Ranzato, and Lior Wolf. 2014. Deepface: Closing the gap to human-level performance in face verification. In CVPR.
[26]
Fei Wang, Liren Chen, Cheng Li, Shiyao Huang, Yanjie Chen, Chen Qian, and Chen Change Loy. 2018. The devil of face recognition is in the noise. In ECCV.
[27]
Feng Wang, Xiang Xiang, Jian Cheng, and Alan Loddon Yuille. 2017. Normface: L2 hypersphere embedding for face verification. In Proceedings of the 25th ACM international conference on Multimedia.
[28]
Lior Wolf, Tal Hassner, and Itay Maoz. 2011. Face recognition in unconstrained videos with matched background similarity. In CVPR.
[29]
Xiangyu Xu, Deqing Sun, Jinshan Pan, Yujin Zhang, Hanspeter Pfister, and Ming-Hsuan Yang. 2017. Learning to super-resolve blurry face and text images. In ICCV.
[30]
Jiaolong Yang, Peiran Ren, Dongqing Zhang, Dong Chen, Fang Wen, Hongdong Li, and Gang Hua. 2017. Neural Aggregation Network for Video Face Recognition. In CVPR.
[31]
Xin Yu and Fatih Porikli. 2017. Hallucinating very low-resolution unaligned and noisy face images by transformative discriminative autoencoders. In CVPR.
[32]
Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters 23, 10 (2016), 1499--1503.
[33]
Shizhan Zhu, Sifei Liu, Chen Change Loy, and Xiaoou Tang. 2016. Deep cascaded bi-network for face hallucination. In ECCV.
Index Terms
- Improving face recognition in surveillance video with judicious selection and fusion of representative frames
Recommendations
Towards Understanding Cross Resolution Feature Matching for Surveillance Face Recognition
MM '22: Proceedings of the 30th ACM International Conference on MultimediaCross-resolution face recognition (CRFR) in an open-set setting is a practical application for surveillance scenarios where low-resolution (LR) probe faces captured via surveillance cameras require being matched to a watchlist of high-resolution (HR) ...
Face Recognition Based Person Specific Identification for Video Surveillance Applications
WCI '15: Proceedings of the Third International Symposium on Women in Computing and InformaticsFace detection is an important aspect for applications like biometrics, video surveillance and human computer interaction. Videos provide abundant information and also that can be leveraged by temporal variations in pose, expression changes and ...
Improving fusion with optimal weight selection in Face Recognition
Face recognition has a large number of applications, including security/counterterrorism, person identification, Internet communications, E-commerce, and computer entertainment. Although research in automatic face recognition has been conducted since ...
Comments
Information & Contributors
Information
Published In
March 2021
512 pages
ISBN:9781450383080
DOI:10.1145/3444685
- General Chairs:
- Tat-Seng Chua,
- Jingdong Wang,
- Qi Tian,
- Program Chairs:
- Cathal Gurrin,
- Jia Jia,
- Hanwang Zhang,
- Qianru Sun
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 03 May 2021
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- State Key Laboratory of Mobile Network and Mobile Multimedia Technology, ZTE Corporation
Conference
MMAsia '20
Sponsor:
Acceptance Rates
Overall Acceptance Rate 59 of 204 submissions, 29%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 96Total Downloads
- Downloads (Last 12 months)11
- Downloads (Last 6 weeks)0
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in