skip to main content
10.1145/3444685.3446259acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Improving face recognition in surveillance video with judicious selection and fusion of representative frames

Published: 03 May 2021 Publication History

Abstract

Face recognition in unconstrained surveillance videos is challenging due to the different acquisition settings and face variations. We propose to utilize the complementary correlation between multi-frames to improve face recognition performance. We design an algorithm to build a representative frame set from the video sequence, selecting faces with high quality and large appearance diversity. We also devise a refined Deep Residual Equivariant Mapping (DREAM) block to improve the discriminative power of the extracted deep features. Extensive experiments on two relevant face recognition benchmarks, YouTube Face and IJB-A, show the effectiveness of the proposed method. Our work is also lightweight, and can be easily embedded into existing CNN based face recognition systems.

References

[1]
Xiang An, Xuhan Zhu, Yang Xiao, Lan Wu, Ming Zhang, Yuan Gao, Bin Qin, Debing Zhang, and Ying Fu. 2020. Partial FC: Training 10 Million Identities on a Single Machine. arXiv preprint arXiv:2010.05222 (2020).
[2]
Kaidi Cao, Rong Yu, Li Cheng, Xiaoou Tang, and Change Loy Chen. 2018. Pose-Robust Face Recognition via Deep Residual Equivariant Mapping. In CVPR.
[3]
Qingxing Cao, Liang Lin, Yukai Shi, Xiaodan Liang, and Guanbin Li. 2017. Attention-aware face hallucination via deep reinforcement learning. In CVPR.
[4]
Sheng Chen, Yang Liu, Xiang Gao, and Zhen Han. 2018. Mobile-facenets: Efficient cnns for accurate real-time face verification on mobile devices. In Chinese Conference on Biometric Recognition.
[5]
Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. 2015. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv preprint arXiv:1512.01274 (2015).
[6]
Yu Chen, Ying Tai, Xiaoming Liu, Chunhua Shen, and Jian Yang. 2018. Fsrnet: End-to-end learning face super-resolution with facial priors. In CVPR.
[7]
Zhiyi Cheng, Xiatian Zhu, and Shaogang Gong. 2018. Surveillance face recognition challenge. arXiv preprint arXiv:1804.09691 (2018).
[8]
Jiankang Deng, Guo Jia, and Stefanos Zafeiriou. 2018. ArcFace: Additive Angular Margin Loss for Deep Face Recognition. In CVPR.
[9]
Changxing Ding and Dacheng Tao. 2017. Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE transactions on pattern analysis and machine intelligence 40, 4 (2017), 1002--1014.
[10]
Sixue Gong, Yichu Shi, Nathan D Kalka, and Anil K Jain. 2019. Video face recognition: Component-wise feature aggregation network (C-FAN). In International Conference on Biometrics.
[11]
Yandong Guo, Zhang Lei, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. 2016. MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. In ECCV.
[12]
Tal Hassner, Shai Harel, Eran Paz, and Roee Enbar. 2015. Effective Face Frontalization in Unconstrained Images. In CVPR.
[13]
Tal Hassner, Iacopo Masi, Jungyeon Kim, Jongmoo Choi, Shai Harel, Prem Natarajan, and Gerard Medioni. 2016. Pooling faces: Template based face recognition with pooled face images. In CVPR.
[14]
Brendan F. Klare, Ben Klein, Emma Taborsky, Austin Blanton, Jordan Cheney, Kristen Allen, Patrick Grother, Alan Mah, and Anil K. Jain. 2015. Pushing the Frontiers of Unconstrained Face Detection and Recognition:IARPA Janus Benchmark A. In CVPR.
[15]
Pei Li, Loreto Prieto, Domingo Mery, and Patrick Flynn. 2018. Face recognition in low quality images: a survey. arXiv preprint arXiv:1805.11519 (2018).
[16]
Yu Liu, Junjie Yan, and Wanli Ouyang. 2017. Quality aware network for set to set recognition. In CVPR.
[17]
Tran Luan, Yin Xi, and Xiaoming Liu. 2017. Disentangled Representation Learning GAN for Pose-Invariant Face Recognition. In CVPR.
[18]
Iacopo Masi, Anh Tun Trn, Tal Hassner, Jatuporn Toy Leksut, and Grard Medioni. 2016. Do We Really Need to Collect Millions of Faces for Effective Face Recognition?. In ECCV.
[19]
Connor J Parde, Carlos Castillo, Matthew Q Hill, Y Ivette Colon, Swami Sankaranarayanan, Jun-Cheng Chen, and Alice J O'Toole. 2016. Deep convolutional neural network features and the original image. arXiv preprint arXiv:1611.01751 (2016).
[20]
Omkar M Parkhi, Andrea Vedaldi, and Andrew Zisserman. 2015. Deep face recognition. British Machine Vision Association (2015).
[21]
Yongming Rao, Ji Lin, Jiwen Lu, and Jie Zhou. 2017. Learning discriminative aggregation network for video-based face recognition. In ICCV.
[22]
Swami Sankaranarayanan, Azadeh Alavi, Carlos D Castillo, and Rama Chellappa. 2016. Triplet probabilistic embedding for face verification and clustering. In 2016 IEEE 8th international conference on biometrics theory, applications and systems (BTAS).
[23]
Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In CVPR.
[24]
Ziyi Shen, Wei-Sheng Lai, Tingfa Xu, Jan Kautz, and Ming-Hsuan Yang. 2018. Deep semantic face deblurring. In CVPR.
[25]
Yaniv Taigman, Ming Yang, Marc'Aurelio Ranzato, and Lior Wolf. 2014. Deepface: Closing the gap to human-level performance in face verification. In CVPR.
[26]
Fei Wang, Liren Chen, Cheng Li, Shiyao Huang, Yanjie Chen, Chen Qian, and Chen Change Loy. 2018. The devil of face recognition is in the noise. In ECCV.
[27]
Feng Wang, Xiang Xiang, Jian Cheng, and Alan Loddon Yuille. 2017. Normface: L2 hypersphere embedding for face verification. In Proceedings of the 25th ACM international conference on Multimedia.
[28]
Lior Wolf, Tal Hassner, and Itay Maoz. 2011. Face recognition in unconstrained videos with matched background similarity. In CVPR.
[29]
Xiangyu Xu, Deqing Sun, Jinshan Pan, Yujin Zhang, Hanspeter Pfister, and Ming-Hsuan Yang. 2017. Learning to super-resolve blurry face and text images. In ICCV.
[30]
Jiaolong Yang, Peiran Ren, Dongqing Zhang, Dong Chen, Fang Wen, Hongdong Li, and Gang Hua. 2017. Neural Aggregation Network for Video Face Recognition. In CVPR.
[31]
Xin Yu and Fatih Porikli. 2017. Hallucinating very low-resolution unaligned and noisy face images by transformative discriminative autoencoders. In CVPR.
[32]
Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters 23, 10 (2016), 1499--1503.
[33]
Shizhan Zhu, Sifei Liu, Chen Change Loy, and Xiaoou Tang. 2016. Deep cascaded bi-network for face hallucination. In ECCV.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MMAsia '20: Proceedings of the 2nd ACM International Conference on Multimedia in Asia
March 2021
512 pages
ISBN:9781450383080
DOI:10.1145/3444685
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 May 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. face quality
  2. facial prior
  3. feature enhancement
  4. multi frame fusion
  5. surveillance face recognition

Qualifiers

  • Research-article

Funding Sources

  • State Key Laboratory of Mobile Network and Mobile Multimedia Technology, ZTE Corporation

Conference

MMAsia '20
Sponsor:
MMAsia '20: ACM Multimedia Asia
March 7, 2021
Virtual Event, Singapore

Acceptance Rates

Overall Acceptance Rate 59 of 204 submissions, 29%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 96
    Total Downloads
  • Downloads (Last 12 months)11
  • Downloads (Last 6 weeks)0
Reflects downloads up to 28 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media