research-article

Two-Stage Multi-Scale Resolution-Adaptive Network for Low-Resolution Face Recognition

Authors:

Lin FangAuthors Info & Claims

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Pages 4053 - 4062

https://doi.org/10.1145/3503161.3548196

Published: 10 October 2022 Publication History

Abstract

Low-resolution face recognition is challenging due to uncertain input resolutions and the lack of distinguishing details in low-resolution (LR) facial images. Resolution-invariant representations must be learned for optimal performance. Existing methods for this task mainly minimize the distance between the representations of the low-resolution (LR) and corresponding high-resolution (HR) image pairs in a common subspace. However, these works only focus on introducing various distance metrics at the final layer and between HR-LR image pairs. They do not fully utilize the intermediate layers or multi-resolution supervision, yielding only modest performance. In this paper, we propose a novel two-stage multi-scale resolution-adaptive network to learn more robust resolution-invariant representations. In the first stage, the structural patterns and the semantic patterns are distilled from HR images to provide sufficient supervision for LR images. A curriculum learning strategy facilitates the training of HR and LR image matching, smoothly decreasing the resolution of LR images. In the second stage, a multi-resolution contrastive loss is introduced on LR images to enforce intra-class clustering and inter-class separation of the LR representations. By introducing multi-scale supervision and multi-resolution LR representation clustering, our network can produce robust representations despite uncertain input sizes. Experimental results on eight benchmark datasets demonstrate the effectiveness of the proposed method. Code will be released at https://github.com/hhwang98/TMR.

Supplementary Material

MP4 File (MM22-fp1856.mp4)

ACM Publications Presenatation Videos.

Download
27.33 MB

References

[1]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597--1607.

[2]

Zhiyi Cheng, Xiatian Zhu, and Shaogang Gong. 2018a. Low-resolution face recognition. In Asian Conference on Computer Vision. Springer, 605--621.

[3]

Zhiyi Cheng, Xiatian Zhu, and Shaogang Gong. 2018b. Surveillance Face Recognition Challenge. arXiv preprint arXiv:1804.09691 (2018).

[4]

Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4690--4699.

[5]

Han Fang, Weihong Deng, Yaoyao Zhong, and Jiani Hu. 2020. Generate to adapt: Resolution adaption network for surveillance face recognition. In European Conference on Computer Vision. Springer, 741--758.

Digital Library

[6]

Mislav Grgic, Kresimir Delac, and Sonja Grgic. 2011. SCface--surveillance cameras face database. Multimedia tools and applications, Vol. 51, 3 (2011), 863--879.

[7]

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9729--9738.

[8]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[9]

R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. 2018. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670 (2018).

[10]

Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. 2008. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on faces in'Real-Life'Images: detection, alignment, and recognition.

[11]

Yuge Huang, Pengcheng Shen, Ying Tai, Shaoxin Li, Xiaoming Liu, Jilin Li, Feiyue Huang, and Rongrong Ji. 2020a. Improving face recognition from hard samples via distribution distillation loss. In European Conference on Computer Vision. Springer, 138--154.

Digital Library

[12]

Yuge Huang, Yuhan Wang, Ying Tai, Xiaoming Liu, Pengcheng Shen, Shaoxin Li, Jilin Li, and Feiyue Huang. 2020b. Curricularface: adaptive curriculum learning loss for deep face recognition. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5901--5910.

[13]

Qianfen Jiao, Rui Li, Wenming Cao, Jian Zhong, Si Wu, and Hau-San Wong. 2021. DDAT: Dual domain adaptive translation for low-resolution face verification in the wild. Pattern Recognition, Vol. 120 (2021), 108107.

Digital Library

[14]

Syed Safwan Khalid, Muhammad Awais, Zhen-Hua Feng, Chi-Ho Chan, Ammarah Farooq, Ali Akbari, and Josef Kittler. 2020. Resolution invariant face recognition using a distillation approach. IEEE Transactions on Biometrics, Behavior, and Identity Science, Vol. 2, 4 (2020), 410--420.

[15]

Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. 2017. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 212--220.

[16]

Cheng-Yaw Low, Andrew Beng-Jin Teoh, and Jaewoo Park. 2021. MIND-Net: A Deep Mutual Information Distillation Network for Realistic Low-Resolution Face Recognition. IEEE Signal Processing Letters, Vol. 28 (2021), 354--358.

[17]

Ze Lu, Xudong Jiang, and Alex Kot. 2018. Deep coupled resnet for low-resolution face recognition. IEEE Signal Processing Letters, Vol. 25, 4 (2018), 526--530.

[18]

Fabio Valerio Massoli, Giuseppe Amato, and Fabrizio Falchi. 2020. Cross-resolution learning for face recognition. Image and Vision Computing, Vol. 99 (2020), 103927.

[19]

Brianna Maze, Jocelyn Adams, James A Duncan, Nathan Kalka, Tim Miller, Charles Otto, Anil K Jain, W Tyler Niggel, Janet Anderson, Jordan Cheney, et al. 2018. Iarpa janus benchmark-c: Face dataset and protocol. In 2018 International Conference on Biometrics (ICB). IEEE, 158--165.

[20]

Qiang Meng, Shichao Zhao, Zhida Huang, and Feng Zhou. 2021. Magface: A universal representation for face recognition and quality assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14225--14234.

[21]

Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafeiriou. 2017. Agedb: the first manually collected, in-the-wild age database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 51--59.

[22]

Omkar M Parkhi, Andrea Vedaldi, and Andrew Zisserman. 2015. Deep face recognition. (2015).

[23]

Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition. 815--823.

[24]

Soumyadip Sengupta, Jun-Cheng Chen, Carlos Castillo, Vishal M Patel, Rama Chellappa, and David W Jacobs. 2016. Frontal to profile face verification in the wild. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1--9.

[25]

Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7794--7803.

[26]

Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. 2016. A discriminative feature learning approach for deep face recognition. In European conference on computer vision. Springer, 499--515.

[27]

Cameron Whitelam, Emma Taborsky, Austin Blanton, Brianna Maze, Jocelyn Adams, Tim Miller, Nathan Kalka, Anil K Jain, James A Duncan, Kristen Allen, et al. 2017. Iarpa janus benchmark-b face dataset. In proceedings of the IEEE conference on computer vision and pattern recognition workshops. 90--98.

[28]

S. Yi, X. Wang, and X. Tang. 2014. Deep Learning Face Representation by Joint Identification-Verification. Advances in neural information processing systems, Vol. 27 (2014).

[29]

Xi Yin, Ying Tai, Yuge Huang, and Xiaoming Liu. 2020. Fan: Feature adaptation network for surveillance face recognition and normalization. In Proceedings of the Asian Conference on Computer Vision.

[30]

Juan Zha and Hongyang Chao. 2019. Tcn: Transferable coupled network for cross-resolution face recognition. In ICASSP 2019--2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3302--3306.

Cited By

Chen PLiu HDing JLuo JTan PYang LCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Holistic-CAM: Ultra-lucid and Sanity Preserving Visual Interpretation in Holistic Stage of CNNsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681707(5423-5431)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681707
Alipour Talemi NKashiani HNasrabadi N(2024)CATFace: Cross-Attribute-Guided Transformer With Self-Attention Distillation for Low-Quality Face RecognitionIEEE Transactions on Biometrics, Behavior, and Identity Science10.1109/TBIOM.2023.33492186:1(132-146)Online publication date: Jan-2024
https://doi.org/10.1109/TBIOM.2023.3349218
Shi RGuo WGe S(2024)Low-Resolution Face Recognition via Adaptable Instance-Relation Distillation2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10651293(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10651293
Show More Cited By

Index Terms

Two-Stage Multi-Scale Resolution-Adaptive Network for Low-Resolution Face Recognition
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations

Recommendations

Multi-resolution feature fusion for face recognition

For face recognition, image features are first extracted and then matched to those features in a gallery set. The amount of information and the effectiveness of the features used will determine the recognition performance. In this paper, we propose a ...
Low-resolution face recognition based on feature-mapping face hallucination
Abstract
The image recognition approaches based on Convolutional Neural Network (CNN) have already achieved tremendous performance on super-resolution face images. However, there exist several challenges in face recognition field. For example, ...
A robust face super-resolution algorithm and its application in low-resolution face recognition system
Abstract
In real-world surveillance scenario, the face recognition (FR) systems pose a lot of challenges due to the captured low-resolution (LR) and noisy probe images. A new face super-resolution (SR) algorithm is proposed to design a recognition model ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

7537 pages

ISBN:9781450392037

DOI:10.1145/3503161

General Chairs:
João Magalhães
NOVA University of Lisbon, Portugal
,
Alberto del Bimbo
University of Florence, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Xavier Alameda-Pineda
Inria, Grenoble, France
,
Qin Jin
Renmin University of China, China
,
Vincent Oria
New Jersey Institute of Technology, USA
,
Laura Toni
University College London, UK

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

project from Anhui Science and Technology Agency
National Natural Science Foundation of China

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
205
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)4

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen PLiu HDing JLuo JTan PYang LCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Holistic-CAM: Ultra-lucid and Sanity Preserving Visual Interpretation in Holistic Stage of CNNsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681707(5423-5431)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681707
Alipour Talemi NKashiani HNasrabadi N(2024)CATFace: Cross-Attribute-Guided Transformer With Self-Attention Distillation for Low-Quality Face RecognitionIEEE Transactions on Biometrics, Behavior, and Identity Science10.1109/TBIOM.2023.33492186:1(132-146)Online publication date: Jan-2024
https://doi.org/10.1109/TBIOM.2023.3349218
Shi RGuo WGe S(2024)Low-Resolution Face Recognition via Adaptable Instance-Relation Distillation2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10651293(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10651293
Chen LChen JXu ZLiao YChen Z(2024)Two-stage dual-resolution face network for cross-resolution face recognition in surveillance systemsThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03121-440:8(5545-5556)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s00371-023-03121-4
Chen JChen JWang XXu DLiang CHan Z(2023)Learning Degradation for Real-World Face Super-ResolutionAdvances in Computer Graphics10.1007/978-3-031-50072-5_10(120-131)Online publication date: 29-Dec-2023
https://doi.org/10.1007/978-3-031-50072-5_10
Yuan YLi JLi YQiang JZhu YYang YShen X(2022)Fractional Multiset Coherent Super-Resolution Representation for Low Resolution Face Recognition2022 IEEE 8th International Conference on Cloud Computing and Intelligent Systems (CCIS)10.1109/CCIS57298.2022.10016425(155-159)Online publication date: 26-Nov-2022
https://doi.org/10.1109/CCIS57298.2022.10016425

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten