Yuan Wang
ABSTRACT
During the coronavirus pandemic, the demand for contactless biometrics technology has promoted the development of masked face recognition. Training a masked face recognition model needs to address two crucial issues: a lack of large-scale realistic masked face datasets and the difficulty of obtaining robust face representations due to the huge difference between complete faces and masked faces. To tackle with the first issue, this paper proposes to train a 3D masked face recognition network with non-masked face images. For the second issue, this paper utilizes the geometric features of 3D face, namely depth, azimuth, and elevation, to represent the face. The inherent advantages of 3D face enhance the stability and practicability of 3D masked face recognition network. In addition, a facial geometry extractor is proposed to highlight discriminative facial geometric features so that the 3D masked face recognition network can take full advantage of the depth, azimuth and elevation information in distinguishing face identities. The experimental results on four public 3D face datasets show that the proposed 3D masked face recognition network improves the accuracy of the masked face recognition, which verifies the feasibility of training the masked face recognition model with non-masked face images.
- Jeremy Howard, Austin Huang, Zhiyuan Li, Zeynep Tufekci, Vladimir Zdimal, Helene-Mari Van Der Westhuizen, Arne Von Delft, Amy Price, Lex Fridman, Lei-Han Tang, Viola Tang, Gregory L. Watson, Christina E. Bax, Reshama Shaikh, Frederik Questier, Danny Hernandez, Larry F. Chu, Christina M. Ramirez, and Anne W. Rimoin. 2021. An evidence review of face masks against COVID-19. Proceedings of the National Academy of Sciences 118, 4 (January 2021). https://doi.org/10.1073/pnas.2014564118Google Scholar
Cross Ref
- Naser Damer, Jonas Henry Grebe, Cong Chen, Fadi Boutros, Florian Kirchbuchner, and Arjan Kuijper. 2020. The effect of wearing a mask on face recognition performance: an exploratory study. In Proceedings of 2020 International Conference of the Biometrics Special Interest Group (BIOSIG). IEEE, Darmstadt, Germany, 1-6.Google Scholar
- Zhongyuan Wang, Guangcheng Wang, Baojin Huang, Zhangyang Xiong, Qi Hong, Hao Wu, Peng Yi, Kui Jiang, Nanxi Wang, Yingjiao Pei, Heling Chen, Yu Miao, Zhibing Huang, and Jinbi Liang. 2020. Masked face recognition dataset and application. arXiv preprint arXiv:2003.09093v2 (2020).Google Scholar
- Aqeel Anwar and Arijit Raychowdhury. 2020. Masked face recognition for secure authentication. arXiv preprint arXiv:2008.11104v (2020).Google Scholar
- Mengyue Geng, Peixi Peng, Yangru Huang, and Yonghong Tian. 2020. Masked face recognition with generative data augmentation and domain constrained ranking. In Proceedings of the 28th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 2246–2254. https://doi.org/10.1145/3394171.3413723Google ScholarDigital Library
- Adnane Cabani, Karim Hammoudi, Halim Benhabiles, and Mahmoud Melkemi. 2020. MaskedFace-Net–a dataset of correctly/incorrectly masked face images in the context of COVID-19. Smart Health 19 (March 2021). https://doi.org/10.1016/j.smhl.2020.100144Google Scholar
- Feifei Ding, Peixi Peng, Yangru Huang, Mengyue Geng, and Yonghong Tian. 2020. Masked face recognition with latent part detection. In Proceedings of the 28th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 2281–2289. https://doi.org/10.1145/3394171.3413731Google ScholarDigital Library
- Lingxue Song, Dihong Gong, Zhifeng Li, Changsong Liu, and Wei Liu. 2019. Occlusion robust face recognition based on mask learning with pairwise differential siamese network. In Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, Seoul, South Korea, 773-782. https://doi.org/10.1109/ICCV.2019.00086Google ScholarCross Ref
- Hailing Zhou, Ajmal Mian, Lei Wei, Doug Creighton, Mo Hossny, and Saeid Nahavandi. 2014. Recent advances on singlemodal and multimodal face recognition: a survey. IEEE Transactions on Human-Machine Systems 44,6 (December 2014), 701-716. https://doi.org/10.1109/THMS.2014.2340578Google ScholarCross Ref
- Song Zhou and Sheng Xiao. 2018. 3D face recognition: a survey. Hum. Cent. Comput. Inf. Sci. 8, 35 (November 2018). https://doi.org/10.1186/s13673-018-0157-2Google ScholarDigital Library
- Yuge Huang 2020. CurricularFace: adaptive curriculum learning loss for deep face recognition. In Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Seattle, WA, USA, 5900-5909. https://doi.org/10.1109/CVPR42600.2020.00594Google ScholarCross Ref
- David Montero, Marcos Nieto, Peter Leskovsky, and Naiara Aginako. 2021. Boosting masked face recognition with multi-task arcface. arXiv preprint arXiv:2104.09874v2 (2021).Google Scholar
- Md. Sabbir Ejaz, Md. Rabiul Islam, Md Sifatullah, and Ananya Sarker. 2019. Implementation of principal component analysis on masked and non-masked face recognition. In Proceedings of 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT). IEEE, Dhaka, Bangladesh, 1-5. https://doi.org/10.1109/ICASERT.2019.8934543Google Scholar
- Nizam Ud Din, Kamran Javed, Seho Bae, and Juneho Yi. 2020. A novel GAN-based network for unmasking of masked face. IEEE Access 8 (March 2020), 44276–44287. https://doi.org/10.1109/ACCESS.2020.2977386Google Scholar
- Shiming Ge, Chenyu Li, Shengwei Zhao, and Dan Zeng. 2020. Occluded face recognition in the wild by identity-diversity inpainting. IEEE Transactions on Circuits and Systems for Video Technology 30, 10 (October 2020), 3387-3397. https://doi.org/10.1109/TCSVT.2020.2967754Google ScholarCross Ref
- Chenyu Li, Shiming Ge, Daichi Zhang, and Jia Li. 2020. Look through masks: towards masked face recognition with de-occlusion distillation. In Proceedings of the 28th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 3016–3024. https://doi.org/10.1145/3394171.3413960Google ScholarDigital Library
- Walid Hariri. 2021. Efficient masked face recognition method during the COVID-19 pandemic. SIViP (November 2021). https://doi.org/10.1007/s11760-021-02050-wGoogle Scholar
- Hang Du, Hailin Shi, Yinglu Liu, Dan Zeng, and Tao Mei. 2021. Towards NIR-VIS masked face recognition. IEEE Signal Processing Letters 28 (April 2021), 768-772. https://doi.org/10.1109/LSP.2021.3071663Google ScholarCross Ref
- Syed Zulqarnain Gilani, and Ajmal Mian. 2018. Learning from millions of 3D scans for large-scale 3D face recognition. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Salt Lake City, UT, USA, 1896-1905. https://doi.org/0.1109/CVPR.2018.00203Google ScholarCross Ref
- Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local neural networks. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Salt Lake City, UT, USA, 7794-7803. https://doi.org/10.1109/CVPR.2018.00813Google ScholarCross Ref
- Jongchan Park, Sanghyun Woo, Joon-Young Lee, and In So Kweon. 2020. A simple and light-weight attention module for convolutional neural networks. Int J Comput Vis 128 (January 2020), 783-798. https://doi.org/10.1007/s11263-019-01283-0Google Scholar
- Arman Savran, Neşe Alyüz, Hamdi Dibeklioğlu, Oya Çeliktutan, Berk Gökberk, Bülent Sankur, and Lale Akarun. 2008. Bosphorus database for 3D face analysis. Biometrics and Identity Management: First European Workshop, BIOID 2008, Roskilde, Denmark, May 7-9, 2008. Revised Selected Papers. Springer-Verlag, Berlin, Heidelberg, 47–56. https://doi.org/10.1007/978-3-540-89991-4_6Google ScholarDigital Library
- Ajmal S. Mian. 2009. Shade Face: multiple image-based 3D face recognition. In Proceedings of 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops. IEEE, Kyoto, Japan, 1833-1839. https://doi.org/10.1109/ICCVW.2009.5457505Google ScholarCross Ref
- Ajmal Mian. 2011. Illumination invariant recognition and 3D reconstruction of faces using desktop optics. Opt Express 19, 8 (April 2011), 7491-7506. https://doi.org/10.1364/OE.19.007491Google ScholarCross Ref
- Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. ArcFace: additive angular margin loss for deep face recognition. In Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Long Beach, CA, USA, 4685-4694. https://doi.org/10.1109/CVPR.2019.00482Google ScholarCross Ref
- Hao Wang 2018. CosFace: large margin cosine loss for deep face recognition. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Salt Lake City, UT, USA, 5265-5274. https://doi.org/10.1109/CVPR.2018.00552Google ScholarCross Ref
- Omkar M. Parkhi, Andrea Vedaldi, and Andrew Zisserman. 2015. Deep face recognition. In Proceedings of the British Machine Vision Conference (BMVC). BMVA Press, Swansea, UK, 41.1-41.12. https://doi.org/10.5244/C.29.41Google ScholarCross Ref
- Yande Li, Kun Guo, Yonggang Lu, and Li Liu. 2021. Cropping and attention based approach for masked face recognition. Appl. Intell. 51 (February 2021), 3012-3025. https://doi.org/10.1007/s10489-020-02100-9Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Las Vegas, NV, USA, 770-778. https://doi.org/10.1109/CVPR.2016.90Google ScholarCross Ref
- Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. 2018. CBAM: convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV). Springer, Munich, Germany, 3-19. https://doi.org/10.1007/978-3-030-01234-2_1Google ScholarCross Ref
Recommendations
Local Eyebrow Feature Attention Network for Masked Face Recognition
During the COVID-19 coronavirus epidemic, wearing masks has become increasingly popular. Traditional occlusion face recognition algorithms are almost ineffective for such heavy mask occlusion. Therefore, it is urgent to improve the recognition performance ...
3D face recognition: a survey
3D face recognition has become a trending research direction in both industry and academia. It inherits advantages from traditional 2D face recognition, such as the natural recognition process and a wide range of applications. Moreover, 3D face ...
Self-restrained triplet loss for accurate masked face recognition
Highlights- A solution to improve masked face verification performance.
- A novel loss ...
AbstractUsing the face as a biometric identity trait is motivated by the contactless nature of the capture process and the high accuracy of the recognition algorithms. After the current COVID-19 pandemic, wearing a face mask has been imposed ...
Comments