Abstract
Matching sketch facial images to mug-shot images have crucial significance in law enforcement and digital entertainment. Conventional methods always assume that both the sketch and photo face images have the same resolutions. However, in real criminal detection, the target facial sketches obtained by the artist usually have different resolutions against the source photos in the mug-shot database. In this paper, we propose a locality-constrained feature space learning (LCFSL) method to address the above cross-resolution sketch-photo facial images matching problem. The proposed LCFSL approach not only build bridge to associate cross-domain face images, but also can learn resolution robust representation features for cross-resolution sketch-photo face recognition purpose. After common feature space learning, we simply use nearest neighbor classifier to perform recognition based on the projected features obtained from sketch-photo faces with different resolutions. Experiments conducted on CUHK student database and AR database have shown the effectiveness and superiority of our method to some state-of-the-art face recognition approaches.
Similar content being viewed by others
References
Wang N, Gao X, Li J (2018) Random sampling for fast face sketch synthesis. Pattern Recogn 76:215–227
Lei Z, Pietikäinen M, Li SZ (2014) Learning discriminant face descriptor. IEEE Trans Pattern Anal Mach Intell 36:289–302
Peng C, Gao X, Wang N, Li J (2018) Face recognition from multiple stylistic sketches: scenarios, datasets, and evaluation. Pattern Recogn 84:262–272
Peng C, Gao X, Wang N, Li J (2017) Graphical representation for heterogeneous face recognition. IEEE Trans Pattern Anal Mach Intell 39:301–312
Ou W, You X, Tao D, Zhang P, Tang Y, Zhu Z (2014) Robust face recognition via occlusion dictionary learning. Pattern Recogn 47:1559–1572
Ou W, Luan X, Gou J, Zhou Q, Xiao W, Xiong X, Zeng W (2018) Robust discriminative nonnegative dictionary learning for occluded face recognition. Pattern Recogn Lett 107:41–49
Lezama J, Qiu Q, Sapiro G (2017) Not afraid of the dark: NIR-VIS face recognition via cross-spectral hallucination and low-rank embedding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6628–6637
Wang N, Gao X, Sun L, Li J (2018) Anchored neighborhood index for face sketch synthesis. IEEE Trans Circuits Syst Video Technol 28:2154–2163
Jiang J, Yu Y, Wang Z, Liu X, Ma J (2019) Graph-regularized locality-constrained joint dictionary and residual learning for face sketch synthesis. IEEE Trans Image Process 28:628–641
Kan M, Shan S, Zhang H, Lao S, Chen X (2016) Multi-view discriminant analysis. IEEE Trans Pattern Anal Mach Intell 38:188–194
He R, Wu X, Sun Z, Tan T (2017) Learning invariant deep representation for NIR-VIS face recognition. In: Proceeding of AAAI conference on artificial intelligence, pp. 2000–2006
Zhang Y, Lu W, Ou W, Zhang G, Zhang X, Cheng J, Zhang W (2019) Chinese medical question answer selection via hybrid models based on CNN and GRU. Multimed Tools Appl 2:1–26
Oh B-S, Oh K, Teoh ABJ, Lin Z, Toh K-A (2017) A Gabor-based network for heterogeneous face recognition. Neurocomputing 261:253–265
Duan Y, Lu J, Feng J, Zhou J (2018) Context-aware local binary feature learning for face recognition. IEEE Trans Pattern Anal Mach Intell 40:1139–1153
Ma X, Zhang J, Qi C (2010) Hallucinating face by position-patch. Pattern Recogn 43:2224–2236
Jiang J, Ma J, Chen C, Jiang X, Wang Z (2017) Noise robust face image super-resolution through smooth sparse representation. IEEE Trans Cybern 47:3991–4002
Liu L, Chen CP, Li S, Tang YY, Chen L (2018) Robust face hallucination via locality-constrained bi-layer representation. IEEE Trans Cybern 48:1189–1201
Junjun J, Yi Y, Suhua T, Jiayi M, Guo-Jun Q, Akiko A (2019) Context-patch based face hallucination via thresholding locality-constrained representation and reproducing learning. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2018.2868891
Jiang J, Hu R, Wang Z, Han Z (2014) Noise robust face hallucination via locality-constrained representation. IEEE Trans Multimedia 16:1268–1281
Bhatt HS, Singh R, Vatsa M, Ratha NK (2014) Improving cross-resolution face matching using ensemble-based co-transfer learning. IEEE Trans Image Process 23:5654–5669
Mudunuri SP, Biswas S (2016) Low resolution face recognition across variations in pose and illumination. IEEE Trans Pattern Anal Mach Intell 38:1034–1040
Jiang J, Hu R, Wang Z, Cai Z (2016) CDMMA: coupled discriminant multi-manifold analysis for matching low-resolution face images. Signal Process 124:162–172
Haghighat M, Abdel-Mottaleb M (2017) Low resolution face recognition in surveillance systems using discriminant correlation analysis. In: Proceedings of IEEE international conference on automatic face & gesture recognition, pp. 912–917
Lu T, Chen X, Zhang Y, Chen C, Xiong Z (2018) SLR: semi-coupled locality constrained representation for very low resolution face recognition and super resolution. IEEE Access 6:56269–56281
Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19:2861–2873
Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp. 3360–3367
Lu W, Wu H, Jian P, Huang Y, Huang H (2018) An empirical study of classifier combination based word sense disambiguation. IEICE Trans Inf Syst 101:225–233
Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Trans Pattern Anal Mach Intell 31:1955–1967
Martinez AM, Benavente R (1998) The AR face database. CVC technical report, vol. 24
Acknowledgements
This work was supported by the National Key Research and Development Program of China under Project no. 2018AAA0100102; the National Natural Science Foundation of China under Grant nos. 61972212, 61772568, 61806098 and 61833011; the Natural Science Foundation of Jiangsu Province under Grant no. BK20190089; the Six Talent Peaks Project in Jiangsu Province under Grant no. RJFW-011; the open fund project of Science and Technology on Space Intelligent Control Laboratory under Grant no. 6142208180302; and the Open Fund Project of Provincial Key Laboratory for Computer Information Processing Technology (Soochow University) (No. KJS1840).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gao, G., Wang, Y., Huang, P. et al. Locality-constrained feature space learning for cross-resolution sketch-photo face recognition. Multimed Tools Appl 79, 14903–14917 (2020). https://doi.org/10.1007/s11042-019-08488-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-019-08488-y