Abstract
In this paper, we propose a simple but yet effective method for synthesizing a pseudo face sketch (pseudo-sketch) from a photo, to be used for face recognition based on sketches drawn by a forensic artist. In contrast to current methods, the proposed method does not require training samples while fairly maintains the salient facial features as the artist do. We also propose a matching method on the basis of the Histograms of Oriented Gradients (HOG) descriptor and Principal Component Analysis (PCA), called HOG-PCA, to handle the similarities between a forensic sketch and a synthesized pseudo-sketch. In this method, we first extract the HOG features for the sketch and pseudo-sketch at regular grid and overlapped patches. The PCA is then applied to address the redundancy in feature representation due to several overlapped patches. Finally, the Nearest Neighbors Classifier (NNC) with the cosine distance is used to classify the sketch and pseudo-sketch pairs as matched or mismatched. Experimental results on CUHK and AR face sketch databases demonstrate that our proposed methods outperform state-of-the-art methods.
Similar content being viewed by others
Notes
This threshold was calculated based on the histogram of RMS values for 50 images.
We found that a disk-shaped structuring element with 7 pixels radius is sufficient enough to remove eyes, nose, mouth, eyebrows and shadows.
References
Ahonen T, Rahtu E, Ojansivu V, Heikkilä J (2008) Recognition of blurred faces using local phase quantization. In: 19Th international conference on pattern recognition, pp 1–4
Bhatt HS, Bharadwaj S, Singh R, Vatsa M (2010) On matching sketches with digital face images. In: 4Th IEEE international conference on biometrics: Theory applications and systems, pp 1–7
Bhatt HS, Bharadwaj S, Singh R, Vatsa M (2012) Memetic approach for matching sketches with digital face images. Tech. rep
Bhatt HS, Bharadwaj S, Singh R, Vatsa M (2012) Memetically optimized mcwld for matching sketches with digital face images. IEEE Transactions on Information Forensics and Security 7(5):1522–1535
Chen LF, Liao HYM, Ko MT, Lin JC, Yu GJ (2000) A new lda-based face recognition system which can solve the small sample size problem. Pattern Recog 33(10):1713–1726
Chen H, Xu YQ, Shum HY, Zhu SC, Zheng NN (2001) Example-based facial sketch generation with non-parametric sampling. In: 8Th IEEE international conference on computer vision, vol 2, pp 433–438
Chen J, Shan S, He C, Zhao G, Pietikäinen M, Chen X, Gao W (2010) Wld: a robust local image descriptor. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(9):1705–1720
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE Computer society conference on computer vision and pattern recognition, vol 1, pp 886–893
Gao X, Wang N, Tao D, Li X (2012) Face sketch–photo synthesis and retrieval using sparse representation. IEEE Transactions on Circuits and Systems for Video Technology 22(8):1213–1226
Grgic M, Delac K, Grgic S (2011) Scface–surveillance cameras face database. Multimedia Tools and Applications 51(3):863–879
Huang DA, Wang YC (2013) Coupled dictionary and feature space learning with applications to cross-domain image synthesis and recognition. In: IEEE international conference on computer vision, pp 2496–2503
Jolliffe I (2002) Principal component analysis. Wiley Online Library
Kimmel R, Elad M, Shaked D, Keshet R, Sobel I (2003) A variational framework for retinex. Int J Comput Vis 52(1):7–23
Klare BF, Li Z, Jain AK (2011) Matching forensic sketches to mug shot photos. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(3):639–646
Land EH (1983) Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image. Proc Natl Acad Sci 80(16):5163–5169
Li YH, Savvides M, Bhagavatula V (2006) Illumination tolerant face recognition using a novel face from sketch synthesis approach and advanced correlation filters. In: IEEE international conference on acoustics, speech and signal processing, vol 2, pp 357–360
Liu Q, Tang X, Jin H, Lu H, Ma S (2005) A nonlinear approach for face sketch synthesis and recognition. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol 1, pp 1005–1010
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110
Martinez AM (1998) The ar face database. Tech. rep
Mittal P, Jain A, Goswami G, Vatsa M, Singh R (2017) Composite sketch recognition using saliency and attribute feedback. Information Fusion 33:86–99
Moghaddam B, Pentland A (1997) Probabilistic visual learning for object representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7):696–710
Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7):971–987
Paige CC, Saunders MA (1982) Lsqr: an algorithm for sparse linear equations and sparse least squares. ACM Trans Math Softw 8(1):43–71
Phillips PJ, Moon H, Rizvi S, Rauss P (1998) The feret evaluation. In: Face recognition, pp 244–261
Radman A, Zainal N, Suandi SA (2017) Automated segmentation of iris images acquired in an unconstrained environment using hog-svm and growcut. Digital Signal Processing 64:60–70
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326
Shi Z, Xu B, Zheng X, Zhao M (2016) An integrated method for ancient chinese tablet images de-noising based on assemble of multiple image smoothing filters. Multimedia Tools and Applications 75(19):12,245–12,261
Song Y, Bao L, Yang Q, Yang MH (2014) Real-time exemplar-based face sketch synthesis. In: ECCV, pp 800–813
Štruc V, Pavešić N (2009) Gabor-based kernel partial-least-squares discrimination features for face recognition. Informatica 20(1):115–138
Štruc V, Pavešić N (2010) The complete gabor-fisher classifier for robust face recognition. EURASIP Journal on Advances in Signal Processing 2010:1–26
Tang X, Wang X (2002) Face photo recognition using sketch. In: Proceedings of the 2002 international conference on image processing. 2002, vol 1, pp I–I
Tang X, Wang X (2003) Face sketch synthesis and recognition. In: 9Th IEEE international conference on computer vision, pp 687–694
Tang X, Wang X (2004) Face sketch recognition. IEEE Transactions on Circuits and Systems for Video Technology 14(1):50–57
Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3 (1):71–86
Turk M, Pentland AP et al (1991) Face recognition using eigenfaces. In: IEEE computer society conference on computer vision and pattern recognition, pp 586–591
Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(11):1955–1967
Wang S, Zhang L, Liang Y, Pan Q (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: IEEE conference on computer vision and pattern recognition, pp 2216–2223
Wang N, Tao D, Gao X, Li X, Li J (2013) Transductive face sketch-photo synthesis. IEEE Transactions on Neural Networks and Learning Systems 24(9):1364–1376
Wang N, Tao D, Gao X, Li X, Li J (2014) A comprehensive survey to face hallucination. Int J Comput Vis 106(1):9–30
Wang N, Zhang S, Gao X, Li J, Song B, Li Z (2017) Unified framework for face sketch synthesis. Signal Process 130:1–11
Xiao B, Gao X, Tao D, Li X (2009) A new approach for face recognition by sketches in photos. Signal Process 89(8):1576–1588
Yan C, Xie H, Liu S, Yin J, Zhang Y, Dai Q (2017) Effective uyghur language text detection in complex background images for traffic prompt identification. IEEE Transactions on Intelligent Transportation Systems PP(99):1–10
Yan C, Xie H, Yang D, Yin J, Zhang Y, Dai Q (2017) Supervised hash coding with deep neural network for environment perception of intelligent vehicles. IEEE Transactions on Intelligent Transportation Systems PP(99):1–12
Yang MH (2002) Kernel eigenfaces vs. kernel fisherfaces: Face recognition using kernel methods. In: 5Th IEEE international conference on automatic face and gesture recognition, pp 215–220
Zhang L, Zhang L, Mou X, Zhang D (2011) Fsim: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386
Zhang S, Gao X, Wang N, Li J, Zhang M (2015) Face sketch synthesis via sparse representation-based greedy search. IEEE Trans Image Process 24(8):2466–2477
Zhang S, Gao X, Wang N, Li J (2016) Robust face sketch style synthesis. IEEE Trans Image Process 25(1):220–232
Zhou H, Kuang Z, Wong KYK (2012) Markov weight fields for face sketch synthesis. In: IEEE conference on computer vision and pattern recognition, pp 1091–1097
Acknowledgements
The authors highly acknowledge Universiti Sains Malaysia for its fund Universiti Sains Malaysia Research University Grant (RUI) no. 1001/PELECT/814208. We also thank the Chinese University of Hong Kong and the Ohio State University for their CUHK and AR face sketch databases.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Radman, A., Suandi, S.A. Robust face pseudo-sketch synthesis and recognition using morphological-arithmetic operations and HOG-PCA. Multimed Tools Appl 77, 25311–25332 (2018). https://doi.org/10.1007/s11042-018-5786-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-018-5786-y