Skip to main content
Log in

Robust face pseudo-sketch synthesis and recognition using morphological-arithmetic operations and HOG-PCA

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose a simple but yet effective method for synthesizing a pseudo face sketch (pseudo-sketch) from a photo, to be used for face recognition based on sketches drawn by a forensic artist. In contrast to current methods, the proposed method does not require training samples while fairly maintains the salient facial features as the artist do. We also propose a matching method on the basis of the Histograms of Oriented Gradients (HOG) descriptor and Principal Component Analysis (PCA), called HOG-PCA, to handle the similarities between a forensic sketch and a synthesized pseudo-sketch. In this method, we first extract the HOG features for the sketch and pseudo-sketch at regular grid and overlapped patches. The PCA is then applied to address the redundancy in feature representation due to several overlapped patches. Finally, the Nearest Neighbors Classifier (NNC) with the cosine distance is used to classify the sketch and pseudo-sketch pairs as matched or mismatched. Experimental results on CUHK and AR face sketch databases demonstrate that our proposed methods outperform state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. This threshold was calculated based on the histogram of RMS values for 50 images.

  2. We found that a disk-shaped structuring element with 7 pixels radius is sufficient enough to remove eyes, nose, mouth, eyebrows and shadows.

References

  1. Ahonen T, Rahtu E, Ojansivu V, Heikkilä J (2008) Recognition of blurred faces using local phase quantization. In: 19Th international conference on pattern recognition, pp 1–4

  2. Bhatt HS, Bharadwaj S, Singh R, Vatsa M (2010) On matching sketches with digital face images. In: 4Th IEEE international conference on biometrics: Theory applications and systems, pp 1–7

  3. Bhatt HS, Bharadwaj S, Singh R, Vatsa M (2012) Memetic approach for matching sketches with digital face images. Tech. rep

  4. Bhatt HS, Bharadwaj S, Singh R, Vatsa M (2012) Memetically optimized mcwld for matching sketches with digital face images. IEEE Transactions on Information Forensics and Security 7(5):1522–1535

    Article  Google Scholar 

  5. Chen LF, Liao HYM, Ko MT, Lin JC, Yu GJ (2000) A new lda-based face recognition system which can solve the small sample size problem. Pattern Recog 33(10):1713–1726

    Article  Google Scholar 

  6. Chen H, Xu YQ, Shum HY, Zhu SC, Zheng NN (2001) Example-based facial sketch generation with non-parametric sampling. In: 8Th IEEE international conference on computer vision, vol 2, pp 433–438

  7. Chen J, Shan S, He C, Zhao G, Pietikäinen M, Chen X, Gao W (2010) Wld: a robust local image descriptor. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(9):1705–1720

    Article  Google Scholar 

  8. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE Computer society conference on computer vision and pattern recognition, vol 1, pp 886–893

  9. Gao X, Wang N, Tao D, Li X (2012) Face sketch–photo synthesis and retrieval using sparse representation. IEEE Transactions on Circuits and Systems for Video Technology 22(8):1213–1226

    Article  Google Scholar 

  10. Grgic M, Delac K, Grgic S (2011) Scface–surveillance cameras face database. Multimedia Tools and Applications 51(3):863–879

    Article  Google Scholar 

  11. Huang DA, Wang YC (2013) Coupled dictionary and feature space learning with applications to cross-domain image synthesis and recognition. In: IEEE international conference on computer vision, pp 2496–2503

  12. Jolliffe I (2002) Principal component analysis. Wiley Online Library

  13. Kimmel R, Elad M, Shaked D, Keshet R, Sobel I (2003) A variational framework for retinex. Int J Comput Vis 52(1):7–23

    Article  MATH  Google Scholar 

  14. Klare BF, Li Z, Jain AK (2011) Matching forensic sketches to mug shot photos. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(3):639–646

    Article  Google Scholar 

  15. Land EH (1983) Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image. Proc Natl Acad Sci 80(16):5163–5169

    Article  Google Scholar 

  16. Li YH, Savvides M, Bhagavatula V (2006) Illumination tolerant face recognition using a novel face from sketch synthesis approach and advanced correlation filters. In: IEEE international conference on acoustics, speech and signal processing, vol 2, pp 357–360

  17. Liu Q, Tang X, Jin H, Lu H, Ma S (2005) A nonlinear approach for face sketch synthesis and recognition. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol 1, pp 1005–1010

  18. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  MathSciNet  Google Scholar 

  19. Martinez AM (1998) The ar face database. Tech. rep

  20. Mittal P, Jain A, Goswami G, Vatsa M, Singh R (2017) Composite sketch recognition using saliency and attribute feedback. Information Fusion 33:86–99

    Article  Google Scholar 

  21. Moghaddam B, Pentland A (1997) Probabilistic visual learning for object representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7):696–710

    Article  Google Scholar 

  22. Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7):971–987

    Article  MATH  Google Scholar 

  23. Paige CC, Saunders MA (1982) Lsqr: an algorithm for sparse linear equations and sparse least squares. ACM Trans Math Softw 8(1):43–71

    Article  MathSciNet  MATH  Google Scholar 

  24. Phillips PJ, Moon H, Rizvi S, Rauss P (1998) The feret evaluation. In: Face recognition, pp 244–261

  25. Radman A, Zainal N, Suandi SA (2017) Automated segmentation of iris images acquired in an unconstrained environment using hog-svm and growcut. Digital Signal Processing 64:60–70

    Article  MathSciNet  Google Scholar 

  26. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  27. Shi Z, Xu B, Zheng X, Zhao M (2016) An integrated method for ancient chinese tablet images de-noising based on assemble of multiple image smoothing filters. Multimedia Tools and Applications 75(19):12,245–12,261

    Article  Google Scholar 

  28. Song Y, Bao L, Yang Q, Yang MH (2014) Real-time exemplar-based face sketch synthesis. In: ECCV, pp 800–813

  29. Štruc V, Pavešić N (2009) Gabor-based kernel partial-least-squares discrimination features for face recognition. Informatica 20(1):115–138

    MATH  Google Scholar 

  30. Štruc V, Pavešić N (2010) The complete gabor-fisher classifier for robust face recognition. EURASIP Journal on Advances in Signal Processing 2010:1–26

    MATH  Google Scholar 

  31. Tang X, Wang X (2002) Face photo recognition using sketch. In: Proceedings of the 2002 international conference on image processing. 2002, vol 1, pp I–I

  32. Tang X, Wang X (2003) Face sketch synthesis and recognition. In: 9Th IEEE international conference on computer vision, pp 687–694

  33. Tang X, Wang X (2004) Face sketch recognition. IEEE Transactions on Circuits and Systems for Video Technology 14(1):50–57

    Article  Google Scholar 

  34. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3 (1):71–86

    Article  Google Scholar 

  35. Turk M, Pentland AP et al (1991) Face recognition using eigenfaces. In: IEEE computer society conference on computer vision and pattern recognition, pp 586–591

  36. Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(11):1955–1967

    Article  Google Scholar 

  37. Wang S, Zhang L, Liang Y, Pan Q (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: IEEE conference on computer vision and pattern recognition, pp 2216–2223

  38. Wang N, Tao D, Gao X, Li X, Li J (2013) Transductive face sketch-photo synthesis. IEEE Transactions on Neural Networks and Learning Systems 24(9):1364–1376

    Article  Google Scholar 

  39. Wang N, Tao D, Gao X, Li X, Li J (2014) A comprehensive survey to face hallucination. Int J Comput Vis 106(1):9–30

    Article  Google Scholar 

  40. Wang N, Zhang S, Gao X, Li J, Song B, Li Z (2017) Unified framework for face sketch synthesis. Signal Process 130:1–11

    Article  Google Scholar 

  41. Xiao B, Gao X, Tao D, Li X (2009) A new approach for face recognition by sketches in photos. Signal Process 89(8):1576–1588

    Article  MATH  Google Scholar 

  42. Yan C, Xie H, Liu S, Yin J, Zhang Y, Dai Q (2017) Effective uyghur language text detection in complex background images for traffic prompt identification. IEEE Transactions on Intelligent Transportation Systems PP(99):1–10

    Google Scholar 

  43. Yan C, Xie H, Yang D, Yin J, Zhang Y, Dai Q (2017) Supervised hash coding with deep neural network for environment perception of intelligent vehicles. IEEE Transactions on Intelligent Transportation Systems PP(99):1–12

    Google Scholar 

  44. Yang MH (2002) Kernel eigenfaces vs. kernel fisherfaces: Face recognition using kernel methods. In: 5Th IEEE international conference on automatic face and gesture recognition, pp 215–220

  45. Zhang L, Zhang L, Mou X, Zhang D (2011) Fsim: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386

    Article  MathSciNet  MATH  Google Scholar 

  46. Zhang S, Gao X, Wang N, Li J, Zhang M (2015) Face sketch synthesis via sparse representation-based greedy search. IEEE Trans Image Process 24(8):2466–2477

    Article  MathSciNet  Google Scholar 

  47. Zhang S, Gao X, Wang N, Li J (2016) Robust face sketch style synthesis. IEEE Trans Image Process 25(1):220–232

    Article  MathSciNet  Google Scholar 

  48. Zhou H, Kuang Z, Wong KYK (2012) Markov weight fields for face sketch synthesis. In: IEEE conference on computer vision and pattern recognition, pp 1091–1097

Download references

Acknowledgements

The authors highly acknowledge Universiti Sains Malaysia for its fund Universiti Sains Malaysia Research University Grant (RUI) no. 1001/PELECT/814208. We also thank the Chinese University of Hong Kong and the Ohio State University for their CUHK and AR face sketch databases.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Abduljalil Radman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Radman, A., Suandi, S.A. Robust face pseudo-sketch synthesis and recognition using morphological-arithmetic operations and HOG-PCA. Multimed Tools Appl 77, 25311–25332 (2018). https://doi.org/10.1007/s11042-018-5786-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5786-y

Keywords

Navigation