Skip to main content
SpringerLink
Log in

Markov random fields and facial landmarks for handling uncontrolled images of face sketch synthesis

  • Industrial and Commercial Application
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Face sketch synthesis has drawn great attention in many computer vision applications such as law enforcement and digital entertainment. The majority of existing face sketch synthesis techniques are exemplar-based techniques, where a set of training photo–sketch pairs are first divided into patches. For an input photo patch from the face to be synthesized, k similar photo patches are found from the training set. The corresponding sketch patch of the best match is then selected to be synthesized. In such techniques, a multiscale Markov random fields (MRF) model is utilized for synthesizing a sketch using candidate sketch patches; having observed that techniques tend to fail with face photos acquired in uncontrolled imaging conditions like pose and lighting variations. For example, some structures along the lower part of the face sketch contour get lost due to ignoring the global face shape information and illumination changes. In this paper, we propose a reliable face sketch synthesis method based on MRF model and facial landmarks, called MRF-FL that can maintain further structures with uncontrolled face photos. Besides matching the input photo with training photo, the input photo and training sketch are also matched based on the facial landmarks so as to enhance face sketch structures around the lower part of face sketch contour. Experimental results showed that the proposed MRF-FL achieves superior performance compared with recent face sketch synthesis methods on CUHK and AR face sketch databases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. This synthesized sketch obtained by [33].

  2. This threshold was calculated based on the histogram of RMS values for 50 images of AR database.

  3. The results obtained by Wang and Tang [27], Zhou et al. [33], and Song et al. [20] are downloaded from the Web site http://www.cs.cityu.edu.hk/~yibisong/eccv14/index.html.

  4. The source code of GSM is downloaded from the Web site http://www.ntu.edu.sg/home/wslin/GSM.zip.

References

  1. Besag J (1986) On the statistical analysis of dirty pictures. J R Stat Soc Ser B (Methodological) 48:259–302

    MathSciNet  MATH  Google Scholar 

  2. Bhatt HS, Bharadwaj S, Singh R, Vatsa M (2012) Memetically optimized mcwld for matching sketches with digital face images. IEEE Trans Inf Forensics Secur 7(5):1522–1535

    Article  Google Scholar 

  3. Chang L, Rozenholc Y, Deng X, Duan F, Zhou M (2015) Face sketch synthesis using non-local means and patch-based seaming. In: IEEE international conference on image processing (ICIP), pp 1508–1512

  4. Chang L, Zhou M, Han Y, Deng X (2010) Face sketch synthesis via sparse representation. In: The 20th international conference on pattern recognition (ICPR), pp 2146–2149

  5. Chen H, Xu YQ, Shum HY, Zhu SC, Zheng NN (2001) Example-based facial sketch generation with non-parametric sampling. In: Proceedings of eighth IEEE international conference on computer vision, vol 2, pp 433–438

  6. Efros AA, Freeman WT (2001) Image quilting for texture synthesis and transfer. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques, pp 341–346

  7. Gao X, Wang N, Tao D, Li X (2012) Face sketch-photo synthesis and retrieval using sparse representation. IEEE Trans Circuits Syst Video Technol 22(8):1213–1226

    Article  Google Scholar 

  8. Huang DA, Wang YC (2013) Coupled dictionary and feature space learning with applications to cross-domain image synthesis and recognition. In: Proceedings of the IEEE international conference on computer vision, pp 2496–2503

  9. Kiani Galoogahi H, Sim T (2012) Face photo retrieval by sketch example. In: Proceedings of the 20th ACM international conference on multimedia, pp. 949–952

  10. Kimmel R, Elad M, Shaked D, Keshet R, Sobel I (2003) A variational framework for retinex. Int J Comput Vis 52(1):7–23

    Article  MATH  Google Scholar 

  11. Land EH (1983) Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image. Proc Natl Acad Sci 80(16):5163–5169

    Article  Google Scholar 

  12. Lewis JP (1995) Fast normalized cross-correlation. In: Vision interface, vol 10, pp 120–123

  13. Liu A, Lin W, Narwaria M (2012) Image quality assessment based on gradient similarity. IEEE Trans Image Process 21(4):1500–1512

    Article  MathSciNet  MATH  Google Scholar 

  14. Liu Q, Tang X, Jin H, Lu H, Ma S (2005) A nonlinear approach for face sketch synthesis and recognition. In: IEEE computer society conference on computer vision and pattern recognition (CVPR), vol 1, pp 1005–1010

  15. Martinez AM (1998) The ar face database. Technical report

  16. Ng CB, Tay YH, Goi BM (2015) A review of facial gender recognition. Pattern Anal Appl 18(4):739–755

    Article  MathSciNet  Google Scholar 

  17. Peng C, Gao X, Wang N, Tao D, Li X, Li J (2016) Multiple representations-based face sketch-photo synthesis. IEEE Trans Neural Netw Learn Syst 27(11):2201–2215

    Article  Google Scholar 

  18. Radman A, Suandi SA (2018) Robust face pseudo-sketch synthesis and recognition using morphological-arithmetic operations and hog-pca. Multimed Tools Appl 77(19):25311–25332

    Article  Google Scholar 

  19. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  20. Song Y, Bao L, Yang Q, Yang MH (2014) Real-time exemplar-based face sketch synthesis. In: Computer vision-ECCV 2014, pp 800–813

  21. Tang X, Wang X (2002) Face photo recognition using sketch. In: Proceedings of international conference on image processing, vol 1, pp I-257

  22. Tang X, Wang X (2003) Face sketch synthesis and recognition. In: Proceedings of ninth IEEE international conference on computer vision, pp 687–694

  23. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cognit Neurosci 3(1):71–86

    Article  Google Scholar 

  24. Wang N, Tao D, Gao X, Li X, Li J (2013) Transductive face sketch-photo synthesis. IEEE Trans Neural Netw Learn Syst 24(9):1364–1376

    Article  Google Scholar 

  25. Wang N, Zhang S, Gao X, Li J, Song B, Li Z (2017) Unified framework for face sketch synthesis. Signal Process 130:1–11

    Article  Google Scholar 

  26. Wang S, Zhang L, Liang Y, Pan Q (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2216–2223

  27. Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Trans Pattern Anal Mach Intell 31(11):1955–1967

    Article  Google Scholar 

  28. Xiao B, Gao X, Tao D, Li X (2009) A new approach for face recognition by sketches in photos. Signal Process 89(8):1576–1588

    Article  MATH  Google Scholar 

  29. Yu X, Huang J, Zhang S, Yan W, Metaxas D (2013) Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In: Proceedings of the IEEE international conference on computer vision, pp 1944–1951

  30. Zhang S, Gao X, Wang N, Li J (2016) Robust face sketch style synthesis. IEEE Trans Image Process 25(1):220–232

    Article  MathSciNet  Google Scholar 

  31. Zhang S, Gao X, Wang N, Li J, Zhang M (2015) Face sketch synthesis via sparse representation-based greedy search. IEEE Trans Image Process 24(8):2466–2477

    Article  MathSciNet  Google Scholar 

  32. Zhang W, Wang X, Tang X (2010) Lighting and pose robust face sketch synthesis. In: Computer vision-ECCV 2010, pp 420–433

  33. Zhou H, Kuang Z, Wong KYK (2012) Markov weight fields for face sketch synthesis. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1091–1097

Download references

Acknowledgements

The authors highly acknowledge Universiti Sains Malaysia for its fund Universiti Sains Malaysia Research University Grant (RUI) no.1001/PELECT/8014056.

Author information

Authors and Affiliations

  1. Intelligent Biometric Group, School of Electrical and Electronic Engineering, Universiti Sains Malaysia, 14300, Nibong Tebal, Pulau Pinang, Malaysia

    Abduljalil Radman & Shahrel Azmin Suandi

  2. Faculty of Engineering and Information Technology, Taiz University, Taiz, Yemen

    Abduljalil Radman

Authors
  1. Abduljalil Radman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shahrel Azmin Suandi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abduljalil Radman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Radman, A., Suandi, S.A. Markov random fields and facial landmarks for handling uncontrolled images of face sketch synthesis. Pattern Anal Applic 22, 259–271 (2019). https://doi.org/10.1007/s10044-018-0755-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-018-0755-7

Keywords

Search

Navigation