Skip to main content
SpringerLink
Log in

Face Sketch Synthesis Based on Adaptive Similarity Regularization

  • Conference paper
  • First Online:
Book cover Intelligence Science and Big Data Engineering. Visual Data Engineering (IScIDE 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11935))

  • 1357 Accesses

Abstract

Face sketch synthesis plays an important role in public security and digital entertainment. In this paper, we present a novel face sketch synthesis method via local similarity and nonlocal similarity regularization terms. The local similarity can overcome the technological bottlenecks of the patch representation scheme in traditional patch-based face sketch synthesis methods. It improves the quality of synthesized sketches by penalizing the dissimilar training patches (thus have very small weights or are discarded). In addition, taking the redundancy of image patches into account, a global nonlocal similarity regularization is employed to restrain the generation of the noise and maintain primitive facial features during the synthesized process. More robust synthesized results can be obtained. Extensive experiments on the public databases are carried out to validate the generality, effectiveness, and robustness of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Uhl, R.G. da Vitoria Lobo Jr., N.: A framework for recognizing a facial image from a police sketch. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, pp. 586–593 (1996)

    Google Scholar 

  2. Wang, N.N., Tao, D.C., Gao, X.B., Li, X., Li, J.: Transductive face sketch photo synthesis. IEEE Trans. Neural Netw. Learn. Syst. 24(9), 1–13 (2013)

    Article  Google Scholar 

  3. Wang, J., Bao, H., Zhou, W., Peng, Q., Xu, Y.Q.: Automatic image-based pencil sketch rendering. J. Comput. Sci. Technol. 17, 347–355 (2002)

    Article  Google Scholar 

  4. Li, X., Cao, X.: A simple framework for face photo-sketch synthesis. Math. Probl. Eng. 2012 (2012)

    MathSciNet  MATH  Google Scholar 

  5. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)

    Article  Google Scholar 

  6. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  7. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. arXiv preprint (2017)

    Google Scholar 

  8. Ledig, C., Theis, L., Huszár, F., Caballero, J., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, p. 4 (2017)

    Google Scholar 

  9. Huang, C., Li, Y., Loy, C., Tang, X.: Learning deep representation for imbalanced classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 5375–5384 (2016)

    Google Scholar 

  10. Dong, C., Loy, C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  11. Zhang, L., Lin, L., Wu, X., Ding, S., Zhang, L.: End-to-end photo-sketch generation via fully convolutional representation learning. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, Shanghai, China, pp. 627–634 (2015)

    Google Scholar 

  12. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, Montreal, Canada, pp. 2672–2680 (2014)

    Google Scholar 

  13. Dosovitskiy, A., Brox, T.: Generating images with perceptual similarity metrics based on deep networks. In: Advances in Neural Information Processing Systems, Barcelona, Spain, pp. 658–666 (2016)

    Google Scholar 

  14. Wang, N.N., Zha, W., Li, J., Gao, X.B.: Back projection: an effective: postprocessing method for GAN-based face sketch synthesis. Pattern Recognit. Lett. 107, 59–65 (2018)

    Article  Google Scholar 

  15. Tang, X., Wang, X.: Face photo recognition using sketch. In: Proceedings of IEEE International Conference on Image Processing, New York, USA, pp. 257–260 (2002)

    Google Scholar 

  16. Liu, Q., Tang, X., Jin, H., Lu, H., Ma, S.: A nonlinear approach for face sketch synthesis and recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA, pp. 1005–1010 (2005)

    Google Scholar 

  17. Roweis, S.T., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)

    Article  Google Scholar 

  18. Song, Y., Bao, L., Yang, Q., Yang, M.-H.: Real-time exemplar-based face sketch synthesis. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 800–813. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_51

    Chapter  Google Scholar 

  19. Wang, N.N., Gao, X., Li, J.: Random sampling for fast face sketch synthesis. Pattern Recognit. 76, 215–227 (2018)

    Article  Google Scholar 

  20. Tang, S., Xiao, L., Liu, P., Huang, L., Zhou, N., Xu, Y.: Pansharpening via sparse regression. Opt. Eng. 56, 093105-1–093105-13 (2017)

    Article  Google Scholar 

  21. Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T., Yan, S.: Sparse representation for computer vision pattern recognition. Proc. IEEE 98(6), 1031–1044 (2010)

    Article  Google Scholar 

  22. Chang, L., Zhou, M., Han, Y., Deng, X.: Face sketch synthesis via sparse representation. In: Proceedings of International Conference on Pattern Recognition, Istanbul, Turkey, pp. 2146–2149 (2010)

    Google Scholar 

  23. Gao, X.B., Zhong, J., Li, J., Tian, C.: Face sketch synthesis algorithm using E-HMM and selective ensemble. IEEE Trans. Circuits Syst. Video Technol. 18(4), 487–496 (2008)

    Article  Google Scholar 

  24. Zhou, H., Kuang, Z., Wong, K.: Markov weight fields for face sketch synthesis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Providence, USA, pp. 1091–1097 (2012)

    Google Scholar 

  25. Peng, C.L., Gao, X.B., Wang, N.N., Tao, D.C., Li, X., Li, J.: Multiple representations-based face sketch-photo synthesis. IEEE Trans. Neural Netw. Learn. Syst. 27(11), 2201–2215 (2016)

    Article  Google Scholar 

  26. Li, C., Zhao, S., Xiao, K., Wang, Y.: Face recognition based on the combination of enhanced local texture feature and DBN under complex illumination conditions. J. Inf. Process. Syst. 14, 191–204 (2018)

    Google Scholar 

  27. Muntasa, A.: Homogeneous and non-homogeneous polynomial based eigenspaces to extract the features on facial images. J. Inf. Process. Syst. 12, 591–611 (2016)

    Google Scholar 

  28. Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA, pp. 60–65 (2005)

    Google Scholar 

  29. Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This research was supported in part by the National Natural Science Foundation of China under Grant 61702269, and Grant 61671339, in part by the Natural Science Foundation of Jiangsu Province under Grant BK20171074. The Fundamental Research Funds for the Central Universities at Nanjing Forest Police College under Grant No. LGZD201702.

Author information

Authors and Affiliations

  1. Department of Criminal Science and Technology, Nanjing Forest Police College, Nanjing, 210023, China

    Songze Tang

  2. Department of Information and Technology, Nanjing Forest Police College, Nanjing, 210023, China

    Mingyue Qiu

Authors
  1. Songze Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingyue Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Songze Tang .

Editor information

Editors and Affiliations

  1. Nanjing University of Science and Technology, Nanjing, China

    Zhen Cui

  2. Nanjing University of Science and Technology, Nanjing, China

    Jinshan Pan

  3. Nanjing University of Science and Technology, Nanjing, China

    Shanshan Zhang

  4. Nanjing University of Science and Technology, Nanjing, China

    Liang Xiao

  5. Nanjing University of Science and Technology, Nanjing, China

    Jian Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tang, S., Qiu, M. (2019). Face Sketch Synthesis Based on Adaptive Similarity Regularization. In: Cui, Z., Pan, J., Zhang, S., Xiao, L., Yang, J. (eds) Intelligence Science and Big Data Engineering. Visual Data Engineering. IScIDE 2019. Lecture Notes in Computer Science(), vol 11935. Springer, Cham. https://doi.org/10.1007/978-3-030-36189-1_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-36189-1_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-36188-4

  • Online ISBN: 978-3-030-36189-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation