Skip to main content
SpringerLink
Log in

Face Photo-Sketch Synthesis Based on Conditional Adversarial Networks

  • Conference paper
  • First Online:
Web Information Systems and Applications (WISA 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11817))

Included in the following conference series:

  • 2005 Accesses

Abstract

Face Photo-Sketch synthesis is designed to generate an image contains rich personal information and details facial. It is widely used in law enforcement and life areas. Although some existing methods have achieved remarkable results, however, due to the gap between the synthetic and real image distribution, the synthetic image does not achieve the expected effect. To solve this problem, In this paper, we proposed conditional generation adversarial networks (CGAN) based on arts drawing steps constrain. We divide the process into two stages. In the first stage, we take the original face photos which concat noise z as input, generating stage 1 low resolution synthesize sketches. In the second stage take the stage 1 results and original face photo as inputs, yielding stage 2 high resolution synthesize sketches, which can express more natural textures and details. To add realism, we train our network using an adversarial loss. Experiments have shown that, Compared with previous methods, our results generate visually comfortable face sketches. And express more natural textures and details.

Supported by Project of Fujian Science and Technology Department 2019I0036.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chen, T., Cheng, M.M., Shamir, A., Shamir, A., Hu, S.M.: Sketch2photo: internet image montage. In: ACM SIGGRAPH Asia (2009)

    Google Scholar 

  2. Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. Computer Science (2014)

    Google Scholar 

  3. Di, X., Patel, V.M.: Face synthesis from visual attributes via sketch using conditional VAEs and GANs. arXiv preprint arXiv:1801.00077 (2017)

  4. Eitz, M., Richter, R., Hildebrand, K., Boubekeur, T., Alexa, M.: Photosketcher: interactive sketch-based image synthesis. IEEE Comput. Graph. Appl. 31(6), 56 (2011)

    Article  Google Scholar 

  5. Gao, X., Zhong, J., Jie, L., Tian, C.: Face sketch synthesis algorithm based on e-hmm and selective ensemble. IEEE Trans. Circuits Syst. Video Technol. 18(4), 487–496 (2008)

    Article  Google Scholar 

  6. Goodfellow, I.J., et al.: Generative adversarial nets. In: International Conference on Neural Information Processing Systems (2014)

    Google Scholar 

  7. Han, Z., Tao, X., Li, H.: Stackgan: text to photo-realistic image synthesis with stacked generative adversarial networks. In: Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  8. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  9. Iwashita, S., Takeda, Y., Onisawa, T.: Expressive facial caricature drawing. In: IEEE International Fuzzy Systems Conference (1999)

    Google Scholar 

  10. Johnson, J., Alahi, A., Li, F.F.: Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision (2016)

    Google Scholar 

  11. Uhl Jr., R.G., Lobo, N.D.V., Kwon, Y.H.: Recognizing a facial image from a police sketch. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1996)

    Google Scholar 

  12. Jun, Y., Shengjie, S., Fei, G., Dacheng, T., Qingming, H.: Towards realistic face photo-sketch synthesis via composition-aided GANs. In: Computer Vision and Pattern Recognition (2017)

    Google Scholar 

  13. Konen, W.: Comparing facial line drawings with gray-level images: a case study on PHANTOMAS. In: von der Malsburg, C., von Seelen, W., Vorbrüggen, J.C., Sendhoff, B. (eds.) ICANN 1996. LNCS, vol. 1112, pp. 727–734. Springer, Heidelberg (1996). https://doi.org/10.1007/3-540-61510-5_123

    Chapter  Google Scholar 

  14. Koshimizu, H., Tominaga, M., Fujiwara, T., Murakami, K.: On kansei facial image processing for computerized facial caricaturing system PICASSO. In: IEEE International Conference on Systems (1999)

    Google Scholar 

  15. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  16. Li, C., Wand, M.: Precomputed real-time texture synthesis with markovian generative adversarial networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 702–716. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_43

    Chapter  Google Scholar 

  17. Lin, Z., Lei, Z., Xuanqin, M., David, Z.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Processing 20(8), 2378 (2011). A Publication of the IEEE Signal Processing Society

    Article  MathSciNet  Google Scholar 

  18. Liu, Q., Tang, X., Jin, H., Lu, H., Ma, S.: A nonlinear approach for face sketch synthesis and recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005)

    Google Scholar 

  19. Liu, S., Yang, J., Huang, C., Yang, M.H.: Multi-objective convolutional learning for face labeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3451–3459 (2015)

    Google Scholar 

  20. Lucic, M., Kurach, K., Michalski, M., Gelly, S., Bousquet, O.: Are GANs created equal? A large-scale study. In: Advances in Neural Information Processing Systems, pp. 700–709 (2018)

    Google Scholar 

  21. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  22. Phillips, P.J., Moon, H., Rauss, P., Rizvi, S.A.: The FERET evaluation methodology for face-recognition algorithms. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 137–143. IEEE (1997)

    Google Scholar 

  23. Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training. In: Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  24. Tang, X., Wang, X.: Face photo recognition using sketch. In: International Conference on Image Processing (2002)

    Google Scholar 

  25. Tang, X., Wang, X.: Face sketch synthesis and recognition. In: Proceedings Ninth IEEE International Conference on Computer Vision, pp. 687–694. IEEE (2003)

    Google Scholar 

  26. Wang, L., Sindagi, V.A., Patel, V.M.: High-quality facial photo-sketch synthesis using multi-adversarial networks, pp. 83–90 (2018)

    Google Scholar 

  27. Wang, N., Zha, W., Jie, L., Gao, X.: Back projection: an effective postprocessing method for GAN-based face sketch synthesis. Pattern Recogn. Lett. 107, S0167865517302180 (2017)

    Google Scholar 

  28. Wang, X., Tang, X.: Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1955–1967 (2009)

    Article  Google Scholar 

  29. Wang, X., Gupta, A.: Generative image modeling using style and structure adversarial networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 318–335. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_20

    Chapter  Google Scholar 

  30. Yoo, D., Kim, N., Park, S., Paek, A.S., Kweon, I.S.: Pixel-level domain transfer. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 517–532. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_31

    Chapter  Google Scholar 

  31. Zhang, S., Ji, R., Hu, J., Gao, Y., Lin, C.W.: Robust face sketch synthesis via generative adversarial fusion of priors and parametric sigmoid. In: IJCAI, pp. 1163–1169 (2018)

    Google Scholar 

  32. Zhao, H., Xia, S., Zhao, J., Zhu, D., Yao, R., Niu, Q.: Pareto-based many-objective convolutional neural networks. In: Meng, X., Li, R., Wang, K., Niu, B., Wang, X., Zhao, G. (eds.) WISA 2018. LNCS, vol. 11242, pp. 3–14. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02934-0_1

    Chapter  Google Scholar 

  33. Zhou, H., Kuang, Z., Wong, K.Y.K.: Markov weight fields for face sketch synthesis. In: IEEE Conference on Computer Vision and Pattern Recognition (2012)

    Google Scholar 

  34. Zhou, Y., Berg, T.L.: Learning temporal transformations from time-lapse videos. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 262–277. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_16

    Chapter  Google Scholar 

  35. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering, Xiamen University of Technology, Xiamen, China

    Xiaoqi Xie

  2. Computer Science and Technology, Xiamen University of Technology, Xiamen, China

    Huarong Xu

  3. Academy of Design Arts, Xiamen University of Technology, Xiamen, China

    Lifen Weng

Authors
  1. Xiaoqi Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huarong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lifen Weng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lifen Weng .

Editor information

Editors and Affiliations

  1. Southeast University, Nanjing, China

    Weiwei Ni

  2. Tianjin University, Tianjin, China

    Xin Wang

  3. Wuhan University, Wuhan, China

    Wei Song

  4. Tianjin University of Technology, Tianjin, China

    Yukun Li

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xie, X., Xu, H., Weng, L. (2019). Face Photo-Sketch Synthesis Based on Conditional Adversarial Networks. In: Ni, W., Wang, X., Song, W., Li, Y. (eds) Web Information Systems and Applications. WISA 2019. Lecture Notes in Computer Science(), vol 11817. Springer, Cham. https://doi.org/10.1007/978-3-030-30952-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30952-7_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30951-0

  • Online ISBN: 978-3-030-30952-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation