Abstract
Face sketch synthesis plays an important role in public security and digital entertainment. In this paper, we present a novel face sketch synthesis method via local similarity and nonlocal similarity regularization terms. The local similarity can overcome the technological bottlenecks of the patch representation scheme in traditional patch-based face sketch synthesis methods. It improves the quality of synthesized sketches by penalizing the dissimilar training patches (thus have very small weights or are discarded). In addition, taking the redundancy of image patches into account, a global nonlocal similarity regularization is employed to restrain the generation of the noise and maintain primitive facial features during the synthesized process. More robust synthesized results can be obtained. Extensive experiments on the public databases are carried out to validate the generality, effectiveness, and robustness of the proposed algorithm.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Uhl, R.G. da Vitoria Lobo Jr., N.: A framework for recognizing a facial image from a police sketch. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, pp. 586–593 (1996)
Wang, N.N., Tao, D.C., Gao, X.B., Li, X., Li, J.: Transductive face sketch photo synthesis. IEEE Trans. Neural Netw. Learn. Syst. 24(9), 1–13 (2013)
Wang, J., Bao, H., Zhou, W., Peng, Q., Xu, Y.Q.: Automatic image-based pencil sketch rendering. J. Comput. Sci. Technol. 17, 347–355 (2002)
Li, X., Cao, X.: A simple framework for face photo-sketch synthesis. Math. Probl. Eng. 2012 (2012)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. arXiv preprint (2017)
Ledig, C., Theis, L., Huszár, F., Caballero, J., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, p. 4 (2017)
Huang, C., Li, Y., Loy, C., Tang, X.: Learning deep representation for imbalanced classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 5375–5384 (2016)
Dong, C., Loy, C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)
Zhang, L., Lin, L., Wu, X., Ding, S., Zhang, L.: End-to-end photo-sketch generation via fully convolutional representation learning. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, Shanghai, China, pp. 627–634 (2015)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, Montreal, Canada, pp. 2672–2680 (2014)
Dosovitskiy, A., Brox, T.: Generating images with perceptual similarity metrics based on deep networks. In: Advances in Neural Information Processing Systems, Barcelona, Spain, pp. 658–666 (2016)
Wang, N.N., Zha, W., Li, J., Gao, X.B.: Back projection: an effective: postprocessing method for GAN-based face sketch synthesis. Pattern Recognit. Lett. 107, 59–65 (2018)
Tang, X., Wang, X.: Face photo recognition using sketch. In: Proceedings of IEEE International Conference on Image Processing, New York, USA, pp. 257–260 (2002)
Liu, Q., Tang, X., Jin, H., Lu, H., Ma, S.: A nonlinear approach for face sketch synthesis and recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA, pp. 1005–1010 (2005)
Roweis, S.T., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)
Song, Y., Bao, L., Yang, Q., Yang, M.-H.: Real-time exemplar-based face sketch synthesis. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 800–813. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_51
Wang, N.N., Gao, X., Li, J.: Random sampling for fast face sketch synthesis. Pattern Recognit. 76, 215–227 (2018)
Tang, S., Xiao, L., Liu, P., Huang, L., Zhou, N., Xu, Y.: Pansharpening via sparse regression. Opt. Eng. 56, 093105-1–093105-13 (2017)
Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T., Yan, S.: Sparse representation for computer vision pattern recognition. Proc. IEEE 98(6), 1031–1044 (2010)
Chang, L., Zhou, M., Han, Y., Deng, X.: Face sketch synthesis via sparse representation. In: Proceedings of International Conference on Pattern Recognition, Istanbul, Turkey, pp. 2146–2149 (2010)
Gao, X.B., Zhong, J., Li, J., Tian, C.: Face sketch synthesis algorithm using E-HMM and selective ensemble. IEEE Trans. Circuits Syst. Video Technol. 18(4), 487–496 (2008)
Zhou, H., Kuang, Z., Wong, K.: Markov weight fields for face sketch synthesis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Providence, USA, pp. 1091–1097 (2012)
Peng, C.L., Gao, X.B., Wang, N.N., Tao, D.C., Li, X., Li, J.: Multiple representations-based face sketch-photo synthesis. IEEE Trans. Neural Netw. Learn. Syst. 27(11), 2201–2215 (2016)
Li, C., Zhao, S., Xiao, K., Wang, Y.: Face recognition based on the combination of enhanced local texture feature and DBN under complex illumination conditions. J. Inf. Process. Syst. 14, 191–204 (2018)
Muntasa, A.: Homogeneous and non-homogeneous polynomial based eigenspaces to extract the features on facial images. J. Inf. Process. Syst. 12, 591–611 (2016)
Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA, pp. 60–65 (2005)
Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)
Acknowledgments
This research was supported in part by the National Natural Science Foundation of China under Grant 61702269, and Grant 61671339, in part by the Natural Science Foundation of Jiangsu Province under Grant BK20171074. The Fundamental Research Funds for the Central Universities at Nanjing Forest Police College under Grant No. LGZD201702.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Tang, S., Qiu, M. (2019). Face Sketch Synthesis Based on Adaptive Similarity Regularization. In: Cui, Z., Pan, J., Zhang, S., Xiao, L., Yang, J. (eds) Intelligence Science and Big Data Engineering. Visual Data Engineering. IScIDE 2019. Lecture Notes in Computer Science(), vol 11935. Springer, Cham. https://doi.org/10.1007/978-3-030-36189-1_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-36189-1_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36188-4
Online ISBN: 978-3-030-36189-1
eBook Packages: Computer ScienceComputer Science (R0)