Skip to main content
SpringerLink
Log in

Face image retrieval: super-resolution based on sketch-photo transformation

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Considering the crucial role of face image in modern intelligent system, face image retrieval has attracted more attention for authentication, surveillance, law enforcement, and security control. In most cases, we cannot obtain the suspect’s face image directly and the best substitute is a face sketch of criminal suspect drawn by artist according to eyewitness description. It is a key step in the criminal investigation process to narrow down criminal suspect using the face sketch. At first, the face sketch is transformed to a pseudo-photo for subsequent utilization. Transformation is performed according to the classic eigenface algorithm and enhanced by super-resolution. Matching between reconstructed pseudo-photo and real face photographs is performed by Hash encoding and iterative quantization. We carried out our ideas on two public face databases, and the sketch face images are generated by photo-shopping software program. The experimental results show that the corresponding face images can be retrieved according to the input face sketch and super-resolution can effectively enhance the image quality and detail information of the pseudo-photo. Hash encoding and iterative quantization achieve the quick search of approximate face images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Cao Y, Wang C, Zhang L (2011) Edgel index for large-scale sketch-based image search. In: Proceedings of the 2011 IEEE conference on computer vision and pattern recognition (CVPR) Colorado Springs, CO, USA, 2011, pp 761–768

  • Chang H, Yeung D, Xiong Y (2004) Super-resolution through neighbor embedding. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2004:275–282

    Google Scholar 

  • Chen J, Ma H (2014) Privacy-preserving decentralized access control for cloud storage systems. In: The 2014 IEEE 7th international conference on cloud computing

  • Chen X, Li J, Susilo W (2012) Efficient fair conditional payments for outsourcing computations. IEEE Trans Inf Forensics Secur 7(6):1687–1694

    Article  Google Scholar 

  • Chen B, Shu H, Coatrieux G, Chen G, Sun X, Coatrieux J (2015) Color image analysis by quaternion-type moments. J Math Imaging Vis 51(1):124–144

    Article  MathSciNet  MATH  Google Scholar 

  • Dong W, Zhang D, Shi G et al (2011) Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans Image Process 20(7):1838–1857

  • Elad M, Feuer A (1997) Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans Image Process 6(12):1646–1658

    Article  Google Scholar 

  • Elad M, Feuer A (1999) Super-resolution reconstruction of image sequences. IEEE Trans Pattern Anal Mach Intell 21(9):817–834

    Article  Google Scholar 

  • Freeman T, Jones T, Pasztor EC (2002) Example based superresolution. IEEE Comput Graph Appl 22(2):56–65

    Article  Google Scholar 

  • Galea C, Farrugia R (2015) Fusion of intra- and inter-modality algorithms for face-sketch recognition. Comput Anal Images Patterns 9257:700–7110

    MathSciNet  Google Scholar 

  • Galoogahi HK, Sim T (2012) Face sketch recognition by local radon binary pattern: Lrbp. In: The 19th IEEE international conference on image processing, 2012

  • Gong Y, Lazebnik S (2011) Iterative quantization: a procrustean approach to learning binary codes. In: IEEE conference on computer vision and pattern recognition 2011

  • Gruca A, Czachorski T, Kozielski S (2014) Advances in intelligent systems and computing. Springer International Publishing, Berlin

    Google Scholar 

  • Gu B, Sheng V, Tay K, Romano W, Li S (2015) Incremental support vector learning for ordinal regression. IEEE Trans Neural Netw Learn Syst 26(7):1403–1416

    Article  MathSciNet  Google Scholar 

  • He L, Qi H, Zaretzki R (2013) Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2013, pp 345–352

  • Hong C, Yu J, Tao D, Wang M (2015) Image-based three-dimensional human pose recovery by multiview locality-sensitive sparse retrieval. IEEE Trans Ind Electron 62(6):3742–3751

    Google Scholar 

  • Irani M, Peleg S (1991) Improving resolution by image registration. Graph Models Image Process (CVGIP) 53(3):231–239

    Article  Google Scholar 

  • Jin Li, Kwangjo K (2010) Hidden attribute-based signatures without anonymity revocation. Inf Sci 180(9):1681–1689

    Article  MathSciNet  MATH  Google Scholar 

  • Jin L, Xiaofeng C, Mingqiang L (2014) Secure deduplication with efficient and reliable convergent key management. IEEE Trans Parallel Distrib Syst 25(6):1615–1625

    Article  Google Scholar 

  • Li W, Zhou Z (2015) Learning to hash for big data: current status and future trends (in Chinese). Chin Sci Bull 60:485490

    Google Scholar 

  • Li J, Wang Q, Wang C, Ren K (2009) Enhancing attribute-based encryption with attribute hierarchy. In: The fourth international conference on communications and networking in China

  • Li J, Zhao G, Chen X, Xie D, Rong C, Li W, Tang L, Tang Y (2010) Fine-grained data access control systems with user accountability in cloud computing. In: The 2010 IEEE second international conference on cloud computing technology and science

  • Li J, Huang X, Li J, Chen X, Xiang Y (2014) Securely outsourcing attribute-based encryption with checkability. IEEE Trans Parallel Distrib Syst 25(8):2201–2210

    Article  Google Scholar 

  • Li J, Li X, Yang B, Sun X (2015) Segmentation-based image copy-move forgery detection scheme. IEEE Trans Inf Forensics Secur 10(3):507–518

    Article  Google Scholar 

  • Lin D, Tang X (2005) Coupled space learning of image style transformation. In: Proceedings of the 10th IEEE international conference on computer vision. Beijing: IEEE, 2005, pp 1699–1706

  • Lin L, Zeng K, Wang Y, Xu YQ, Zhu SC (2013) Video stylization: painterly rendering and optimization with content extraction. IEEE Trans Circuits Syst Video Technol 23(4):577–590

    Article  Google Scholar 

  • Lin L, Xu Y, Liang X, Lai J (2014) Complex background subtraction by pursuing dynamic spatio-temporal models. IEEE Trans Image Process 23(7):3191–3202

    Article  MathSciNet  MATH  Google Scholar 

  • Lin L, Wang X, Yang W, Lai JH (2015a) Discriminatively trained and-or graph models for object shape detection. IEEE Trans Pattern Anal Mach Intell 37(5):959–972

    Article  Google Scholar 

  • Lin L, Zhang R, Duan X (2015b) Adaptive scene category discovery with generative learning and compositional sampling. IEEE Trans Circuits Syst Video Technol 25(2):251–260

    Article  Google Scholar 

  • Liu X, Lin L, Jin H (2015) Contextualized trajectory parsing with spatio-temporal graph. In: IEEE transactions on pattern analysis and machine intelligence, PP (99) (2015) 1–1

  • Lowe DG (1999) Object recognition from local scale-invariant features. Proceed Int Conf Comput Vis 2:11501157

    Google Scholar 

  • Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  • Matthew T, Alex P (1991) Eigenfaces for recognition. J Cognit Neurosci 3(1):71–86

    Article  Google Scholar 

  • Mrozek D, Maysiak B, Kozielski S (2006) East: energy alignment search tool. Lect Notes Comput Sci 4223:696–705

    Article  Google Scholar 

  • Pan Z, Zhang Y, Kwong S (2015a) Efficient motion and disparity estimation optimization for low complexity multiview video coding. IEEE Trans Broadcast 61(2):166–176

    Article  Google Scholar 

  • Pan Z, Yu J, Xiao C, Sun W (2015b) Single image super resolution based on adaptive multi-dictionary learning (in Chinese). Acta Electron Sin 43:209–216

    Google Scholar 

  • Pawlowski R, Malysiak-Mrozek B, Kozielski S, Mrozek D (2011) Fast and accurate similarity searching of biopolymer sequences with GPU and CUDA. In: 11th International conference on algorithms and architectures for parallel processing (ICA3PP 2011)

  • Silva MAA, Camara-Chavez G (2014) Face sketch recognition from local features. In: The 27th SIBGRAPI conference on graphics, 2014

  • Smith LN, Elad M (2013) Improving dictionary learning: multiple dictionary updates and coefficient reuse. IEEE Sig Process Lett 20:79–82

    Article  Google Scholar 

  • Szanto B, Pozsegovics P, Vamossy Z, Sergyan S (2011) Sketch4match content-based image retrieval system using sketches. In: Proceedings of the 2011 IEEE 9th international symposium on applied machine intelligence and informatics (SAMI), Smolenice, IEEE, 2011, pp 183–188

  • Tang X, Wang X (2002) Face photo recognition using sketch. In: Proceedings of the 2002 international conference on image processing. Rochester, NY: IEEE, 2002, pp 257–260

  • Tang X, Wang X (2004) Face sketch recognition. IEEE Trans Circuits Syst Video Technol 14(1):50–57

    Article  Google Scholar 

  • Timofte R et al (2013) Anchored neighborhood regression for fastexample-based super-resolution. In: IEEE international conference oncomputer vision (ICCV), 2013, pp 1920–1927

  • Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Trans Pattern Anal Mach Intell 31(11):1955–1967

    Article  Google Scholar 

  • Wang Y, Yu M, He G, Jia Q (2011) Query by sketch: an asymmetric sketch-vs-image retrieval system. In: Proceedings of the 4th international congress on image and signal processing, Shanghai, China, 2011, pp 1368–1372

  • Wang S, Zhang L, Y et al (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: Proceedings of the 25th IEEE conference on computer vision and pattern recognition. Providence, RI: IEEE, 2012, pp 2216–2223

  • Wang N, Tao D, Gao X, Li X, Li J (2014) A comprehensive survey to face hallucination. Int J Comput Vis 106(1):9–30

    Article  Google Scholar 

  • Xia Z, Wang X, Sun X, Liu Q, Xiong N (2016) Steganalysis of LSB matching using differences between nonadjacent pixels. Multimed Tools Appl 75(4):1947–1962

    Article  Google Scholar 

  • Yamasaki T, Aizawa K, Silva GC (2009) Sketch-based spatial queries for retrieving human locomotion patterns from continuously archived GPS data. IEEE Trans Multimed 11(7):1240–1253

    Article  Google Scholar 

  • Yang M, Wang YCF (2013) A self-learning approach to single image super-resolution. IEEE Trans Multimed 15(3):498–508

  • Yang J, Wright J, Huang T (2010) Image super-resolution via sparse representation 19:2861–2873

  • Yu J, Rui Y, Tang Y, Tao D (2014a) High-order distance based multiview stochastic learning in image classification. IEEE Trans Cybern 44(12):2431–2442

    Article  Google Scholar 

  • Yu J, Hong R, Wang M, You J (2014b) Image clustering based on sparse patch alignment framework. Pattern Recognit 47(11):3512–3519

    Article  MATH  Google Scholar 

  • Yu J, Rui Y, Chen B (2014c) Exploiting click constraints and multi-view features for image re-ranking. IEEE Trans Multimed 16(1):159–168

    Article  Google Scholar 

  • Yu J, Rui Y, Tao D (2014d) Click prediction for web image reranking using multimodal sparse coding. IEEE Trans Image Process 23(5):2019–2032

    Article  MathSciNet  MATH  Google Scholar 

  • Yu J, Gao X et al (2014e) A unified learning framework for single image. IEEE Trans Neural Netw Learn Syst 25(4):780–792

    Article  MathSciNet  Google Scholar 

  • Yu J, Tao D, Rui Y (2015) Learning to rank using user clicks and visual features for image retrieval. IEEE Trans Cybern 45(4):767–779

    Article  Google Scholar 

  • Zeyde R, Elad M, Protter M et al (2012) On single image scale-up using sparse representations. In: Proceedings of the 7th international conference on curves and surfaces, Avignon, France, 2012, pp 711–730

  • Zhang W, Wang X, Tang X (2011) Coupled information-theoretic encoding for face photo-sketch recognition. In: IEEE conference on computer vision and pattern recognition (CVPR), 2011

  • Zhou X, Cao L (2015) The sketch face recognition combining with adaboost and blocking lbp. J Image Graph 20(1):50–58

    Google Scholar 

  • Zhao J, Mao Y, Fang Q, Liang Z, Yang F, Zhan S (2015) Heterogeneous face recognition based on super resolution reconstruction by adaptive multi-dictionary learning, vol 9428. The 10th Chinese conference on biometric recognition (CCBR 2015), 2015, pp 143–150

  • Zhao J, Fang Q, Liang Z, Hu C, Yang F, Zhan S (2016) Sketch face recognition based on super-resolution reconstruction. J Image Graphics 21(2):0218–0224

    Google Scholar 

  • Zheng Y, Jeon B, Xu D, Wu Q, Zhang H (2015) Image segmentation by generalized hierarchical fuzzy C-means algorithm. J Intell Fuzzy Syst 28(2):961–973

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grants 61371156, Anhui Province Science and Technology Research Programs under Grant 1401B042019. The authors would like to thank the anonymous reviewers for their helpful and constructive comments and suggestions.

Author information

Authors and Affiliations

  1. Hefei University of Technology, Hefei, China

    Shu Zhan, Jingjing Zhao, Yucheng Tang & Zhenzhu Xie

Authors
  1. Shu Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingjing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yucheng Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenzhu Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shu Zhan.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhan, S., Zhao, J., Tang, Y. et al. Face image retrieval: super-resolution based on sketch-photo transformation. Soft Comput 22, 1351–1360 (2018). https://doi.org/10.1007/s00500-016-2427-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2427-0

Keywords

Search

Navigation