Skip to main content
SpringerLink
Log in

A Comprehensive Survey to Face Hallucination

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper comprehensively surveys the development of face hallucination (FH), including both face super-resolution and face sketch-photo synthesis techniques. Indeed, these two techniques share the same objective of inferring a target face image (e.g. high-resolution face image, face sketch and face photo) from a corresponding source input (e.g. low-resolution face image, face photo and face sketch). Considering the critical role of image interpretation in modern intelligent systems for authentication, surveillance, law enforcement, security control, and entertainment, FH has attracted growing attention in recent years. Existing FH methods can be grouped into four categories: Bayesian inference approaches, subspace learning approaches, a combination of Bayesian inference and subspace learning approaches, and sparse representation-based approaches. In spite of achieving a certain level of development, FH is limited in its success by complex application conditions such as variant illuminations, poses, or views. This paper provides a holistic understanding and deep insight into FH, and presents a comparative analysis of representative methods and promising future directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Ahmed, S., Ghafoor, A., & Sheri, A. (2008). Direct hallucination: Direct locality preserving projections (dlpp) for face super-resolution. In Proceedings of international conference on advanced computer theory and engineering (pp. 105–110).

  • Baker, S., & Kanade, T. (2000a). Hallucinating faces. In Proceedings of IEEE international conference on automatic face and gesture recognition (pp. 83–88).

  • Baker, S., & Kanade, T. (2000b). Limits on super-resolution and how to break them. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 372–379).

  • Baker, S., & Kanade, T. (2002). Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9), 1167–1183.

    Article  Google Scholar 

  • Belkin, M. & Niyogi, P. (2001). Laplacian eigenmaps and spectral techniques for embedding and clustering. In Proceedings of advances in neural information processing systems (pp. 585–591).

  • Bishop, C., Blake, A., & Marthi, B. (2003). Super-resolution enhancement of video. In Proceedings of IEEE workshop on artificial intelligence and statistics.

  • Bonet, J. (1997). Multiresolution sampling procedure for analysis and synthesis of texture images. In Proceedings of SIGGRAPH (pp. 361–368).

  • Brown, M., & Lowe, D. (2007). Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 74(1), 59–73.

    Google Scholar 

  • Brox, T., Bruhn, A., Papenberg, N., & Weicket, J. (2004). High accuracy optical flow estimation based on a theory for warping. In Proceedings of European conference on computer vision (pp. 25–36).

  • Burt, P. (1981). Fast filter transforms for image processing. Computer Graphics and Image Processing, 16(1), 20–51.

    Article  Google Scholar 

  • Burt, P., & Adelson, E. (1983). The laplacian pyramid as a compact image code. IEEE Transactions on Communications, 31(4), 532–540.

    Article  Google Scholar 

  • Cai, D., He, X., Han, J., & Zhang, H. (2006). Orthogonal laplacianfaces for face recognition. IEEE Transactions on Image Processing, 15(11), 3608–3614.

    Article  Google Scholar 

  • Capel, D., & Zisserman, A. (2001). Super-resolution from multiple views using learnt image models. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 627– 634).

  • Chakrabarti, A., Rajagopaian, A., & Chellappa, R. (2007). Super-resolution of face images using kernel-pca-based prior. IEEE Transactions on Multimedia, 9(4), 888–892.

    Article  Google Scholar 

  • Chang, D., Yeung, H., & Xiong, Y. (2004). Super-resolution through neighbor embedding. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 275–282).

  • Chang, M., Zhou, L., Han, Y., & Deng, X. (2010). Face sketch synthesis via sparse representation. In Proceedings of international conference on pattern recognition (pp. 2146–2149).

  • Chang, M., Zhou, L., Deng, X., & Han, Y. (2011). Face sketch synthesis via multivariate output regression. In Proceedings of international conference on human–computer interaction (pp. 555–561).

  • Chellappa, R., Wilson, C., & Sirohey, S. (1995). Human and machine recognition of faces: A survey. Proceedings of the IEEE, 83(5), 705–740.

    Article  Google Scholar 

  • Chen, H., Xu, Y., Shum, H., Zhu, S., & Zheng, N. (2001). Example-based face sketch generation with non-parametric sampling. In Proceedings of IEEE international conference on computer vision (pp. 433–438).

  • Chen, J., Yi, D., Yang, J., & Zhao, G. (2009). Learning mappings for face synthesis from near infrared to visual light images. In Proceedings of IEEE international conference on computer vision and pattern recognition (pp. 156–163).

  • Dedeoglu, G., Kanade, T., & August, J. (2004). High-zoom video hallucination by exploiting spatial-temporal regularities. In Proceedings of IEEE international conference on computer vision and pattern recognition (pp. 151–158).

  • Dong, H., & Gu, N. (2001). Asian face image database pf01. Technical report, Pohang University of Science and Technology.

  • Donoho, D. (2006). For most large underdetermined systems of linear equations, the minimal l1-norm near-solution approximates the sparsest near-solution. Communications on Pure and Applied Mathematics, 59(7), 907–934.

    Article  MathSciNet  Google Scholar 

  • Efros, A., & Freeman, W. (2001). Image quilting for texture synthesis and transfer. In Proceedings of SIGGRAPH (pp. 341–346).

  • Efros, A., & Leung, T. (1999). Texture synthesis by non-parametric sampling. In Proceedings of IEEE international conference on computer vision (pp. 1033–1038).

  • Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 15(12), 3736–3745.

    Article  MathSciNet  Google Scholar 

  • Elad, M., & Feuer, A. (1997). Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing, 6(12), 1646–1658.

    Article  Google Scholar 

  • Elad, M., & Feuer, A. (1999). Super-resolution reconstruction of image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(9), 817–834.

    Article  Google Scholar 

  • Fan, W., & Yeung, D. (2007). Image hallucination using neighbor embedding over visual primitive manifolds. In Proceedings of IEEE international conference on computer vision and pattern recognition (pp. 1–7).

  • Fransens, R., Strecha, C., & Gool, L. (2005). Parametric stereo for multi-pose face recognition and 3d-face modeling. In Proceedings of IEEE international conference on computer vision workshop analysis and modeling of faces and gestures (pp. 109–124).

  • Freeman, W., & Pasztor, E. (1999). Learning low-level vision. In Proceedings of IEEE international conference on computer vision (pp. 1182–1189).

  • Freeman, W., Pasztor, E., & Carmichael, O. (2000). Learning low-level vision. International Journal of Computer Vision, 40(1), 25–47.

    Article  MATH  Google Scholar 

  • Freeman, W., Jones, T., & Pasztor, E. (2002). Example-based super-resolution. IEEE Computer Graphics and Applications, 22(2), 56–65.

    Article  Google Scholar 

  • Fu, Y., Guo, G., & Huang, T. (2010). Age synthesis and estimation via faces: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(11), 1955–1976.

    Article  Google Scholar 

  • Gao, W., Cao, B., Shan, S., Chen, X., Zhou, D., Zhang, X., et al. (2008a). The cas-peal large-scale Chinese face database and baseline evaluations. IEEE Transactions on Systems, Man, and Cybernetics A, 38(1), 149–161.

    Article  Google Scholar 

  • Gao, X., Zhong, J., Li, J., & Tian, C. (2008b). Face sketch synthesis using e-hmm and selective ensemble. IEEE Transactions on Circuits and Systems for Video Technology, 18(4), 487–496.

    Article  Google Scholar 

  • Gao, X., Zhong, J., Tao, D., & Li, X. (2008c). Local face sketch synthesis learning. Neurocomputing, 71(10–12), 1921–1930.

    Article  Google Scholar 

  • Gao, X., Wang, N., Tao, D., & Li, X. (2012). Face sketch-photo synthesis and retrieval using sparse representation. IEEE Transactions on Circuits and Systems for Video Technology, 22(8), 1213– 1226.

    Google Scholar 

  • Gelman, A., Carlin, H., Stern, J., & Rubin, D. (2003). Bayesian data analysis. Boca Raton: Chapman & Hall/CRC.

  • Georghiades, A., Belhumeur, P., & Kriegman, D. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643–660.

    Article  Google Scholar 

  • Gunturk, B., Batur, A., & Altunbasak, Y. (2003). Eigenface-domain super-resolution for face recognition. IEEE Transactions on Image Processing, 12(5), 597–606.

    Article  Google Scholar 

  • Hallinan, P. (1994). A low dimensional representation of human faces for arbitrary lighting conditions. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 995–999).

  • Hardie, R., Barnard, K., & Armstrong, E. (1997). Joint map registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing, 6(12), 1621–1633.

    Article  Google Scholar 

  • He, X. (2005). Locality preserving projections. Technical report, PhD Thesis, University of Chicago.

  • Hertzmann, A., Jacobs, C., Oliver, N., Curless, B., & Salesin, D. (2001). Image analogies. In Proceedings of SIGGRAPH (pp. 327–340).

  • Hsu, C., Lin, C., & Liao, H. (2009). Cooperative face hallucination using multiple references. In Proceedings of IEEE international conference on multimedia & expo (pp. 818–821).

  • Hu, Y., Lam, K., Qiu, G., & Shen, T. (2010). Learning local pixel structure for face hallucination. In Proceedings of IEEE international conference on image processing (pp. 26–29).

  • Hu, Y., Lam, K., Qiu, G., & Shen, T. (2011). From local pixel structure to global image super-resolution: A new face hallucination framework. IEEE Transactions on Image Processing, 20(2), 433–445.

    Article  MathSciNet  Google Scholar 

  • Hwang, B., & Lee, S. (2003). Reconstruction of partially damaged face images based on a morphable face model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(3), 365–372.

    Article  Google Scholar 

  • Iwashita, S., Takeda, Y., & Onisawa, T. (1999). Expressive face caricature drawing. In Proceedings of IEEE international conference on fuzzy systems (pp. 1597–1602).

  • Jain, A., Duin, R., & Mao, J. (2000). Statistical pattern recognition: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1), 4–37.

    Article  Google Scholar 

  • Ji, N., Chai, X., Shan, S., & Chen, X. (2011). Local regression model for automatic face sketch generation. In Proceedings of international conference on image and graphics (pp. 412–417).

  • Jia, K., & Gong, S. (2005). Multi-modal tensor face for simultaneous super-resolution and recognition. In Proceedings of IEEE international conference on computer vision (pp. 1683–1690).

  • Jia, K., & Gong, S. (2006). Multi-resolution patch tensor for face expression hallucination. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 395–402).

  • Jia, K., & Gong, S. (2008). Generalized face super-resolution. IEEE Transactions on Image Processing, 17(6), 873–886.

    Article  MathSciNet  Google Scholar 

  • Jones, M., Sinha, P., Vetter, T., & Poggio, T. (1997). Top–down learning of low-level vision tasks. Current Biology, 7(12), 991–994.

    Article  Google Scholar 

  • Jung, C., Jiao, L., Liu, B., & Gong, M. (2011). Position-patch based face hallucination using convex optimization. IEEE Signal Processing Letters, 18(6), 367–370.

    Article  Google Scholar 

  • Kanade, T., Cohn, J., & Tian, Y. (2000). Comprehensive database for face expression analysis. In Proceedings of IEEE international conference on automatic face and gesture recognition (pp. 46– 53).

  • Kang, H., He, W., Chui, C., & Chakraborty, U. (2005). Interactive sketch generation. The Visual Computer, 21(8–10), 821–830.

    Article  Google Scholar 

  • Komarek, P. (2004). Logistic regression for data mining and high-dimensional classification. Technical report, PhD Thesis, Carnegie Mellon University.

  • Koshimizu, H., & Tominaga, M. (1999). On kanse face processing for computerized face caricaturing system picasso. In Proceedings of IEEE international conference on systems, man, and cybernetics (pp. 294–299).

  • Kumar, B., & Aravind, R. (2008a). A 2d model for face superresolution. In Proceedings of international conference on pattern recognition (pp. 1–4).

  • Kumar, B., & Aravind, R. (2008b). Face hallucination using olpp and kernel ridge regression. In Proceedings of IEEE international conference on image processing (pp. 353–356).

  • Lee, D., & Seung, H. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755), 788–791.

    Article  Google Scholar 

  • Li, B., Chang, H., Shan, S., & Chen, X. (2009). Aligning coupled manifolds for face hallucination. IEEE Signal Processing Letters, 16(11), 957–960.

    Article  Google Scholar 

  • Li, S. (2010). Markov random field modeling in image analysis. Berlin: Springer.

  • Li, Y., & Lin, X. (2004). Face hallucination with pose variation. In Proceedings of international conference on automatic face and gesture recognition (pp. 723–728).

  • Li, Y., Savvides, M., & Bhagavatula, V. (2006). Illumination tolearn face recognition using a novel face from sketch synthesis approach and advanced correlation filters. In Proceedings of IEEE international conference on acoustics, speech, and signal processing (pp. 357–360).

  • Liang, L., Liu, C., Xu, Y., & Guo, B. (2001). Real-time texture synthesis by patch-based sampling. ACM Transactions on Graphics, 20, 127–150.

    Google Scholar 

  • Liang, Y., Lai, J., Xie, X., & Liu, W. (2010). Face hallucination under an image decomposition perspective. In Proceedings of international conference on pattern recognition (pp. 2158–2161).

  • Lin, Z., He, J., Tang, X., & Tang, C. (2007). Limits of learning-based superresolution algorithms. In Proceedings of IEEE international conference on computer vision (pp. 1–8).

  • Lin, Z., He, J., Tang, X., & Tang, C. (2008). Limits of learning-based superresolution alogrithms. International Journal of Computer Vision, 80(3), 406–420.

    Article  Google Scholar 

  • Liu, C., Shum, H., & Zhang, C. (2001). A two-step approach to hallucinating faces: Global parametric model and local nonparametric model. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 192–198).

  • Liu, C., Shum, H., & Freeman, W. (2007a). Face hallucination: Theory and practice. International Journal of Computer Vision, 75(1), 115–134.

    Article  Google Scholar 

  • Liu, Q., Tang, X., Jin, H., Lu, H., & Ma, S. (2005a). A nonlinear approach for face sketch synthesis and recognition. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1005–1010).

  • Liu, W., Lin, D., & Tang, X. (2005b). Face hallucination through dual associative learning. In Proceedings of IEEE international conference on image processing (pp. 873–876).

  • Liu, W., Lin, D., & Tang, X. (2005c). Hallucinating faces: Tensorpatch super-resolution and coupled residue compensation. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 478–484).

  • Liu, W., Lin, D., & Tang, X. (2005d). Neighbor combination and transformation for hallucinating faces. In Proceedings of IEEE international conference on multimedia & expo (pp. 145–148).

  • Liu, W., Tang, X., & Liu, J. (2007b). Bayesian tensor inference for sketch-based face photo hallucination. In Proceedings of international joint conference on artificial intelligence (pp. 2141– 2146).

  • Liu, X. (2009). Discriminative face alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(11), 1941–1954.

    Article  Google Scholar 

  • Luo, P., Wang, X., & Tang, X. (2012). Hierarchical face parsing via deep learning. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 2480–2487).

  • Ma, H. X., Huang, H., Wang, S., & Qi, C. (2010a). A simple approach to multiview face hallucination. IEEE Signal Processing Letters, 17(6), 579–582.

    Google Scholar 

  • Ma, X., Zhang, J., & Qi, C. (2009). Position-based face hallucination method. In Proceedings of IEEE international conference on multimedia & expo (pp. 290–293).

  • Ma, X., Zhang, J., & Qi, C. (2010b). Hallucinating face by position-patch. Pattern Recognition, 43(6), 2224–2236.

    Article  Google Scholar 

  • Mairal, J., Sapiro, G., & Elad, M. (2008a). Learning multiscale sparse representations for image and video restoration. SIAM Multiscale Modeling and Simulation, 17, 214–241.

    Google Scholar 

  • Mairal, J., Elad, M., & Sapiro, G. (2008b). Sparse representation for color image restoration. IEEE Transactions on Image Processing, 17(1), 53–69.

    Google Scholar 

  • Martinez, A., Benavente, R. (1998). The ar face database. Technical report, CVC Technical, Report no. 24.

  • Messer, K., Matas, J., Kittler, J., Luettin, J., & Maitre, G. (1999). Xm2vtsdb: the extended m2vts database. In Proceedings of international conference on audio- and video-based biometric Person authentication (pp. 72–77).

  • Mika, S., Ratsch, G., & Weston, J. (1999). Fisher discriminant analysis with kernels. In Proceedings of IEEE workshop on neural networks for signal processing (pp. 41–48).

  • Moghaddam, B., Jebara, T., & Pentland, A. (2000). Bayesian face recognition. Pattern Recognition, 33(11), 1771–1782.

    Article  Google Scholar 

  • Nefian, A. (1997). Georgia tech face database. http://www.anefian.com/research/face_reco.htm. Accessed 3 Aug 2013.

  • Nefian, A., & Hayes, M. (1999). Face recognition using an embedded hmm. In Proceedings of international conference on audio- and video-based biometric person authentication (pp. 19–24).

  • Ong, E., & Bowden, R. (2011). Robust face feature tracking using shape-constrained multiresolution-selected linear predictors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1), 1–16.

    Article  Google Scholar 

  • Park, J., & Lee, S. (2003). Resolution enhancement of face image based on top–down learning. In Proceedings of SIGMM workshop on video surveillance (pp. 59–64).

  • Park, J., & Lee, S. (2008). An example-based face hallucination method for single-frame, low-resolution face images. IEEE Transactions on Image Processing, 17(10), 1806–1816.

    Article  MathSciNet  Google Scholar 

  • Park, S., & Savvides, M. (2007). Breaking the limitation of manifold analysis for super-resoluton of face images. In Proceedings of IEEE international conference on acoustics, speech and signal processing (pp. 573–576).

  • Pear, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Francisco, CA: Morgan Kaufmann.

  • Philips, P., Moon, H., Rauss, P., & Rizvi, S. (1997). The feret evaluation methodology for face-recognition algorithms In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 137–143).

  • Philips, P., Flynn, P., Scruggs, T., Bowyer, K., Chang, J., & Hoffman, K., et al. (2005). Overview of face recognition grand challenge. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 947–954).

  • Phillips, P., Moon, H., Rauss, P., & Rizvi, S. (2000). The feret evaluation methodology for face recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10), 1090–1104.

    Google Scholar 

  • Rabiner, L. (1989). A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.

    Article  Google Scholar 

  • Roh, M., & Lee, S. (2007). Performance analysis of face recognition alogrithms on Korean face database. International Journal of Pattern Recognition and Artificial Intelligence, 21(6), 1017–1033.

    Article  Google Scholar 

  • Roweis, S., & Saul, L. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500), 2323–2326.

    Article  Google Scholar 

  • Rowley, H., Baluja, S., & Kanade, T. (1998). Neural network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1), 137–143.

    Article  Google Scholar 

  • Samaria, F. (1994). Face recognition using hidden markov models. Techical report, PhD Thesis, University of Cambridge.

  • Sheikh, H., & Bovik, A. (2006). Image information and visual quality. IEEE Transactions on Image Processing, 15(2), 430–444.

    Article  Google Scholar 

  • Sim, T., Baker, S., & Bsat, M. (2002). The cmu pose, illumination, and expression (pie) database. In Proceedings of IEEE international conference on automatic face and gesture recognition (pp. 46– 51).

  • Stephenson, T., & Chen, T. (2006). Adaptive markov random fields for example-based super-resolution of faces. EURASIP Jouranl on Applied Signal Processing, 2006, 1–11.

    Article  Google Scholar 

  • Su, C., Zhuang, Y., Huang, L., & Wu, F. (2005). Steerable pyramid-based face hallucination. Pattern Recognition, 38(6), 813–824.

    Article  Google Scholar 

  • Sun, J., Zheng, N., Tao, H., & Shum, H. (2003). Image hallucination with primal sketch priors. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 729–736).

  • Tang, X., & Wang, X. (2002). Face photo recognition using sketches. In Proceedings of IEEE international conference on image processing (pp. 257–260).

  • Tang, X., & Wang, X. (2003). Face sketch synthesis and recognition. In Proceedings of IEEE international conference on computer vision (pp. 687–694).

  • Tang, X., & Wang, X. (2004). Face sketch recognition. IEEE Transactions on Circuits and Systems for Video Technology, 14(1), 1–7.

    Article  Google Scholar 

  • Tanveer, M., & Iqbal, N. (2010). A bayesian approach to face hallucination using dlpp and krr. In Proceedings of international conference on pattern recognition (pp. 2154–2157).

  • Tao, D., Li, X., Wu, X., Hu, W., & Maybank, S. (2007a). Supervised tensor learning. Knowledge and Information Systems, 13(1), 1–42.

    Article  Google Scholar 

  • Tao, D., Li, X., Wu, X., & Maybank, S. (2007b). General tensor discriminant analysis and Gabor features for gait recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(10), 1700–1715.

    Article  Google Scholar 

  • Tao, D., Song, M., Li, X., Shen, J., Sun, J., Wu, X., et al. (2008). Bayesian tensor approach for 3-D face modeling. IEEE Transactions on Circuits and Systems for Video Technology, 18(10), 1397–1410.

    Article  Google Scholar 

  • Tibshirani, R. (1996). Regression shrinkge and selection via the lasso. Journal of Royal Statistics Society B, 58(1), 267–288.

    MATH  MathSciNet  Google Scholar 

  • Tipping, M. (1991). Sparse bayesian learning and the relevance vector machine. Journal of Machine Learning Research, 1, 586–591.

    Google Scholar 

  • Turk, M., & Pentland, A. (1991). Face recognition using eigenfaces. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 586–591).

  • Vetter, T., & Troje, N. (1997). Separation of texture and shape in images of faces for image coding and synthesis. Journal of Optical Society of America, 14(9), 2152–2161.

    Google Scholar 

  • Wang, N., Gao, X., Tao, D., & Li, X. (2011). Face sketch-photo synthesis under multi-dictionary sparse representation framework. In Proceedings of international conference on image and graphics (pp. 82–87).

  • Wang, N., Li, J., Tao, D., Li, X., & Gao, X. (2013a). Heterogeneous image transformation. Pattern Recognition Letters, 34(1), 77–84.

  • Wang, N., Tao, D., Gao, X., Li, X., & Li, J. (2013b). Transductive face sketch-photo synthesis. IEEE Transactions on Neural Networks and Learning Systems, 24(9), 1–13.

    Article  Google Scholar 

  • Wang, S., Zhang, L., Liang, Y., & Pan, Q. (2012). Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 2216–2223).

  • Wang, X., & Tang, X. (2003). Face hallucination and recognition. In Proceedings of international conference on audio- and video-based biometric person authentication (pp. 486–494).

  • Wang, X., & Tang, X. (2005). Hallucinating face by eigentransformation. IEEE Transactions on Systems, Man, and Cybernetics C, 35(3), 425–434.

    Article  Google Scholar 

  • Wang, X., & Tang, X. (2006). Random sampling for subspace face recognition. International Journal of Computer Vision, 70(1), 91–104.

    Article  Google Scholar 

  • Wang, X., & Tang, X. (2009). Face photo-sketch synthesis and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(11), 1955–1967.

    Article  Google Scholar 

  • Wang, Z., & Bovik, A. (2002). A universal image quality index. IEEE Signal Processing Letters, 9(3), 81–84.

    Article  Google Scholar 

  • Wang, Z., & Bovik, A. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.

    Article  Google Scholar 

  • Wang, Z., & Bovik, A. (2009). Mean squared error: Love it or leave it? a new look at signal fidelity measures. IEEE Signal Processing Magazine, 26(1), 98–117.

    Article  Google Scholar 

  • Wen, F., Luan, Q., Liang, L., Xu, Y., & Shum, H. (2006). Color sketch generation. In Proceedings of international symposium on non-photorealistic animation and rendering (pp. 47–54).

  • Wright, J., Yang, A., Ganesh, A., Sastry, S., & Ma, Y. (2009). Robust face recognition via sparse represnetation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2), 210–227.

    Article  Google Scholar 

  • Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T., & Yan, S. (2010). Sparse representation for computer vision and pattern recognition. Proceedings of the IEEE, 98(6), 1031–1044.

    Article  Google Scholar 

  • Xiao, B., Gao, X., Tao, D., & Li, X. (2009). A new approach for face recognition by sketches in photos. Signal Processing, 89(8), 1576–1588.

    Article  MATH  Google Scholar 

  • Xiao, B., Gao, X., Tao, D., Yuan, Y., & Li, J. (2010). Photo-sketch synthesis and recognition based on subspace learning. Neurocomputing, 73(4–6), 840–852.

    Article  Google Scholar 

  • Xiong, Z., Sun, X., & Wu, F. (2009). Image hallucination with feature enhancement. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 2704–2081).

  • Yang, J., Tang, H., Ma, Y., & Huang, T. (2008a). Face hallucination via sparse coding. In Proceedings of IEEE international conference on image processing (pp. 1264–1267).

  • Yang, J., Wright, J., Huang, T., & Ma, Y. (2008b). Image super-resolution as sparse representation of raw image patches. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1–8).

  • Yedidia, J., Freeman, W., & Weiss, Y. (2001). Generalized belief propagation. In Proceedings of advances in neural information processing systems (pp. 689–695).

  • Yu, J., Liu, D., Tao, D., & Seah, H. (2012a). On combining multiple features for cartoon character retrieval and clip synthesis. IEEE Transactions on Systems, Man, and Cybernetics B, 42(5), 1413–1427.

    Article  Google Scholar 

  • Yu, J., Wang, M., & Tao, D. (2012b). Semisupervised multiview distance metric learning for cartoon synthesis. IEEE Transactions on Image Processing, 21(11), 4636–4648.

    Article  MathSciNet  Google Scholar 

  • Zalesny, A., Ferrari, V., Caenen, G., & Gool, L. (2005). Composite texture synthesis. International Journal of Computer Vision, 62(1–2), 161–176.

    Google Scholar 

  • Zhang, C., & Zhang, Z. (2010). A survey of recent advances in face detection. Research technical report.

  • Zhang, D., & Zhou, Z. (2005). \((2d)^2\) pca: 2-directional 2-dimensional pca for efficient face representation and recognition. Neurocomputing, 69(1–3), 224–231.

  • Zhang, J., Wang, N., Gao, X., Tao, D., & Li, X. (2011a). Face sketch-photo synthesis based on support vector regression. In Proceedings of IEEE international conference on image processing (pp. 1149–1152).

  • Zhang, L., Zhang, L., Mou, X., & Zhang, D. (2011b). Fsim: A feature similarity index for image quality assessment. IEEE Transactions on Image Processing, 20(8), 2378–2386.

    Article  MathSciNet  Google Scholar 

  • Zhang, T., Tao, D., Li, X., & Yang, J. (2009). Patch alignment for dimensionality reduction. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1299–1313.

    Article  Google Scholar 

  • Zhang, W., & Cham, W. (2008). Learning-based face hallucination in dct domain. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1–8).

  • Zhang, W., & Cham, W. (2011). Hallucinating face in the dct domain. IEEE Transactions on Image Processing, 20(10), 2769–2779.

    Article  MathSciNet  Google Scholar 

  • Zhang, W., Wang, X., & Tang, X. (2010). Lighting and pose robust face sketch synthesis. In Proceedings of European conference on computer vision (pp. 420–423).

  • Zhang, W., Wang, X., & Tang, X. (2011c). Coupled information-theoretic encoding for face photo-sketch recognition. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 513–520).

  • Zhang, X., Peng, S., & Jiang, J. (2008). An adaptive learning method for face hallucination using locality preserving projections. In Proceedings of IEEE international conference on automatic face and gesture recognition (pp. 1–8).

  • Zhao, W., Chellappa, R., Phillips, P., & Rosenfeld, A. (2003). Face recognition: A literature survey. ACM Computing Surveys, 35(4), 399–458.

    Article  Google Scholar 

  • Zhong, J., Gao, X., & Tian, C. (2007). Face sketch synthesis using e-hmm and selective ensemble. In Proceedings of IEEE international conference on acoustics, speech, and signal processing (pp. 485–488).

  • Zhou, H., Kuang, Z., & Wong, K. (2012). Markov weight fields for face sketch synthesis. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1091–1097).

  • Zhou, T., & Tao, D. (2013). Double shrinking sparse dimension reduction. IEEE Transactions on Image Processing, 22(1), 244–257.

    Article  MathSciNet  Google Scholar 

  • Zhuang, Y., Zhang, J., & Wu, F. (2007). Hallucinating faces: Lph super-resolution and neighbor reconstruction for residue compensation. Pattern Recognition, 40(11), 3178–3194.

    Article  MATH  Google Scholar 

Download references

Acknowledgments

We want to thank the helpful comments and suggestions from the anonymous reviewers. This work was supported in part by the National Basic Research Program of Chinas 973 Program under Grant 2012CB316400, the National Natural Science Foundation of China under Grants 61125204, 61125106, 61172146, 91120302, and 61072093, the Australia Research Council Discovery Project under Grant ARC DP-120103730, the Fundamental Research Funds for the Central Universities under Grant K5051202048, and Grant K50513100009, and the Shaanxi Innovative Research Team for Key Science and Technology under Grant 2012KCT-02.

Author information

Authors and Affiliations

  1. VIPS Lab, School of Electronic Engineering, Xidian University, Xi’an, 710071, People’s Republic of China

    Nannan Wang, Xinbo Gao & Jie Li

  2. Centre for Quantum Computation & Intelligent Systems, Faculty of Engineering & Information Technology, University of Technology Sydney, 235 Jones Street, Ultimo, NSW, 2007, Australia

    Dacheng Tao

  3. Center for OPTical IMagery Analysis and Learning (OPTIMAL), State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an, 710119, Shaanxi, People’s Republic of China

    Xuelong Li

Authors
  1. Nannan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dacheng Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinbo Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuelong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuelong Li.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, N., Tao, D., Gao, X. et al. A Comprehensive Survey to Face Hallucination. Int J Comput Vis 106, 9–30 (2014). https://doi.org/10.1007/s11263-013-0645-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-013-0645-9

Keywords

Search

Navigation