Skip to main content
SpringerLink
Log in

Color face image decomposition under complex lighting conditions

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

An Erratum to this article was published on 06 August 2014

Abstract

In this paper, we proposed a method to recover the reflectance and shading images of face images, such as portraits, captured under complex lighting conditions. Under such lighting conditions, traditional intrinsic image decomposition can hardly be applied on face images especially with cast shadows. Inspired by the fact that the face symmetry is an inherent property of reflectance image for human face, face symmetry is firstly exploited as a constraint into traditional intrinsic decomposition framework to obtain the initial reflectance image. To remove the artifacts of the initial reflectance image caused by cast shadows, we use the skin color model to refine the reflectance image. Experiments show that our method is effective for face images under complex lighting conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Adini, Y., Moses, Y., Ullman, S.: Face recognition: The problem of compensating for changes in illumination direction. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 721–732 (1997)

    Article  Google Scholar 

  2. Barron, J., Malik, J.: Shape, albedo, and illumination from a single image of an unknown object. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 334–341 (2012)

  3. Barron, J., Malik, J.: Shape, illumination, and reflectance from shading. Tech. rep, Berkeley Tech Report (2013)

  4. Barron, J.T., Malik, J.: Color constancy, intrinsic images, and shape estimation. In: Computer Vision-ECCV 2012, pp. 57–70. Springer (2012)

  5. Barrow, H.G., Tenenbaum, J.M.: Recovering intrinsic scene characteristics from images. Academic Press, London (1978)

    Google Scholar 

  6. Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003)

    Article  Google Scholar 

  7. Blanz, V., Vetter, T.: Face recognition based on fitting a 3d morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1063–1074 (2003)

    Article  Google Scholar 

  8. Casati, J.P.B., Moraes, D.R., Rodrigues, E.L.L.: SFA: A human skin image database based on FERET and AR facial images. In: IX workshop de Visao Computational, Rio de Janeiro (2013)

  9. Chen, J., Su, G., He, J., Ben, S.: Face image relighting using locally constrained global optimization. In: Computer Vision-ECCV 2010, pp. 44–57. Springer (2010)

  10. Chen, X., Chen, M., Jin, X., Zhao, Q.: Face illumination transfer through edge-preserving filters. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 281–287. IEEE, New York (2011)

  11. Chen, X., Rynn, P., Bowyer, K.W.: Fully automated facial symmetry axis detection in frontal color images. In: Fourth IEEE Workshop on Automatic Identification Advanced Technologies, 2005, pp. 106–111. IEEE (2005)

  12. Deng, W., Hu, J., Guo, J.: Extended src: Undersampled face recognition via intraclass variant dictionary. IEEE Trans. Pattern Anal. Mach. Intell. 34(9), 1864–1870 (2012)

    Article  Google Scholar 

  13. Deng, W., Hu, J., Guo, J.: In defense of sparsity based face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 399–406. IEEE, New York (2013)

  14. Gehler, P.V., Rother, C., Kiefel, M., Zhang, L., Schölkopf, B.: Recovering intrinsic images with a global sparsity prior on reflectance. In: NIPS, vol. 2, p. 4 (2011)

  15. Han, H., Shan, S., Qing, L., Chen, X., Gao, W.: Lighting aware preprocessing for face recognition across varying illumination. In: Computer Vision-ECCV 2010, pp. 308–321. Springer (2010)

  16. Harguess, J., Aggarwal, J.: Is there a connection between face symmetry and face recognition? In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2011, pp. 66–73. IEEE, New York (2011)

  17. Jones, M.J., Rehg, J.M.: Statistical color models with application to skin detection. Int. J. Comput. Vision 46(1), 81–96 (2002)

    Article  MATH  Google Scholar 

  18. Kakumanu, P., Makrogiannis, S., Bourbakis, N.: A survey of skin-color modeling and detection methods. Pattern Recognit. 40(3), 1106–1122 (2007)

    Article  MATH  Google Scholar 

  19. Kemelmacher-Shlizerman, I., Basri, R.: 3D face reconstruction from a single image using a single reference face shape. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 394–405 (2011)

    Article  Google Scholar 

  20. Land, E.H., McCann, J.: Lightness and retinex theory. JOSA 61(1), 1–11 (1971)

    Article  Google Scholar 

  21. Li, Q., Yin, W., Deng, Z.: Image-based face illumination transferring using logarithmic total variation models. Visual Comput. 26(1), 41–49 (2010)

    Article  Google Scholar 

  22. Luo, Y., Gavrilova, M.L., Wang, P.S.: Facial metamorphosis using geometrical methods for biometric applications. Int. J. Pattern Recognit. Artif. Intell. 22(03), 555–584 (2008)

    Article  Google Scholar 

  23. Milborrow, S., Nicolls, F.: Locating facial features with an extended active shape model. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) Computer Vision-ECCV 2008. Lecture Notes in Computer Science, vol. 5305, pp. 504–513. Springer, Berlin Heidelberg (2008)

  24. Patel, A., Smith, W.A.: Shape-from-shading driven 3D morphable models for illumination insensitive face recognition. In: BMVC, pp. 1–10 (2009)

  25. Peers, P., Tamura, N., Matusik, W., Debevec, P.: Post-production facial performance relighting using reflectance transfer. In: ACM Transactions on Graphics (TOG), vol. 26, p. 52. ACM (2007)

  26. Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. In: ACM Transactions on Graphics (TOG), vol. 22, pp. 313–318. ACM (2003)

  27. Phung, S.L., Bouzerdoum, A., Chai Sr, D.: Skin segmentation using color pixel classification: analysis and comparison. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 148–154 (2005)

    Article  Google Scholar 

  28. Ramamoorthi, R., Hanrahan, P.: An efficient representation for irradiance environment maps. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 497–500. ACM (2001)

  29. Shashua, A., Riklin-Raviv, T.: The quotient image: class-based re-rendering and recognition with varying illuminations. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 129–139 (2001)

    Article  Google Scholar 

  30. Shen, L., Yeo, C.: Intrinsic images decomposition using a local and global sparse representation of reflectance. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 697–704. IEEE (2011)

  31. Smith, W.A.P., Hancock, E.: Estimating the albedo map of a face from a single image. In: IEEE International Conference on Image Processing, 2005. ICIP 2005, vol. 3, pp. III-780-3 (2005)

  32. Štruc, A.V.: Performance evaluation of photometric normalization techniques for illumination invariant face recognition. CiteSeer (2010)

  33. Tan, X., Triggs, B.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans. Image Process. 19(6), 1635–1650 (2010)

    Article  MathSciNet  Google Scholar 

  34. Tappen, M.F., Freeman, W.T., Adelson, E.H.: Recovering intrinsic images from a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27(9), 1459 (2005)

    Article  Google Scholar 

  35. Vezhnevets, V., Sazonov, V., Andreeva, A.: A survey on pixel-based skin color detection techniques. In: Proceedings Graphicon, vol. 3, pp. 85–92. Moscow, Russia (2003)

  36. Wang, H., Li, S.Z., Wang, Y.: Generalized quotient image. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004, vol. 2, pp. II-498. IEEE (2004)

  37. Wang, Y., Liu, Z., Hua, G., Wen, Z., Zhang, Z., Samaras, D.: Face re-lighting from a single image under harsh lighting conditions. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR ’07, pp. 1–8 (2007)

  38. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)

    Article  Google Scholar 

  39. Yang, L., Waibel, A.: A real-time face tracker. In: Applications of Computer Vision, 1996. WACV’96., Proceedings 3rd IEEE Workshop on, pp. 142–147. IEEE (1996)

  40. Zhang, R., Tsai, P.S., Cryer, J., Shah, M.: Shape-from-shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21(8), 690–706 (1999). doi:10.1109/34.784284

    Article  Google Scholar 

  41. Zhao, W., Chellappa, R.: Illumination-insensitive face recognition using symmetric shape-from-shading. In: IEEE Conference on Computer Vision and Pattern Recognition, 2000. Proceedings, vol. 1, pp. 286–293. IEEE, New York (2000)

  42. Zhao, X., Shah, S.K., Kakadiaris, I.A.: Illumination normalization using self-lighting ratios for 3d2d face recognition. In: Proceedings of the 12th International Conference on Computer Vision, Vol. 2, ECCV’12, pp. 220–229. Springer-Verlag (2012)

Download references

Acknowledgments

We would like to thank the anonymous reviewers for helping us to improve this paper. The work is supported by NSFC of China (No. 61103137), 863 Program of China (No. 2013AA013902) and Key Projects in the National Science and Technology Pillar Program of China (No. 2012BAH62F02).

Author information

Authors and Affiliations

  1. College of Computer Science, Sichuan University, Chengdu, China

    Kexin Hu, Yanli Liu, Qi Dong & Hao Liu

  2. National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu, China

    Kexin Hu, Yanli Liu, Qi Dong & Hao Liu

  3. School of Computer Science and Engineering, University of Electronic Science and Technology, Chengdu, China

    Guanyu Xing

Authors
  1. Kexin Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanli Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qi Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guanyu Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanli Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, K., Liu, Y., Dong, Q. et al. Color face image decomposition under complex lighting conditions. Vis Comput 30, 685–695 (2014). https://doi.org/10.1007/s00371-014-0962-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-014-0962-1

Keywords

Search

Navigation