Skip to main content
SpringerLink
Log in

Multi-Cue Illumination Estimation via a Tree-Structured Group Joint Sparse Representation

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

A multi-cue illumination estimation method based on tree-structured group joint sparse representation is proposed. Tests show that the proposed method works better than existing methods, most of which are based on using only a single cue type, for example, a binarized color histogram or simple image statistic such as the mean RGB. Most existing illumination estimation methods make their estimates using only one of three kinds of cues. They differ in which cue type they use, but the chosen cue is either based on (1) properties of the low-level RGB color distribution, (2) mid-level initial illuminant estimates provided by subordinate methods, or (3) high-level knowledge of scene content (e.g., indoor versus outdoor scene). The proposed multi-cue method combines the information provided by cues of all three of these types within the framework of a tree-structured group joint sparse representation (TGJSR). In TGJSR, the training data is grouped into a tree of subgroups. A test image under an unknown illuminant has its features reconstructed in terms of a joint sparse representation model derived from the grouped training data. The test image’s illumination is then estimated based on the weights involved in the joint sparse representation model. As a general framework, the proposed TGJSR framework can also easily be extended to incorporate any new features or cues that might be discovered in the future for illumination estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. \(O_1 =\frac{R-G}{\sqrt{2}},O_2 =\frac{R+G-2B}{\sqrt{6}},O_3 =\frac{R+B+G}{\sqrt{3}}\)

  2. http://colorconstancy.com/.

  3. http://www.kyb.mpg.de/bs/people/pgehler/colour/index.html.

  4. http://www.cvc.uab.es/color_calibration/Database.html.

References

  • Barnard, K., Cardei, V., & Funt, B. (2002). A comparison of computational color constancy algorithms-part 1: Methodology and experiments with synthesized data. IEEE Transactions on Image Processing, 11(9), 972–983.

    Article  Google Scholar 

  • Barnard, K., Martin, L., Coath, A., & Funt, B. (2002). Comparison of computational color constancy algorithms-part 2: Experiments with Image Data. IEEE Transactions on Image Processing, 11(9), 985–996.

    Article  Google Scholar 

  • Beigpour, S., Riess, C., Weijer, J. V., & Angelopoulou, E. (2014). Multi-Illuminant estimation with conditional random fields. IEEE Transactions on Image Processing, 23(1), 83–96.

    Article  MathSciNet  Google Scholar 

  • Bengio, S., Pereira, F., Singer, Y., Strelow, D. (2009). Group sparse coding. In: Proceedings of neural information processing systems conference (pp. 1–8).

  • Bianco, S., Gasparini, F., & Schettini, R. (2008). Consensus-based framework for illuminant chromaticity estimation. Journal of Electronic Imaging, 17(2), 023013.

    Article  Google Scholar 

  • Bianco, S., Ciocca, G., Cusano, C., & Schettini, R. (2009). Improving Color Constancy using Indoor-outdoor Image Classification. IEEE Transactions on Image Processing, 17(12), 2381–2392.

    Article  MathSciNet  Google Scholar 

  • Bianco, S., Ciocca, G., Cusano, C., & Schettini, R. (2009). Automatic color constancy algorithm selection and combination. Pattern Recognition, 43(3), 695–705.

    Article  Google Scholar 

  • Brainard, D. H., & Freeman, W. T. (1997). Bayesian color constancy. Journal of the Optical Society of America A, 14(7), 1393–1411.

    Article  Google Scholar 

  • Buchsbaum, G. (1980). A spatial processor model for object colour perception. Journal of the Franklin Institute, 310(1), 1–26.

    Article  MathSciNet  Google Scholar 

  • Cardei, V., & Funt, B. (1999). Committee-based color constancy. In: Proceedings of IS&T/SID color imaging conference (CIC) (pp. 311–313).

  • Cardei, V., Funt, B., & Barnard, K. (2002). Estimating the scene illumination chromaticity using a neural network. Journal of the Optical Society of America A, 19(12), 2374–2386.

    Article  Google Scholar 

  • Chakrabarti, A., Hirakawa, K., & Zickler, T. (2012). Color constancy with spatio-spectral statistics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(8), 1509–1519.

    Article  Google Scholar 

  • Chen, X., Pan, W., Kwok, J., & Garbonell, J. (2009). Accelerated gradient method for multi-task sparse learning problem. In: Proceedings of international conference of data mining (pp. 746–751).

  • Ciurea, F., & Funt, B. (2003). A large image database for color constancy research. In: Proceedings of IS&T/SID color imaging conference (pp. 160–164).

  • Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.

    MATH  Google Scholar 

  • Finlayson, G.G. (2013). Corrected-moment illuminant estimation. In: Proceedings of international conference on computer vision (ICCV) (pp. 1904–1911).

  • Finlayson, G., & Trezzi, E. (2004). Shades of gray and color constancy. In Proceedings of IS&T/SID color imaging conference (pp. 37–41).

  • Finlayson, G., Hordley, S., & Hubel, P. (2001). Color by correlation: A simple, unifying framework for color constancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), 1209–1221.

    Article  Google Scholar 

  • Forsyth, D. A. (1990). A novel algorithm for color constancy. International Journal of Computer Vision, 5(1), 5–36.

    Article  MathSciNet  Google Scholar 

  • Foster, D. H. (2010). Colour constancy. Vision Research, 51, 674–700.

    Article  Google Scholar 

  • Funt, B., & Mosny, M. (2012). Removing outliers in illumination estimation. In: Proceedings of IS&T color imaging conference (pp. 105–110).

  • Funt, B., & Shi, L. (2012). MaxRGB reconsidered. Journal of Imaging Science and Technology, 56(2), 020501-1–020501-10.

    Google Scholar 

  • Gao, S., Chia, L., & Tsang, I. (2011). Multi-layer group sparse coding for concurrent image classification and annotation. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 2809–2816).

  • Gao, S., Yang, K., Li, C., & Li, Y. (2013). A color constancy model with double-opponency mechanisms. In: Proceedings of international conference on computer vision (pp. 929–936).

  • Gehler, P. V., Rother, C., Blake, A., & Minka, T. (2008). Bayesian color constancy revisited. In: Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1–8).

  • Geusebroek, J. M., & Smeulders, A. W. M. (2005). A six stimulus theory for stochastic texture. International Journal of Computer Vision, 62(1–2), 7–16.

    Article  Google Scholar 

  • Gijsenij, A., Gevers, T., & Lucassen, M. (2009). A Perceptual analysis of distance measures for color constancy algorithms. Journal of the Optical Society of America A, 26(10), 2243–2256.

    Article  Google Scholar 

  • Gijsenij, A., Gevers, T., & van de Weijer, J. (2010). Generalized gamut mapping using image derivative structures for color constancy. International Journal of Computer Vision, 86(2–3), 127–139.

    Article  Google Scholar 

  • Gijsenij, A., Gevers, T., & van de Weijer, J. (2011). Computational color constancy: Survey and experiments. IEEE Transactions on Image Processing, 20(9), 2475–2489.

    Article  MathSciNet  Google Scholar 

  • Gijsenij, A., Gevers, T., & van de Weijer, J. (2011). Improving color constancy by photometric edge weighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(5), 918–929.

    Article  Google Scholar 

  • Gijsenij, A., Lu, R., & Gevers, T. (2012). Color constancy for multiple light sources. IEEE Transactions on Image Processing, 21(2), 697–707.

    Article  MathSciNet  Google Scholar 

  • Gijsenij, A., & Gevers, T. (2011). Color constancy using natural image statistics and scene semantics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4), 687–698.

    Article  Google Scholar 

  • Jenatton, R., Audibert, J., & Bach, F. (2011). Structured variable selection with sparsity-inducing norms. Journal of Machine Learning Research, 12(10), 2777–2824.

    MathSciNet  MATH  Google Scholar 

  • Joze, H. R. V., & Drew, M. S. (2014). Exemplar-based colour constancy and multiple illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(5), 860–873.

    Article  Google Scholar 

  • Land, E. H. (1977). The retinex theory of color vision. Scientific America, 237(6), 108–128.

    Article  Google Scholar 

  • Li, B., Xiong, W., Hu, W., & Peng, H. (2013). Illumination estimation based on bilayer sparse coding. In: Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1423–1429).

  • Li, B., Xiong, W., & Xu, D. (2010). A supervised combination strategy for illumination chromaticity estimation. ACM Transactions on Applied Perception, 8(1), 5.

    Article  Google Scholar 

  • Li, B., Xiong, W., Hu, W., & Funt, B. (2014). Evaluating combinational illumination estimation methods on real-world images. IEEE Transactions on Image Processing, 23(3), 1194–1209.

    Article  MathSciNet  Google Scholar 

  • Liu, J., & Ye, J. (2010). Moreau-Yosida regularization for grouped tree structure learning. In: Proceedings of neural information processing systems conference (pp. 1–8).

  • Logvinenko, A. D., Funt, B., & Godau, C. (2014). Metamer mismatching. IEEE Transactions on Image Processing, 23(1), 34–43.

    Article  MathSciNet  Google Scholar 

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant key-points. International Journal of Computer Vision, 60(2), 91–110.

  • Lu, R., Gijsenij, A., Gevers, T., Nedovic, V., & Xu, D. (2009). Color Constancy using 3D Scene Geometry. In: Proceedings of international conference on computer vision (pp. 1749-1756).

  • Nedovic, V., Smeulders, A. W. M., Redert, A., & Geusebroek, J. M. (2010). Stages as models of scene geometry. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), 1673–1687.

    Article  Google Scholar 

  • Nesterov, Y. (2007). Gradient methods for minimizing composite objective function. CORE, Catholic University Louvain, Louvain-la-Neuve, Technical Report.

  • Parraga, C. A., Vazquez-Corral, J., & Vanrell, M. (2009). A new cone activation-based natural images dataset. Perception, 36, 180.

    Google Scholar 

  • Ruderman, D. L., Cronin, T. W., & Chiao, C. C. (1998). Statistics of cone responses to natural images: implications for visual coding. Journal of the Optical Society of America A, 15(8), 2036–2045.

    Article  Google Scholar 

  • Shi, L. (2012). Personal communication.

  • Shi, L., & Funt, B. (2011). Re-processed version of the gehler color constancy dataset of 568 Images. Retrieved from http://www.cs.sfu.ca/~colour/data/

  • Tan, R., Nishino, K., & Ikeuchi, K. (2004). Color constancy through inverse-intensity chromaticity space. Journal of the Optical Society of America A, 21(3), 321–334.

    Article  Google Scholar 

  • Tseng, P. (2008). On accelerated proximal gradient methods for convex-concave optimization. SIAM Journal on Optimization, 6, 1–20.

    Google Scholar 

  • van de Weijer, J., Schmid, C., & Verbeek, J. (2007). Using High-level visual information for color constancy. In: Proceedings of international conference on computer vision (pp. 1–8).

  • van de Weijer, J., Gevers, T., & Gijsenij, A. (2007). Edge based color constancy. IEEE Transactions on Image Processing, 16(9), 2207–2214.

    Article  MathSciNet  Google Scholar 

  • Vazquez-Corral, J., Alejandro Párraga, C., Vanrell, M., & Baldrich, R. (2009). Color constancy algorithms: Psychophysical evaluation on a new dataset. Journal of Imaging Science and Technology, 53(3), 031105–031109.

    Article  Google Scholar 

  • Vazquez-Corral, J., Vanrell, M., Baldrich, R., & Tous, F. (2012). Color constancy by category correlation. IEEE Transactions on Image Processing, 21(4), 1997–2007.

    Article  MathSciNet  Google Scholar 

  • Wright, J., Ma, Y., Mairal, J., Sapiro, G. ( 2010). Sparse representation for computer vision and pattern recognition. In: Proceedings of the IEEE.

  • Xiong, W., & Funt, B. (2006). Estimating illumination chromaticity via support vector regression. Journal of Imaging Science and Technology, 50(4), 341–348.

    Article  Google Scholar 

  • Yang, A., Zhou, Z., Balasubramanian, A., Sastry, S., & Ma, Y. (2013). Fast L1-minimization algorithms for robust face recognition. IEEE Transactions on Image Processing, 22(8), 3234–3246.

    Article  Google Scholar 

  • Yosida, K. (1964). Functional analysis. Berlin: Springer.

    Google Scholar 

  • Yuan, X., & Yan, S. (2010). Visual classification with multi-task joint sparse representation. In: Proceedings of IEEE conference on computer vision and pattern recognition (pp. 3493–3500).

Download references

Acknowledgments

This work was supported by the National Nature Science Foundation of China (Nos. 61370038, and 61472421), the 973 Basic Research Program of China (No. 2014CB349303), and Chinese National Programs for High Technology Research and Development (863 Program) (Nos. 2012AA012503 and 2012AA012504), as well as the Natural Sciences and Engineering Research Council of Canada.

Author information

Authors and Affiliations

  1. National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Bing Li, Weihua Xiong, Weiming Hu & Junliang Xing

  2. School of Computing Science, Simon Fraser University, Burnaby, V5A 1S6, Canada

    Brian Funt

Authors
  1. Bing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weihua Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weiming Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Brian Funt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Junliang Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weiming Hu.

Additional information

Communicated by Ko Nishino.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, B., Xiong, W., Hu, W. et al. Multi-Cue Illumination Estimation via a Tree-Structured Group Joint Sparse Representation. Int J Comput Vis 117, 21–47 (2016). https://doi.org/10.1007/s11263-015-0844-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-015-0844-7

Keywords

Search

Navigation