Skip to main content
SpringerLink
Log in

Color constancy for non-uniform illumination estimation with variable number of illuminants

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

A Correction to this article was published on 09 May 2023

This article has been updated

Abstract

Image white-balancing is an integral part of every camera’s processing pipeline. White-balancing is used to remove illumination chromaticity from an image. Most research in this field has been limited to images with a single uniform illuminant. In this paper, we introduce a novel method for illumination estimation for situations where the scene is illuminated by a variable number of different illuminants and where the illumination in the scene can be non-uniform. The proposed method uses a lightweight convolutional neural network that achieves state-of-the-art results. The method performs illumination estimation on a patch-by-patch basis. We use the assumption that only one illuminant affects each patch since they are so small. Unlike other such methods, our method uses features extracted from the entire image to perform patch illumination estimation. The paper also shows how the image features improve method accuracy with a minimal increase in complexity. The proposed method has around 42 k parameters, and it was tested on three different cameras from the Large-Scale Multi-Illuminant dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

The dataset analyzed during the current study is available upon request, https://github.com/DY112/LSMI-dataset [7].

Change history

References

  1. Land EH (1977) The retinex theory of color vision. Sci Am 237(6):108–129. https://doi.org/10.1038/scientificamerican1277-108

    Article  Google Scholar 

  2. Buchsbaum G (1980) A spatial processor model for object colour perception. J Frankl Inst 310(1):1–26. https://doi.org/10.1016/0016-0032(80)90058-7

    Article  Google Scholar 

  3. Hu Y, Wang B, Lin S (2017) Fc4: fully convolutional color constancy with confidence-weighted pooling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4085–4094 . https://doi.org/10.1109/CVPR.2017.43

  4. Wang K, Chen Z, Wu QMJ, Liu C (2019) Face recognition using AMVP and WSRC under variable illumination and pose. Neural Comput Appl 31(8):3805–3818. https://doi.org/10.1007/s00521-017-3316-x

    Article  Google Scholar 

  5. Bianco S, Cusano C, Schettini R (2017) Single and multiple illuminant estimation using convolutional neural networks. IEEE Trans Image Process 26(9):4347–4362. https://doi.org/10.1109/TIP.2017.2713044

    Article  MathSciNet  MATH  Google Scholar 

  6. Shi W, Loy CC, Tang X (2016) Deep specialized network for illuminant estimation. In: European conference on computer vision. Springer, pp 371–387 . https://doi.org/10.1007/978-3-319-46493-0_23

  7. Kim D, Kim J, Nam S, Lee D, Lee Y, Kang N, Lee H-E, Yoo B, Han J-J, Kim SJ (2021) Large scale multi-illuminant (lsmi) dataset for developing white balance algorithm under mixed illumination. In: 2021 IEEE/CVF international conference on computer vision (ICCV), pp 2390–2399. https://doi.org/10.1109/ICCV48922.2021.00241

  8. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical image computing and computer-assisted intervention - MICCAI 2015. Springer, Cham, pp 234–241. https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  9. Xiao J, Gu S, Zhang L (2020) Multi-domain learning for accurate and few-shot color constancy. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3258–3267. https://doi.org/10.1109/CVPR42600.2020.00332

  10. Rizzi A, Bonanomi C (2017) Milano Retinex family. J Electron Imaging 26(3):1–7. https://doi.org/10.1117/1.JEI.26.3.031207

    Article  Google Scholar 

  11. Banić N, Lončarić S (2013) Light random sprays retinex: exploiting the noisy illumination estimation. IEEE Signal Process Lett 20(12):1240–1243. https://doi.org/10.1109/LSP.2013.2285960

    Article  Google Scholar 

  12. Gijsenij A, Gevers T, Van De Weijer J (2011) Computational color constancy: survey and experiments. IEEE Trans Image Process 20(9):2475–2489. https://doi.org/10.1109/TIP.2011.2118224

    Article  MathSciNet  MATH  Google Scholar 

  13. von Kries J (1905) Influence of adaptation on the effects produced by luminous stimuli. handbuch der Physiologie des Menschen 3:109–282. https://doi.org/10.1016/0016-0032(80)90058-7

  14. Finlayson GD, Trezzi E (2004) Shades of gray and colour constancy. In: Color and Imaging Conference. Society for Imaging Science and Technology, vol 2004, pp 37–41. https://ueaeprints.uea.ac.uk/id/eprint/23682

  15. Van De Weijer J, Gevers T, Gijsenij A (2007) Edge-based color constancy. IEEE Trans Image Process 16(9):2207–2214. https://doi.org/10.1109/TIP.2007.901808

    Article  MathSciNet  Google Scholar 

  16. Gijsenij A, Gevers T, van de Weijer J (2012) Improving color constancy by photometric edge weighting. IEEE Trans Pattern Anal Mach Intell 34(5):918–929. https://doi.org/10.1109/TPAMI.2011.197

    Article  Google Scholar 

  17. Domislović I, Vršnak D, Subašić M, Lončarić S (2022) One-net: convolutional color constancy simplified. Pattern Recognit Lett 159:31–37. https://doi.org/10.1016/j.patrec.2022.04.035

    Article  Google Scholar 

  18. Van Rossum G, Drake FL (2009) Python 3 reference manual. CreateSpace, Scotts Valley, CA

    Google Scholar 

  19. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. Software available from tensorflow.org. https://www.tensorflow.org/

  20. Li Z, Ma Z (2021) Robust white balance estimation using joint attention and angular loss optimization. In: Thirteenth international conference on machine vision. International Society for Optics and Photonics, vol 11605, p 116051. https://doi.org/10.1117/12.2586930

  21. Loshchilov I, Hutter F (2017) Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. https://doi.org/10.48550/ARXIV.1711.05101

  22. Smith LN (2017) Cyclical learning rates for training neural networks. In: 2017 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 464–472. https://doi.org/10.1109/WACV.2017.58

  23. Gijsenij A, Lu R, Gevers T (2011) Color constancy for multiple light sources. IEEE Trans Image Process 21(2):697–707. https://doi.org/10.1109/TIP.2011.2165219

    Article  MathSciNet  MATH  Google Scholar 

  24. Beigpour S, Riess C, Van De Weijer J, Angelopoulou E (2013) Multi-illuminant estimation with conditional random fields. IEEE Trans Image Process 23(1):83–96. https://doi.org/10.1109/CVPR42600.2020.00332

    Article  MathSciNet  MATH  Google Scholar 

  25. Hordley SD (2006) Scene illuminant estimation: past, present, and future. Color Res Appl: Endorsed by Inter-Society Color Council, The Colour Group (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation, Colour Society of Australia, Centre Français de la Couleur 31(4):303–314 . https://doi.org/10.1002/col.20226

  26. Li J, Fang P (2019) Hdrnet: single-image-based hdr reconstruction using channel attention cnn. In: Proceedings of the 2019 4th international conference on multimedia systems and signal processing. ICMSSP 2019. Association for Computing Machinery, New York, NY, USA, pp 119–124. https://doi.org/10.1145/3330393.3330426

  27. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 5967–5976. https://doi.org/10.1109/CVPR.2017.632

  28. Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE international conference on computer vision (ICCV), pp 2242–2251. https://doi.org/10.1109/ICCV.2017.244

  29. Wang T-C, Liu M-Y, Zhu J-Y, Tao A, Kautz J, Catanzaro B (2018) High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  30. Park T, Efros AA, Zhang R, Zhu J-Y (2020) Contrastive learning for conditional image synthesis. In: ECCV

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, 10000, Zagreb, Croatia

    Ilija Domislović, Donik Vršnjak, Marko Subašić & Sven Lončarić

Authors
  1. Ilija Domislović
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Donik Vršnjak
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marko Subašić
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sven Lončarić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ilija Domislović.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Domislović, I., Vršnjak, D., Subašić, M. et al. Color constancy for non-uniform illumination estimation with variable number of illuminants. Neural Comput & Applic 35, 14825–14835 (2023). https://doi.org/10.1007/s00521-023-08487-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08487-z

Keywords

Search

Navigation