Skip to main content
Springer Nature Link
Log in

SCD: Statistical Color Distribution-Based Objective Image Colorization Quality Assessment

  • Conference paper
  • First Online:
Advances in Computer Graphics (CGI 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15338))

Included in the following conference series:

  • 81 Accesses

Abstract

Colorization research has long been a focal point in computer vision and image processing. However, due to its inherently ill-posed nature, a reasonable assessment of the quality of their outcomes remains a challenge. Subjective evaluations are often restricted to a limited number of participants due to the high costs. This along with the existence of individual differences and subjective biases makes it difficult to derive convincing conclusions. Despite no need for participants in objective evaluation metrics, the currently widely applied objective metrics fail to accurately reflect the quality of colorization results, thereby impeding the attainment of consistency with subjective user opinions. Facing the above problems, we propose a novel Statistical Color Distribution-based Objective Evaluation Metric (SCD) for better consistency with human opinions. We first segment images into semantic regions. For each semantic type, a novel two-dimensional natural color distribution w.r.t. hue and saturation is collected to better align with human perceptual observations during image assessment. An adjacency weighted matrix considering surrounding neighboring regions smooths the color distribution table, enabling a more reliable quality assessment. The application of probability density eliminates the issue of frequency anomalies caused by human visual insensitivity, ensuring more accurate evaluation.Through extensive and comprehensive experiments involving two distinct datasets with the participation of 1321 volunteers, this paper demonstrates that the proposed SCD is more consistent with subjective user opinions compared with current objective metrics for evaluating colorization.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cao, Y., Meng, X., Mok, P., Liu, X., Lee, T.Y., Li, P.: AnimeDiffusion: anime face line drawing colorization via diffusion models. arXiv preprint arXiv:2303.11137 (2023)

  2. Chen, S.Y., et al.: Active colorization for cartoon line drawings. IEEE Trans. Visual Comput. Graphics 28(2), 1198–1208 (2020)

    Article  MATH  Google Scholar 

  3. Dou, Z., Wang, N., Li, B., Wang, Z., Li, H., Liu, B.: Dual color space guided sketch colorization. IEEE Trans. Image Process. 30, 7292–7304 (2021)

    Article  MATH  Google Scholar 

  4. Gurney, J.: Color and Light: A Guide for the Realist Painter, vol. 2. Andrews McMeel Publishing (2010)

    Google Scholar 

  5. Hasler, D., Suesstrunk, S.E.: Measuring colorfulness in natural images. In: Human vision and electronic imaging VIII, vol. 5007, pp. 87–95. SPIE (2003)

    Google Scholar 

  6. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs Trained By a Two Time-scale Update Rule Converge to a Local Nash Equilibrium. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  7. Hristova, H., Le Meur, O., Cozot, R., Bouatouch, K.: Perceptual metric for color transfer methods. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 1237–1241. IEEE (2017)

    Google Scholar 

  8. Huang, Z., Zhao, N., Liao, J.: UniColor: a unified framework for multi-modal colorization with transformer. ACM Trans. Graph. (TOG) 41(6), 1–16 (2022)

    Article  MATH  Google Scholar 

  9. Iizuka, S., Simo-Serra, E., Ishikawa, H.: Let there be color! Joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Trans. Graph. (ToG) 35(4), 1–11 (2016)

    Article  MATH  Google Scholar 

  10. Jayasumana, S., Ramalingam, S., Veit, A., Glasner, D., Chakrabarti, A., Kumar, S.: Rethinking FID: towards a better evaluation metric for image generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9307–9315 (2024)

    Google Scholar 

  11. Kim, E., Lee, S., Park, J., Choi, S., Seo, C., Choo, J.: Deep edge-aware interactive colorization against color-bleeding effects. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14667–14676 (2021)

    Google Scholar 

  12. Kim, E., Suk, H.J.: Hue extraction and tone match: Generating a theme color to enhance the emotional quality of an image. In: ACM SIGGRAPH 2015 Posters, pp. 1–1 (2015)

    Google Scholar 

  13. Larsson, G., Maire, M., Shakhnarovich, G.: Learning Representations for Automatic Colorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 577–593. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_35

  14. Li, Z., Geng, Z., Kang, Z., Chen, W., Yang, Y.: Eliminating Gradient Conflict in Reference-based Line-art Colorization. In: European Conference on Computer Vision, pp. 579–596. Springer (2022). https://doi.org/10.1007/978-3-031-19790-1_35

  15. Lynch, D.K., Livingston, W.C.: Color and Light in Nature. Cambridge University P\(^{.}\)ress (2001)

    Google Scholar 

  16. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 2, pp. 416–423. IEEE (2001)

    Google Scholar 

  17. Min, L., Li, Z., Jin, Z., Cui, Q.: Color edge preserving image colorization with a coupled natural vectorial total variation. Comput. Vis. Image Underst. 196, 102981 (2020)

    Article  MATH  Google Scholar 

  18. Mullery, S., Whelan, P.F.: Human vs objective evaluation of colourisation performance. arXiv preprint arXiv:2204.05200 (2022)

  19. Rodriguez-Pardo, C., Casas, D., Garces, E., Lopez-Moreno, J.: TexTile: a differentiable metric for texture tileability. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4439–4449 (2024)

    Google Scholar 

  20. Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  21. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  22. Smith, A.R.: Color gamut transform pairs. ACM Siggraph Comput. Graph. 12(3), 12–19 (1978)

    Article  MATH  Google Scholar 

  23. Strudel, R., Garcia, R., Laptev, I., Schmid, C.: Segmenter: transformer for Semantic Segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7262–7272 (2021)

    Google Scholar 

  24. Szegedy, C., et al.: Going Deeper with Convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

    Google Scholar 

  25. Teng, X., Li, Z., Liu, Q., Pointer, M.R., Huang, Z., Sun, H.: Subjective evaluation of colourized images with different colorization models. Color Res. Appl. 46(2), 319–331 (2021)

    Article  Google Scholar 

  26. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  MATH  Google Scholar 

  27. Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, vol. 2, pp. 1398–1402. IEEE (2003)

    Google Scholar 

  28. Weng, S., Zhang, P., Li, Y., Li, S., Shi, B., et al.: L-CAD: language-based colorization with any-level descriptions using diffusion priors. In: Advances in Neural Information Processing Systems, vol. 36 (2024)

    Google Scholar 

  29. Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 649–666. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_40

  30. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The Unreasonable Effectiveness of Deep Features As a Perceptual Metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)

    Google Scholar 

  31. Zhang, R., et al.: Real-time user-guided image colorization with learned deep priors. arXiv preprint arXiv:1705.02999 (2017)

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Informatics, Cardiff University, Wales, UK

    Hongjin Lyu, Hareeharan Elangovan, Paul L. Rosin & Yu-Kun Lai

Authors
  1. Hongjin Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hareeharan Elangovan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul L. Rosin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu-Kun Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Hongjin Lyu , Paul L. Rosin or Yu-Kun Lai .

Editor information

Editors and Affiliations

  1. University of Geneva, Geneva, Switzerland

    Nadia Magnenat-Thalmann

  2. The University of Sydney, Sydney, NSW, Australia

    Jinman Kim

  3. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

  4. University of Houston, Houston, TX, USA

    Zhigang Deng

  5. EPFL, Lausanne, Switzerland

    Daniel Thalmann

  6. The Hong Kong Polytechnic University, Kowloon, Hong Kong

    Ping Li

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 450 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lyu, H., Elangovan, H., Rosin, P., Lai, YK. (2025). SCD: Statistical Color Distribution-Based Objective Image Colorization Quality Assessment. In: Magnenat-Thalmann, N., Kim, J., Sheng, B., Deng, Z., Thalmann, D., Li, P. (eds) Advances in Computer Graphics. CGI 2024. Lecture Notes in Computer Science, vol 15338. Springer, Cham. https://doi.org/10.1007/978-3-031-81806-6_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-81806-6_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-81805-9

  • Online ISBN: 978-3-031-81806-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics