Skip to main content
SpringerLink
Log in

Removal of spatial inconsistencies in automated image colorization using parameter-free clustering and convolutional neural networks

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Given a grayscale photograph, this paper tackles the issue of fantasizing about a conceivable shading rendition of the photograph. This issue is underconstrained, so past methodologies have either depended on significant human cooperation or came about in desaturated colorizations. We propose a wholly automated approach that produces lively furthermore, sensible colorizations. This approach utilizes the combination of CNN-based colorization and parameter-free k-means clustering to identify the color spills, so that the image can be recolored to produce color images that are aesthetically more pleasing and plausible than the images produced by the state-of-the-art methods. The performance of the proposed model is evaluated in terms of mean square error and structural similarity and found to be superior to the related works.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Vieira, L.F.M., Vilela, R.D., Nascimento, E.R.D., Fernandes, F.A., Carceroni, R.L. and Araújo, A.D.A.: Automatically choosing source color images for coloring grayscale images. In: 16th Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2003), pp. 151–158. IEEE (2003)

  2. Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization. In: European Conference on Computer Vision, Amsterdam, Netherlands, pp. 649–666 (2016)

  3. Zhang, R., Zhu, Y., Isola, P., Geng, X., Lin, A. S., Yu, T, Efros, A.A.: Real-Time User-Guided Image Colorization with Learned Deep Priors (2017). https://arxiv.org/abs/1705.02999v1

  4. Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. ACM SIGGRAPH 2004 Papers. Los Angeles, USA, vol. 14(1), pp. 689–694 (2004)

  5. Huang, Y.C., Tung, Y.S., Chen, J.C. et al.: An adaptive edge detection based colorization algorithm and its applications. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, Singapore, vol. 14(1), pp. 351–354 (2005)

  6. Yatziv, L., Sapiro, G.: Fast image and video colorization using chrominance blending. IEEE Trans. Img. Proc. 15(5), 1120–1129 (2006)

    Article  Google Scholar 

  7. Qu, Y., Wong, T.-T., Heng, P.-A.: Natural image colorization. ACM SIGGRAPH 2006 Papers, SIGGRAPH 06, Boston, USA, vol. 3, pp. 1214–1220 (2006)

  8. Luan, Q., Wen, F., Cohen-Or, D., et al.: Natural image colorization. In: Proceedings of the 18th Eurographics Conf. on Rendering Techniques, EGSR 07, Grenoble, France, vol. 42(3), pp. 309–320 (2007)

  9. Charpiat, G., Hofmann, M., Scholkopf, B.: Automatic image colorization via multimodal predictions. In: European Conference on Computer Vision ECCV, pp. 126–139 (2008)

  10. Gupta, R.K., Chia, A.Y.-S., Rajan, D., et al.: Image colorization using similar images. ACM Int. Conf. Multimedia 1, 369–378 (2012)

    Google Scholar 

  11. Liu, X., Wan, L., Qu, Y., et al.: Intrinsic colorization. Trans. Graph. 27, 152 (2008)

    Article  Google Scholar 

  12. Chia, A.Y.-S., Zhuo, S., Gupta, R.K., et al.: Semantic colorization with internet images. ACM Trans. Graph. 30(156), 1–8 (2011)

    Article  Google Scholar 

  13. Cheng, Z., Yang, Q., Sheng, B.: Deep colorization. In: International Conference on Computer Vision (ICCV), Las Condes, Chile, vol. 1 (2015)

  14. Deshpande, A., Rock, J., Forsyth, D.: Learning large-scale automatic image colorization. In: International Conference on Computer Vision (ICCV), Las Condes, Chile, vol. 1 (2015)

  15. Larsson, G., Gustav, M.M., Shakhnarovich, G.: Learning representations for automatic colorization. In: European Conference on Computer Vision, Amsterdam, Netherlands, pp. 577–593 (2016)

  16. Iizuka, S., Edgar, S.S., Ishikawa, H.: Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Trans. Graph. (TOG) 35(4), 110 (2016)

    Article  Google Scholar 

  17. Liu, H., Fu, Z., Han, J., Shao, L., Liu, H.: Single satellite imagery simultaneous super-resolution and colorization using multi-task deep neural networks. J. Vis. Commun. Image Represent. 53, 20–30 (2018)

    Article  Google Scholar 

  18. Hussein, A.A., Yang, X.: Colorization using edge-preserving smoothing filter. SIViP 8(8), 1681–1689 (2014)

    Article  Google Scholar 

  19. Ju, H.J., Lee, D.K., Park, R.H.: Color fringe removal in narrow color regions of digital images. SIViP 8(8), 1651–1662 (2014)

    Article  Google Scholar 

  20. Dong, X., Liu, C., Li, W., Hu, X., Wang, X., Wanga, Y.: Self-supervised colorization towards monochrome-color camera systems using cycle CNN. IEEE Trans. Image Process. 30, 6609–6622 (2021)

    Article  Google Scholar 

  21. Kong, G., Tian, H., Duan, X., Long, H.: Adversarial edge-aware image colorization with semantic segmentation. IEEE Access 9, 28194–28203 (2021)

    Article  Google Scholar 

  22. Nguyen-Quynh, T.T., Kim, S.H., Do, N.T.: Image colorization using the global scene-context style and pixel-wise semantic segmentation. IEEE Access 8, 214098–214114 (2020)

    Article  Google Scholar 

  23. Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics (2007).

  24. Davies, D. L., Bouldin, D. W.: A cluster separation measure. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-1(2): 224–227 (1979)

  25. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv preprint https://arxiv.org/abs/1409.1556

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Information Technology Allahabad, Prayagraj, 211015, India

    Navjot Singh

  2. Motilal Nehru National Institute of Technology Allahabad, Prayagraj, 211004, India

    Garvit Gupta, Anubhav Singh, Anshul Kishore, Kumud Kumar & Deepak Bharti

Authors
  1. Navjot Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Garvit Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anubhav Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anshul Kishore
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kumud Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Deepak Bharti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Navjot Singh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, N., Gupta, G., Singh, A. et al. Removal of spatial inconsistencies in automated image colorization using parameter-free clustering and convolutional neural networks. SIViP 16, 1011–1018 (2022). https://doi.org/10.1007/s11760-021-02047-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-02047-5

Keywords

Search

Navigation