Skip to main content

Advertisement

SpringerLink
Log in

Colorization using edge-preserving smoothing filter

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Colorization is the process of replacing a scalar value stored at each pixel of a grayscale image or film by a vector in a multidimensional color space. Mapping between scalar and color is therefore not unique, and colorization is ambiguous in nature and requires some amount of human interaction or external information. This paper presents a new method of interactive colorization by introducing a new concept to the bilateral filter. We adopt the bilateral filter as a range weights filter (bilateral filter without Gaussian spatial domain), and we show that the Nikolaou filter is equivalent to the range weights bilateral filter, but has better behavior near the edges. In our algorithm, the user selects grayscale image regions by directly painting these regions; the user does not need to paint over the whole object. Each selection can be automatically expanded from the user’s paint brush and aligned with the object boundary. Robustness and quality of the results obtained over a collection of several challenging images demonstrate the efficiency of this new method for some difficult cases, such as human faces or images with confusing lighting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The images are adopted from Pouli et al. [15] and reprinted with permission from Elsevier

  2. The image is publically available at:http://www.f11digital.com/

Abbreviations

\(I\) :

Digital image

\(s\) :

The spatial locations of pixels at s

\(p\) :

The spatial locations of pixels at p

\(I_{s}\) :

Image at position s

\(I_{p}\) :

Image at position p

\(W_{s}\) :

Normalization at position s

\(\varOmega \) :

Set of image pixels

\(\sum _{p\in \varOmega }\) :

Sum over all pixels p

\(f\) :

The spatial domain

\(g\) :

The range domain

\(\sigma _{s}\) :

Standard deviation of the spatial similarity

\(\sigma _{r}\) :

Standard deviation of the range (intensity/color) similarity

\(|.|\) :

Absolute value

\(d_{i}\) :

Intensity differences

\(k\) :

Parameter (controls the amount of colors that will diffuse across edges)

\(t\) :

Discrete time (iterations)

\(Y\) :

Luminance of the pixels in a grayscale image

\(Y_{p}\) :

Luminance of the pixels at position p

\(Y_{s}\) :

Luminance of the pixels at position s

\(U,V\) :

Chrominance (color information)

\(U_{p},V_{p}\) :

Color hints

References

  1. Yatziv, L., Sapiro, G.: Fast image and video colorization using chrominance blending. In: IEEE Trans. Image Process. 15(5), 1120–1129 (2006)

  2. Vieira, L.F.M., do Nascimento, E.R., Fernandes Jr., F.A., Carceroni, R.L., Vilela, R.D., Araújo, A. de A.: Fully automatic coloring of grayscale images. Image Vis Comput. 25(1), 50–60 (2007)

  3. Quang, M.H., Kang, S.H., Le, T.M.: Image and video colorization using vector-valued reproducing Kernel Hilbert spaces. J Math. Imaging Vis. 37(1), 49–65 (2010)

    Article  Google Scholar 

  4. Fornasier, M., Ramlau, R., Teschke, G.: The application of joint sparsity and total variation minimization algorithms in a real-life art restoration problem. Adv. Comput. Math. 31(1–3), 157–184 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  5. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings 6th International Conference Computer Vision. Bombay, India, Jan 4–7, pp 839–846 (1998)

  6. Nikolaou, N., Papamarkos, N.: Color reduction for complex document images. Int. J. Imaging Syst. Technol. 19(1), 14–26 (2009)

    Article  Google Scholar 

  7. Markle, W., Hunt, B.: Coloring a Black and White Signal Using Motion Detection. US Colorization Inc. Canadian patent no.1291260, Dec. (1987)

  8. Welsh, T., Ashikhmin, M., Mueller, K.: Transferring color to grayscale images. ACM Trans. Graphics 21(3), 277–280 (2002)

    Article  Google Scholar 

  9. Eisemann, E., Durand, F.: Flash photography enhancement via intrinsic relighting. ACM Trans. Graphics 23(3), 673–678 (2004)

    Article  Google Scholar 

  10. Wang, J., Cohen, M.F.: Image and video matting: a survey. Trends Comput. Graph. Vis. 3(2), 97–175 (2007)

    Article  Google Scholar 

  11. Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. In: IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008)

  12. He, K., Sun, J., Tang, X.: Fast matting using large kernel matting laplacian matrices. In: Proceedings 23rd IEEE Conference Computer Vision and Pattern Recognition. San Francisco, USA, Jun. 13–18, pp. 2165–2172 (2010)

  13. Charpiat, G., Hofmann, M., Scholkopf, B.: Automatic image colorization via multimodal predictions. In: Proceedings 10th European Conference Computer Vision. Marseille, France, Oct. 12–18, pp. 126–139 (2008)

  14. Kekre, H.B., Thepade, S.D., Bhandari, N.: Colorization of grayscale images using Kekre’s biorthogonal color spaces and Kekre’s fast codebook generation. Adv. Multimed. Int. J. 1(3), 49–58 (2011)

    Google Scholar 

  15. Pouli, T., Reinhard, E.: Progressive color transfer for images of arbitrary dynamic range. Comput. Graphics 35(1), 67–80 (2011)

    Article  Google Scholar 

  16. Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. ACM Trans. Graphics 23(3), 689–694 (2004)

    Article  Google Scholar 

  17. Drew, M.S., Finlayson, G.D.: Improvement of colorization realism via the structure tensor. Int. J. Image Graphics 11(4), 589–609 (2011)

    Article  MathSciNet  Google Scholar 

  18. Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graphics 26(3), pp 103:1–10 (2007)

    Google Scholar 

  19. Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. Int. J. Comput. Vis. 81(1), 24–52 (2009)

    Article  Google Scholar 

  20. Heinrich, S.B., Snyder, W.E.: Improved Edge Awareness in Discontinuity Preserving Smoothing. arXiv, Vol. arXiv:1103.5808, (2011) Available at: http://arxiv.org/abs/1103.5808

  21. Jack, K.: Video Demystified: A Handbook for the Digital Engineer(3rd) Ch.3, pp. 15–17. LLH Technology Publishing, Eagle Rock (2001)

  22. Criminisi, A., Sharp, T., Rother, C., Pérez, P.: Geodesic image and video editing. ACM Trans. Graphics 29(5), 1–15 (2010)

    Google Scholar 

  23. Park, K., Singhal, N., Lee, M.H., Cho, S., Kim, C.W.: Design and performance evaluation of image processing algorithms on GPUs. In: IEEE Trans. Parallel Distrib. Syst. 22(1), 91–104 (2011)

  24. Martin, D.R., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings 8th International Conference Computer Vision vol. 2. Vancouver, British Columbia, Canada, July 7–14, pp. 416–425 (2001)

Download references

Acknowledgments

The authors would like to thank the anonymous reviewers for their comments. The work is partially supported by the National Basic Research Program of China (973 Program) (No.2012CB316201), the National Natural Science Foundation of China (Nos. 60973018, 60973020), the Doctoral Fund of Ministry of Education of China (No. 20110042110028), and the Fundamental Research Funds for the Central Universities (No. N110804002).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Northeastern University, #3-11 Wenhua Rd., Heping District, Shenyang, 110819, Liaoning Province, People’s Republic of China

    Ahmed A. Hussein & Xiaochun Yang

Authors
  1. Ahmed A. Hussein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaochun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed A. Hussein.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hussein, A.A., Yang, X. Colorization using edge-preserving smoothing filter. SIViP 8, 1681–1689 (2014). https://doi.org/10.1007/s11760-012-0402-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-012-0402-5

Keywords

Search

Navigation