Skip to main content
SpringerLink
Log in

Image Colorization Based on a Generalization of the Low Dimensional Manifold Model

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

In this paper, we introduce a novel model that restores a color image from a grayscale image with color values given in small regions. The model is based on the idea of the generalization of the low dimensional manifold model (Shi et al. in J Sci Comput, 2017. https://doi.org/10.1007/s10915-017-0549-x) and the YCbCr color space. It involves two prior terms, a weighted nonlocal Laplacian (WNLL) and a weighted total variation (WTV). The WNLL allows regions without color information to be interpolated smoothly from given sparse color data, while the WTV assists to inhibit the diffusion of color values across edges. To cope with various types of sampled data, we introduce an updating rule for the weight function in the WNLL. Furthermore, we present an efficient iterative algorithm for solving the proposed model. Lastly, numerical experiments validate the superior performance of the proposed model over that of the other state-of-the-art models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Barrett, R., Berry, M., Chan, T.F., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., Van der Vorst, H.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. SIAM, Philadelphia (1994)

    Book  Google Scholar 

  2. Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)

    Article  Google Scholar 

  3. Bhat, P., Zitnick, C.L., Cohen, M., Curless, B.: Gradientshop: a gradient-domain optimization framework for image and video filtering. ACM Trans. Graph. (TOG) 29(2), 10 (2010)

    Article  Google Scholar 

  4. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends\({\textregistered }\) Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  5. Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 60–65 (2005)

  6. Cai, J.F., Osher, S., Shen, Z.: Split Bregman methods and frame based image restoration. Multisc. Model. Simul. 8(2), 337–369 (2009)

    Article  MathSciNet  Google Scholar 

  7. Chatterjee, P., Milanfar, P.: Patch-based near-optimal image denoising. IEEE Trans. Image Process. 21(4), 1635–1649 (2012)

    Article  MathSciNet  Google Scholar 

  8. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    Article  MathSciNet  Google Scholar 

  9. Deng, W., Yin, W.: On the global and linear convergence of the generalized alternating direction method of multipliers. J. Sci. Comput. 66(3), 889–916 (2016)

    Article  MathSciNet  Google Scholar 

  10. Fornasier, M.: Nonlinear projection recovery in digital inpainting for color image restoration. J. Math. Imaging Vis. 24(3), 359–373 (2006)

    Article  MathSciNet  Google Scholar 

  11. Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multisc. Model. Simul. 7(3), 1005–1028 (2008)

    Article  MathSciNet  Google Scholar 

  12. Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  Google Scholar 

  13. Han, D., Yuan, X.: A note on the alternating direction method of multipliers. J. Optim. Theory Appl. 155(1), 227–238 (2012)

    Article  MathSciNet  Google Scholar 

  14. Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur Stand. 49(6), 409–436 (1952)

    Article  MathSciNet  Google Scholar 

  15. Jin, Z., Zhou, C., Ng, M.K.: A coupled total variation model with curvature driven for image colorization. Inverse Probl. Imaging 10(4), 1037–1055 (2016)

    Article  MathSciNet  Google Scholar 

  16. Jung, M., Kang, M.: Variational image colorization models using higher-order Mumford–Shah regularizers. J. Sci. Comput. 68(2), 864–888 (2016)

    Article  MathSciNet  Google Scholar 

  17. Kang, S.H., March, R.: Variational models for image colorization via chromaticity and brightness decomposition. IEEE Trans. Image Process. 16(9), 2251–2261 (2007)

    Article  MathSciNet  Google Scholar 

  18. Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. ACM Trans. Graph. (TOG) 23, 689–694 (2004)

    Article  Google Scholar 

  19. Li, F., Bao, Z., Liu, R., Zhang, G.: Fast image inpainting and colorization by Chambolles dual method. J. Vis. Commun. Image Represent. 22(6), 529–542 (2011)

    Article  Google Scholar 

  20. Li, Z., Shi, Z.: A convergent point integral method for isotropic elliptic equations on a point cloud. Multisc. Model. Simul. 14(2), 874–905 (2016)

    Article  MathSciNet  Google Scholar 

  21. Li, Z., Shi, Z., Sun, J.: Point integral method for solving Poisson-type equations on manifolds from point clouds with convergence guarantees. Commun. Comput. Phys. 22(1), 228–258 (2017)

    Article  MathSciNet  Google Scholar 

  22. Matviychuk, Y., Hughes, S.M.: Regularizing inverse problems in image processing with a manifold-based model of overlapping patches. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5352–5356 (2014)

  23. Ni, J., Turaga, P., Patel, V.M., Chellappa, R.: Example-driven manifold priors for image deconvolution. IEEE Trans. Image Process. 20(11), 3086–3096 (2011)

    Article  MathSciNet  Google Scholar 

  24. Osher, S., Shi, Z., Zhu, W.: Low dimensional manifold model for image processing. SIAM J. Imaging Sci. 10(4), 1669–1690 (2017)

    Article  MathSciNet  Google Scholar 

  25. Peyré, G.: Image processing with nonlocal spectral bases. Multisc. Model. Simul. 7(2), 703–730 (2008)

    Article  MathSciNet  Google Scholar 

  26. Peyré, G.: Manifold models for signals and images. Comput. Vis. Image Underst. 113(2), 249–260 (2009)

    Article  Google Scholar 

  27. Peyré, G.: Sparse modeling of textures. J. Math. Imaging Vis. 34(1), 17–31 (2009)

    Article  MathSciNet  Google Scholar 

  28. Quang, M.H., Kang, S.H., Le, T.M.: Image and video colorization using vector-valued reproducing kernel Hilbert spaces. J. Math. Imaging Vis. 37(1), 49–65 (2010)

    Article  MathSciNet  Google Scholar 

  29. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    Article  MathSciNet  Google Scholar 

  30. Sapiro, G.: Inpainting the colors. In: IEEE International Conference on Image Processing (ICIP), vol. 2, pp. II-698–701 (2005)

  31. Shi, Z., Osher, S., Zhu, W.: Generalization of the weighted nonlocal Laplacian in low dimensional manifold model. J. Sci. Comput. 75(2), 638–656 (2018)

    Article  MathSciNet  Google Scholar 

  32. Shi, Z., Osher, S., Zhu, W.: Weighted nonlocal Laplacian on interpolation from sparse data. J. Sci. Comput. 73(2–3), 1164–1177 (2017)

    Article  MathSciNet  Google Scholar 

  33. Shi, Z., Sun, J.: Convergence of the point integral method for Poisson equation on point cloud. arXiv preprint arXiv:1403.2141 (2014)

  34. Yang, Y., Li, C., Kao, C.Y., Osher, S.: Split Bregman method for minimization of region-scalable fitting energy for image segmentation. In: International Symposium on Visual Computing (ISVC 2010): Advances in Visual Computing, vol. 6454, pp. 117–128 (2010)

    Chapter  Google Scholar 

  35. Yatziv, L., Sapiro, G.: Fast image and video colorization using chrominance blending. IEEE Trans. Image Process. 15(5), 1120–1129 (2006)

    Article  Google Scholar 

  36. Yin, R., Gao, T., Lu, Y.M., Daubechies, I.: A tale of two bases: local–nonlocal regularization on image patches with convolution framelets. SIAM J. Imaging Sci. 10(2), 711–750 (2017)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

Myeongmin Kang was supported by the NRF (2016R1C1B1009808). Myungjoo Kang was supported by the NRF (2015R1A5A1009350, 2017R1A2A1A17069644), and IITP-MSIT (B0717-16-0107). Miyoun Jung was supported by Hankuk University of Foreign Studies Research Fund and the NRF (2017R1A2B1005363).

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, Seoul National University, Seoul, Korea

    Myeongmin Kang & Myungjoo Kang

  2. Department of Mathematics, Hankuk University of Foreign Studies, Yongin, Korea

    Miyoun Jung

Authors
  1. Myeongmin Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Myungjoo Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miyoun Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miyoun Jung.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, M., Kang, M. & Jung, M. Image Colorization Based on a Generalization of the Low Dimensional Manifold Model. J Sci Comput 77, 911–935 (2018). https://doi.org/10.1007/s10915-018-0732-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-018-0732-8

Keywords

Search

Navigation