Abstract
In this paper, we introduce a novel model that restores a color image from a grayscale image with color values given in small regions. The model is based on the idea of the generalization of the low dimensional manifold model (Shi et al. in J Sci Comput, 2017. https://doi.org/10.1007/s10915-017-0549-x) and the YCbCr color space. It involves two prior terms, a weighted nonlocal Laplacian (WNLL) and a weighted total variation (WTV). The WNLL allows regions without color information to be interpolated smoothly from given sparse color data, while the WTV assists to inhibit the diffusion of color values across edges. To cope with various types of sampled data, we introduce an updating rule for the weight function in the WNLL. Furthermore, we present an efficient iterative algorithm for solving the proposed model. Lastly, numerical experiments validate the superior performance of the proposed model over that of the other state-of-the-art models.


















Similar content being viewed by others
References
Barrett, R., Berry, M., Chan, T.F., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., Van der Vorst, H.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. SIAM, Philadelphia (1994)
Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)
Bhat, P., Zitnick, C.L., Cohen, M., Curless, B.: Gradientshop: a gradient-domain optimization framework for image and video filtering. ACM Trans. Graph. (TOG) 29(2), 10 (2010)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends\({\textregistered }\) Mach. Learn. 3(1), 1–122 (2011)
Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 60–65 (2005)
Cai, J.F., Osher, S., Shen, Z.: Split Bregman methods and frame based image restoration. Multisc. Model. Simul. 8(2), 337–369 (2009)
Chatterjee, P., Milanfar, P.: Patch-based near-optimal image denoising. IEEE Trans. Image Process. 21(4), 1635–1649 (2012)
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)
Deng, W., Yin, W.: On the global and linear convergence of the generalized alternating direction method of multipliers. J. Sci. Comput. 66(3), 889–916 (2016)
Fornasier, M.: Nonlinear projection recovery in digital inpainting for color image restoration. J. Math. Imaging Vis. 24(3), 359–373 (2006)
Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multisc. Model. Simul. 7(3), 1005–1028 (2008)
Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)
Han, D., Yuan, X.: A note on the alternating direction method of multipliers. J. Optim. Theory Appl. 155(1), 227–238 (2012)
Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur Stand. 49(6), 409–436 (1952)
Jin, Z., Zhou, C., Ng, M.K.: A coupled total variation model with curvature driven for image colorization. Inverse Probl. Imaging 10(4), 1037–1055 (2016)
Jung, M., Kang, M.: Variational image colorization models using higher-order Mumford–Shah regularizers. J. Sci. Comput. 68(2), 864–888 (2016)
Kang, S.H., March, R.: Variational models for image colorization via chromaticity and brightness decomposition. IEEE Trans. Image Process. 16(9), 2251–2261 (2007)
Levin, A., Lischinski, D., Weiss, Y.: Colorization using optimization. ACM Trans. Graph. (TOG) 23, 689–694 (2004)
Li, F., Bao, Z., Liu, R., Zhang, G.: Fast image inpainting and colorization by Chambolles dual method. J. Vis. Commun. Image Represent. 22(6), 529–542 (2011)
Li, Z., Shi, Z.: A convergent point integral method for isotropic elliptic equations on a point cloud. Multisc. Model. Simul. 14(2), 874–905 (2016)
Li, Z., Shi, Z., Sun, J.: Point integral method for solving Poisson-type equations on manifolds from point clouds with convergence guarantees. Commun. Comput. Phys. 22(1), 228–258 (2017)
Matviychuk, Y., Hughes, S.M.: Regularizing inverse problems in image processing with a manifold-based model of overlapping patches. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5352–5356 (2014)
Ni, J., Turaga, P., Patel, V.M., Chellappa, R.: Example-driven manifold priors for image deconvolution. IEEE Trans. Image Process. 20(11), 3086–3096 (2011)
Osher, S., Shi, Z., Zhu, W.: Low dimensional manifold model for image processing. SIAM J. Imaging Sci. 10(4), 1669–1690 (2017)
Peyré, G.: Image processing with nonlocal spectral bases. Multisc. Model. Simul. 7(2), 703–730 (2008)
Peyré, G.: Manifold models for signals and images. Comput. Vis. Image Underst. 113(2), 249–260 (2009)
Peyré, G.: Sparse modeling of textures. J. Math. Imaging Vis. 34(1), 17–31 (2009)
Quang, M.H., Kang, S.H., Le, T.M.: Image and video colorization using vector-valued reproducing kernel Hilbert spaces. J. Math. Imaging Vis. 37(1), 49–65 (2010)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)
Sapiro, G.: Inpainting the colors. In: IEEE International Conference on Image Processing (ICIP), vol. 2, pp. II-698–701 (2005)
Shi, Z., Osher, S., Zhu, W.: Generalization of the weighted nonlocal Laplacian in low dimensional manifold model. J. Sci. Comput. 75(2), 638–656 (2018)
Shi, Z., Osher, S., Zhu, W.: Weighted nonlocal Laplacian on interpolation from sparse data. J. Sci. Comput. 73(2–3), 1164–1177 (2017)
Shi, Z., Sun, J.: Convergence of the point integral method for Poisson equation on point cloud. arXiv preprint arXiv:1403.2141 (2014)
Yang, Y., Li, C., Kao, C.Y., Osher, S.: Split Bregman method for minimization of region-scalable fitting energy for image segmentation. In: International Symposium on Visual Computing (ISVC 2010): Advances in Visual Computing, vol. 6454, pp. 117–128 (2010)
Yatziv, L., Sapiro, G.: Fast image and video colorization using chrominance blending. IEEE Trans. Image Process. 15(5), 1120–1129 (2006)
Yin, R., Gao, T., Lu, Y.M., Daubechies, I.: A tale of two bases: local–nonlocal regularization on image patches with convolution framelets. SIAM J. Imaging Sci. 10(2), 711–750 (2017)
Acknowledgements
Myeongmin Kang was supported by the NRF (2016R1C1B1009808). Myungjoo Kang was supported by the NRF (2015R1A5A1009350, 2017R1A2A1A17069644), and IITP-MSIT (B0717-16-0107). Miyoun Jung was supported by Hankuk University of Foreign Studies Research Fund and the NRF (2017R1A2B1005363).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kang, M., Kang, M. & Jung, M. Image Colorization Based on a Generalization of the Low Dimensional Manifold Model. J Sci Comput 77, 911–935 (2018). https://doi.org/10.1007/s10915-018-0732-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10915-018-0732-8