Color Image Super Resolution in Real Noise
Article No.: 61, Pages 1 - 9
Abstract
In practice, images can contain different amounts of noise for different color channels, which is not acknowledged by existing super-resolution approaches. In this paper, we propose to super-resolve noisy color images by considering the color channels jointly. Noise statistics are blindly estimated from the input low-resolution image and are used to assign different weights to different color channels in the data cost. Implicit low-rank structure of visual data is enforced via nuclear norm minimization in association with adaptive weights, which is added as a regularization term to the cost. Additionally, multi-scale details of the image are added to the model through another regularization term that involves projection onto PCA basis, which is constructed using similar patches extracted across different scales of the input image. The results demonstrate the super-resolving capability of the approach in real scenarios.
References
[1]
Abdelrahman Abdelhamed, Stephen Lin, and Michael S. Brown. 2018. A High-Quality Denoising Dataset for Smartphone Cameras. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1692--1700.
[2]
M. Aharon, M. Elad, and A. Bruckstein. Nov. 2006. K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Transactions on Signal Processing 54, 11 (Nov. 2006), 4311--4322.
[3]
Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. 2011. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Found. Trends Mach. Learn. 3, 1 (Jan. 2011), 1--122. https://doi.org/10.1561/2200000016
[4]
G. Chen, F. Zhu, and P. A. Heng. 2015. An Efficient Statistical Method for Image Noise Level Estimation. In IEEE International Conference on Computer Vision (ICCV). 477--485. https://doi.org/10.1109/ICCV.2015.62
[5]
K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. 2007. Color Image Denoising via Sparse 3D Collaborative Filtering with Grouping Constraint in Luminance-Chrominance Space. In IEEE International Conference on Image Processing (ICIP), Vol. 1. I-313--I-316. https://doi.org/10.1109/ICIP.2007.4378954
[6]
Chao Dong, ChenChange Loy, Kaiming He, and Xiaoou Tang. 2014. Learning a Deep Convolutional Network for Image Super-Resolution. In Computer Vision -ECCV 2014. Lecture Notes in Computer Science, Vol. 8692. Springer International Publishing, 184--199. https://doi.org/10.1007/978-3-319-10593-2_13
[7]
Chao Dong, Chen Change Loy, and Xiaoou Tang. 2016. Accelerating the Super-Resolution Convolutional Neural Network. In Computer Vision - ECCV 2016, Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, Cham, 391--407.
[8]
W. Dong, L. Zhang, G. Shi, and X. Li. 2013. Nonlocally Centralized Sparse Representation for Image Restoration. IEEE Transactions on Image Processing 22, 4 (April 2013), 1620--1630. https://doi.org/10.1109/TIP.2012.2235847
[9]
Weisheng Dong, Lei Zhang, Guangming Shi, and Xiaolin Wu. 2011. Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization. IEEE Transactions on Image Processing 20, 7 (Jul. 2011), 1838--1857. https://doi.org/10.1109/TIP.2011.2108306
[10]
W.T. Freeman, T.R. Jones, and E.C. Pasztor. 2002. Example-based super-resolution. IEEE, Computer Graphics and Applications 22, 2 (mar/apr 2002), 56--65. https://doi.org/10.1109/38.988747
[11]
D. Glasner, S. Bagon, and M. Irani. 2009. Super-resolution from a single image. In IEEE International Conference on Computer Vision (ICCV). 349--356. https://doi.org/10.1109/ICCV.2009.5459271
[12]
Shuhang Gu, Qi Xie, Deyu Meng, Wangmeng Zuo, Xiangchu Feng, and Lei Zhang. 2017. Weighted Nuclear Norm Minimization and Its Applications to Low Level Vision. International Journal of Computer Vision 121, 2 (01 Jan 2017), 183--208. https://doi.org/10.1007/s11263-016-0930-5
[13]
K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep Residual Learning for Image Recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770--778. https://doi.org/10.1109/CVPR.2016.90
[14]
J. B. Huang, A. Singh, and N. Ahuja. 2015. Single image super-resolution from transformed self-exemplars. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5197--5206. https://doi.org/10.1109/CVPR.2015.7299156
[15]
G. Jeon and E. Dubois. 2013. Demosaicking of Noisy Bayer-Sampled Color Images With Least-Squares Luma-Chroma Demultiplexing and Noise Level Estimation. IEEE Transactions on Image Processing 22, 1 (Jan 2013), 146--156. https://doi.org/10.1109/TIP.2012.2214041
[16]
Atsunori Kanemura, Shin ichi Maeda, and Shin Ishii. 2009. Superresolution with compound Markov random fields via the variational {EM} algorithm. Neural Networks 22, 7 (2009), 1025--1034. https://doi.org/10.1016/j.neunet.2008.12.005
[17]
Hakki Can Karaimer and Michael S. Brown. 2016. A Software Platform for Manipulating the Camera Imaging Pipeline. In Computer Vision - ECCV, Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, Cham, 429--444.
[18]
J. Kim, J. K. Lee, and K. M. Lee. 2016. Accurate Image Super-Resolution Using Very Deep Convolutional Networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1646--1654. https://doi.org/10.1109/CVPR.2016.182
[19]
J. Kim, J. K. Lee, and K. M. Lee. 2016. Deeply-Recursive Convolutional Network for Image Super-Resolution. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1637--1645. https://doi.org/10.1109/CVPR.2016.181
[20]
Marc Lebrun, Miguel Colom, and Jean-Michel Morel. 2015. The Noise Clinic: a Blind Image Denoising Algorithm. Image Processing On Line 5 (2015), 1--54. https://doi.org/10.5201/ipol.2015.125
[21]
C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi. 2017. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 105--114. https://doi.org/10.1109/CVPR.2017.19
[22]
B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee. 2017. Enhanced Deep Residual Networks for Single Image Super-Resolution. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1132--1140. https://doi.org/10.1109/CVPRW.2017.151
[23]
C. Liu, R. Szeliski, S. Bing Kang, C. L. Zitnick, and W. T. Freeman. 2008. Automatic Estimation and Removal of Noise from a Single Image. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 2 (Feb 2008), 299--314. https://doi.org/10.1109/TPAMI.2007.1176
[24]
J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman. 2009. Non-local sparse models for image restoration. In IEEE 12th International Conference on Computer Vision. 2272--2279. https://doi.org/10.1109/ICCV.2009.5459452
[25]
S. Mandal, A. Bhavsar, and A.K. Sao. 2014. Super-resolving a Single Intensity/Range Image via Non-local Means and Sparse Representation. In Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), 2014. 1--8. https://doi.org/10.1145/2683483.2683541
[26]
S. Mandal, A. Bhavsar, and A. K. Sao. 2017. Depth Map Restoration From Undersampled Data. IEEE Transactions on Image Processing 26, 1 (Jan 2017), 119--134. https://doi.org/10.1109/TIP.2016.2621410
[27]
Srimanta Mandal, Arnav Bhavsar, and Anil Kumar Sao. 2017. Noise adaptive super-resolution from single image via non-local mean and sparse representation. Signal Processing 132 (2017), 134--149. https://doi.org/10.1016/j.sigpro.2016.09.017
[28]
Srimanta Mandal and A. N. Rajagopalan. 2018. Single Noisy Image Super Resolution by Minimizing Nuclear Norm in Virtual Sparse Domain. In Computer Vision, Pattern Recognition, Image Processing, and Graphics, Renu Rameshan, Chetan Arora, and Sumantra Dutta Roy (Eds.). Springer Singapore, Singapore, 163--176.
[29]
Srimanta Mandal and Anil Kumar Sao. 2016. Employing structural and statistical information to learn dictionary(s) for single image super-resolution in sparse domain. Signal Processing: Image Communication 48 (2016), 63--80. https://doi.org/10.1016/j.image.2016.08.006
[30]
Xiao-Jiao Mao, Chunhua Shen, and Yu-Bin Yang. 2016. Image Restoration Using Very Deep Convolutional Encoder-decoder Networks with Symmetric Skip Connections. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16). Curran Associates Inc., USA, 2810--2818. http://dl.acm.org/citation.cfm?id=3157382.3157412
[31]
Antonio Marquina and Stanley J. Osher. 2008. Image Super-Resolution by TV-Regularization and Bregman Iteration. Journal of Scientific Computing 37 (2008), 367--382. Issue 3. https://doi.org/10.1007/s10915-008-9214-8
[32]
S. Nam, Y. Hwang, Y. Matsushita, and S. J. Kim. 2016. A Holistic Approach to Cross-Channel Image Noise Modeling and Its Application to Image Denoising. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1683--1691. https://doi.org/10.1109/CVPR.2016.186
[33]
Sung Cheol Park, Min Kyu Park, and Moon Gi Kang. 2003. Super-resolution image reconstruction: a technical overview. IEEE, Signal Processing Magazine 20, 3 (may 2003), 21--36. https://doi.org/10.1109/MSP.2003.1203207
[34]
T. Peleg and M. Elad. 2014. A Statistical Prediction Model Based on Sparse Representations for Single Image Super-Resolution. IEEE Transactions on Image Processing 23, 6 (June 2014), 2569--2582. https://doi.org/10.1109/TIP.2014.2305844
[35]
T. Plötz and S. Roth. 2017. Benchmarking Denoising Algorithms with Real Photographs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2750--2759. https://doi.org/10.1109/CVPR.2017.294
[36]
R. Rubinstein, A.M. Bruckstein, and M. Elad. 2010. Dictionaries for Sparse Representation Modeling. Proc. IEEE 98, 6 (june 2010), 1045--1057. https://doi.org/10.1109/JPROC.2010.2040551
[37]
W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang. 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1874--1883. https://doi.org/10.1109/CVPR.2016.207
[38]
Assaf Shocher, Nadav Cohen, and Michal Irani. 2018. "Zero-Shot" Super-Resolution Using Deep Internal Learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3118--3126.
[39]
A. Singh, F. Porikli, and N. Ahuja. 2014. Super-resolving Noisy Images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2846--2853. https://doi.org/10.1109/CVPR.2014.364
[40]
Henry Stark and Peyma Oskoui. 1989. High-resolution image recovery from image-plane arrays, using convex projections. J. Opt. Soc. Am. A 6, 11 (Nov. 1989), 1715--1726.
[41]
Radu Timofte, Vincent De Smet, and Luc Van Gool. 2015. A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution. In Computer Vision - ACCV 2014. Lecture Notes in Computer Science, Vol. 9006. Springer International Publishing, 111--126. https://doi.org/10.1007/978-3-319-16817-3_8
[42]
S. Vishnukumar, Madhu S. Nair, and M. Wilscy. 2014. Edge preserving single image super-resolution with improved visual quality. Signal Processing 105, 0 (2014), 283--297. https://doi.org/10.1016/j.sigpro.2014.05.033
[43]
J. Xu, L. Zhang, D. Zhang, and X. Feng. 2017. Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising. In IEEE International Conference on Computer Vision (ICCV). 1105--1113. https://doi.org/10.1109/ICCV.2017.125
[44]
Chih-Yuan Yang, Jia-Bin Huang, and Ming-Hsuan Yang. 2011. Exploiting Self-similarities for Single Frame Super-Resolution. In Computer Vision - ACCV 2010, Ron Kimmel, Reinhard Klette, and Akihiro Sugimoto (Eds.). Lecture Notes in Computer Science, Vol. 6494. Springer Berlin Heidelberg, 497--510. https://doi.org/10.1007/978-3-642-19318-7_39
[45]
Jianchao Yang, J. Wright, T. Huang, and Yi Ma. Jun. 2008. Image super-resolution as sparse representation of raw image patches. In IEEE Conference on Computer Vision and Pattern Recognition. 1--8. https://doi.org/10.1109/CVPR.2008.4587647
[46]
Jianchao Yang, J. Wright, T.S. Huang, and Yi Ma. Nov. 2010. Image Super-Resolution Via Sparse Representation. IEEE Transactions on Image Processing 19, 11 (Nov. 2010), 2861--2873. https://doi.org/10.1109/TIP.2010.2050625
[47]
Roman Zeyde, Michael Elad, and Matan Protter. 2012. On Single Image Scale-Up Using Sparse-Representations. In Curves and Surfaces. Vol. 6920. Springer, 711--730. https://doi.org/10.1007/978-3-642-27413-8_47
[48]
Xin Zhang, Edmund Y. Lam, EdX. Wu, and Kenneth K.Y. Wong. 2008. Application of Tikhonov Regularization to Super-Resolution Reconstruction of Brain MRI Images. In Medical Imaging and Informatics, Xiaohong Gao, Henning Müller, MartinJ. Loomes, Richard Comley, and Shuqian Luo (Eds.). Lecture Notes in Computer Science, Vol. 4987. Springer Berlin Heidelberg, 51--56. https://doi.org/10.1007/978-3-540-79490-5_8
Index Terms
- Color Image Super Resolution in Real Noise
Recommendations
Single image super-resolution based on nonlocal sparse and low-rank regularization
PRICAI'16: Proceedings of the 14th Pacific Rim International Conference on Trends in Artificial IntelligenceImage super resolution (SR) is an active research topic to obtain an high resolution (HR) image from the low resolution (LR) observation. Many results of existing methods may be corrupted by some artifacts. In this paper, we propose an SR reconstruction ...
Colorization for single image super resolution
ECCV'10: Proceedings of the 11th European conference on Computer vision: Part VIThis paper introduces a new procedure to handle color in single image super resolution (SR). Most existing SR techniques focus primarily on enforcing image priors or synthesizing image details; less attention is paid to the final color assignment. As a ...
Real Color Image Enhanced by Illumination - Reflectance Model and Wavelet Transformation
ITCS '09: Proceedings of the 2009 International Conference on Information Technology and Computer Science - Volume 01Because the greatly most algorithms of real color image enhancement did not relate to the spectral sensitivity of most human vision, the color restoration functions of some real color image enhancement algorithms are greatly at random and not proved , ...
Comments
Information & Contributors
Information
Published In
December 2018
659 pages
ISBN:9781450366151
DOI:10.1145/3293353
Copyright © 2018 ACM.
© 2018 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 03 May 2020
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
ICVGIP 2018
ICVGIP 2018: 11th Indian Conference on Computer Vision, Graphics and Image Processing
December 18 - 22, 2018
Hyderabad, India
Acceptance Rates
Overall Acceptance Rate 95 of 286 submissions, 33%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 69Total Downloads
- Downloads (Last 12 months)7
- Downloads (Last 6 weeks)0
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in