skip to main content
10.1145/3503161.3547962acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

CRNet: Unsupervised Color Retention Network for Blind Motion Deblurring

Published: 10 October 2022 Publication History

Abstract

Blind image deblurring is still a challenging problem due to the inherent ill-posed properties. To improve the deblurring performance, many supervised methods have been proposed. However, obtaining labeled samples from a specific distribution (or a domain) is usually expensive, and the data-driven training-based model also cannot be generalized to the blurry images in all domains. These challenges have given birth to certain unsupervised deblurring methods. However, there is a great chromatic aberration between the latent and original images, directly degrading the performance. In this paper, we therefore propose a novel unsupervised color retention network termed CRNet to perform blind motion deblurring. In addition, new concepts of blur offset estimation and adaptive blur correction are proposed to retain the color information when deblurring. As a result, unlike the previous studies, CRNet does not learn a mapping directly from the blurry image to the restored latent image, but from the blurry image to a motion offset. An adaptive blur correction operation is then performed on the blurry image to restore the latent image, thereby retaining the color information of the original image to the greatest extent. To further effectively retain the color information and extract the blur information, we also propose a new module called pyramid global blur feature perception (PGBFP). To quantitatively prove the effectiveness of our network in color retention, we propose a novel chromatic aberration quantization metrics in line with the human perception. Extensive quantitative and visualization experiments show that CRNet can obtain the state-of-the-art performance in unsupervised deblurring tasks.

Supplementary Material

MP4 File (MM22-fp0873.mp4)
Presentation video for "Unsupervised Color Retention Network for Blind Motion Deblurring"

References

[1]
Yuanchao Bai, Huizhu Jia, Ming Jiang, Xianming Liu, Xiaodong Xie, and Wen Gao. 2019. Single-image blind deblurring using multi-scale latent structure prior. IEEE Trans. Circuits Syst. Video Technol., Vol. 30, 7 (2019), 2033--2045.
[2]
RE Bedford and G Wyszecki. 1957. Axial chromatic aberration of the human eye. JOSA, Vol. 47, 6 (1957), 564_1--565.
[3]
Siavash Arjomand Bigdeli, Meiguang Jin, Paolo Favaro, and Matthias Zwicker. 2017. Deep mean-shift priors for image restoration. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA (2017).
[4]
Liang Chen, Faming Fang, Tingting Wang, and Guixu Zhang. 2019. Blind image deblurring with local maximum gradient prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.
[5]
Liang Chen, Faming Fang, Jiawei Zhang, Jun Liu, and Guixu Zhang. 2020. Oid: Outlier identifying and discarding in blind image deblurring. In Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK. 598--613.
[6]
Grigorios Chrysos, Paolo Favaro, and Stefanos Zafeiriou. 2019. Motion Deblurring of Faces. Int. J. Comput. Vis., Vol. 127, 6--7 (2019), 801--823.
[7]
Rajshekhar Das, Anurag Bajpai, and Shankar M Venkatesan. 2017. Fast non-blind image deblurring with sparse priors. In Proceedings of the International Conference on Computer Vision and Image Processing, Indian Institute of Technology Roorkee.
[8]
Mauricio Delbracio, Ignacio Garcia-Dorado, Sungjoon Choi, Damien Kelly, and Peyman Milanfar. 2021. Polyblur: Removing mild blur by polynomial reblurring. IEEE Trans. Computational Imaging, Vol. 7 (2021), 837--848.
[9]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Miami, Florida, USA. 248--255.
[10]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the Advances in Neural Information Processing Systems, Montreal, Quebec, Canada. 2672--2680.
[11]
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. 2017. Improved training of wasserstein gans. In Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
[12]
Akash Gupta, Abhishek Aich, and Amit K. Roy-Chowdhury. 2020. ALANET: Adaptive Latent Attention Network for Joint Video Deblurring and Interpolation. In Proceedings of the ACM International Conference on Multimedia, Seattle, WA, USA. 256--264.
[13]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA. 770--778.
[14]
Yuzhu Ji, Haijun Zhang, Zhao Zhang, and Ming Liu. 2021. CNN-based encoder-decoder networks for salient object detection: A comprehensive review and recent advances. Inf. Sci., Vol. 546 (2021), 835--857.
[15]
Micah K Johnson and Hany Farid. 2006. Exposing digital forgeries through chromatic aberration. In Proceedings of the 8th workshop on Multimedia & Security, MM&Sec, Geneva, Switzerland. 48--55.
[16]
Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim. 2017. Learning to discover cross-domain relations with generative adversarial networks. In Proceedings of the 34th International Conference on Machine Learning, Sydney, NSW, Australia. 1857--1865.
[17]
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA (2015).
[18]
Dilip Krishnan and Rob Fergus. 2009. Fast Image Deconvolution using Hyper-Laplacian Priors. In Proceedings of the Advances in Neural Information Processing Systems, Vancouver, British Columbia, Canada. 1033--1041.
[19]
Orest Kupyn, Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jivr 'i Matas. 2018. Deblurgan: Blind motion deblurring using conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. 8183--8192.
[20]
Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang. 2019. Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea (South).
[21]
Jun Liu, Ming Yan, and Tieyong Zeng. 2021. Surface-Aware Blind Image Deblurring. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 43, 3 (2021), 1041--1055.
[22]
Risheng Liu, Yi He, Shichao Cheng, Xin Fan, and Zhongxuan Luo. 2018. Learning Collaborative Generation Correction Modules for Blind Image Deblurring and Beyond. In Proceedings of the ACM International Conference on Multimedia, Seoul, Republic of Korea. 1921--1929.
[23]
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile. 3730--3738.
[24]
Boyu Lu, Jun-Cheng Chen, and Rama Chellappa. 2019. Unsupervised domain-specific deblurring via disentangled representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.
[25]
Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. 2017. Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA. 3883--3891.
[26]
Thekke Madam Nimisha, Kumar Sunil, and AN Rajagopalan. 2018. Unsupervised class-specific deblurring. In Proceedings of the 16th European Conference on Computer Vision, Munich, Germany. 353--369.
[27]
Jinshan Pan, Deqing Sun, Hanspeter Pfister, and Ming-Hsuan Yang. 2016. Blind image deblurring using dark channel prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA. 1628--1636.
[28]
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. (2017).
[29]
Dongwei Ren, Wangmeng Zuo, David Zhang, Jun Xu, and Lei Zhang. 2017. Partial deconvolution with inaccurate blur kernel. IEEE Trans. Image Process., Vol. 27, 1 (2017).
[30]
William Hadley Richardson. 1972. Bayesian-based iterative method of image restoration. JOSA, Vol. 62, 1 (1972), 55--59.
[31]
Gabriel Rioux, Christopher Scarvelis, Rustum Choksi, Tim Hoheisel, and Pierre Marechal. 2021. Blind Deblurring of Barcodes via Kullback-Leibler Divergence. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 43, 4 (2021), 77--88.
[32]
Denys Rozumnyi, Martin R Oswald, Vittorio Ferrari, Jiri Matas, and Marc Pollefeys. 2021. DeFMO: Deblurring and Shape Recovery of Fast Moving Objects. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, virtual. 3456--3465.
[33]
Qi Shan, Jiaya Jia, and Aseem Agarwala. 2008. High-quality motion deblurring from a single image. ACM Trans. Graph., Vol. 27, 3 (2008), 1--10.
[34]
Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA (2015).
[35]
Hyeongseok Son, Junyong Lee, Jonghyeop Lee, Sunghyun Cho, and Seungyong Lee. 2021. Recurrent Video Deblurring with Blur-Invariant Motion Estimation and Pixel Volumes. ACM Trans. Graph., Vol. 40, 5 (2021), 185:1--185:18.
[36]
Libin Sun, Sunghyun Cho, Jue Wang, and James Hays. 2013. Edge-based blur kernel estimation using patch priors. In Proceedings of the IEEE International Conference on Computational Photography, Cambridge, MA, USA. 1--8.
[37]
Libin Sun, Sunghyun Cho, Jue Wang, and James Hays. 2014. Good image priors for non-blind deconvolution. In Proceedings of the 13th European Conference on Computer Vision, Zurich, Switzerland. 231--246.
[38]
LN Thibos, A Bradley, DL Still, X Zhang, and PA Howarth. 1990. Theory and measurement of ocular chromatic aberration. Vision Research, Vol. 30, 1 (1990), 33--49.
[39]
Fu-Jen Tsai, Yan-Tsung Peng, Yen-Yu Lin, Chung-Chi Tsai, and Chia-Wen Lin. 2021. BANet: Blur-aware attention networks for dynamic scene deblurring. CoRR (2021).
[40]
Yanyan Wei, Zhao Zhang, Yang Wang, Mingliang Xu, Yi Yang, Shuicheng Yan, and Meng Wang. 2021. Deraincyclegan: Rain attentive cyclegan for single image deraining and rainmaking. IEEE Trans. Image Process., Vol. 30 (2021), 4788--4801.
[41]
Norbert Wiener. 1964. Extrapolation, interpolation, and smoothing of stationary time series: with engineering applications. Vol. 8.
[42]
Zhenhua Xu, Huasong Chen, and Zhenhua Li. 2021. Fast blind deconvolution using a deeper sparse patch-wise maximum gradient prior. Signal Process. Image Commun., Vol. 90 (2021), 116050.
[43]
Minyuan Ye, Dong Lyu, and Gengsheng Chen. 2020. Scale-iterative upscaling network for image deblurring. IEEE Access, Vol. 8 (2020), 18316--18325.
[44]
Zili Yi, Hao Zhang, Ping Tan, and Minglun Gong. 2017. Dualgan: Unsupervised dual learning for image-to-image translation. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy. 2849--2857.
[45]
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. 2021. Multi-stage progressive image restoration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, virtual. 14821--14831.
[46]
Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. 2019. Self-Attention generative adversarial networks. In Proceedings of the 36th International Conference on Machine Learning, Long Beach, California, USA. 7354--7363.
[47]
Kai Zhang, Wangmeng Zuo, Shuhang Gu, and Lei Zhang. 2017. Learning deep CNN denoiser prior for image restoration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA. 3929--3938.
[48]
Yan Zhang, Zhao Zhang, Yang Wang, Zheng Zhang, Li Zhang, Shuicheng Yan, and Meng Wang. 2021. Dual-Constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior. Int. J. Comput. Vis., Vol. 129, 12 (2021), 3233--3254.
[49]
Zhao Zhang, Huan Zheng, Richang Hong, Mingliang Xu, Shuicheng Yan, and Meng Wang. 2022. Deep Color Consistent Network for Low-Light Image Enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1899--1908.
[50]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.io

Cited By

View all
  • (2025)Noise Self-Regression: A New Learning Paradigm to Enhance Low-Light Images Without Task-Related DataIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.348736147:2(1073-1088)Online publication date: Feb-2025
  • (2025)Passive Non-Line-of-Sight Imaging With Light Transport ModulationIEEE Transactions on Image Processing10.1109/TIP.2024.351809734(410-424)Online publication date: 2025
  • (2025)When low-light meets flares: Towards Synchronous Flare Removal and Brightness EnhancementNeural Networks10.1016/j.neunet.2025.107149185(107149)Online publication date: May-2025
  • Show More Cited By

Index Terms

  1. CRNet: Unsupervised Color Retention Network for Blind Motion Deblurring

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      MM '22: Proceedings of the 30th ACM International Conference on Multimedia
      October 2022
      7537 pages
      ISBN:9781450392037
      DOI:10.1145/3503161
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 10 October 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. color retention
      2. image restoration
      3. new chromatic aberration quantization metric
      4. unsupervised deblurring

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      MM '22
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)70
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 05 Mar 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)Noise Self-Regression: A New Learning Paradigm to Enhance Low-Light Images Without Task-Related DataIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.348736147:2(1073-1088)Online publication date: Feb-2025
      • (2025)Passive Non-Line-of-Sight Imaging With Light Transport ModulationIEEE Transactions on Image Processing10.1109/TIP.2024.351809734(410-424)Online publication date: 2025
      • (2025)When low-light meets flares: Towards Synchronous Flare Removal and Brightness EnhancementNeural Networks10.1016/j.neunet.2025.107149185(107149)Online publication date: May-2025
      • (2024)Motion Deblur by Learning Residual From EventsIEEE Transactions on Multimedia10.1109/TMM.2024.335563026(6632-6647)Online publication date: 18-Jan-2024
      • (2024)Multi-Scale Interaction Network for Low-Light Stereo Image EnhancementIEEE Transactions on Consumer Electronics10.1109/TCE.2023.328022970:1(3626-3634)Online publication date: Feb-2024
      • (2024)Unsupervised Blind Image Deblurring Based on Self-Enhancement2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02427(25691-25700)Online publication date: 16-Jun-2024
      • (2024)Semi-Supervised Atmospheric Turbulence Mitigation Based on Hybrid ModelsIEEE Access10.1109/ACCESS.2024.349995712(174527-174538)Online publication date: 2024
      • (2024)CD-iNet: Deep Invertible Network for Perceptual Image Color Difference MeasurementInternational Journal of Computer Vision10.1007/s11263-024-02087-7132:12(5983-6003)Online publication date: 12-Jul-2024
      • (2023)Uncertainty-Aware Variate Decomposition for Self-supervised Blind Image DeblurringProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612535(252-260)Online publication date: 26-Oct-2023
      • (2023)CropCap: Embedding Visual Cross-Partition Dependency for Image CaptioningProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612245(1750-1758)Online publication date: 26-Oct-2023
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media