Abstract
Despite the rapid development of photography equipment, shooting high-definition RAW images in extreme low-light environments has always been a difficult problem to solve. Existing methods use neural networks to automatically learn the mapping from extreme low-light noise RAW images to long-exposure RGB images for jointly denoising and demosaicing of extreme low-light images, but the performance on other datasets is unpleasant. In order to address this problem, we present a separable Unet++ (SUnet++) network structure to improve the generalization ability of the joint denoising and demosaicing method for extreme low-light images. We introduce Unet++ to adapt the model to other datasets, and then replace the conventional convolutions of Unet++ with M sets of depthwise separable convolutions, which greatly reduced the number of parameters without losing performance. Experimental results on SID and ELD dataset demonstrate our proposed SUnet++ outperform the state-of-the-arts methods in term of subjective and objective results, which further validates the robust generalization of our proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)
Brooks, T., Mildenhall, B., Xue, T., Chen, J., Sharlet, D., Barron, J.T.: Unprocessing images for learned raw denoising. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11036–11045 (2019)
Chen, C., Chen, Q., Xu, J., Koltun, V.: Learning to see in the dark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3291–3300 (2018)
Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)
Ehret, T., Davy, A., Arias, P., Facciolo, G.: Joint demosaicking and denoising by fine-tuning of bursts of raw images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8868–8877 (2019)
Guo, S., Liang, Z., Zhang, L.: Joint denoising and demosaicking with green channel prior for real-world burst images. arXiv preprint arXiv:2101.09870 (2021)
Jin, Q., Facciolo, G., Morel, J.M.: A review of an old dilemma: demosaicking first, or denoising first? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 514–515 (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Koskinen, S., Yang, D., Kämäräinen, J.K.: Reverse imaging pipeline for raw RGB image augmentation. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 2896–2900. IEEE (2019)
Liu, L., Jia, X., Liu, J., Tian, Q.: Joint demosaicing and denoising with self guidance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2240–2249 (2020)
Liu, S., Chen, J., Xun, Y., Zhao, X., Chang, C.H.: A new polarization image demosaicking algorithm by exploiting inter-channel correlations with guided filtering. IEEE Trans. Image Process. 29, 7076–7089 (2020)
Oktay, O., et al.: Attention u-net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 (2018)
Pan, Z., Li, B., Cheng, H., Bao, Y.: Deep residual network for MSFA raw image denoising. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2413–2417. IEEE (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Wei, K., Fu, Y., Yang, J., Huang, H.: A physics-based noise formation model for extreme low-light raw denoising. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2758–2767 (2020)
Yang, C.C., Guo, S.M., Tsai, J.S.H.: Evolutionary fuzzy block-matching-based camera raw image denoising. IEEE Trans. Cybern. 47(9), 2862–2871 (2016)
Man, Y., Huang, Y., Feng, J., Li, X., Wu, F.: Deep Q learning driven CT pancreas segmentation with geometry-aware u-net. IEEE Trans. Med. Imaging (2019)
Zamir, S.W., et al.: Cycleisp: real image restoration via improved data synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2696–2705 (2020)
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imaging 39(6), 1856–1867 (2019)
Acknowledgments
This work was supported by the National Natural Science Foundation of China under Grant 61906009, the Scientific Research Common Program of Beijing Municipal Commission of Education KM202010005018, and the International Research Cooperation Seed Fund of Beijing University of Technology (Project No. 2021B06).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Qi, J., Qi, N., Zhu, Q. (2022). SUnet++:Joint Demosaicing and Denoising of Extreme Low-Light Raw Image. In: Þór Jónsson, B., et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13142. Springer, Cham. https://doi.org/10.1007/978-3-030-98355-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-98355-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98354-3
Online ISBN: 978-3-030-98355-0
eBook Packages: Computer ScienceComputer Science (R0)