Abstract
Recently, transformer based deep neural networks have been found useful in solving various image restoration tasks like image denoising, deblurring, deraining etc., producing significant improvement in PSNR and SSIM over CNN based techniques on benchmark datasets. These networks have effectively addressed quadratic computational complexity issue with increasing image resolution by making use of novel self attention strategies on local image windows. In this paper, we propose a fast and efficient UNet based architecture using transformer modules for the image demoireing task. The proposed architecture is computationally very efficient as the transformer blocks perform non-overlapping window-based self-attention instead of global self attention. We further improve upon the computational complexity by using decreasing window sizes across scales under the proposed U-Net multi resolution framework. To the best of our knowledge, ours is the first deep network architecture using transformer blocks for the image demoireing problem producing comparable results with state of the art techniques both visually and quantitatively on the CFAMoire challenge dataset [23].
All the authors have equally contributed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anwar, S., Barnes, N.: Real image denoising with feature attention. CoRR abs/1904.07396 (2019). http://arxiv.org/abs/1904.07396
Cheng, X., Fu, Z., Yang, J.: Multi-scale dynamic feature encoding network for image demoireing (2019)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3–7, 2021. OpenReview.net (2021). https://openreview.net/forum?id=YicbFdNTTy
Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution (03 2018)
He, B., Wang, C., Shi, B., Duan, L.Y.: Mop moire patterns using MOPNET. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019
Kim, Y., Soh, J.W., Park, G.Y., Cho, N.I.: Transfer learning from synthetic to real-noise denoising with adaptive instance normalization. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., Matas, J.: Deblurgan: blind motion deblurring using conditional adversarial networks. CoRR abs/1711.07064 (2017). http://arxiv.org/abs/1711.07064
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105–114 (2017)
Li, Y., Zhang, K., Cao, J., Timofte, R., Van Gool, L.: Localvit: bringing locality to vision transformers (2021). https://doi.org/10.48550/ARXIV.2104.05707, https://arxiv.org/abs/2104.05707
Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: Swinir: image restoration using swin transformer. In: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pp. 1833–1844 (2021). https://doi.org/10.1109/ICCVW54120.2021.00210
Liu, B., Shu, X., Wu, X.: Demoiréing of camera-captured screen images using deep convolutional neural network (2018)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10–17, 2021. pp. 9992–10002. IEEE (2021). https://doi.org/10.1109/ICCV48922.2021.00986
Luo, X., Zhang, J., Hong, M., Qu, Y., Xie, Y., Li, C.: Deep wavelet network with domain adaptation for single image demoireing. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1687–1694 (2020). https://doi.org/10.1109/CVPRW50498.2020.00218
Sun, Y., Yu, Y., Wang, W.: Moiré photo restoration using multiresolution convolutional neural networks. IEEE Trans. Image Process. 27, 4160–4172 (2018)
Vaswani, A., et al.: Attention is all you need. CoRR abs/1706.03762 (2017). http://arxiv.org/abs/1706.03762
Wang, Z., Cun, X., Bao, J., Liu, J.: Uformer: a general u-shaped transformer for image restoration. CoRR abs/2106.03106 (2021). https://arxiv.org/abs/2106.03106
Xia, Z., Chakrabarti, A.: Identifying recurring patterns with deep neural networks for natural image denoising. In: IEEE Winter Conference on Applications of Computer Vision, WACV 2020, Snowmass Village, CO, USA, March 1–5, 2020, pp. 2415–2423. IEEE (2020). https://doi.org/10.1109/WACV45572.2020.9093586
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M., Luo, P.: Segformer: simple and efficient design for semantic segmentation with transformers. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems, vol. 34, pp. 12077–12090. Curran Associates, Inc. (2021). https://proceedings.neurips.cc/paper/2021/file/64f1f27bf1b4ec22924fd0acb550c235-Paper.pdf
Yang, F., Yang, H., Fu, J., Lu, H., Guo, B.: Learning texture transformer network for image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
Yang, J., Liu, F., Yue, H., Fu, X., Hou, C., Wu, F.: Textured image demoiréing via signal decomposition and guided filtering. IEEE Trans. Image Process. 26(7), 3528–3541 (2017)
Yuan, S., et al.: Ntire 2020 challenge on image demoireing: methods and results. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1882–1893 (2020)
Yuan, S., Timofte, R., Slabaugh, G., Leonardis, A.: Aim 2019 challenge on image demoireing: Dataset and study. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3526–3533 (2019)
Yuan, S., Timofte, R., Leonardis, A., Slabaugh, G.: Ntire 2020 challenge on image demoireing: methods and results. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020
Yuan, S., et al.: Aim 2019 challenge on image demoireing: methods and results (2019). https://doi.org/10.48550/ARXIV.1911.03461
Zhang, H., Dai, Y., Li, H., Koniusz, P.: Deep stacked hierarchical multi-patch network for image deblurring. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5971–5979 (2019)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. CoRR abs/1807.02758 (2018). http://arxiv.org/abs/1807.02758
Zheng, B., Yuan, S., Slabaugh, G., Leonardis, A.: Image demoireing with learnable bandpass filters (2020). https://doi.org/10.48550/ARXIV.2004.00406. https://arxiv.org/abs/2004.00406
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable DETR: deformable transformers for end-to-end object detection. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3–7, 2021. OpenReview.net (2021). https://openreview.net/forum?id=gZ9hCDWe6ke
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Puthussery, D., Hrishikesh, P.S., Jiji, C.V. (2023). A Transformer-Based U-Net Architecture for Fast and Efficient Image Demoireing. In: Gupta, D., Bhurchandi, K., Murala, S., Raman, B., Kumar, S. (eds) Computer Vision and Image Processing. CVIP 2022. Communications in Computer and Information Science, vol 1777. Springer, Cham. https://doi.org/10.1007/978-3-031-31417-9_40
Download citation
DOI: https://doi.org/10.1007/978-3-031-31417-9_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31416-2
Online ISBN: 978-3-031-31417-9
eBook Packages: Computer ScienceComputer Science (R0)