Uncertainty-Aware Sparse Transformer Network for Single-Image Deraindrop | IEEE Journals & Magazine | IEEE Xplore

Uncertainty-Aware Sparse Transformer Network for Single-Image Deraindrop


Abstract:

Image Deraindrop aims to enhance the visibility and clarity of the image by eliminating unwanted visual artifacts, such as raindrops or rain streaks. Despite remarkable a...Show More

Abstract:

Image Deraindrop aims to enhance the visibility and clarity of the image by eliminating unwanted visual artifacts, such as raindrops or rain streaks. Despite remarkable advancements in image raindrop removal, the sparse distribution of raindrops and the various levels of degradation within raindrop regions are still not fully considered: 1) globally, raindrops often exhibit a unique sparse distribution in images, but the existing methods apply a uniform treatment of pixels and 2) locally, raindrops have specific degradation within raindrop regions, such as variations in shape, size, and transparency, but current approaches fail to effectively model them. To address these problems, this work proposes an uncertainty-aware sparse Transformer network (USTN) for image Deraindrop. Specifically, to characterize the sparsity of raindrops, we develop a sparse Transformers backbone in which several sparse Transformers blocks (STBs) are deployed at each scale. To ensure effective sparse feature learning, we introduce a top-k sparse attention (TSA) layer in each STB, which dynamically selects high-score attention and generates corresponding sparse feature responses. To effectively model various degrees of degradation in local raindrop regions, we incorporate image uncertainty estimation, which can explicitly show the observation that worse degradation exhibits higher uncertainty. Motivated by this, we design two decoding branches, one for estimating uncertainty maps and the other for generating raindrop-free images. We then formulate an uncertainty-driven loss to better push the USTN to remove raindrops guided by uncertainty maps. In addition, to further refine the learned sparse features, we propose a pyramid feature refinement (PFR) module to fully mine the local features under multiscale receptive fields and a residual channel-spatial attention (RCSA) module to stimulate the effective expression of the deepest features. Extensive experiments demonstrate that the proposed UST...
Article Sequence Number: 5035313
Date of Publication: 03 October 2024

ISSN Information:

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.