Abstract
All-in-one image restoration aims to handle multiple degradation types using one model. This paper proposes a simple pipeline for all-in-one blind image restoration to Restore Anything with Masks (RAM). We focus on the image content by utilizing Mask Image Modeling to extract intrinsic image information rather than distinguishing degradation types like other methods. Our pipeline consists of two stages: masked image pre-training and fine-tuning with mask attribute conductance. We design a straightforward masking pre-training approach specifically tailored for all-in-one image restoration. This approach enhances networks to prioritize the extraction of image content priors from various degradations, resulting in a more balanced performance across different restoration tasks and achieving stronger overall results. To bridge the gap of input integrity while preserving learned image priors as much as possible, we selectively fine-tuned a small portion of the layers. Specifically, the importance of each layer is ranked by the proposed Mask Attribute Conductance (MAC), and the layers with higher contributions are selected for finetuning. Extensive experiments demonstrate that our method achieves state-of-the-art performance. Our code and model will be released at https://github.com/Dragonisss/RAM.
Chu-Jie Qin: A part of this work is done during Chu-Jie Qin’s internship at Samsung.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, H., et al.: Pre-trained image processing transformer. In: CVPR, pp. 12299–12310 (2021)
Chen, H., et al.: Masked image training for generalizable deep image denoising. In: CVPR, pp. 1692–1703 (2023)
Chen, L., Chu, X., Zhang, X., Sun, J.: Simple baselines for image restoration. In: ECCV, pp. 17–33. Springer (2022)
Chen, W.T., Huang, Z.K., Tsai, C.C., Yang, H.H., Ding, J.J., Kuo, S.Y.: Learning multiple adverse weather removal via two-stage knowledge learning and multi-contrastive regularization: toward a unified model. In: CVPR, pp. 17653–17662 (2022)
Wei, C., Wang, W., Yang, W., Liu, J.: Deep retinex decomposition for low-light enhancement. In: BMVC. British Machine VLOLision Association (2018)
Dhamdhere, K., Sundararajan, M., Yan, Q.: How important is a neuron. In: ICLR (2019)
Duan, H., et al.: Masked autoencoders as image processors. arXiv preprint arXiv:2303.17316 (2023)
Fan, Q., Chen, D., Yuan, L., Hua, G., Yu, N., Chen, B.: A general decoupled learning framework for parameterized image operators. PAMI 43(1), 33–47 (2019)
Fang, Y., Zhang, H., Wong, H.S., Zeng, T.: A robust non-blind deblurring method using deep denoiser prior. In: CVPRW, pp. 735–744 (June 2022)
Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. TIP 26(6), 2944–2956 (2017)
Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: CVPR, pp. 3855–3863 (2017)
Gu, J., Dong, C.: Interpreting super-resolution networks with local attribution maps. In: CVPR, pp. 9199–9208 (2021)
Guo, C.L., Yan, Q., Anwar, S., Cong, R., Ren, W., Li, C.: Image dehazing transformer with transmission-aware 3d position embedding. In: CVPR, pp. 5812–5820 (2022)
Guo, C., et al.: Zero-reference deep curve estimation for low-light image enhancement. In: CVPR, pp. 1780–1789 (2020)
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick., R.: Masked autoencoders are scalable vision learners. In: CVPR, pp. 16000–16009 (2022)
Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR, pp. 5197–5206 (2015)
Jin, X., Han, L.H., Li, Z., Guo, C.L., Chai, Z., Li, C.: Dnf: Decouple and feedback network for seeing in the dark. In: CVPR, pp. 18135–18144 (2023)
Kenton, J.D.M.W.C., Toutanova, L.K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL, pp. 4171–4186 (2019)
Leino, K., Sen, S., Datta, A., Fredrikson, M., Li, L.: Influence-directed explanations for deep convolutional networks. In: ITC, pp. 1–8. IEEE (2018)
Li, B., et al.: Benchmarking single-image dehazing and beyond. TIP 28(1), 492–505 (2018)
Li, B., Liu, X., Hu, P., Wu, Z., Lv, J., Peng, X.: All-in-one image restoration for unknown corruption. In: CVPR, pp. 17452–17462 (2022)
Li, C., et al.: Embedding fourier for ultra-high-definition low-light image enhancement. In: ICLR (2022)
Li, D., Zhang, Y., Cheung, K.C., Wang, X., Qin, H., Li, H.: Learning degradation representations for image deblurring. In: ECCV, pp. 736–753. Springer (2022)
Li, R., Tan, R.T., Cheong, L.F.: All in one bad weather removal using architectural search. In: CVPR, pp. 3175–3185 (2020)
Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: ECCV, pp. 254–269 (2018)
Li, Y., et al.: Lsdir: a large scale dataset for image restoration. In: CVPR, pp. 1775–1787 (2023)
Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: CVPR (2016)
Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: Swinir: image restoration using swin transformer. In: CVPR, pp. 1833–1844 (2021)
Lin, X., Ren, C., Liu, X., Huang, J., Lei, Y.: Unsupervised image denoising in real-world scenarios via self-collaboration parallel generative adversarial branches. In: ICCV, pp. 12642–12652 (2023)
Lin, X., Yue, J., Ren, C., Guo, C.L., Li, C.: Unlocking low-light-rainy image restoration by pairwise degradation feature vector guidance. arXiv preprint arXiv:2305.03997 (2023)
Liu, L., et al.: Tape: Task-agnostic prior embedding for image restoration. In: ECCV, pp. 447–464. Springer (2022)
Liu, Y., He, J., Gu, J., Kong, X., Qiao, Y., Dong, C.: Degae: a new pretraining paradigm for low-level vision. In: CVPR, pp. 23292–23303 (2023)
Liu, Y., et al.: Discovering distinctive “semantics” in super-resolution networks. arXiv preprint arXiv:2108.00406 (2021)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: CVPR, pp. 10012–10022 (2021)
Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: ICCV, pp. 3397–3405 (2015)
Luo, Y., et al.: Mowe: mixture of weather experts for multiple adverse weather removal. arXiv preprint arXiv:2303.13739 (2023)
Magid, S.A., Lin, Z., Wei, D., Zhang, Y., Gu, J., Pfister, H.: Texture-based error analysis for image super-resolution. In: CVPR, pp. 2118–2127 (2022)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423. IEEE (2001)
Mehri, A., Ardakani, P.B., Sappa, A.D.: Mprnet: multi-path residual network for lightweight image super resolution. In: CVPR, pp. 2704–2713 (2021)
Nah, S., Hyun Kim, T., Mu Lee, K.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: CVPR (2017)
Park, D., Lee, B.H., Chun, S.Y.: All-in-one image restoration for unknown degradations using adaptive discriminative filters for specific degradations. In: CVPR, pp. 5815–5824 (2023)
Potlapalli, V., Zamir, S.W., Khan, S.H., Shahbaz Khan, F.: Promptir: prompting for all-in-one image restoration. NeurIPS 36 (2024)
Radford, A., et al.: Improving language understanding by generative pre-training (2018)
Shrikumar, A., Su, J., Kundaje, A.: Computationally efficient measures of internal neuron importance. arXiv preprint arXiv:1807.09946 (2018)
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: ICML, pp. 3319–3328. PMLR (2017)
Sundararajan, M., Taly, A., Yan, Q.: Gradients of counterfactuals. ICLR (2017)
Wang, X., Wang, W., Cao, Y., Shen, C., Huang, T.: Images speak in images: a generalist painter for in-context visual learning. In: CVPR, pp. 6830–6839 (2023)
Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., Li, H.: Uformer: a general u-shaped transformer for image restoration. In: CVPR, pp. 17683–17693 (2022)
Wu, R.Q., Duan, Z.P., Guo, C.L., Chai, Z., Li, C.: Ridcp: revitalizing real image dehazing via high-quality codebook priors. In: CVPR, pp. 22282–22291 (2023)
Xie, L., Wang, X., Dong, C., Qi, Z., Shan, Y.: Finding discriminative filters for specific degradations in blind super-resolution. NeurIPS 34, 51–61 (2021)
Xie, Z., et al.: Simmim: a simple framework for masked image modeling. In: CVPR, pp. 9653–9663 (2022)
Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: CVPR, pp. 1357–1366 (2017)
Yang, W., Tan, R.T., Wang, S., Fang, Y., Liu, J.: Single image deraining: from model-based to data-driven and beyond. PAMI 43(11), 4059–4077 (2020)
Yang, W., Wang, W., Huang, H., Wang, S., Liu, J.: Sparse gradient regularized deep retinex network for robust low-light image enhancement. TIP 30, 2072–2086 (2021)
Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H.: Restormer: Efficient transformer for high-resolution image restoration. In: CVPR, pp. 5728–5739 (2022)
Zhang, C., Zhu, Y., Yan, Q., Sun, J., Zhang, Y.: All-in-one multi-degradation image restoration network via hierarchical degradation representation. In: ACMMM, pp. 2285–2293 (2023)
Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. In: CVPR, pp. 695–704 (2018)
Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. TCSVT 30(11), 3943–3956 (2019)
Zhang, J., et al.: Ingredient-oriented multi-degradation learning for image restoration. In: CVPR, pp. 5825–5835 (2023)
Zheng, N., et al.: Empowering low-light image enhancer through customized learnable priors. In: ICCV, pp. 12559–12569 (2023)
Zhu, Y., et al.: Learning weather-general and weather-specific features for image restoration under multiple adverse weather conditions. In: CVPR, pp. 21747–21758 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Qin, CJ. et al. (2025). Restore Anything with Masks: Leveraging Mask Image Modeling for Blind All-in-One Image Restoration. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15103. Springer, Cham. https://doi.org/10.1007/978-3-031-72995-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-72995-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72994-2
Online ISBN: 978-3-031-72995-9
eBook Packages: Computer ScienceComputer Science (R0)