skip to main content
10.1145/3633637.3633700acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccprConference Proceedingsconference-collections
research-article

Single Image Reflection Removal with Diffusion Model

Published: 28 February 2024 Publication History

Abstract

The removal of undesirable reflections from workpiece surface images captured under industrial conditions is a challenging task in image enhancement for the industry. While existing reflection removal methods have shown promising results in natural situations, they often struggle with image reflections encountered in industrial settings. This is mainly due to the evaluation of these methods on synthetic or unrepresentative datasets. To address this issue and advance research in this area, we introduce a new dataset called the Workpiece Surface Image Reflection Dataset (WSIRD). This dataset comprises 1000 image pairs captured under various imaging conditions in industrial environments. With WISRD, we aim to facilitate the development of new techniques for reflection removal specifically tailored to workpiece surface images. In line with this, we propose a novel diffusion-based model called ReflectDiffusion. This model leverages a reflection image as a constraint and employs a guided denoising process during the inference stage to remove reflections from workpiece surface images. Through comprehensive experiments, we demonstrate that our proposed method effectively eliminates reflections in workpiece surface images.

References

[1]
Nikolaos Arvanitopoulos, Radhakrishna Achanta, and Sabine Süsstrunk. 2017. Single Image Reflection Suppression. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1752–1760.
[2]
Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. 2021. ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 14347–14356.
[3]
Prafulla Dhariwal and Alexander Quinn Nichol. 2021. Diffusion Models Beat GANs on Image Synthesis. In Advances in Neural Information Processing Systems, A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (Eds.).
[4]
Yilun Du and Igor Mordatch. 2019. Implicit Generation and Generalization in Energy-Based Models. (2019).
[5]
Q. Fan, J. Yang, G. Hua, B. Chen, and D. Wipf. 2017. A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing. In 2017 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, USA, 3258–3267.
[6]
Kun Gai, Zhenwei Shi, and Changshui Zhang. 2012. Blind Separation of Superimposed Moving Images Using Image Statistics. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 1 (2012), 19–32.
[7]
Xiaojie Guo, Xiaochun Cao, and Yi Ma. 2014. Robust Separation of Reflection from Multiple Images. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2195–2202. https://doi.org/10.1109/CVPR.2014.281
[8]
Byeong-Ju Han and Jae-Young Sim. 2017. Reflection Removal Using Low-Rank Matrix Completion. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3872–3880.
[9]
Geoffrey E. Hinton. 2002. Training Products of Experts by Minimizing Contrastive Divergence. Neural Comput. 14, 8 (aug 2002), 1771–1800.
[10]
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., 6840–6851.
[11]
Jonathan Ho, Chitwan Saharia, William Chan, David J. Fleet, Mohammad Norouzi, and Tim Salimans. 2022. Cascaded Diffusion Models for High Fidelity Image Generation. J. Mach. Learn. Res. 23 (2022), 47:1–47:33.
[12]
Chenyang Lei and Qifeng Chen. 2021. Robust Reflection Removal with Reflection-free Flash-only Cues. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 14806–14815.
[13]
Anat Levin, Assaf Zomet, and Yair Weiss. 2002. Learning to Perceive Transparency from the Statistics of Natural Scenes. In Advances in Neural Information Processing Systems, S. Becker, S. Thrun, and K. Obermayer (Eds.). Vol. 15. MIT Press.
[14]
A. Levin, A. Zomet, and Y. Weiss. 2004. Separating reflections from a single image using local features. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., Vol. 1. I–I.
[15]
Chao Li, Yixiao Yang, Kun He, Stephen Lin, and John E. Hopcroft. 2020. Single Image Reflection Removal Through Cascaded Refinement. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3562–3571.
[16]
Yu Li and Michael S. Brown. 2013. Exploiting Reflection Change for Automatic Reflection Removal. In 2013 IEEE International Conference on Computer Vision. 2432–2439.
[17]
Yu Li and Michael S. Brown. 2014. Single Image Layer Separation Using Relative Smoothness. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2752–2759.
[18]
Yu Li, Ming Liu, Yaling Yi, Qince Li, Dongwei Ren, and Wangmeng Zuo. 2023. Two-stage single image reflection removal with reflection-aware guidance. Applied Intelligence (03 2023), 1–16.
[19]
Chongyu Liu, Yuliang Liu, Lianwen Jin, Shuaitao Zhang, Canjie Luo, and Yongpan Wang. 2020. EraseNet: End-to-End Text Removal in the Wild. IEEE Transactions on Image Processing 29 (2020), 8760–8775.
[20]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis with Latent Diffusion Models. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10674–10685.
[21]
Bernard Sarel and Michal Irani. 2004. Separating Transparent Layers through Layer Information Exchange. In Computer Vision - ECCV 2004, Tomás Pajdla and Jiří Matas (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 328–341.
[22]
Junghoon Seo, Taewon Yoon, Jinwoo Kim, and Kin Choong Yow. 2017. One-to-one Example-based Automatic Image Coloring Using Deep Convolutional Generative Adversarial Network. Journal of Advances in Information Technology (2017), 80–85.
[23]
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 37), Francis Bach and David Blei (Eds.). PMLR, Lille, France, 2256–2265.
[24]
Yang Song and Stefano Ermon. 2019. Generative Modeling by Estimating Gradients of the Data Distribution. In Advances in Neural Information Processing Systems, Vol. 32. Curran Associates, Inc.
[25]
Ofer Springer and Yair Weiss. 2017. Reflection separation using guided annotation. In 2017 IEEE International Conference on Image Processing (ICIP). 1192–1196.
[26]
Chao Sun, Shuaicheng Liu, Taotao Yang, Bing Zeng, Zhengning Wang, and Guanghui Liu. 2016. Automatic Reflection Removal Using Gradient Intensity and Motion Cues. Association for Computing Machinery, New York, NY, USA.
[27]
R. Szeliski, S. Avidan, and P. Anandan. 2000. Layer extraction from multiple images containing reflections and transparency. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662), Vol. 1. 246–253 vol.1.
[28]
Tijmen Tieleman and Geoffrey Hinton. 2009. Using Fast Weights to Improve Persistent Contrastive Divergence. In Proceedings of the 26th Annual International Conference on Machine Learning(ICML ’09). Association for Computing Machinery, 1033–1040.
[29]
Mehmet Ali Cagri Tuncer and Ali Cafer Gurbuz. 2012. Ground Reflection Removal in Compressive Sensing Ground Penetrating Radars. IEEE Geoscience and Remote Sensing Letters 9, 1 (2012), 23–27.
[30]
Pascal Vincent. 2011. A Connection Between Score Matching and Denoising Autoencoders. Neural Computation 23, 7 (2011), 1661–1674.
[31]
Renjie Wan, Boxin Shi, Tan Ah Hwee, and Alex C. Kot. 2016. Depth of field guided reflection removal. In 2016 IEEE International Conference on Image Processing (ICIP). 21–25.
[32]
Kaixuan Wei, Jiaolong Yang, Ying Fu, David Wipf, and Hua Huang. 2019. Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8170–8179.
[33]
Max Welling and Yee Whye Teh. 2011. Bayesian Learning via Stochastic Gradient Langevin Dynamics(ICML’11). Omnipress, 681–688.
[34]
Tianfan Xue, Michael Rubinstein, Ce Liu, and William T. Freeman. 2015. A Computational Approach for Obstruction-Free Photography. 34, 4 (2015).
[35]
Tao Yu, Zongyu Guo, Xin Jin, Shilin Wu, Zhibo Chen, Weiping Li, Zhizheng Zhang, and Sen Liu. 2020. Region Normalization for Image Inpainting. Proceedings of the AAAI Conference on Artificial Intelligence 34, 07 (apr 2020), 12733–12740.
[36]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In 2017 IEEE International Conference on Computer Vision (ICCV). 2242–2251.

Index Terms

  1. Single Image Reflection Removal with Diffusion Model

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICCPR '23: Proceedings of the 2023 12th International Conference on Computing and Pattern Recognition
    October 2023
    589 pages
    ISBN:9798400707988
    DOI:10.1145/3633637
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 February 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. denoising diffusion models
    2. machine vision
    3. reflection removal
    4. workpiece surface image

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICCPR 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 126
      Total Downloads
    • Downloads (Last 12 months)126
    • Downloads (Last 6 weeks)34
    Reflects downloads up to 19 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media