Abstract
The key challenge of image layer separation stems from recognizing different components in a single image. Typical methods optimize the modeling of different components by performing low-level supervision on the separated image to minimize its per-pixel difference from the groundtruth, which relies on substantial training samples to learn diverse components robustly and avoid overfitting spurious coupled patterns. In this work, we perform supervision on the contrastive distribution between the predicted separated images. Specifically, our proposed method separates components in parallel and seeks to maximize the distribution consistency between the separated components’ contrast and their corresponding groundtruth contrast in the latent space. Such supervision pushes the model to focus on contrastive modeling between different components of the input image. Besides, the learned latent representations of different components directly guide the weight optimization of convolution kernels in the decoder, which achieves more comprehensive separation than traditional skip connection while reconstructing target images by the decoder. We validate the effectiveness and generalization of our CFDNet on two single image layer separation tasks, including image reflection separation and intrinsic image decomposition. Extensive experiments demonstrate that our CFDNet consistently outperforms other state-of-the-art methods specifically designed for either of image separation applications.













Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.Data availability
The data generated and analyzed during the current study are available from the corresponding author on reasonable request.
References
Fan Q, Yang J, Hua G, Chen B, Wipf D (2017) A generic deep architecture for single image reflection removal and image smoothing. In: Proceedings of the IEEE international conference on computer vision, pp 3238–3247
Levin A, Weiss Y (2007) User assisted separation of reflections from a single image using a sparsity prior. IEEE Trans Pattern Anal Mach Intell 29(9):1647–1654
Wan R, Shi B, Hwee TA, Kot AC (2016) Depth of field guided reflection removal. In: 2016 IEEE international conference on image processing (ICIP). IEEE, pp 21–25
Yan Q, Xu Y, Yang X, Nguyen T (2014) Separation of weak reflection from a single superimposed image. IEEE Signal Process Lett 21(10):1173–1176
Arvanitopoulos N, Achanta R, Susstrunk S (2017) Single image reflection suppression. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4498–4506
Li C, Yang Y, He K, Lin S, Hopcroft JE (2020) Single image reflection removal through cascaded refinement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3565–3574
Wen Q, Tan Y, Qin J, Liu W, Han G, He S (2019) Single image reflection removal beyond linearity. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3771–3779
Wei K, Yang J, Fu Y, Wipf D, Huang H (2019) Single image reflection removal exploiting misaligned training data and network enhancements. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8178–8187
Yang J, Gong D, Liu L, Shi Q (2018) Seeing deeply and bidirectionally: a deep learning approach for single image reflection removal. In: Proceedings of the European conference on computer vision, pp 654–669
Feng X, Pei W, Jia Z, Chen F, Zhang D, Lu G (2021) Deep-masking generative network: a unified framework for background restoration from superimposed images. IEEE Trans Image Process 30:4867–4882
Zou Z, Lei S, Shi T, Shi Z, Ye J (2020) Deep adversarial decomposition: a unified framework for separating superimposed images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12806–12816
Feng X, Ji H, Jiang B, Pei W, Chen F, Lu G (2021) Contrastive feature decomposition for image reflection removal. In: 2021 IEEE international conference on multimedia and expo. IEEE, pp 1–6
Wan R, Shi B, Tan A-H, Kot AC (2017) Sparsity based reflection removal using external patch search. In: 2017 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1500–1505
Ma S, Zhang H, Miao Z (2021) Blind source separation for the analysis sparse model. Neural Comput Appl 33(14):8543–8553
Li C, He W, Liao N, Gong J, Hou S, Guo B (2022) Superpixels with contour adherence via label expansion for image decomposition. Neural Comput Appl 34(19):16223–16237
Shen L, Tan P, Lin S (2008) Intrinsic image decomposition with non-local texture cues. In: 2008 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–7
Chen Q, Koltun V (2013) A simple model for intrinsic image decomposition with depth cues. In: Proceedings of the IEEE international conference on computer vision, pp 241–248
Shih Y, Krishnan D, Durand F, Freeman WT (2015) Reflection removal using ghosting cues. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3193–3201
Huang Y, Quan Y, Xu Y, Xu R, Ji H (2019) Removing reflection from a single image with ghosting effect. IEEE Trans Comput Imaging 6:34–45
Yang, Y, Ma W, Zheng Y, Cai J-F, Xu W (2019) Fast single image reflection suppression via convex optimization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8141–8149
Zhang X, Ng R, Chen Q (2018) Single image reflection separation with perceptual losses. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4786–4794
Fan Q, Yang J, Hua G, Chen B, Wipf D (2018) Revisiting deep intrinsic image decompositions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8944–8952
Wang C, Xu D, Wan R, He B, Shi B, Duan L-Y (2022) Background scene recovery from an image looking through colored glass. IEEE Trans Multimedia
Zhang H, Xu X, He H, He S, Han G, Qin J, Wu D (2019) Fast user-guided single image reflection removal via edge-aware cascaded networks. IEEE Trans Multimedia 22(8):2012–2023
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30
Ji H, Feng X, Pei W, Li J, Lu G (2021) U2-former: A nested u-shaped transformer for image restoration. arXiv preprint arXiv:2112.02279
Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Proceedings of the 27th international conference on neural information processing systems, pp 2672–2680
Xiao J, Zhang S, Yao Y, Wang Z, Zhang Y, Wang Y-F (2022) Generative adversarial network with hybrid attention and compromised normalization for multi-scene image conversion. Neural Comput Appl 34(9):7209–7225
Huang G, Jafari AH (2021) Enhanced balancing gan: minority-class image generation. Neural Comput Appl:1–10
Ma D, Wan R, Shi B, Kot AC, Duan L-Y (2019) Learning to jointly generate and separate reflections. In: Proceedings of the IEEE international conference on computer vision, pp 2444–2452
Zhang L, Lu Y, Li J, Chen F, Lu G, Zhang D (2023) Deep adaptive hiding network for image hiding using attentive frequency extraction and gradual depth extraction. Neural Comput Appl:1–19
Wu Z, Zhuang C, Shi J, Guo J, Xiao J, Zhang X, Yan D-M (2021) Single-image specular highlight removal via real-world dataset construction. IEEE Trans Multimedia 24:3782–3793
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Gatys L, Ecker AS, Bethge M (2015) Texture synthesis using convolutional neural networks. Adv Neural Inf Process Syst 28
Hui Z, Gao X, Yang Y, Wang X (2019) Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th Acm international conference on multimedia, pp 2024–2032
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708
Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision. Springer, pp 694–711
Jolicoeur-Martineau A (2018) The relativistic discriminator: a key element missing from standard gan. arXiv preprint arXiv:1807.00734
Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134
Hu Q, Guo X (2021) Trash or treasure? an interactive dual-stream strategy for single image reflection separation. Adv Neural Inf Process Syst 34:24683–24694
Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980
Wan R, Shi B, Duan L-Y, Tan A-H, Kot AC (2017) Benchmarking single-image reflection removal algorithms. In: Proceedings of the IEEE international conference on computer vision, pp 3922–3930
Dong Z, Xu K, Yang Y, Bao H, Xu W, Lau RW (2021) Location-aware single image reflection removal. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 5017–5026
Zhang Y-N, Shen L, Li Q (2022) Content and gradient model-driven deep network for single image reflection removal. In: Proceedings of the 30th ACM international conference on multimedia, pp 6802–6812
Das P, Karaoglu S, Gevers T (2022) Pie-net: Photometric invariant edge guided network for intrinsic image decomposition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 19790–19799
Butler DJ, Wulff J, Stanley GB, Black MJ (2012) A naturalistic open source movie for optical flow evaluation. In: Computer Vision–ECCV 2012: 12th European conference on computer vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12. Springer, pp 611–625
Chang AX, Funkhouser T, Guibas L, Hanrahan P, Huang Q, Li Z, Savarese S, Savva M, Song S, Su H, et al (2015) Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012
Grosse R, Johnson MK, Adelson EH, Freeman WT (2009) Ground truth dataset and baseline evaluations for intrinsic image algorithms. In: 2009 IEEE 12th international conference on computer vision, pp 2335–2342 . IEEE
Grosse R, Johnson MK, Adelson EH, Freeman WT (2009) Ground truth dataset and baseline evaluations for intrinsic image algorithms. In: 2009 IEEE 12th international conference on computer vision, pp 2335–2342 . IEEE
Barron JT, Malik J (2014) Shape, illumination, and reflectance from shading. IEEE Trans Pattern Anal Mach Intell 37(8):1670–1687
Li Z, Snavely N (2018) Learning intrinsic image decomposition from watching the world. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9039–9048
Liu Y, Li Y, You S, Lu F (2020) Unsupervised learning for intrinsic image decomposition from a single image. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3248–3257
Li Y, Brown MS (2014) Single image layer separation using relative smoothness. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2752–2759
Ma W-C, Chu H, Zhou B, Urtasun R, Torralba A (2018) Single image intrinsic decomposition without a single intrinsic image. In: Proceedings of the European conference on computer vision (ECCV), pp 201–217
Liu Y, Lu F (2020) Separate in latent space: Unsupervised single image layer separation. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 11661–11668
Acknowledgements
This work was supported in part by the NSFC fund (NO. 62176077), in part by the Shenzhen Key Technical Project (NO. 2022N001, 2020N046), in part by the Guangdong International Science and Technology Cooperation Project (NO. 20220505), in part by the Shenzhen Fundamental Research Fund (NO. JCYJ20210324132210025), and in part by the Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies (NO. 2022B1212010005).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Feng, X., Li, J., Ji, H. et al. Contrastive feature decomposition for single image layer separation. Neural Comput & Applic 36, 8039–8053 (2024). https://doi.org/10.1007/s00521-024-09478-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-024-09478-4