Skip to main content

Advertisement

Log in

Contrastive feature decomposition for single image layer separation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The key challenge of image layer separation stems from recognizing different components in a single image. Typical methods optimize the modeling of different components by performing low-level supervision on the separated image to minimize its per-pixel difference from the groundtruth, which relies on substantial training samples to learn diverse components robustly and avoid overfitting spurious coupled patterns. In this work, we perform supervision on the contrastive distribution between the predicted separated images. Specifically, our proposed method separates components in parallel and seeks to maximize the distribution consistency between the separated components’ contrast and their corresponding groundtruth contrast in the latent space. Such supervision pushes the model to focus on contrastive modeling between different components of the input image. Besides, the learned latent representations of different components directly guide the weight optimization of convolution kernels in the decoder, which achieves more comprehensive separation than traditional skip connection while reconstructing target images by the decoder. We validate the effectiveness and generalization of our CFDNet on two single image layer separation tasks, including image reflection separation and intrinsic image decomposition. Extensive experiments demonstrate that our CFDNet consistently outperforms other state-of-the-art methods specifically designed for either of image separation applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Data availability

The data generated and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Fan Q, Yang J, Hua G, Chen B, Wipf D (2017) A generic deep architecture for single image reflection removal and image smoothing. In: Proceedings of the IEEE international conference on computer vision, pp 3238–3247

  2. Levin A, Weiss Y (2007) User assisted separation of reflections from a single image using a sparsity prior. IEEE Trans Pattern Anal Mach Intell 29(9):1647–1654

    Article  Google Scholar 

  3. Wan R, Shi B, Hwee TA, Kot AC (2016) Depth of field guided reflection removal. In: 2016 IEEE international conference on image processing (ICIP). IEEE, pp 21–25

  4. Yan Q, Xu Y, Yang X, Nguyen T (2014) Separation of weak reflection from a single superimposed image. IEEE Signal Process Lett 21(10):1173–1176

    Article  Google Scholar 

  5. Arvanitopoulos N, Achanta R, Susstrunk S (2017) Single image reflection suppression. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4498–4506

  6. Li C, Yang Y, He K, Lin S, Hopcroft JE (2020) Single image reflection removal through cascaded refinement. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3565–3574

  7. Wen Q, Tan Y, Qin J, Liu W, Han G, He S (2019) Single image reflection removal beyond linearity. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3771–3779

  8. Wei K, Yang J, Fu Y, Wipf D, Huang H (2019) Single image reflection removal exploiting misaligned training data and network enhancements. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8178–8187

  9. Yang J, Gong D, Liu L, Shi Q (2018) Seeing deeply and bidirectionally: a deep learning approach for single image reflection removal. In: Proceedings of the European conference on computer vision, pp 654–669

  10. Feng X, Pei W, Jia Z, Chen F, Zhang D, Lu G (2021) Deep-masking generative network: a unified framework for background restoration from superimposed images. IEEE Trans Image Process 30:4867–4882

    Article  Google Scholar 

  11. Zou Z, Lei S, Shi T, Shi Z, Ye J (2020) Deep adversarial decomposition: a unified framework for separating superimposed images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12806–12816

  12. Feng X, Ji H, Jiang B, Pei W, Chen F, Lu G (2021) Contrastive feature decomposition for image reflection removal. In: 2021 IEEE international conference on multimedia and expo. IEEE, pp 1–6

  13. Wan R, Shi B, Tan A-H, Kot AC (2017) Sparsity based reflection removal using external patch search. In: 2017 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1500–1505

  14. Ma S, Zhang H, Miao Z (2021) Blind source separation for the analysis sparse model. Neural Comput Appl 33(14):8543–8553

    Article  Google Scholar 

  15. Li C, He W, Liao N, Gong J, Hou S, Guo B (2022) Superpixels with contour adherence via label expansion for image decomposition. Neural Comput Appl 34(19):16223–16237

    Article  Google Scholar 

  16. Shen L, Tan P, Lin S (2008) Intrinsic image decomposition with non-local texture cues. In: 2008 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–7

  17. Chen Q, Koltun V (2013) A simple model for intrinsic image decomposition with depth cues. In: Proceedings of the IEEE international conference on computer vision, pp 241–248

  18. Shih Y, Krishnan D, Durand F, Freeman WT (2015) Reflection removal using ghosting cues. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3193–3201

  19. Huang Y, Quan Y, Xu Y, Xu R, Ji H (2019) Removing reflection from a single image with ghosting effect. IEEE Trans Comput Imaging 6:34–45

    Article  Google Scholar 

  20. Yang, Y, Ma W, Zheng Y, Cai J-F, Xu W (2019) Fast single image reflection suppression via convex optimization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8141–8149

  21. Zhang X, Ng R, Chen Q (2018) Single image reflection separation with perceptual losses. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4786–4794

  22. Fan Q, Yang J, Hua G, Chen B, Wipf D (2018) Revisiting deep intrinsic image decompositions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8944–8952

  23. Wang C, Xu D, Wan R, He B, Shi B, Duan L-Y (2022) Background scene recovery from an image looking through colored glass. IEEE Trans Multimedia

  24. Zhang H, Xu X, He H, He S, Han G, Qin J, Wu D (2019) Fast user-guided single image reflection removal via edge-aware cascaded networks. IEEE Trans Multimedia 22(8):2012–2023

    Article  Google Scholar 

  25. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30

  26. Ji H, Feng X, Pei W, Li J, Lu G (2021) U2-former: A nested u-shaped transformer for image restoration. arXiv preprint arXiv:2112.02279

  27. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Proceedings of the 27th international conference on neural information processing systems, pp 2672–2680

  28. Xiao J, Zhang S, Yao Y, Wang Z, Zhang Y, Wang Y-F (2022) Generative adversarial network with hybrid attention and compromised normalization for multi-scene image conversion. Neural Comput Appl 34(9):7209–7225

    Article  Google Scholar 

  29. Huang G, Jafari AH (2021) Enhanced balancing gan: minority-class image generation. Neural Comput Appl:1–10

  30. Ma D, Wan R, Shi B, Kot AC, Duan L-Y (2019) Learning to jointly generate and separate reflections. In: Proceedings of the IEEE international conference on computer vision, pp 2444–2452

  31. Zhang L, Lu Y, Li J, Chen F, Lu G, Zhang D (2023) Deep adaptive hiding network for image hiding using attentive frequency extraction and gradual depth extraction. Neural Comput Appl:1–19

  32. Wu Z, Zhuang C, Shi J, Guo J, Xiao J, Zhang X, Yan D-M (2021) Single-image specular highlight removal via real-world dataset construction. IEEE Trans Multimedia 24:3782–3793

    Article  Google Scholar 

  33. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  34. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  35. Gatys L, Ecker AS, Bethge M (2015) Texture synthesis using convolutional neural networks. Adv Neural Inf Process Syst 28

  36. Hui Z, Gao X, Yang Y, Wang X (2019) Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th Acm international conference on multimedia, pp 2024–2032

  37. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  38. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  39. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision. Springer, pp 694–711

  40. Jolicoeur-Martineau A (2018) The relativistic discriminator: a key element missing from standard gan. arXiv preprint arXiv:1807.00734

  41. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  42. Hu Q, Guo X (2021) Trash or treasure? an interactive dual-stream strategy for single image reflection separation. Adv Neural Inf Process Syst 34:24683–24694

    Google Scholar 

  43. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  44. Wan R, Shi B, Duan L-Y, Tan A-H, Kot AC (2017) Benchmarking single-image reflection removal algorithms. In: Proceedings of the IEEE international conference on computer vision, pp 3922–3930

  45. Dong Z, Xu K, Yang Y, Bao H, Xu W, Lau RW (2021) Location-aware single image reflection removal. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 5017–5026

  46. Zhang Y-N, Shen L, Li Q (2022) Content and gradient model-driven deep network for single image reflection removal. In: Proceedings of the 30th ACM international conference on multimedia, pp 6802–6812

  47. Das P, Karaoglu S, Gevers T (2022) Pie-net: Photometric invariant edge guided network for intrinsic image decomposition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 19790–19799

  48. Butler DJ, Wulff J, Stanley GB, Black MJ (2012) A naturalistic open source movie for optical flow evaluation. In: Computer Vision–ECCV 2012: 12th European conference on computer vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12. Springer, pp 611–625

  49. Chang AX, Funkhouser T, Guibas L, Hanrahan P, Huang Q, Li Z, Savarese S, Savva M, Song S, Su H, et al (2015) Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012

  50. Grosse R, Johnson MK, Adelson EH, Freeman WT (2009) Ground truth dataset and baseline evaluations for intrinsic image algorithms. In: 2009 IEEE 12th international conference on computer vision, pp 2335–2342 . IEEE

  51. Grosse R, Johnson MK, Adelson EH, Freeman WT (2009) Ground truth dataset and baseline evaluations for intrinsic image algorithms. In: 2009 IEEE 12th international conference on computer vision, pp 2335–2342 . IEEE

  52. Barron JT, Malik J (2014) Shape, illumination, and reflectance from shading. IEEE Trans Pattern Anal Mach Intell 37(8):1670–1687

    Article  Google Scholar 

  53. Li Z, Snavely N (2018) Learning intrinsic image decomposition from watching the world. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9039–9048

  54. Liu Y, Li Y, You S, Lu F (2020) Unsupervised learning for intrinsic image decomposition from a single image. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3248–3257

  55. Li Y, Brown MS (2014) Single image layer separation using relative smoothness. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2752–2759

  56. Ma W-C, Chu H, Zhou B, Urtasun R, Torralba A (2018) Single image intrinsic decomposition without a single intrinsic image. In: Proceedings of the European conference on computer vision (ECCV), pp 201–217

  57. Liu Y, Lu F (2020) Separate in latent space: Unsupervised single image layer separation. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 11661–11668

Download references

Acknowledgements

This work was supported in part by the NSFC fund (NO. 62176077), in part by the Shenzhen Key Technical Project (NO. 2022N001, 2020N046), in part by the Guangdong International Science and Technology Cooperation Project (NO. 20220505), in part by the Shenzhen Fundamental Research Fund (NO. JCYJ20210324132210025), and in part by the Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies (NO. 2022B1212010005).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Guangming Lu.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, X., Li, J., Ji, H. et al. Contrastive feature decomposition for single image layer separation. Neural Comput & Applic 36, 8039–8053 (2024). https://doi.org/10.1007/s00521-024-09478-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09478-4

Keywords