Skip to main content
SpringerLink
Log in

Two-Image Approach to Reflection Removal with Deep Learning

  • Conference paper
  • First Online:
Computer Vision and Image Processing (CVIP 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1377))

Included in the following conference series:

Abstract

Reflection in an image is not always desirable due to the loss of information. Conventional methods to remove reflection are based on priors that require certain conditions to be fulfilled. Recent advancements of deep learning in many fields have revolutionized these traditional approaches. Using input images more than one reduces the ill-posedness of the problem statement. Standard assumption in numerous methods assumes background is stationary and only reflection layer is varying. However, images at different angles have slightly different backgrounds. Considering this a new dataset is created where both reflection and background layer is varying. In this paper, a two-image based method with an end to end mapping between the observed images and background is presented. The key feature is the practicability of the method, wherein a sequence of images at slightly different angles can easily be captured using modern dual camera mobile devices. A combination of feature loss with MSE maintains the content and quality of the resultant image.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Levin, A., Weiss, Y.: User assisted separation of reflections from a single image using a sparsity prior. IEEE Trans. Pattern Anal. Mach. Intell. 29(9), 1647–1654 (2007). https://doi.org/10.1109/TPAMI.2007.1106

    Article  Google Scholar 

  2. Li, Y., Brown, M-S.: Single image layer separation using relative smoothness. In: IEEE Conference on Computer Vision and Pattern Recognition, Columbus, pp. 2752–2759 (2014). https://doi.org/10.1109/CVPR.2014.346

  3. Shih, Y., Krishnan, D., Durand, F., Freeman, W.: Reflection removal using ghosting cues. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, pp. 3193–3201 (2015). https://doi.org/10.1109/CVPR.2015.7298939

  4. Tao, M.W., Hadap, S., Malik, J., Ramamoorthi, R.: Depth from combining defocus and correspondence using light-field cameras. In: 2013 IEEE International Conference on Computer Vision, Sydney, pp. 673–680 (2013). https://doi.org/10.1109/ICCV.2013.89

  5. Wan, R., Shi, B., Tan, A.H., Kot, A.C.: Depth of field guided reflection removal. In: Paper presented at the meeting of the ICIP), Phoenix, AZ, pp. 21–25 (2016). https://doi.org/10.1109/ICIP.2016.7532311

  6. Fergus, R., Singh, B., Hertzmann, A., Roweis, S., Freeman, W.: Removing camera shake from a single photograph. ACM Trans. Graph. 25, 787–794 (2006). https://doi.org/10.1145/1179352.1141956

    Article  MATH  Google Scholar 

  7. Fan, Q., Yang, J., Hua, G., Chen, B., Wipf, D.: A generic deep architecture for single image reflection removal and image smoothing. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice. Italy (2017). https://doi.org/10.1109/ICCV.2017.351

  8. Kong, N., Tai, Y.-W., Shin, J.S.: A physically-based approach to reflection separation: from physical modeling to constrained optimization. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 209–221 (2014). https://doi.org/10.1109/TPAMI.2013.45

    Article  Google Scholar 

  9. Wolff, L-B.: Using polarization to separate reflection components. In: Proceedings CVPR 1989: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA¸pp. 363–369 (1989). https://doi.org/10.1109/CVPR.1989.37873

  10. Li, Y., Brown, S-M.: Exploiting reflection change for automatic reflection removal. In: IEEE International Conference on Computer Vision, Sydney, NSW, pp. 2432–2439 (2013). https://doi.org/10.1109/ICCV.2013.302

  11. Gai, K., Shi, Z., Zhang, C.: Blind Separation of superimposed moving images using image statistics. IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 19–32 (2012). https://doi.org/10.1109/TPAMI.2011.87

    Article  Google Scholar 

  12. Schechner, Y-Y., Shamir, J., Kiryati, N.: Polarization-based decorrelation of transparent layers: the inclination angle of an invisible surface. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, Greece, vol. 2, pp. 814–819 (1999). https://doi.org/10.1109/ICCV.1999.790305

  13. Chi, Z., Wu, X., Shu, X., Gu, J.: Single Image Reflection Removal Using Deep Encoder-Decoder Network. CoRR (2018). arXiv preprint arXiv:1802.00094v1

  14. Lee, D., Yang, M-H., Oh, S.: Generative Single Image Reflection Separation (2018). arXiv preprint arXiv:1801.04102

  15. Kuanar, S., Rao, K., Mahapatra, D., Bilas, M.: Night Time Haze and Glow Removal using Deep Dilated Convolutional Network (2019). arXiv preprint arXiv:1902.00855v1

  16. Yang, J., Gong, D., Liu, L., Shi, Q.: Seeing deeply and bidirectionally: a deep learning approach for single image reflection removal. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 675–691. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_40

    Chapter  Google Scholar 

  17. Jin, M., Süsstrunk, S., Favaro, P.: Learning to see through reflections. In: IEEE International Conference on Computational Photography (ICCP), Pittsburgh, PA, p. 12 (2018). https://doi.org/10.1109/ICCPHOT.2018.8368464

  18. Wan, R., Shi, B., Duan, L.Y., Tan, A.-H., Kot, A.C.: CRRN: multi-scale guided concurrent reflection removal network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, pp. 4777–4785 (2018). https://doi.org/10.1109/CVPR.2018.00502

  19. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  20. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  21. Zhang, X., Ng, R., Chen, Q.: Single image reflection separation with perceptual losses. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, pp. 4786–4794 (2018). https://doi.org/10.1109/CVPR.2018.00503

  22. Dumoulin, V., Visin, F.: A guide to convolution arithmetic for deep learning (2016). arXiv preprint arXiv:1603.07285v2

  23. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning, Lille, France, vol. 37, pp. 448–456 (2015). arXiv preprint arXiv:1502.03167v3

  24. Maas, A.L., Hannun, A.-Y., Ng, A.-Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of the International Conference on Machine Learning (ICML), vol. 30, no. 1, p. 3 (2013). arXiv preprint arXiv:1804.02763v1

  25. Chen, Q., Koltun, V.: Photographic image synthesis with cascaded refinement networks. In: IEEE International Conference on Computer Vision (ICCV), Venice, pp. 1520–1529 (2017). https://doi.org/10.1109/ICCV.2017.168

  26. Ledig, C., et al.: In photo-realistic single image super-resolution using a generative adversarial network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 105–114 (2017). https://doi.org/10.1109/CVPR.2017.19

  27. Gatys, L.-A., Ecker, A.-S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 1, pp. 262–270 (2015). https://doi.org/10.5555/2969239.2969269

  28. Gatys, L.-A., Ecker, A.-S., Bethge, M.: A neural algorithm of artistic style (2015). arXiv preprint arXiv:1508.06576

  29. Chang, Y., Jung, C.: Single image reflection removal using convolutional neural networks. IEEE Trans. Image Process. 28(4), 1954–1966 (2019). https://doi.org/10.1109/TIP.2018.2880088

    Article  MathSciNet  Google Scholar 

  30. Arvanitopoulos, N., Achanta, R., Süsstrunk, S.: Single image reflection suppression. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 1752–1760 (2017). https://doi.org/10.1109/CVPR.2017.190

  31. Wei, K., Yang, J., Fu, Y., Wipf, D., Huang, H.: Single image reflection removal exploiting misaligned training data and network enhancements. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp. 8170–8179 (2019). https://doi.org/10.1109/CVPR.2019.00837

  32. Li, C., Yang, Y., He, K., Lin, S., Hopcroft, J.: Single Image Reflection Removal through Cascaded Refinement (2019). arXiv preprint arXiv:1911.06634v2

Download references

Author information

Authors and Affiliations

  1. IGDTUW, Kashmere Gate, New Delhi, 110006, India

    Rashmi Chaurasiya & Dinesh Ganotra

Authors
  1. Rashmi Chaurasiya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dinesh Ganotra
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Indian Institute of Information Technology Allahabad, Prayagraj, India

    Satish Kumar Singh

  2. Indian Institute of Technology Roorkee, Roorkee, India

    Partha Roy

  3. Indian Institute of Technology Roorkee, Roorkee, India

    Balasubramanian Raman

  4. Indian Institute of Information Technology Allahabad, Prayagraj, India

    P. Nagabhushan

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chaurasiya, R., Ganotra, D. (2021). Two-Image Approach to Reflection Removal with Deep Learning. In: Singh, S.K., Roy, P., Raman, B., Nagabhushan, P. (eds) Computer Vision and Image Processing. CVIP 2020. Communications in Computer and Information Science, vol 1377. Springer, Singapore. https://doi.org/10.1007/978-981-16-1092-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-1092-9_6

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-1091-2

  • Online ISBN: 978-981-16-1092-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation