Skip to main content
Springer Nature Link
Log in

Dehazing Cost Volume for Deep Multi-view Stereo in Scattering Media

  • Conference paper
  • First Online:
Computer Vision – ACCV 2020 (ACCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12622))

Included in the following conference series:

  • 1152 Accesses

Abstract

We propose a learning-based multi-view stereo (MVS) method in scattering media such as fog or smoke with a novel cost volume, called the dehazing cost volume. An image captured in scattering media degrades due to light scattering and attenuation caused by suspended particles. This degradation depends on scene depth; thus it is difficult for MVS to evaluate photometric consistency because the depth is unknown before three-dimensional reconstruction. Our dehazing cost volume can solve this chicken-and-egg problem of depth and scattering estimation by computing the scattering effect using swept planes in the cost volume. Experimental results on synthesized hazy images indicate the effectiveness of our dehazing cost volume against the ordinary cost volume regarding scattering media. We also demonstrated the applicability of our dehazing cost volume to real foggy scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Furukawa, Y., Hernández, C.: Multi-view stereo: a tutorial. Found. Trends® Comput. Graph. Vis. 9, 1–148 (2015)

    Article  Google Scholar 

  2. Yao, Y., Luo, Z., Li, S., Fang, T., Quan, L.: MVSNet: depth inference for unstructured multi-view stereo. In: The European Conference on Computer Vision (ECCV), pp. 767–783 (2018)

    Google Scholar 

  3. Huang, P., Matzen, K., Kopf, J., Ahuja, N., Huang, J.: DeepMVS: learning multi-view stereopsis. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2821–2830 (2018)

    Google Scholar 

  4. Im, S., Jeon, H., Lin, S., Kweon, I.S.: DPSNet: end-to-end deep plane sweep stereo (2019)

    Google Scholar 

  5. Wang, K., Shen, S.: Mvdepthnet: real-time multiview depth estimation neural network. In: International Conference on 3D Vision (3DV), pp. 248–257 (2018)

    Google Scholar 

  6. Collins, R.T.: A space-sweep approach to true multi-image matching. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 358–363 (1996)

    Google Scholar 

  7. Zheng, E., Dunn, E., Jojic, V., Frahm, J.: PatchMatch based joint view selection and depthmap estimation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1510–1517 (2014)

    Google Scholar 

  8. Schönberger, J.L., Zheng, E., Pollefeys, M., Frahm, J.: Pixelwise view selection for unstructured multi-view stereo. In: The European Conference on Computer Vision (ECCV), pp. 501–518 (2016)

    Google Scholar 

  9. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33, 2341–2353 (2011)

    Article  Google Scholar 

  10. Nishino, K., Kratz, L., Lombardi, S.: Bayesian defogging. Int. J. Comput. Vision 98, 263–278 (2012)

    Article  MathSciNet  Google Scholar 

  11. Fattal, R.: Dehazing using color-lines. ACM Trans. Graph. (TOG) 34, 1–14 (2014)

    Article  Google Scholar 

  12. Berman, D., Treibitz, T., Avidan, S.: Non-local image dehazing. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1674–1682 (2016)

    Google Scholar 

  13. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: DehazeNet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25, 5187–5198 (2016)

    Article  MathSciNet  Google Scholar 

  14. Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.-H.: Single image dehazing via multi-scale convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 154–169. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_10

    Chapter  Google Scholar 

  15. Zhang, H., Patel, V.M.: Densely connected pyramid dehazing network. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3194–3203 (2018)

    Google Scholar 

  16. Yang, D., Sun, J.: Proximal Dehaze-Net: a prior learning-based deep network for single image dehazing. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 729–746. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_43

    Chapter  Google Scholar 

  17. Liu, Y., Pan, J., Ren, J., Su, Z.: Learning deep priors for image dehazing. In: The IEEE International Conference on Computer Vision (ICCV), pp. 2492–2500 (2019)

    Google Scholar 

  18. Qin, X., Wang, Z., Bai, Y., Xie, X., Jia, H.: FFA-Net: feature fusion attention network for single image dehazing. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020), pp. 11908–11915 (2020)

    Google Scholar 

  19. Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: AOD-Net: all-in-one dehazing network. In: The IEEE International Conference on Computer Vision (ICCV), pp. 4770–4778 (2017)

    Google Scholar 

  20. Narasimhan, S.G., Nayar, S.K., Sun, B., Koppal, S.J.: Structured light in scattering media. In: Proceedings of the Tenth IEEE International Conference on Computer Vision, vol. I, pp. 420–427 (2005)

    Google Scholar 

  21. Tsiotsios, C., Angelopoulou, M.E., Kim, T., Davison, A.J.: Backscatter compensated photometric stereo with 3 sources. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2259–2266 (2014)

    Google Scholar 

  22. Murez, Z., Treibitz, T., Ramamoorthi, R., Kriegman, D.J.: Photometric stereo in a scattering medium. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1880–1891 (2017)

    Article  Google Scholar 

  23. Fujimura, Y., Iiyama, M., Hashimoto, A., Minoh, M.: Photometric stereo in participating media considering shape-dependent forward scatter. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7445–7453 (2018)

    Google Scholar 

  24. Heide, F., Xiao, L., Kolb, A., Hullin, M.B., Heidrich, W.: Imaging in scattering media using correlation image sensors and sparse convolutional coding. Opt. Express 22, 26338–26350 (2014)

    Article  Google Scholar 

  25. Satat, G., Tancik, M., Rasker, R.: Towards photography through realistic fog. In: The IEEE International Conference on Computational Photography (ICCP), pp. 1–10 (2018)

    Google Scholar 

  26. Wang, J., Bartels, J., Whittaker, W., Sankaranarayanan, A.C., Narasimhan, S.G.: Programmable triangulation light curtains. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 20–35. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_2

    Chapter  Google Scholar 

  27. Caraffa, L., Tarel, J.: Stereo reconstruction and contrast restoration in daytime fog. In: Asian Conference on Computer Vision (ACCV), pp. 13–25 (2012)

    Google Scholar 

  28. Song, T., Kim, Y., Oh, C., Sohn, K.: Deep network for simultaneous stereo matching and dehazing. In: British Machine Vision Conference (BMVC) (2018)

    Google Scholar 

  29. Li, Z., Tan, P., Tang, R.T., Zou, D., Zhou, S.Z., Cheong, L.: Simultaneous video defogging and stereo reconstruction. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4988–4997 (2015)

    Google Scholar 

  30. Tan, R.T.: Visibility in bad weather from a single image. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2008)

    Google Scholar 

  31. Berman, D., Treibitz, T., Avidan, S.: Air-light estimation using haze-lines. In: The IEEE International Conference on Computational Photography (ICCP) (2017)

    Google Scholar 

  32. Ummenhofer, B., et al.: DeMoN: depth and motion network for learning monocular stereo. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5038–5047 (2017)

    Google Scholar 

  33. Xiao, J., Owens, A., Torralba, A.: SUN3D: a database of big spaces reconstructed using SFM and object labels. In: The IEEE International Conference on Computer Vision (ICCV), pp. 1625–1632 (2013)

    Google Scholar 

  34. Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: The International Conference on Intelligent Robot Systems (IROS) (2012)

    Google Scholar 

  35. Fuhrmann, S., Langguth, F., Goesel, M.: MVE: a multi-view reconstruction environment. Eurographics Workshop on Graphics and Cultural Heritage, pp. 11–18 (2014)

    Google Scholar 

  36. Chang, A.X., et al.: ShapeNet: an information-rich 3D model repository. arXiv:1512.03012 (2015)

  37. Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. In: Twenty-eighth Conference on Neural Information Processing Systems (NeurIPS) (2014)

    Google Scholar 

  38. Tateno, K., Tombari, F., Laina, I., Navab, N.: CNN-SLAM: real-time dense monocular slam with learned depth prediction. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6243–6252 (2017)

    Google Scholar 

  39. Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4104–4113 (2016)

    Google Scholar 

  40. Gur, S., Wolf, L.: Single image depth estimation trained via depth from defocus cues. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7683–7692 (2019)

    Google Scholar 

  41. Maximov, M., Galim, K., Leal-Taixe, L.: Focus on defocus: bridging the synthetic to real domain gap for depth estimation. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1071–1080 (2020)

    Google Scholar 

Download references

Acknowledgements

This work was supported by JSPS KAKENHI Grant Number 18H03263 and 19J10003.

Author information

Authors and Affiliations

  1. Kyoto University, Kyoto-shi, 606-8501, Japan

    Yuki Fujimura, Motoharu Sonogashira & Masaaki Iiyama

Authors
  1. Yuki Fujimura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Motoharu Sonogashira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Masaaki Iiyama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuki Fujimura .

Editor information

Editors and Affiliations

  1. Waseda University, Tokyo, Japan

    Hiroshi Ishikawa

  2. Institute of Automation of Chinese Academy of Sciences, Beijing, China

    Cheng-Lin Liu

  3. Czech Technical University in Prague, Prague, Czech Republic

    Tomas Pajdla

  4. University of Pennsylvania, Philadelphia, PA, USA

    Jianbo Shi

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fujimura, Y., Sonogashira, M., Iiyama, M. (2021). Dehazing Cost Volume for Deep Multi-view Stereo in Scattering Media. In: Ishikawa, H., Liu, CL., Pajdla, T., Shi, J. (eds) Computer Vision – ACCV 2020. ACCV 2020. Lecture Notes in Computer Science(), vol 12622. Springer, Cham. https://doi.org/10.1007/978-3-030-69525-5_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69525-5_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69524-8

  • Online ISBN: 978-3-030-69525-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics