Skip to main content
Springer Nature Link
Log in

Bilateral Normal Integration

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13661))

Included in the following conference series:

  • 3978 Accesses

Abstract

This paper studies the discontinuity preservation problem in recovering a surface from its surface normal map. To model discontinuities, we introduce the assumption that the surface to be recovered is semi-smooth, i.e., the surface is one-sided differentiable (hence one-sided continuous) everywhere in the horizontal and vertical directions. Under the semi-smooth surface assumption, we propose a bilaterally weighted functional for discontinuity preserving normal integration. The key idea is to relatively weight the one-sided differentiability at each point’s two sides based on the definition of one-sided depth discontinuity. As a result, our method effectively preserves discontinuities and alleviates the under- or over-segmentation artifacts in the recovered surfaces compared to existing methods. Further, we unify the normal integration problem in the orthographic and perspective cases in a new way and show effective discontinuity preservation results in both cases (Source code is available at https://github.com/hoshino042/bilateral_normal_integration.).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    This requirement is stricter than jump discontinuity, which requires the one-sided limits exist but are unequal at a point’s two sides. Figure 2(b), (c), and (e) are jump discontinuity examples, but a semi-smooth surface allows only Fig. 2(b) and (c).

  2. 2.

    See experiments in the supplementary material.

  3. 3.

    The method we call robust estimator is called non-convex estimator in [25].

  4. 4.

    https://github.com/hoshino042/NormalIntegrationhttps://github.com/yqueau/normal_integration.

References

  1. Agrawal, A., Raskar, R., Chellappa, R.: What is the range of surface reconstructions from a gradient field? In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 578–591. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_45

    Chapter  Google Scholar 

  2. Badri, H., Yahia, H., Aboutajdine, D.: Robust surface reconstruction via triple sparsity. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 2283–2290 (2014)

    Google Scholar 

  3. Cao, X., Shi, B., Okura, F., Matsushita, Y.: Normal integration via inverse plane fitting with minimum point-to-plane distance. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 2382–2391 (2021)

    Google Scholar 

  4. Chabert, C.F., et al.: Relighting human locomotion with flowed reflectance fields. In: Proceedings of ACM SIGGRAPH (2006)

    Google Scholar 

  5. Deschaintre, V., Lin, Y., Ghosh, A.: Deep polarization imaging for 3D shape and SVBRDF acquisition. In: CVPR, pp. 15567–15576 (2021)

    Google Scholar 

  6. Durou, J.D., Aujol, J.F., Courteille, F.: Integrating the normal field of a surface in the presence of discontinuities. In: Workshops of the Computer Vision and Pattern Recognition (CVPRW) (2009)

    Google Scholar 

  7. Durou, J.D., Courteille, F.: Integration of a normal field without boundary condition. In: Workshops of the International Conference on Computer Vision (ICCVW) (2007)

    Google Scholar 

  8. Harker, M., O’Leary, P.: Least squares surface reconstruction from measured gradient fields. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2008)

    Google Scholar 

  9. Harker, M., O’Leary, P.: Least squares surface reconstruction from gradients: direct algebraic methods with spectral, Tikhonov, and constrained regularization. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2011)

    Google Scholar 

  10. Harker, M., O’leary, P.: Regularized reconstruction of a surface from its measured gradient field. J. Math. Imaging Vision 51, 46–70 (2015)

    Article  MathSciNet  Google Scholar 

  11. Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bureau Stand. 49(6), 409 (1952)

    Article  MathSciNet  Google Scholar 

  12. Holland, P.W., Welsch, R.E.: Robust regression using iteratively reweighted least-squares. Commun. Stat.-theory Methods 6(9), 813–827 (1977)

    Article  Google Scholar 

  13. Horn, B.K., Brooks, M.J.: The variational approach to shape from shading. In: Computer Vision, Graphics, and Image Processing(1986)

    Google Scholar 

  14. Horovitz, I., Kiryati, N.: Depth from gradient fields and control points: bias correction in photometric stereo. Image Vision Comput. 22, 681–694 (2004)

    Article  Google Scholar 

  15. Ikehata, S.: CNN-PS: CNN-based photometric stereo for general non-convex surfaces. In: Proceedings of European Conference on Computer Vision (ECCV), pp. 3–18 (2018)

    Google Scholar 

  16. Ikeuchi, K.: Constructing a depth map from images. Technical report, Massachusetts Inst of Tech Cambridge Artificial Intelligence Lab (1983)

    Google Scholar 

  17. Kadambi, A., Taamazyan, V., Shi, B., Raskar, R.: Polarized 3D: high-quality depth sensing with polarization cues. In: Proceedings of International Conference on Computer Vision (ICCV) (2015)

    Google Scholar 

  18. Karacali, B., Snyder, W.: Reconstructing discontinuous surfaces from a given gradient field using partial integrability. Comput. Vision Image Underst. 92, 78–111 (2003)

    Article  Google Scholar 

  19. Klette, R., Schluens, K.: Height data from gradient maps. In: Machine Vision Applications, Architectures, and Systems Integration V, vol. 2908, pp. 204–215. SPIE (1996)

    Google Scholar 

  20. Li, M., Zhou, Z., Wu, Z., Shi, B., Diao, C., Tan, P.: Multi-view photometric stereo: a robust solution and benchmark dataset for spatially varying isotropic materials. IEEE Trans. Image Process. 29, 4159–4173 (2020)

    Article  Google Scholar 

  21. Ng, H.S., Wu, T.P., Tang, C.K.: Surface-from-gradients without discrete integrability enforcement: a Gaussian kernel approach. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 32(11), 2085–2099 (2009)

    Google Scholar 

  22. Nimier-David, M., Vicini, D., Zeltner, T., Jakob, W.: Mitsuba 2: a retargetable forward and inverse renderer. ACM Trans. Graph. (TOG) 38(6), 1–17 (2019)

    Article  Google Scholar 

  23. Quéau, Y., Durou, J.-D.: Edge-preserving integration of a normal field: weighted least-squares, TV and \(L^1\) approaches. In: Aujol, J.-F., Nikolova, M., Papadakis, N. (eds.) SSVM 2015. LNCS, vol. 9087, pp. 576–588. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-18461-6_46

    Chapter  MATH  Google Scholar 

  24. Quéau, Y., Durou, J.D., Aujol, J.F.: Normal integration: a survey. J. Math. Imaging Vision 60, 576–593 (2018)

    Article  MathSciNet  Google Scholar 

  25. Quéau, Y., Durou, J.D., Aujol, J.F.: Variational methods for normal integration. J. Math. Imaging Vision 60, 609–632 (2018)

    Article  MathSciNet  Google Scholar 

  26. Saracchini, R.F., Stolfi, J., Leitão, H.C., Atkinson, G.A., Smith, M.L.: A robust multi-scale integration method to obtain the depth from gradient maps. Comput. Vision Image Underst. 116(8), 882–895 (2012)

    Article  Google Scholar 

  27. Shi, B., Mo, Z., Wu, Z., Duan, D., Yeung, S.K., Tan, P.: A benchmark dataset and evaluation for non-Lambertian and uncalibrated photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) (2019)

    Google Scholar 

  28. Wang, Y., Bu, J., Li, N., Song, M., Tan, P.: Detecting discontinuities for surface reconstruction. In: Proceedings of International Conference on Pattern Recognition (ICPR), pp. 2108–2111. IEEE (2012)

    Google Scholar 

  29. Woodham, R.J.: Photometric stereo: a reflectance map technique for determining surface orientation from image intensity. In: Proceedings of Image Understanding Systems and Industrial Applications I (1979)

    Google Scholar 

  30. Wu, T.P., Tang, C.K.: Visible surface reconstruction from normals with discontinuity consideration. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 1793–1800. IEEE (2006)

    Google Scholar 

  31. Xie, W., Wang, M., Wei, M., Jiang, J., Qin, J.: Surface reconstruction from normals: a robust DGP-based discontinuity preservation approach. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  32. Xiu, Y., Yang, J., Tzionas, D., Black, M.J.: ICON: implicit clothed humans obtained from normals. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 13296–13306 (2022)

    Google Scholar 

  33. Zhu, D., Smith, W.A.P.: Least squares surface reconstruction on arbitrary domains. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12367, pp. 530–545. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58542-6_32

    Chapter  Google Scholar 

Download references

Acknowledgments

This work was supported by JSPS KAKENHI Grant Number JP19H01123, and National Natural Science Foundation of China under Grant No. 62136001, 61872012.

Author information

Authors and Affiliations

  1. Osaka University, Suita, Japan

    Xu Cao, Hiroaki Santo, Fumio Okura & Yasuyuki Matsushita

  2. NERCVT, School of Computer Science, Peking University, Beijing, China

    Boxin Shi

  3. Peng Cheng Laboratory, Shenzhen, China

    Boxin Shi

Authors
  1. Xu Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hiroaki Santo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Boxin Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fumio Okura
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yasuyuki Matsushita
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xu Cao .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 7952 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cao, X., Santo, H., Shi, B., Okura, F., Matsushita, Y. (2022). Bilateral Normal Integration. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13661. Springer, Cham. https://doi.org/10.1007/978-3-031-19769-7_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19769-7_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19768-0

  • Online ISBN: 978-3-031-19769-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics