Skip to main content
SpringerLink
Log in

Online Illumination Planning for Shadow-Robust Photometric Stereo

  • Conference paper
  • First Online:
Frontiers of Computer Vision (IW-FCV 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1578))

Included in the following conference series:

Abstract

Photometric stereo is a technique for estimating normals of an object surface from its images taken under different light source directions. In general, photometric stereo suffers from shadows, because almost no information on surface normals is available from shadowed pixels. In this paper, we propose an illumination planning for shadow-robust Lambertian photometric stereo; it optimizes the light source directions adaptively for an object of interest, because cast shadows depend on the entire shape of the object. More specifically, our proposed method iteratively adds the optimal light source for surface normal estimation by taking the visibility and linear independence of light source directions into consideration on the basis of the previously captured images of the object. We implemented our illumination planning with a programmable light source in an online manner, and achieve shadow-robust surface normal estimation from a small number of images.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    It is known that surface normals can be recovered from attached shadows if a large number of images taken under varying light source directions are given [12].

  2. 2.

    The visibility is often used for representing cast shadows, but we use it for representing both attached shadows and cast shadows in this paper.

  3. 3.

    Note that not the noise \(\delta _{pl}\) but the visibility \(v_{pl}\) is used for representing shadows as shown in Eq. (1).

  4. 4.

    When multiple pixels have the same visibility matrix and therefore have the same MSE, we randomly select one of the pixels as the worst pixel.

  5. 5.

    We set the lower limit not to 0 but to 1 so that the gradient of the cost function C(x, y) does not vanish for gradient-based optimization.

References

  1. Ackermann, J., Michael, G.: A survey of photometric stereo techniques. Found. Trends Comput. Graph. Vis. 9(3–4), 149–254 (2015)

    Article  Google Scholar 

  2. Barsky, S., Petrou, M.: The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Trans. PAMI 25(10), 1239–1252 (2003)

    Article  Google Scholar 

  3. Chandraker, M., Agarwal, S., Kriegman, D.: ShadowCuts: photometric stereo with shadows. In: Proceedings of the IEEE CVPR 2007, pp. 1–8 (2007)

    Google Scholar 

  4. Chen, G., Han, K., Shi, B., Matsushita, Y., Wong, K.Y.K.: Self-calibrating deep photometric stereo networks. In: Proceedings of the IEEE CVPR 2019, pp. 8739–8747 (2019)

    Google Scholar 

  5. Chen, G., Han, K., Wong, K.-Y.K.: PS-FCN: a flexible learning framework for photometric stereo. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11213, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01240-3_1

    Chapter  Google Scholar 

  6. Drbohlav, O., Chantler, M.: On optimal light configurations in photometric stereo. In: Proceedings of the IEEE ICCV 2005, vol. 2, pp. 1707–1712 (2005)

    Google Scholar 

  7. Ikehata, S.: CNN-PS: CNN-based photometric stereo for general non-convex surfaces. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11219, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01267-0_1

    Chapter  Google Scholar 

  8. Ikehata, S., Wipf, D., Matsushita, Y., Aizawa, K.: Robust photometric stereo using sparse regression. In: Proceedings of the IEEE CVPR 2012, pp. 318–325 (2012)

    Google Scholar 

  9. Li, J., Robles-Kelly, A., You, S., Matsushita, Y.: Learning to minify photometric stereo. In: Proceedings of the IEEE CVPR 2019, pp. 7568–7576 (2019)

    Google Scholar 

  10. Mukaigawa, Y., Ishii, Y., Shakunaga, T.: Analysis of photometric factors based on photometric linearization. JOSA A 24(10), 3326–3334 (2007)

    Article  Google Scholar 

  11. Murase, H., Nayar, S.K.: Illumination planning for object recognition using parametric eigenspaces. IEEE Trans. PAMI 16(12), 1219–1227 (1994)

    Article  Google Scholar 

  12. Okabe, T., Sato, I., Sato, Y.: Attached shadow coding: estimating surface normals from shadows under unknown reflectance and lighting conditions. In: Proceedings of the IEEE ICCV 2009, pp. 1693–1700 (2009)

    Google Scholar 

  13. Okabe, T., Sato, Y.: Object recognition based on photometric alignment using RANSAC. In: Proceedings of the IEEE CVPR 2003, pp. 221–228 (2003)

    Google Scholar 

  14. Shi, B., Mo, Z., Wu, Z., Duan, D., Yeung, S.K., Tan, P.: A benchmark dataset and evaluation for non-Lambertian and uncalibrated photometric stereo. In: Proceedings of the IEEE CVPR 2016, pp. 3707–3716 (2016)

    Google Scholar 

  15. Silver, W.M.: Determining shape and reflectance using multiple images. Master’s thesis, MIT (1980)

    Google Scholar 

  16. Sunkavalli, K., Zickler, T., Pfister, H.: Visibility subspaces: uncalibrated photometric stereo with shadows. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 251–264. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15552-9_19

    Chapter  Google Scholar 

  17. Taniai, T., Maehara, T.: Neural inverse rendering for general reflectance photometric stereo. In: Proceedings of the ICML 2018, pp. 4857–4866 (2018)

    Google Scholar 

  18. Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 139–144 (1980)

    Article  Google Scholar 

  19. Wu, L., Ganesh, A., Shi, B., Matsushita, Y., Wang, Y., Ma, Y.: Robust photometric stereo via low-rank matrix completion and recovery. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6494, pp. 703–717. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19318-7_55

    Chapter  Google Scholar 

  20. Zheng, Q., Jia, Y., Shi, B., Jiang, X., Duan, L.Y., Kot, A.C.: SPLINE-Net: sparse photometric stereo through lighting interpolation and normal estimation networks. In: Proceedings of the IEEE ICCV 2019, pp. 8549–8558 (2019)

    Google Scholar 

Download references

Acknowledgement

This work was supported by JSPS KAKENHI Grant Number JP20H00612.

Author information

Authors and Affiliations

  1. Department of Artificial Intelligence, Kyushu Institute of Technology, 680-4 Kawazu, Iizuka, Fukuoka, 820-8502, Japan

    Hirochika Tanikawa, Ryo Kawahara & Takahiro Okabe

Authors
  1. Hirochika Tanikawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ryo Kawahara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Takahiro Okabe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Takahiro Okabe .

Editor information

Editors and Affiliations

  1. Aoyama Gakuin University, Kanagawa, Japan

    Kazuhiko Sumi

  2. Chosun University, Gwangju, Korea (Republic of)

    In Seop Na

  3. Aoyama Gakuin University, Kanagawa, Japan

    Naoshi Kaneko

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tanikawa, H., Kawahara, R., Okabe, T. (2022). Online Illumination Planning for Shadow-Robust Photometric Stereo. In: Sumi, K., Na, I.S., Kaneko, N. (eds) Frontiers of Computer Vision. IW-FCV 2022. Communications in Computer and Information Science, vol 1578. Springer, Cham. https://doi.org/10.1007/978-3-031-06381-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06381-7_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06380-0

  • Online ISBN: 978-3-031-06381-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation