Skip to main content
SpringerLink
Log in

Multispectral Photometric Stereo Using Intrinsic Image Decomposition

  • Conference paper
  • First Online:
Frontiers of Computer Vision (IW-FCV 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1212))

Included in the following conference series:

Abstract

One of the main problems faced by the photometric stereo method is that several measurements are required, as this method needs illumination from light sources from different directions. A solution to this problem is the color photometric stereo method, which conducts one-shot measurements by simultaneously illuminating lights of different wavelengths. However, the classic color photometric stereo method only allows measurements of white objects, while a surface-normal estimation of a multicolored object using this method is theoretically impossible. Therefore, it is necessary to convert a multi-colored object to a single-colored object before applying the photometric stereo. In this study, we employ the intrinsic image decomposition for conversion. Intrinsic image decomposition can produce the intrinsic image which is not affected by the reflectance. Since the intrinsic image is the image with white object, we can obtain the surface normal by applying the conventional photometric stereo algorithm to the intrinsic image. To demonstrate the effectiveness of this study, a measurement device that can realize the multispectral photometric stereo method with seven colors is employed instead of the classic color photometric stereo method with three colors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Anderson, R., Stenger, B., Cipolla, R.: Color photometric stereo for multicolored surfaces. In: International Conference on Computer Vision, pp. 2182–2189 (2011)

    Google Scholar 

  2. Bell, M., Freeman, E.T.: Learning local evidence for shading and reflectance. In: IEEE International Conference on Computer Vision (2001)

    Google Scholar 

  3. Brostow, G.J., Stenger, B., Vogiatzis, G., Hernández, C., Cipolla, R.: Video normals from colored lights. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 2104–2114 (2011)

    Article  Google Scholar 

  4. Chakrabarti, A., Sunkavalli, K.: Single-image RGB photometric stereo with spatially-varying albedo. In: International Conference on 3D Vision, pp. 258–266 (2016)

    Google Scholar 

  5. Drew, M.S.: Reduction of rank-reduced orientation-from-color problem with many unknown lights to two-image known-illuminant photometric stereo. In: Proceedings of International Symposium on Computer Vision, pp. 419–424 (1995)

    Google Scholar 

  6. Drew, M.S.: Direct solution of orientation-from-color problem using a modification of Pentland’s light source direction estimator. Comput. Vis. Image Underst. 64(2), 286–299 (1996)

    Article  Google Scholar 

  7. Drew, M.S., Brill, M.H.: Color from shape from color: a simple formalism with known light sources. J. Opt. Soc. Am. A: 17(8), 1371–1381 (2000)

    Article  Google Scholar 

  8. Drew, M., Kontsevich, L.: Closed-form attitude determination under spectrally varying illumination. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 985–990 (1994)

    Google Scholar 

  9. Funt, B.V., Drew, M.S., Brockington, M.: Recovering shading from color images. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 124–132. Springer, Heidelberg (1992). https://doi.org/10.1007/3-540-55426-2_15

    Chapter  Google Scholar 

  10. Fyffe, G., Yu, X., Debevec, P.: Single-shot photometric stereo by spectral multiplexing. In: IEEE International Conference on Computational Photography, pp. 1–6 (2011)

    Google Scholar 

  11. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, p. 716. Addison Wesley, Reading (1993)

    Google Scholar 

  12. Gotardo, P.F.U., Simon, T., Sheikh, Y., Mathews, I.: Photogeometric scene flow for high-detail dynamic 3D reconstruction. In: IEEE International Conference on Computer Vision, pp. 846–854 (2015)

    Google Scholar 

  13. Hernandez, C., Vogiatzis, G., Brostow, G.J., Stenger, B., Cipolla, R.: Non-rigid photometric stereo with colored lights. In: IEEE International Conference on Computer Vision, p. 8 (2007)

    Google Scholar 

  14. Hernández, C., Vogiatzis, G., Cipolla, R.: Shadows in three-source photometric stereo. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 290–303. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88682-2_23

    Chapter  Google Scholar 

  15. Horn, B.K.P., Brooks, M.J.: The variational approach to shape from shading. Comput. Vis. Graph. Image Process. 33(2), 174–208 (1986)

    Article  Google Scholar 

  16. Ikehata, S., Wipf, D., Matsushita, Y., Aizawa, K.: Photometric stereo using sparse Bayesian regression for general diffuse surfaces. IEEE Trans. Pattern Anal. Mach. Intell. 39(9), 1816–1831 (2014)

    Article  Google Scholar 

  17. Ikeuchi, K., Horn, B.K.P.: Numerical shape from shading and occluding boundaries. Artif. Intell. 17(1–3), 141–184 (1981)

    Article  Google Scholar 

  18. Jiao, H., Luo, Y., Wang, N., Qi, L., Dong, J., Lei, H.: Underwater multi-spectral photometric stereo reconstruction from a single RGBD image. In: Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, pp. 1–4 (2016)

    Google Scholar 

  19. Kim, H., Wilburn, B., Ben-Ezra, M.: Photometric stereo for dynamic surface orientations. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 59–72. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_5

    Chapter  Google Scholar 

  20. Kontsevich, L., Petrov, A., Vergelskaya, I.: Reconstruction of shape from shading in color images. J. Opt. Soc. Am. A: 11, 1047–1052 (1994)

    Article  Google Scholar 

  21. Landstrom, A., Thurley, M.J., Jonsson, H.: Sub-millimeter crack detection in casted steel using color photometric stereo. In: International Conference on Digital Image Computing: Techniques and Applications, pp. 1–7 (2013)

    Google Scholar 

  22. Levin, A., Weiss, Y.: User assisted separation of reflections from a single image using a sparsity prior. IEEE Trans. Pattern Anal. Mach. Intell. 29(9), 1647–1654 (2007)

    Article  Google Scholar 

  23. Matsushita, Y., Nishino, K., Ikeuchi, K., Sakauchi, M.: Illumination normalization with time-dependent intrinsic images for video surveillance. IEEE Trans. Pattern Anal. Mach. Intell. 26(10), 1336–1347 (2004)

    Article  Google Scholar 

  24. Miyazaki, D., Ikeuchi, K.: Photometric stereo under unknown light sources using robust SVD with missing data. In: Proceedings of IEEE International Conference on Image Processing, pp. 4057–4060 (2010)

    Google Scholar 

  25. Miyazaki, D., Onishi, Y., Hiura, S.: Color photometric stereo using multi-band camera constrained by median filter and ollcuding boundary. J. Imaging 5(7), 29 (2019). Article no. 64

    Article  Google Scholar 

  26. Mori, T., Taketa, R., Hiura, S., Sato, K.: Photometric linearization by robust PCA for shadow and specular removal. In: Csurka, G., Kraus, M., Laramee, R.S., Richard, P., Braz, J. (eds.) VISIGRAPP 2012. CCIS, vol. 359, pp. 211–224. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38241-3_14

    Chapter  Google Scholar 

  27. Mukaigawa, Y., Ishii, Y., Shakunaga, T.: Analysis of photometric factors based on photometric linearization. J. Opt. Soc. Am. A: 24(10), 3326–3334 (2007)

    Article  Google Scholar 

  28. Nicodemus, F.E., Richmond, J.C., Hsia, J.J., Ginsberg, I.W., Limperis, T.: Geometrical considerations and nomenclature of reflectance. In: Wolff, L.B., Shafer, S.A., Healey, G. (eds.) Radiometry, pp. 940–145. Jones and Bartlett Publishers Inc. (1992)

    Google Scholar 

  29. Petrov, A.P., Kontsevich, L.L.: Properties of color images of surfaces under multiple illuminants. J. Opt. Soc. Am. A: 11(10), 2745–2749 (1994)

    Article  Google Scholar 

  30. Quéau, Y., Mecca, R., Durou, J.-D.: Unbiased photometric stereo for colored surfaces: a variational approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4359–4368 (2016)

    Google Scholar 

  31. Quéau, Y., Durou, J.-D., Aujol, J.-F.: Normal integration: a survey. J. Math. Imaging Vis. 60(4), 576–593 (2018). https://doi.org/10.1007/s10851-017-0773-x

    Article  MathSciNet  MATH  Google Scholar 

  32. Rahman, S., Lam, A., Sato, I., Robles-Kelly, A.: Color photometric stereo using a rainbow light for non-lambertian multicolored surfaces. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9003, pp. 335–350. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16865-4_22

    Chapter  Google Scholar 

  33. Roubtsova, N., Guillemaut, J.Y.: Colour Helmholtz Stereopsis for reconstruction of complex dynamic scenes. In: International Conference on 3D Vision, pp. 251–258 (2014)

    Google Scholar 

  34. Shum, H.-Y., Ikeuchi, K., Reddy, R.: Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Trans. Pattern Anal. Mach. Intell. 17(9), 854–867 (1995)

    Article  Google Scholar 

  35. Silver, W.M.: Determining shape and reflectance using multiple images. Master’s thesis, Massachusetts Institute of Technology (1980)

    Google Scholar 

  36. Tan, R.T., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27(2), 178–193 (2005)

    Article  Google Scholar 

  37. Tappen, M.F., Freeman, W.T., Adelson, E.H.: Recovering intrinsic images from a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27(9), 1459–1472 (2005)

    Article  Google Scholar 

  38. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. B 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  39. Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9(2), 137–154 (1992). https://doi.org/10.1007/BF00129684

    Article  Google Scholar 

  40. Vogiatzis, G., Hernández, C.: Practical 3D reconstruction based on photometric stereo. In: Cipolla, R., Battiato, S., Farinella, G.M. (eds.) Computer Vision: Detection, Recognition and Reconstruction. SCI, vol. 285, pp. 313–345. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12848-6_12

    Chapter  Google Scholar 

  41. Vogiatzis, G., Hernandez, C.: Self-calibrated, multi-spectral photometric stereo for 3D face capture. Int. J. Comput. Vis. 97, 91–103 (2012). https://doi.org/10.1007/s11263-011-0482-7

    Article  Google Scholar 

  42. Weiss, Y.: Deriving intrinsic images from image sequences. In: IEEE International Conference on Computer Vision (2001)

    Google Scholar 

  43. Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 139–144 (1980)

    Article  Google Scholar 

  44. Woodham, R.J.: Gradient and curvature from photometric stereo including local confidence estimation. J. Opt. Soc. Am. 11, 3050–3068 (1994)

    Article  Google Scholar 

  45. Wu, L., Ganesh, A., Shi, B., Matsushita, Y., Wang, Y., Ma, Y.: Robust photometric stereo via low-rank matrix completion and recovery. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6494, pp. 703–717. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19318-7_55

    Chapter  Google Scholar 

  46. Xie, X., Zheng, W., Lai, J., Yuen, P.C.: Face illumination normalization on large and small scale features. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Hiroshima City University, Hiroshima, 731-3194, Japan

    Koumei Hamaen & Daisuke Miyazaki

  2. University of Hyogo, Kobe, Hyogo, 651-2197, Japan

    Shinsaku Hiura

Authors
  1. Koumei Hamaen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daisuke Miyazaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shinsaku Hiura
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daisuke Miyazaki .

Editor information

Editors and Affiliations

  1. Saitama Institute of Technology, Saitama, Japan

    Wataru Ohyama

  2. Kyungpook National University, Daegu, Korea (Republic of)

    Soon Ki Jung

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hamaen, K., Miyazaki, D., Hiura, S. (2020). Multispectral Photometric Stereo Using Intrinsic Image Decomposition. In: Ohyama, W., Jung, S. (eds) Frontiers of Computer Vision. IW-FCV 2020. Communications in Computer and Information Science, vol 1212. Springer, Singapore. https://doi.org/10.1007/978-981-15-4818-5_22

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-4818-5_22

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-4817-8

  • Online ISBN: 978-981-15-4818-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation