Skip to main content
SpringerLink
Log in

Fast Spectral Reflectance Recovery Using DLP Projector

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Spectral reflectance is an intrinsic characteristic of objects that is independent of illumination and the used imaging sensors. This direct representation of objects is useful for various computer vision tasks, such as color constancy and material discrimination. In this work, we present a novel system for spectral reflectance recovery with high temporal resolution by exploiting the unique color-forming mechanism of digital light processing (DLP) projectors. DLP projectors use color wheels, which are composed of a number of color segments and rotate quickly to produce the desired colors. Making effective use of this mechanism, we show that a DLP projector can be used as a light source with spectrally distinct illuminations when the appearance of a scene under the projector’s irradiation is captured with a high-speed camera. Based on the measurements, the spectral reflectance of scene points can be recovered using a linear approximation of the surface reflectance. Our imaging system is built from off-the-shelf devices, and is capable of taking multi-spectral measurements as fast as 100 Hz. We carefully evaluated the accuracy of our system and demonstrated its effectiveness by spectral relighting of static as well as dynamic scenes containing different objects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Conventional RGB cameras cannot capture very fast moving objects and would not be able to get images under distinct illuminations produced by the fast switching DLP projector, i.e. we would only get an image under white light.

References

  • Abrardo, A., Alparone, L., Cappellini, V., & Prosperi, A. (1999). Color constancy from multispectral images. In Proc. International Conference on Image Processing, (Vol. 3, pp. 570–574).

  • Chiao, C. C., Cronin, T. W., & Osorio, D. (2000). Color signals in natural scenes:Characteristics of reflectance spectra and effects of natural illuminants. Journal of the Optical Society of America A, 17(2), 218–224.

    Article  Google Scholar 

  • Cohen, J. (1964). Dependency of the spectral reflectance curves of the munsell color chips. Psychonomic Science, 1, 369–370.

    Article  Google Scholar 

  • Cui, C., Hyunjin, Y., & Moshe, B. E. (2010). Multi-spectral imaging by optimized wide band illumination. International Journal on Computer Vision, 86, 140–151.

    Article  Google Scholar 

  • Dannemiller, J. L. (1992). Spectral reflectance of natural objects: How many basis functions are necessary? Journal of the Optical Society of America A, 9(4), 507–515.

    Article  Google Scholar 

  • DiCarlo, J. M., Xiao, F., & Wandell, B. A. (2000). Illuminating illumination. In Proc. Ninth Color Imaging Conference (pp. 27–34).

  • DiCarlo, J. M., Xiao, F., & Wandell, B. A. (2003). Spectral estimation theory: Beyond linear but before bayesian. Journal of the Optical Society of America A, 20(7), 1261–1270.

    Article  Google Scholar 

  • Du, H., Tong, X., Cao, X., & Lin, S. (2009). A prism-based system for multispectral video acquisition. In Proc. IEEE International Conference Computer Vision.

  • D’Zmura, M. (1992). Color constancy: Surface color from changing illumination. Journal of the Optical Society of America A, 9(3), 490–493.

    Article  Google Scholar 

  • Finlayson, G. D., Hordley, S. D., & Morovic, P. (2005). Colour constancy using the chromagenic constraint. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (Vol. 1, pp. 1079–1086).

  • Gat, N. (2000). Imaging spectroscopy using tunable filters: A review. In Proc. SPIE 4056 (Vol. 4056, pp. 50–64).

  • Han, S., Sato, I., Okabe, T., & Sato, Y. (2010). Fast spectral reflectance recovery using dlp projector. In Proc. Asian Conference on Computer Vision (pp. 323–335).

  • Jasinski, M. F. (1996). Estimation of subpixel vegetation density of natural regions using satellite multispectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 34, 804–813.

    Article  Google Scholar 

  • Jiang, J., & Gu, J. (2012). Recovering spectral reflectance under commonly available lighting conditions. In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (pp. 1–8).

  • Jiang, J., Liu, D., Gu, J., & Susstrunk, S. (2013) What is the space of spectral sensitivity functions for digital color cameras? In Proc. IEEE Workshop on the Applications of Computer Vision 2013 (pp. 4321–4328).

  • Kohonen, O., Parkkinen, J., & Jääskeläinen, T. (2006). Databases for spectral color science. Color Research and Application, 31(5), 381–390.

    Article  Google Scholar 

  • Maloney, L. T. (1986). Evaluation of linear models of surface spectral reflectance with small numbers of parameters. Journal of the Optical Society of America A, 3(10), 1673–1683.

    Article  Google Scholar 

  • Maloney, L. T., & Wandell, B. A. (1986). Color constancy: A method for recovering surface spectral reflectance. Journal of the Optical Society of America A, 3(1), 29–33.

    Article  Google Scholar 

  • Narasimhan, S. G., Koppal, S. J., Yamazaki, S. (2008). Temporal dithering of illumination for fast active vision. In Proc. European Conference Computer Vision.

  • Nayar, S. K., Branzoi, V., & Boult, T. E. (2006). Programmable imaging: Towards a flexible camera. International Journal on Computer Vision, 70, 7–22.

    Article  Google Scholar 

  • Park, J., Lee, M., Grossberg, M. D., & Nayar, S. K. (2007). Multispectral imaging using multiplexed illumination. In Proc. IEEE International Conference on Computer Vision.

  • Parkkinen, J. P. S., Hallikainen, J., & Jaaskelainen, T. (1989). Characteristic spectra of munsell colors. Journal of the Optical Society of America A, 6(2), 318–322.

    Article  Google Scholar 

  • Schechner, Y. Y., & Nayar, S. K. (2002). Generalized mosaicing: Wide field of view multispectral imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(10), 1334–1348.

    Article  Google Scholar 

  • Tominaga, S. (1996). Multichannel vision system for estimating surface and illumination functions. Journal of the Optical Society of America A, 13(11), 2163–2173.

    Article  Google Scholar 

  • Tominaga, S., & Okajima, R. (2000). Object recognition by multi-spectral imaging with a liquid crystal filter. In Proc. International Conference on Pattern Recognition (Vol. 1).

  • Wenger, A., Hawkins, T., & Debevec, P. (2003). Optimizing color matching in a lighting reproduction system for complex subject and illuminant spectra. In Proc. 14th Eurographics workshop on Rendering.

  • Xing, X., Dong, W., Zhang, X., & Paul, J. C. (2010). Spectrally-based single image relighting. In Proc. International conference on E-learning and games.

  • Yamaguchi, M., Haneishi, H., & Fukuda, H. (2006). High-fidelity video and still-image communication based on spectral information: Natural vision system and its applications. In Proc. SPIE 6062 (Vol. 6062).

  • Zhang, S., & Huang, P. (2004). High-resolution, real-time 3d shape acquisition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition Workshops (Vol. 3, pp. 28–37).

Download references

Acknowledgments

This research was supported in part by Grant-in-Aid for Scientific Research on Innovative Areas from the Ministry of Education, Culture, Sports, Science and Technology.

Author information

Authors and Affiliations

  1. Institute of Industrial Science, The University of Tokyo, 4-6-1 Komaba, Meguro, Tokyo, 153-8505, Japan

    Shuai Han, Takahiro Okabe & Yoichi Sato

  2. National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda, Tokyo, 101-8430, Japan

    Imari Sato

Authors
  1. Shuai Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Imari Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Takahiro Okabe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yoichi Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Han.

Appendix

Appendix

Our system divides 25 consecutively captured images into a sequence. One reason is that using our system, basis images exist in a sequence composed of as few as 25 consecutively captured images. The color wheel rotates at 120 rps and the camera works at 500 fps; thus, the color wheel makes \(120/500\)th of a rotation, i.e., \(6/25\)th of a rotation within a frame. Therefore, the offset of the color wheel for every four consecutively captured frames changes \(1-6/25\times 4\), i.e., \(1/25\)th rotation. Inputting the projector (255, 255, 255), since \((6/25+1/25)<1/3\) (rotation), there must be at least one basis image among 25 consecutively captured images for each distinct illumination. An example of the “green” basis images is shown in Fig. 12. Another reason is that the illumination in the captured images repeats every 25 frames because the our high-speed camera captures 25 frames while the color wheel turns six times.

Fig. 12
figure 12

Offset variation every four images. In this example, there are two green basis images (Color figure online)

Full size image

Rights and permissions

Reprints and permissions

About this article

Cite this article

Han, S., Sato, I., Okabe, T. et al. Fast Spectral Reflectance Recovery Using DLP Projector. Int J Comput Vis 110, 172–184 (2014). https://doi.org/10.1007/s11263-013-0687-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-013-0687-z

Keywords

Search

Navigation