Skip to main content
SpringerLink
Log in

From the Rendering Equation to Stratified Light Transport Inversion

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Recent advances in fast light transport acquisition have motivated new applications for forward and inverse light transport. While forward light transport enables image relighting, inverse light transport provides new possibilities for analyzing and cancelling interreflections, to enable applications like projector radiometric compensation and light bounce separation. With known scene geometry and diffuse reflectance, inverse light transport can be easily derived in closed form. However, with unknown scene geometry and reflectance properties, we must acquire and invert the scene’s light transport matrix to undo the effects of global illumination. For many photometric setups such as that of a projector-camera system, the light transport matrix often has a size of 105×105 or larger. Direct matrix inversion is accurate but impractical computationally at these resolutions.

In this work, we explore a theoretical analysis of inverse light transport, relating it to its forward counterpart, expressed in the form of the rendering equation. It is well known that forward light transport has a Neumann series that corresponds to adding bounces of light. In this paper, we show the existence of a similar inverse series, that zeroes out the corresponding physical bounces of light. We refer to this series solution as stratified light transport inversion, since truncating to a certain number of terms corresponds to cancelling the corresponding interreflection bounces. The framework of stratified inversion is general and may provide insight for other problems in light transport and beyond, that involve large-size matrix inversion. It is also efficient, requiring only sparse matrix-matrix multiplications. Our practical application is to radiometric compensation, where we seek to project patterns onto real-world surfaces, undoing the effects of global illumination. We use stratified light transport inversion to efficiently invert the acquired light transport matrix for a static scene, after which interreflection cancellation is a simple matrix-vector multiplication to compensate the input image for projection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Arvo, J., Torrance, K., & Smits, B. (1994). A framework for the analysis of error in global illumination algorithms. In Proceedings of ACM SIGGRAPH (pp. 75–84).

    Chapter  Google Scholar 

  • Ashdown, M., Okabe, T., Sato, I., & Sato, Y. (2006). Robust content-dependent photometric projector compensation. In Proceedings of IEEE international workshop on projector camera systems.

    Google Scholar 

  • Bai, J., Chandraker, M., Ng, T.-T., & Ramamoorthi, R. (2010). A dual theory of inverse and forward light transport. In Proceedings of European conference on computer vision.

    Google Scholar 

  • Bimber, O. (2006). Emerging technologies of augmented reality: interfaces and design. In Idea group pub. (pp. 64–89). Chapter Projector-based augmentation.

    Google Scholar 

  • Bimber, O., Emmerling, A., & Klemmer, T. (2005). Embedded entertainment with smart projectors. IEEE Computer, 38(1), 48–55.

    Article  Google Scholar 

  • Bimber, O., Grundhofer, A., Zeidler, T., Danch, D., & Kapakos, P. (2006). Compensating indirect scattering for immersive and semi-immersive projection displays. In Proceedings of the IEEE virtual reality conference (pp. 151–158).

    Google Scholar 

  • Debevec, P., Hawkins, T., Tchou, C., Duiker, H., Sarokin, W., & Sagar, M. (2000). Acquiring the reflectance field of a human face. In Proceedings of ACM SIGGRAPH (pp. 145–156).

    Google Scholar 

  • Ding, Y., Xiao, J., Tan, K.-H., & Yu, J. (2009). Catadioptric projectors. In Proceedings of computer vision and pattern recognition.

    Google Scholar 

  • Fujii, K., Grossberg, M., & Nayar, S. (2005). A projector-camera system with real-time photometric adaptation for dynamic environments. In Proceedings of computer vision and pattern recognition.

    Google Scholar 

  • Gortler, S., Schröder, P., Cohen, M., & Hanrahan, P. (1993). Wavelet radiosity. In Proceedings of ACM SIGGRAPH (pp. 221–230).

    Google Scholar 

  • Habe, H., Saeki, N., & Matsuyama, T. (2007). Inter-reflection compensation for immersive projection display. In Proceedings of computer vision and pattern recognition.

    Google Scholar 

  • Hanrahan, P., Salzman, D., & Aupperle, L. (1991). A rapid hierarchical radiosity algorithm. In Proceedings of ACM SIGGRAPH (pp. 197–206).

    Google Scholar 

  • Kajiya, J.-T. (1986). The rendering equation. In Proceedings of ACM SIGGRAPH (pp. 143–150).

    Google Scholar 

  • Langville, A. N., & Meyer, C. D. (2003). Relighting with 4D incident light fields (Report CRSC-TR03-34). Center for Research in Scientific Computation, North Carolina State University.

  • Liu, S., Ng, T.-T., & Matsushita, Y. (2010). Shape from second-bounce of light transport. In Proceedings of European conference on computer vision.

    Google Scholar 

  • Majumder, A., Gopi, M., Seales, B. W., & Fuchs, H. (1999). Geometric stitching for real-time panoramic image generation using texture maps. In Proceedings of ACM international conference on multimedia.

    Google Scholar 

  • Marschner, S. R. (1998). Inverse rendering for computer graphics. PhD Thesis, Cornell University.

  • Masselus, V., Peers, P., Dutré, P., & Willems, Y. D. (2003). Fiddling with pagerank. In Proceedings of ACM SIGGRAPH (pp. 613–620).

    Google Scholar 

  • Mukaigawa, Y., Kakinuma, T., & Ohta, Y. (2006). Analytical compensation of inter-reflection for pattern projection. In Proceedings of the ACM symposium on virtual reality software and technology (p. 268).

    Google Scholar 

  • Nayar, S., Peri, H., Grossberg, M., & Belhumeur, P. (2003). A projection system with radiometric compensation for screen imperfections. In Proceedings of IEEE international workshop on projector camera systems.

    Google Scholar 

  • Nayar, S., Krishnan, G., Grossberg, M., & Raskar, R. (2006). Fast separation of direct and global components of a scene using high frequency illumination. ACM Transactions on Graphics, 25(3), 935–944.

    Article  Google Scholar 

  • Ng, R., Ramamoorthi, R., & Hanrahan, P. (2003). All-frequency shadows using non-linear wavelet lighting approximation. Proceedings of ACM SIGGRAPH, 22(3), 376–381.

    Article  Google Scholar 

  • Ng, T.-T., Pahwa, R. S., Bai, J., Quek, Q.-S., & Tan, K.-H. (2009). Radiometric compensation using stratified inverses. In Proceedings of international conference on computer vision.

    Google Scholar 

  • O’Toole, M., & Kutulakos, K. N. (2010). Optical computing for fast light transport analysis. In Proceedings of ACM SIGGRAPH Asia.

    Google Scholar 

  • Peers, P., Mahajan, D., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., & Debevec, P. (2009). Compressive light transport sensing. ACM Transactions on Graphics, 28(1). doi:10.1145/1477926.1477929.

  • Ramamoorthi, R. (2007). Precomputation-based rendering. Foundations and Trends in Computer Graphics and Vision, 3(4), 281–369.

    Article  MATH  Google Scholar 

  • Ramamoorthi, R., & Hanrahan, P. (2001). A signal-processing framework for inverse rendering. In Proceedings of ACM SIGGRAPH (pp. 117–128).

    Google Scholar 

  • Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., & Fuchs, H. (1998). The office of the future: a unified approach to image-based modeling and spatially immersive displays. In Proceedings of ACM SIGGRAPH (pp. 179–188).

    Google Scholar 

  • Raskar, R., Welch, G., & Fuchs, H. (1999). Spatially augmented reality. Wellesley: AK Peters.

    Google Scholar 

  • Seitz, S., Matsushita, Y., & Kutulakos, K. (2005). A theory of inverse light transport. In Proceedings of international conference on computer vision (pp. 1440–1447).

    Google Scholar 

  • Sen, P., & Darabi, S. (2009). Compressive dual photography. Computer Graphics Forum, 28(2), 609–618.

    Article  Google Scholar 

  • Sen, P., Chen, B., Garg, G., Marschner, S., Horowitz, M., Levoy, M., & Lensch, H. (2005). Dual photography. ACM Transactions on Graphics, 24(3), 745–755.

    Article  Google Scholar 

  • Song, P., & Cham, T. (2005). A theory for photometric self-calibration of multiple overlapping projectors and cameras. In Proceedings of IEEE international workshop on projector camera systems.

    Google Scholar 

  • Wang, J., Dong, Y., Tong, X., Lin, Z., & Guo, B. (2009). Kernel Nyström method for light transport. ACM Transactions on Graphics, 28(3). doi:10.1145/1531326.1531335.

  • Wetzstein, G., & Bimber, O. (2007). Radiometric compensation through inverse light transport. In Proceedings of Pacific conference on computer graphics and applications (pp. 391–399).

    Google Scholar 

  • Yang, R., Majumder, A., & Brown, M. (2004). Camera based calibration techniques for seamless flexible multi-projector displays. In Proceedings of applications of computer vision workshop.

    Google Scholar 

  • Yu, Y., Debevec, P., Malik, J., & Hawkins, T. (1999). Inverse global illumination: recovering reflectance models of real scenes from photographs. In Proceedings of ACM SIGGRAPH (pp. 215–224).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Infocomm Research, 1 Fusionopolis Way, 21-01 Connexis, Singapore, 128632, Singapore

    Tian-Tsong Ng

  2. HP Labs, 1501 Page Mill Road, Bldg. 3U, Palo Alto, CA, 94304, USA

    Kar-Han Tan

  3. EECS Department, University of California at Berkeley, 525 Soda Hall, MC 1776, Berkeley, CA, 94720-1776, USA

    Jiamin Bai & Ravi Ramamoorthi

  4. Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, 61801-2918, USA

    Ramanpreet Singh Pahwa

Authors
  1. Tian-Tsong Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ramanpreet Singh Pahwa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiamin Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kar-Han Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ravi Ramamoorthi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tian-Tsong Ng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ng, TT., Pahwa, R.S., Bai, J. et al. From the Rendering Equation to Stratified Light Transport Inversion. Int J Comput Vis 96, 235–251 (2012). https://doi.org/10.1007/s11263-011-0467-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-011-0467-6

Keywords

Search

Navigation