Abstract
Recent advances in fast light transport acquisition have motivated new applications for forward and inverse light transport. While forward light transport enables image relighting, inverse light transport provides new possibilities for analyzing and cancelling interreflections, to enable applications like projector radiometric compensation and light bounce separation. With known scene geometry and diffuse reflectance, inverse light transport can be easily derived in closed form. However, with unknown scene geometry and reflectance properties, we must acquire and invert the scene’s light transport matrix to undo the effects of global illumination. For many photometric setups such as that of a projector-camera system, the light transport matrix often has a size of 105×105 or larger. Direct matrix inversion is accurate but impractical computationally at these resolutions.
In this work, we explore a theoretical analysis of inverse light transport, relating it to its forward counterpart, expressed in the form of the rendering equation. It is well known that forward light transport has a Neumann series that corresponds to adding bounces of light. In this paper, we show the existence of a similar inverse series, that zeroes out the corresponding physical bounces of light. We refer to this series solution as stratified light transport inversion, since truncating to a certain number of terms corresponds to cancelling the corresponding interreflection bounces. The framework of stratified inversion is general and may provide insight for other problems in light transport and beyond, that involve large-size matrix inversion. It is also efficient, requiring only sparse matrix-matrix multiplications. Our practical application is to radiometric compensation, where we seek to project patterns onto real-world surfaces, undoing the effects of global illumination. We use stratified light transport inversion to efficiently invert the acquired light transport matrix for a static scene, after which interreflection cancellation is a simple matrix-vector multiplication to compensate the input image for projection.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Arvo, J., Torrance, K., & Smits, B. (1994). A framework for the analysis of error in global illumination algorithms. In Proceedings of ACM SIGGRAPH (pp. 75–84).
Ashdown, M., Okabe, T., Sato, I., & Sato, Y. (2006). Robust content-dependent photometric projector compensation. In Proceedings of IEEE international workshop on projector camera systems.
Bai, J., Chandraker, M., Ng, T.-T., & Ramamoorthi, R. (2010). A dual theory of inverse and forward light transport. In Proceedings of European conference on computer vision.
Bimber, O. (2006). Emerging technologies of augmented reality: interfaces and design. In Idea group pub. (pp. 64–89). Chapter Projector-based augmentation.
Bimber, O., Emmerling, A., & Klemmer, T. (2005). Embedded entertainment with smart projectors. IEEE Computer, 38(1), 48–55.
Bimber, O., Grundhofer, A., Zeidler, T., Danch, D., & Kapakos, P. (2006). Compensating indirect scattering for immersive and semi-immersive projection displays. In Proceedings of the IEEE virtual reality conference (pp. 151–158).
Debevec, P., Hawkins, T., Tchou, C., Duiker, H., Sarokin, W., & Sagar, M. (2000). Acquiring the reflectance field of a human face. In Proceedings of ACM SIGGRAPH (pp. 145–156).
Ding, Y., Xiao, J., Tan, K.-H., & Yu, J. (2009). Catadioptric projectors. In Proceedings of computer vision and pattern recognition.
Fujii, K., Grossberg, M., & Nayar, S. (2005). A projector-camera system with real-time photometric adaptation for dynamic environments. In Proceedings of computer vision and pattern recognition.
Gortler, S., Schröder, P., Cohen, M., & Hanrahan, P. (1993). Wavelet radiosity. In Proceedings of ACM SIGGRAPH (pp. 221–230).
Habe, H., Saeki, N., & Matsuyama, T. (2007). Inter-reflection compensation for immersive projection display. In Proceedings of computer vision and pattern recognition.
Hanrahan, P., Salzman, D., & Aupperle, L. (1991). A rapid hierarchical radiosity algorithm. In Proceedings of ACM SIGGRAPH (pp. 197–206).
Kajiya, J.-T. (1986). The rendering equation. In Proceedings of ACM SIGGRAPH (pp. 143–150).
Langville, A. N., & Meyer, C. D. (2003). Relighting with 4D incident light fields (Report CRSC-TR03-34). Center for Research in Scientific Computation, North Carolina State University.
Liu, S., Ng, T.-T., & Matsushita, Y. (2010). Shape from second-bounce of light transport. In Proceedings of European conference on computer vision.
Majumder, A., Gopi, M., Seales, B. W., & Fuchs, H. (1999). Geometric stitching for real-time panoramic image generation using texture maps. In Proceedings of ACM international conference on multimedia.
Marschner, S. R. (1998). Inverse rendering for computer graphics. PhD Thesis, Cornell University.
Masselus, V., Peers, P., Dutré, P., & Willems, Y. D. (2003). Fiddling with pagerank. In Proceedings of ACM SIGGRAPH (pp. 613–620).
Mukaigawa, Y., Kakinuma, T., & Ohta, Y. (2006). Analytical compensation of inter-reflection for pattern projection. In Proceedings of the ACM symposium on virtual reality software and technology (p. 268).
Nayar, S., Peri, H., Grossberg, M., & Belhumeur, P. (2003). A projection system with radiometric compensation for screen imperfections. In Proceedings of IEEE international workshop on projector camera systems.
Nayar, S., Krishnan, G., Grossberg, M., & Raskar, R. (2006). Fast separation of direct and global components of a scene using high frequency illumination. ACM Transactions on Graphics, 25(3), 935–944.
Ng, R., Ramamoorthi, R., & Hanrahan, P. (2003). All-frequency shadows using non-linear wavelet lighting approximation. Proceedings of ACM SIGGRAPH, 22(3), 376–381.
Ng, T.-T., Pahwa, R. S., Bai, J., Quek, Q.-S., & Tan, K.-H. (2009). Radiometric compensation using stratified inverses. In Proceedings of international conference on computer vision.
O’Toole, M., & Kutulakos, K. N. (2010). Optical computing for fast light transport analysis. In Proceedings of ACM SIGGRAPH Asia.
Peers, P., Mahajan, D., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., & Debevec, P. (2009). Compressive light transport sensing. ACM Transactions on Graphics, 28(1). doi:10.1145/1477926.1477929.
Ramamoorthi, R. (2007). Precomputation-based rendering. Foundations and Trends in Computer Graphics and Vision, 3(4), 281–369.
Ramamoorthi, R., & Hanrahan, P. (2001). A signal-processing framework for inverse rendering. In Proceedings of ACM SIGGRAPH (pp. 117–128).
Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., & Fuchs, H. (1998). The office of the future: a unified approach to image-based modeling and spatially immersive displays. In Proceedings of ACM SIGGRAPH (pp. 179–188).
Raskar, R., Welch, G., & Fuchs, H. (1999). Spatially augmented reality. Wellesley: AK Peters.
Seitz, S., Matsushita, Y., & Kutulakos, K. (2005). A theory of inverse light transport. In Proceedings of international conference on computer vision (pp. 1440–1447).
Sen, P., & Darabi, S. (2009). Compressive dual photography. Computer Graphics Forum, 28(2), 609–618.
Sen, P., Chen, B., Garg, G., Marschner, S., Horowitz, M., Levoy, M., & Lensch, H. (2005). Dual photography. ACM Transactions on Graphics, 24(3), 745–755.
Song, P., & Cham, T. (2005). A theory for photometric self-calibration of multiple overlapping projectors and cameras. In Proceedings of IEEE international workshop on projector camera systems.
Wang, J., Dong, Y., Tong, X., Lin, Z., & Guo, B. (2009). Kernel Nyström method for light transport. ACM Transactions on Graphics, 28(3). doi:10.1145/1531326.1531335.
Wetzstein, G., & Bimber, O. (2007). Radiometric compensation through inverse light transport. In Proceedings of Pacific conference on computer graphics and applications (pp. 391–399).
Yang, R., Majumder, A., & Brown, M. (2004). Camera based calibration techniques for seamless flexible multi-projector displays. In Proceedings of applications of computer vision workshop.
Yu, Y., Debevec, P., Malik, J., & Hawkins, T. (1999). Inverse global illumination: recovering reflectance models of real scenes from photographs. In Proceedings of ACM SIGGRAPH (pp. 215–224).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ng, TT., Pahwa, R.S., Bai, J. et al. From the Rendering Equation to Stratified Light Transport Inversion. Int J Comput Vis 96, 235–251 (2012). https://doi.org/10.1007/s11263-011-0467-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-011-0467-6