Rui Xia
Abstract
We present a novel integrated approach for estimating both spatially-varying surface reflectance and detailed geometry from a video of a rotating object under unknown static illumination. Key to our method is the decoupling of the recovery of normal and surface reflectance from the estimation of surface geometry. We define an apparent normal field with corresponding reflectance for each point (including those not on the object's surface) that best explain the observations. We observe that the object's surface goes through points where the apparent normal field and corresponding reflectance exhibit a high degree of consistency with the observations. However, estimating the apparent normal field requires knowledge of the unknown incident lighting. We therefore formulate the recovery of shape, surface reflectance, and incident lighting, as an iterative process that alternates between estimating shape and lighting, and simultaneously recovers surface reflectance at each step. To recover the shape, we first form an initial surface that passes through locations with consistent apparent temporal traces, followed by a refinement that maximizes the consistency of the surface normals with the underlying apparent normal field. To recover the lighting, we rely on appearance-from-motion using the recovered geometry from the previous step. We demonstrate our integrated framework on a variety of synthetic and real test cases exhibiting a wide variety of materials and shape.
Supplemental Material
Available for Download
Supplemental file.
- Ackermann, J., Ritz, M., Stork, A., and Goesele, M. 2012. Removing the example from example-based photometric stereo. In ECCV, 197--210. Google Scholar
Digital Library
- Adato, Y., Vasilyev, Y., Zickler, T., and Ben-Shahar, O. 2010. Shape from specular flow. IEEE PAMI 32, 11 (Nov), 2054--2070. Google ScholarDigital Library
- Ashikhmin, M., Premoze, S., and Shirley, P. 2000. A microfacet-based BRDF generator. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, 65--74. Google ScholarDigital Library
- Barron, J. T., and Malik, J. 2015. Shape, illumination, and reflectance from shading. IEEE PAMI.Google Scholar
- Basri, R., Jacobs, D. W., and Kemelmacher, I. 2007. Photometric stereo with general, unknown lighting. IJCV 72, 3, 239--257. Google ScholarDigital Library
- Chandraker, M. 2014. On shape and material recovery from motion. In ECCV, 202--217.Google Scholar
- Cook, R. L., and Torrance, K. E. 1982. A reflectance model for computer graphics. ACM Trans. Graph. 1, 1, 7--24. Google ScholarDigital Library
- Dong, Y., Wang, J., Tong, X., Snyder, J., Lan, Y., BenEzra, M., and Guo, B. 2010. Manifold bootstrapping for SVBRDF capture. ACM Trans. Graph. 29, 4, 98:1--98:10. Google ScholarDigital Library
- Dong, Y., Chen, G., Peers, P., Zhang, J., and Tong, X. 2014. Appearance-from-motion: Recovering spatially varying surface reflectance under unknown lighting. ACM Trans. Graph. 33, 6, 193:1--193:12. Google ScholarDigital Library
- Fiala, M. 2005. Artag, a fiducial marker system using digital techniques. In CVPR, vol. 2, 590--596. Google ScholarDigital Library
- Hertzmann, A., and Seitz, S. M. 2003. Shape and materials by example: A photometric stereo approach. In CVPR, 533--540. Google ScholarDigital Library
- Kazhdan, M., and Hoppe, H. 2013. Screened poisson surface reconstruction. ACM Trans. Graph. 32, 3, 29:1--29:13. Google ScholarDigital Library
- Lombardi, S., and Nishino, K. 2016. Reflectance and illumination recovery in the wild. IEEE PAMI 38, 1, 129--141. Google ScholarDigital Library
- Lu, F., Matsushita, Y., Sato, I., Okabe, T., and Sato, Y. 2013. Uncalibrated photometric stereo for unknown isotropic reflectances. In CVRP, 1490--1497. Google ScholarDigital Library
- Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Trans. Graph. 24, 3, 536--543. Google ScholarDigital Library
- Nicodemus, F. E., Richmond, J. C., Hsia, J. J., Ginsberg, I. W., and Limperis, T. 1977. Geometric considerations and nomenclature for reflectance. Monograph 161,National Bureau of Standards (US).Google Scholar
- Oxholm, G., and Nishino, K. 2012. Shape and reflectance from natural illumination. In ECCV, 528--541. Google ScholarDigital Library
- Oxholm, G., and Nishino, K. 2014. Multiview shape and reflectance from natural illumination. In CVPR, 2163--2170. Google ScholarDigital Library
- Palma, G., Callieri, M., Dellepiane, M., and Scopigno, R. 2012. A statistical method for svbrdf approximation from video sequences in general lighting conditions. Comput. Graph. Forum 31, 4, 1491--1500. Google ScholarDigital Library
- Romeiro, F., and Zickler, T. 2010. Blind reflectometry. In ECCV, 45--58. Google ScholarDigital Library
- Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. In CVPR, 519--528. Google ScholarDigital Library
- Treuille, A., Hertzmann, A., and Seitz, S. M. 2004. Example-based stereo with general BRDFs. In ECCV, 457--469.Google Scholar
- Triggs, B., McLauchlan, P. F., Hartley, R. I., and Fitzgibbon, A. W. 1999. Bundle adjustment - a modern synthesis. In ICCV, 298--372. Google ScholarDigital Library
- Valgaerts, L., Wu, C., Bruhn, A., Seidel, H.-P., and Theobalt, C. 2012. Lightweight binocular facial performance capture under uncontrolled lighting. ACM Trans. Graph. 31, 6, 187:1--187:11. Google ScholarDigital Library
- Wang, J., Zhao, S., Tong, X., Snyder, J., and Guo, B. 2008. Modeling anisotropic surface reflectance with example-based microfacet synthesis. ACM Trans. Graph. 27, 3, 41:1--41:9. Google ScholarDigital Library
- Wang, T.-C., Chandraker, M., Efros, A., and Ramamoorthi, R. 2016. Svbrdf-invariant shape and reflectance estimation from light-field cameras. In CVPR.Google Scholar
- Weinmann, M., and Klein, R. 2015. Advances in geometry and reflectance acquisition. In ACM SIGGRAPH Asia, Course Notes. Google ScholarDigital Library
- Woodham, R. J. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering 19, 1, 3050--3068.Google ScholarCross Ref
- Wu, C., Wilburn, B., Matsushita, Y., and Theobalt, C. 2011. High-quality shape from multi-view stereo and shading under general illumination. In CVPR, 969--976. Google ScholarDigital Library
- Wu, H., Wang, Z., and Zhou, K. 2016. Simultaneous localization and appearance estimation with a consumer RGB-D camera. IEEE Trans. Vis. and Comp. Graph. 2, 8, 2012--2023. Google ScholarDigital Library
- Xu, D., Duan, Q., Zheng, J., Zhang, J., Cai, J., and Cham, T.-J. 2014. Recovering surface details under general unknown illumination using shading and coarse multi-view stereo.Google Scholar
- Zhang, Z. 2000. A flexible new technique for camera calibration. In IEEE PAMI, vol. 22, 1330--1334. Google ScholarDigital Library
Index Terms
- Recovering shape and spatially-varying surface reflectance under unknown illumination
Recommendations
Acquiring reflectance and shape from continuous spherical harmonic illumination
We present a novel technique for acquiring the geometry and spatially-varying reflectance properties of 3D objects by observing them under continuous spherical harmonic illumination conditions. The technique is general enough to characterize either ...
Estimating Facial Reflectance Properties Using Shape-from-Shading
In this paper we show how to estimate facial surface reflectance properties (a slice of the BRDF and the albedo) in conjunction with the facial shape from a single image. The key idea underpinning our approach is to iteratively interleave the two ...
All-frequency rendering of dynamic, spatially-varying reflectance
We describe a technique for real-time rendering of dynamic, spatially-varying BRDFs in static scenes with all-frequency shadows from environmental and point lights. The 6D SVBRDF is represented with a general microfacet model and spherical lobes fit to ...
Comments