Mohamad Ivan Fanany
Itsuo Kumazawa
ABSTRACT
This paper presents a new neural network (NN) scheme for recovering three dimensional (3D) transparent surface. We view the transparent surface modeling, not as a separate problem, but as an extension of opaque surface modeling. The main insight of this work is we simulate transparency not only for generating visually realistic images, but for recovering the object shape. We construct a formulation of transparent surface modeling using ray tracing framework into our NN. We compared this ray tracing method, with a texture mapping method that simultaneously map the silhouette images and smooth shaded images (obtained form our NN), and textured images (obtained from the teacher image) to an initial 3D model. By minimizing the images error between the output images of our NN and the teacher images, observed in multiple views, we refine vertices position of the initial 3D model. We show that our NN can refine the initial 3D model obtained by polarization images and converge into more accurate surface.
- Chuang, Y.-Y., Zongker, D. E., Hindorff, J., Curless, B., Salesin, D., and Szeliski, R. 2000. Environment matting extensions: towards higher accuracy and real-time capture. In SIGGRAPH, 121--130. Google Scholar
Digital Library
- Cook, R. L., Porter, T., and Carpenter, L. 1988. Distributed ray tracing. 139--147. Google ScholarDigital Library
- Fanany, M. I., and Kumazawa, I. 2003. SA-optimized multiple view smooth polyhedron representation nn. In Discovery Science, 306--310.Google Scholar
- Fanany, M. I., and Kumazawa, I. 2004. Multiple-view shape extraction from shading as local regression by analytic nn scheme. Mathematical and Computer Modelling 40, 9-10, 959--975. Google ScholarDigital Library
- Fanany, M. I., and Kumazawa, I. 2004. A neural network for recovering 3d shape from erroneous and few depth maps of shaded images. Pattern Recognition Letters 25, 4, 377--389. Google ScholarDigital Library
- Fanany, M. I., Ohno, M., and Kumazawa, I. 2002. A Scheme for Reconstructing Face from Shading using Smooth Projected Polygon Representation NN. In Proc. of the IEEEICIP (volume II), 305--308.Google Scholar
- Fanany, M. I., Kobayashi, K., and Kumazawa, I. 2004. A combinatorial transparent surface modeling from polarization images. In IWCIA, 65--76. Google ScholarDigital Library
- Faugeras, O. D., Luong, Q.-T., and Maybank, S. J. 1992. Camera self-calibration: Theory and experiments. In European Conference on Computer Vision, 321--334. Google ScholarDigital Library
- Hata, S., Saitoh, Y., Kumamura, S., and Kaida, K. 1996. Shape extraction of transparent object using genetic algorithm. In ICPR96, D93.6. Google ScholarDigital Library
- Hearn, D., and Baker, M. 1998. Computer graphics: C version, prentice hall, upper saddle river. In NJ 98, 1997. Google ScholarDigital Library
- Interrante, V., Fuchs, H., and Pizer, S. 1997. Conveying the 3D shape of transparent surfaces via texture. Tech. Rep. TR-97-27. Google ScholarDigital Library
- Matusik, W., Pfister, H., Ziegler, R., Ngan, A., and McMillan, L. 2002. Acquisition and rendering of transparent and refractive objects. In Rendering Techniques, 267--278. Google ScholarDigital Library
- Miyazaki, D., Saito, M., Sato, Y., and Ikeuchi, K. 2002. Determining surface orientations of transparent objects based on polarization degrees in visible and infrared wavelengths. JOSAA 19, 4 (April), 687--694.Google ScholarCross Ref
- Miyazaki, D., Kagesawa, M., and Ikeuchi, K. 2003. Polarization-based transparent surface modeling from two views. In ICCV, 1381--1386. Google ScholarDigital Library
- Murase, H. 1992. Surface shape reconstruction of a nonrigid transport object using refraction and motion. IEEE Trans. Pattern Anal. Mach. Intell. 14, 10, 1045--1052. Google ScholarDigital Library
- Nakayama, K., Shimojo, S., and Ramachandran, V. 1990. Transparency: Relation to depth, subjective contours, luminance and neon color spreading. Perception 19, 497-513, 497--513.Google ScholarCross Ref
- Rahmann, S., and Canterakis, N. 2001. Reconstruction of specular surfaces using polarization imaging. In CVPR01, I:149--155.Google Scholar
- Saito, M., Kashiwagi, H., Sato, Y., and Ikeuchi, K. 1999. Measurement of surface orientations of transparent objects using polarization in highlight. In CVPR, 1381-.Google Scholar
- Smith, A. R., and Blinn, J. F. 1996. Blue screen matting. In SIGGRAPH '96: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, ACM Press, New York, NY, USA, 259--268. Google ScholarDigital Library
- Szeliski, R., Avidan, S., and Anandan, P. 2000. Layer extraction from multiple images containing reflections and transparency. In CVPR, 1246-.Google Scholar
- Wexler, Y., Fitzgibbon, A. W., and Zisserman, A. 2002. Image-based environment matting. In Rendering Techniques, 279--290. Google ScholarDigital Library
- Wolff, L. B., and Kurlander, D. J. 1990. Ray tracing with polarization parameters. IEEE Comput. Graph. Appl. 10, 6, 44--55. Google ScholarDigital Library
- Wolff, L. 1987. Shape from polarization images. In CVWS87, 79--85.Google Scholar
- Zongker, D. E., Werner, D. M., Curless, B., and Salesin, D. 1999. Environment matting and compositing. In SIGGRAPH, 205--214. Google ScholarDigital Library
Index Terms
- A neural network scheme for transparent surface modelling
Recommendations
Shape Recovery of Hybrid Reflectance Surface using Neural Network
ICIP '97: Proceedings of the 1997 International Conference on Image Processing (ICIP '97) 3-Volume Set-Volume 3 - Volume 3The main interest of previous researches for shape from shading was only on Lambertian surface which consists of diffuse reflectance surface. In practice, natural objects have hybrid reflectance, which limits the recovery of shape from intensity ...
Real-time 3D surface modeling for image based relighting
This paper proposes to obtain 3D modeling of object surface and to create a relit image in real-time. The proposed algorithm uses a single camera and synchronized stereo lights to capture the light field images by turning on and off the lights ...
Neural-network-based adaptive hybrid-reflectance model for 3-D surface reconstruction
This paper proposes a novel neural-network-based adaptive hybrid-reflectance three-dimensional (3-D) surface reconstruction model. The neural network automatically combines the diffuse and specular components into a hybrid model. The proposed model ...
Comments