Yuanhong Xu
ABSTRACT
Obtaining dense 3D reconstruction with low computational cost is one of the important goals in the field of Simultaneous Localization and Mapping (SLAM). In this paper we propose a dense 3D reconstruction framework from monocular multi-spectral video sequences using jointly semi-dense SLAM and depth-estimation-based Multi-spectral Photometric Stereo approaches. Starting from multi-spectral video, we use SALM to reconstruct a semi-dense 3D shape that will be densified. Then the depth maps estimated via conditional Generative Adversarial Nets (cGAN) are fed as priors into optimization-based multi-spectral photometric stereo for dense surface normal recovery. Finally, we use camera poses for view conversion in fusion procedure where we combine the relative sparse point cloud with the dense surface normal to get a dense point cloud. Experiments show that our method can effectively obtain denser 3D reconstruction.
- Achtelik, M. and M. Achtelik, et al. (2011). Onboard IMU and monocular vision-based control for MAVs in unknown in- and outdoor environments. IEEE International Conference on Robotics and Automation.Google Scholar
- Akbarzadeh, A. and J. M. Frahm, et al. (2006). Towards Urban 3D Reconstruction from Video. International Symposium on 3d Data Processing, Visualization, and Transmission. Google ScholarDigital Library
- Davison (2003). Real-time simultaneous localisation and mapping with a single camera. IEEE International Conference on Computer Vision. Google ScholarDigital Library
- Klein, G. and D. Murray (2007). Parallel Tracking and Mapping for Small AR Workspaces. IEEE and ACM International Symposium on Mixed and Augmented Reality. Google ScholarDigital Library
- Newcombe, R. A. and S. J. Lovegrove, et al. (2011). DTAM: Dense tracking and mapping in real-time. Proceedings of the 2011 International Conference on Computer Vision. Google ScholarDigital Library
- Newcombe, R. A. and S. J. Lovegrove, et al. (2011). DTAM: Dense tracking and mapping in real-time. Proceedings of the 2011 International Conference on Computer Vision. Google ScholarDigital Library
- Engel, Jakob, T. Schöps, and D. Cremers. LSD-SLAM: Large-Scale Direct Monocular SLAM. Computer Vision - ECCV 2014. Springer International Publishing, 2014:834--849.Google Scholar
- Engel, Jakob, V. Koltun, and D. Cremers. "Direct Sparse Odometry." IEEE Transactions on Pattern Analysis & Machine Intelligence PP.99(2017):1--1.Google Scholar
- Wu, C. and Y. Liu, et al. (2011). "Fusing multiview and photometric stereo for 3D reconstruction under uncalibrated illumination." IEEE Transactions on Visualization & Computer Graphics 17 (8): 1082. Google ScholarDigital Library
- Jiao, H. and Y. Luo, et al. (2017). Underwater multi-spectral photometric stereo reconstruction from a single RGBD image. Signal and Information Processing Association Summit and Conference.Google Scholar
- Kim, Jae Hak, C. Cadena, and I. Reid. "Direct semi-dense SLAM for rolling shutter cameras." IEEE International Conference on Robotics and Automation IEEE, 2016:1308--1315.Google Scholar
- Liu, F.; Chung, S.; Ng, A.Y. Learning depth from single monocular images. IEEE Trans. Pattern Anal.Mach. Intell. 2005, 18, 1--8.Google Scholar
- Drew, M. S., and L. L. Kontsevich. "Closed-form attitude determination under spectrally varying illumination." Computer Vision and Pattern Recognition, 1994. Proceedings CVPR '94. 1994 IEEE Computer Society Conference on IEEE, 1994:985--990.Google Scholar
- Petrov, A. P., I. S. Vergelskaya, and L. L. Kontsevich. "Petrov, A. P., I. S. Vergelskaya, and L. L. Kontsevich. "Reconstruction of shape from shading in color images." Journal of the Optical Society of America A 12.11(1994):1047--1052.Google Scholar
- Woodham, Robert J. "Gradient and Curvature from Photometric Stereo Including Local Condence Estimation." J.opt.soc.am.a 11.11(1994):3050--3068.Google ScholarCross Ref
- Chen, Chia Yen, R. Klette, and C. F. Chen. Shape from Photometric Stereo and Contours. Computer Analysis of Images and Patterns. Springer Berlin Heidelberg, 2003:377--384.Google Scholar
- Sabzevari, R. and A. D. Bue, et al. (2011). Structure from Motion and Photometric Stereo for Dense 3D Shape Recovery. International Conference on Image Analysis and Processing. Google ScholarDigital Library
- Phillip Isola, et al. Image-to-Image Translation with Conditional Adversarial Networks. 2016: 5967--5976.Google Scholar
- Yu, Yang, et al. Unsupervised Representation Learning with Deep Convolutional Neural Network for Remote Sensing Images. International Conference on Image and Graphics Springer 2017; Cham:97--108.Google Scholar
- Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. International Conference on International Conference on Machine Learning 2015; JMLR.org:448--456. Google ScholarDigital Library
- Nehab, Diego, et al. "Efficiently combining positions and normals for precise 3D geometry." Acm Transactions on Graphics 24.3(2005):536--543. Google ScholarDigital Library
Index Terms
- Combining depth-estimation-based multi-spectral photometric stereo and SLAM for real-time dense 3D reconstruction
Recommendations
A 3D Imaging Framework Based on High-Resolution Photometric-Stereo and Low-Resolution Depth
This paper introduces a 3D imaging framework that combines high-resolution photometric stereo and low-resolution depth. Our approach targets imaging scenarios based on either macro-lens photography combined with focal stacking or a large-format camera ...
Dense 3D reconstruction combining depth and RGB information
Dense 3D reconstruction has important applications in many fields. The existing depth information based methods are typically constrained in their effective camera-object distance which should be from 0.4m to 4m. We present a novel method that can ...
High Quality Photometric Reconstruction Using a Depth Camera
CVPR '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern RecognitionIn this paper we present a depth-guided photometric 3D reconstruction method that works solely with a depth camera like the Kinect. Existing methods that fuse depth with normal estimates use an external RGB camera to obtain photometric information and ...
Comments