Abstract
Depth refinement is critical for consumer depth cameras to remove the inherent noises and unreliable or missing data. In this paper, we propose a simple yet effective approach to achieve high-resolution refinement of low-cost Kinect depth map. The key of our approach is a unified energy function with two new constraints terms, that is, the photometric and multi-shading gradients constraints. Specifically, photometric constraint term helps to enrich the faithful local details of scene surface; while multi-shading gradients constraint term suppresses the effect of inaccurate normal estimation under multiple illuminations. Besides, smoothness and initial depth constraints are also included. We design an adaptive weighting strategy to further increase the robustness of approach for the depth-missing and non-Lambertian regions. Since energy function can be optimized directly in the shading domain, the refined depth map has the same higher resolution of the shading images. Experiments validate the effectiveness of our approach with reliable accuracy and faithfully richer 3D details than competitor methods.
Y. Zhang and Q. Zhang—Contributed equally to this work.
W. Feng—This work is supported by NSFC 61671325, 61572354, 61672376.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahmed, A.H., Farag, A.A.: Shape from shading under various imaging conditions. In: CVPR (2007)
Bai, J., Yang, J., Ye, X., Hou, C.: Depth refinement for binocular kinect RGB-D cameras. In: VCIP (2016)
Chatterjee, A., Govindu, V.M.: Photometric refinement of depth maps for multi-albedo objects. In: CVPR (2015)
Feng, W., Tian, F., Zhang, Q., Sun, J.: 6D dynamic camera relocalization from single reference image. In: CVPR (2016)
Feng, W., Tian, F., Zhang, Q., Zhang, N., Wan, L., Sun, J.: Fine-grained change detection of misaligned scenes with varied illuminations. In: ICCV (2015)
Han, Y., Lee, J.Y., So Kweon, I.: High quality shape from a single RGB-D image under uncalibrated natural illumination. In: ICCV (2013)
Haque, M., Chatterjee, A., Madhav Govindu, V.: High quality photometric reconstruction using a depth camera. In: CVPR (2014)
Horn, B.K., Brooks, M.J.: The variational approach to shape from shading. Comput. Vis. Graph. Image Process. 33(2), 174–208 (1986)
Liu, S., Do, M.N.: Inverse rendering and relighting from multiple color plus depth images. IEEE TIP 26(10), 4951–4961 (2017)
Maier, R., Kim, K., Cremers, D., Kautz, J., Niener, M.: Intrinsic3D: high-quality 3D reconstruction by joint appearance and geometry optimization with spatially-varying lighting. In: ICCV (2017)
Miao, D., Tian, F., Feng, W.: Active camera relocalization with RGBD camera from a single 2D image. In: ICASSP (2018)
Or-El, R., Rosman, G., Wetzler, A., Kimmel, R., Bruckstein, A.M.: RGBD-fusion: real-time high precision depth recovery. In: CVPR (2015)
Papadhimitri, T., Favaro, P.: A closed-form, consistent and robust solution to uncalibrated photometric stereo via local diffuse reflectance maxima. IJCV 107(2), 139–154 (2014)
Park, J., Sinha, S.N., Matsushita, Y., Tai, Y.W., Kweon, I.S.: Multiview photometric stereo using planar mesh parameterization. In: ICCV (2013)
Shi, Y., Tian, F., Miao, D., Feng, W.: Fast and reliable computational rephotography on mobile device. In: ICME (2018)
Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 1–22 (1980)
Wu, C., Zollhfer, M., Niener, M., Stamminger, M., Izadi, S., Theobalt, C.: Real-time shading-based refinement for consumer depth cameras. ACM TOG 33(6), 200 (2014)
Zhang, Q., Feng, W., Wan, L., Tian, F., Tan, P.: Active recurrence of lighting condition for fine-grained change detection. In: IJCAI (2018)
Zhang, Q., Ye, M., Yang, R., Matsushita, Y., Wilburn, B., Yu, H.: Edge-preserving photometric stereo via depth fusion. In: CVPR (2012)
Zhang, S., Wang, C., Chan, S.C.: A new high resolution depth map estimation system using stereo vision and kinect depth sensing. J. Signal Process. Syst. 79(1), 19–31 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Zhang, Y., Zhang, Q., Feng, W. (2018). High-Resolution Depth Refinement by Photometric and Multi-shading Constraints. In: Geng, X., Kang, BH. (eds) PRICAI 2018: Trends in Artificial Intelligence. PRICAI 2018. Lecture Notes in Computer Science(), vol 11013. Springer, Cham. https://doi.org/10.1007/978-3-319-97310-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-97310-4_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97309-8
Online ISBN: 978-3-319-97310-4
eBook Packages: Computer ScienceComputer Science (R0)