Abstract
Raw depth images acquired by low-cost range imaging sensors, such as Kinect v1 and Kinect v2, usually contain invalid regions without depth information and suffer from noise. Although many approaches are proposed to address the problems, the robustness of these approaches still needs enhancement. This paper introduces a dual-way guided inpainting method with RGBD image pairs to restore the missing depth values of invalid regions and to eliminate noise caused by acquisition apparatus. By leveraging the structural difference between the colour and depth images, the colour image is first segmented with watershed segmentation and then merged under the guidance of the simultaneously captured depth image, followed by inpainting the depth image guided by the merged colour image using Radial Basis Functions (RBFs). The proposed framework of the dual-way guided approach with the RBFs is new for depth image inpainting and outperforms the existing state-of-the-art approaches in the experimental evaluations.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bae, S., Paris, S., Durand, F.: Two-scale tone management for photographic look. ACM TOG 25(3), 637–645 (2006)
Kepski, M., Kwolek, B.: Human fall detection on embedded platform using depth maps and wireless accelerometer. Comput. Methods Progr. Biomed. 117(3), 489–501 (2014)
Carr, J.C., Beatson, R.K., Cherrie, J.B., Mitchell, T.J., Fright, W.R., McCallum, B.C., Evans, T.R.: Reconstruction and representation of 3D objects with radial basis functions. In: SIGGRAPH 2001, pp. 67–76. ACM (2001)
Chen, L., Lin, H., Li, S.: Depth image enhancement for kinect using region growing and bilateral filter. In: Proceedings of 21st ICPR, pp. 3070–3073. IEEE (2012)
Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graph. (TOG) 21(3), 257–266 (2002)
Gong, X., Liu, J., Zhou, W., Liu, J.: Guided depth enhancement via a fast marching method. Image Vis. Comput. 31(10), 695–703 (2013)
He, K., Sun, J., Tang, X.: Guided image filtering. IEEE Trans. PAMI 35(6), 1397–1409 (2013)
Hodaň, T., Haluza, P., Obdržálek, Š., Matas, J., Lourakis, M., Zabulis, X.: T-LESS: an RGB-D dataset for 6D pose estimation of texture-less objects. In: IEEE Winter Conference on Applications of Computer Vision (WACV) (2017)
Kim, S.-Y., Kim, M., Ho, Y.-S.: Depth image filter for mixed and noisy pixel removal in RGB-D camera systems. IEEE Trans. Consum. Electron. 59(3), 681–689 (2013)
Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. ACM TOG 26(3), 96 (2007)
Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. Int. J. Comput. Vis. 81(1), 24–52 (2009)
Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: Proceedings of CVPR, pp. 1–8. IEEE (2007)
Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: Computer Vision and Pattern Recognition, vol. 1, pp. I–195. IEEE (2003)
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of ICCV, pp. 839–846. IEEE (1998)
Yang, J., Ye, X., Li, K., Hou, C., Wang, Y.: Color-guided depth recovery from RGB-D data using an adaptive autoregressive model. IEEE Trans. Image Process. 23(8), 3443–3458 (2014)
Zhang, G., Jia, J., Wong, T.-T., Bao, H.: Recovering consistent video depth maps via bundle optimization. In: Proceedings of CVPR, pp. 1–8. IEEE (2008)
Zhang, G., Jia, J., Wong, T.-T., Bao, H.: Consistent depth maps recovery from a video sequence. IEEE Trans. PAMI 31(6), 974–988 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Yuan, H., Zhou, Y., Sheng, Y., Zhang, G. (2018). Dual-Way Guided Depth Image Inpainting with RGBD Image Pairs. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10704. Springer, Cham. https://doi.org/10.1007/978-3-319-73603-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-73603-7_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73602-0
Online ISBN: 978-3-319-73603-7
eBook Packages: Computer ScienceComputer Science (R0)