Abstract
Current active 3D range sensors, such as time-of-flight cameras, enable acquiring of range maps at video frame rate. Unfortunately, the resolution of the range maps is quite limited and the captured data are typically contaminated by noise. We therefore present a simple pipeline to enhance the quality as well as improve the spatial and depth resolution of range data in real time. To improve the spatial resolution of range data, we first upsample the depth information with the data from high resolution video camera. And then, a new strategy is utilized to increase the sub-pixel accuracy. We show that these techniques can greatly improve the reconstruction quality, boost the resolution of the range data to that of video sensor while achieving high computational efficiency for a real-time application.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ogger, T., Griesbach, K., et al.: 3D-Imaging in real-time with miniaturized optical range camera. In: Proc. OPTO, pp. 89–94 (2004)
SwissRangerTM SR-4000, MESA Imaging inc., http://www.mesa-imaging.ch
Yang, Q., Wang, L., Yang, R., Stewnius, H., Nistr, D.: Stereo matching with color-weighted correlation, hierarchical belief propagation, and Occlusion Handling. IEEE Trans. PAMI 31(3), 492–504 (2009)
Liang, C.-K., Cheng, C.-C., Lai, Y.-C., Chen, L.-G., Chen, H.: Hardware efficient belief propagation. In: Proc. CVPR, pp. 80–87 (2009)
Diebel, J., Thrun, S.: An application of markov random fields to range sensing. In: Proc. NIPS (2005)
Yang, Q., Yang, R., Davis, J.: Spatial-depth super resolution for range images. In: Proc. CVPR, pp. 1–8 (2007)
Tomasi, C., Manduchi, R.: Bilateral ltering for gray and color images. In: Proc. ICCV, pp. 839–846 (1998)
Kopf, J., Cohen, M., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. ACM Transactions on Graphics (TOG) 26(3), 96(1–5) (2007)
Nvidia Corporation. CUDA: compute unified device architecture programming guide. Technical report (2008)
Schuon, S., Theobalt, C., Davis, J.: Thrun. S.: High-quality scanning using time-of-flight depth superresolution. In: Proc. CVPRW 2008, pp. 1–7 (2008)
Riemens, A.K., Gangwal, O.P., Barenbrug, B., Berretty, R.-P.M.: Multistep joint bilateral depth upsampling. In: SPIE 7257: Proc. VCIP (2009)
Porikli, F.: Constant time O(1) bilateral ltering. In: Proc. CVPR, pp. 1–8 (2008)
Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics (TOG) 26(3), 103:1–10 (2007)
Yang, Q., Tan, H.-H., Ahuja, N.: Real-time O(1) bilateral ltering. In: Proc. CVPR, pp. 557–564 (2009)
Bhme, M., Haker, M., Martinetz, T., Barth, E.: Shading constraint improves accuracy of time-of-flight measurements. In: Proc. CVPR 2008 (2008)
Chan, D., Buisman, H., Theobalt, C., Thrun, S.: A noise-aware lter for real-time depth upsampling. In: M2SFA 208: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications (2008)
Weiss, B.: Fast median and bilateral ltering. In: Siggraph., vol. 25, pp. 519–526 (2006)
Gehrig, S.K., Franke, U.: Improving stereo sub-pixel accuracy for long range stereo. In: Proc. ICCV, pp. 1–7 (2007)
Shimizu, M., Okutomi, M.: Precise sub-pixel estimation on area-based matching. In: Proc. ICCV, pp. 90–97 (2001)
Szeliski, R., Scharstein, D.: Sampling the disparity space image. IEEE Trans. PAMI 26(3), 419–425 (2004)
Zhu, J., Wang, L., Yang, R., Davis, J.: Fusion of time-of-flight depth and stereo for high accuracy depth maps. In: Proc. CVPR (2008)
Guizar-Sicairos, M., Thurman, S.T., Fienup, J.R.: Efficient subpixel image registration algorithms. Opt. Lett. 33, 156–158 (2008)
Lindner, M., Lambers, M., Kolb, A.: Data fusion and edge-enhanced distance fefinement for 2D RGB and 3D range images. IJISTA, Special Issue on Dynamic 3D Imaging 5(1), 344–354 (2008)
Kutulakos, K.N., Steger, E.: A theory of refractive and specular 3D shape by light-path triangulation. In: Proc. ICCV, pp. 1448–1455 (2005)
Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. In: Siggraph., vol. 21, pp. 1–7 (2002)
Barash, D., Camiciu, D.: A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean shift. Image and Vision Computing 22(1), 73–81 (2004)
Alvarez, L., Deriche, R., Sanchez, J., Weickert, J.: Dense disparity map estimation respecting image discontinuities: A pde and scale-space based approach. Technical report. Research Report 3874, INRIA Sophia Antipolis, France (January 2000), 2,3
Freeman, W.T., Pasztor, E.C., Carmichael, O.T.: Learning low-level vision. IJCV 40(1), 25–47 (2000)
Middlebury datasets, http://vision.middlebury.edu/stereo
Tseng, Y.C., Chang, N., Chang, T.S.: Low memory cost block-based belief propagation for stereo correspondence. In: Proc. ICME, pp. 1415–1418 (2007)
Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters. In: Proc. ICCV, pp. 900–906 (2003)
Byrd, R., Lu, P., Nocedal, J., Zhu. C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comp. 16(5), 1190–1208 (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Xiang, X., Li, G., Tong, J., Zhang, M., Pan, Z. (2011). Real-Time Spatial and Depth Upsampling for Range Data. In: Gavrilova, M.L., Tan, C.J.K., Sourin, A., Sourina, O. (eds) Transactions on Computational Science XII. Lecture Notes in Computer Science, vol 6670. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22336-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-22336-5_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22335-8
Online ISBN: 978-3-642-22336-5
eBook Packages: Computer ScienceComputer Science (R0)