Abstract
In this work, we propose a new depth image super-resolution method. We use low resolution depth image, refined high resolution color image and generated HR depth image to conduct iterative joint trilateral up-sampling. During the process of up-sampling, we put forward an algorithm to smooth the area in color image with overmuch texture to solve the texture copying problem. Based on the assumption that LR image is a counterpart of HR image with missing pixels, we defined an evaluation criterion to ensure the convergence of iteration and simultaneously make the final generated image close to the true HR depth image as far as possible. Our approach can generate HR depth image with sharp edges, none texture copying and little noises. Experiments are conducted on various datasets including Middlebury to demonstrate the superiority of the proposed method and show the improvement over state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Gribbon, K.T., Bailey, D.G.: A novel approach to real-time bilinear interpolation. In: Second IEEE International Workshop on Electronic Design, Test and Applications, Perth, Australia, pp. 126–131, January 2004
Keys, R.G.: Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust. Speech Signal Process. 29(6), 1153–1160 (1981)
Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Trans. Image Process. 10, 1521–1527 (2001)
Wang, Q., Ward, R.K.: A new orientation-adaptive interpolation method. IEEE Trans. Image Process. 16(4), 889–900 (2007)
Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22(2), 56–65 (2002)
Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(22), 2323–2326 (2000)
Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: CVPR, vol. 1, pp. 275–282 (2004)
Yang, J.C., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)
Kiechle, M., Hawe, S., Kleinsteuber, M.: A joint intensity and depth co-sparse analysis model for depth map super-resolution. In: ICCV, pp. 1545–1552 (2013)
Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., Chenin, P., Cohen, A., Gout, C., Lyche, T., Mazure, M.-L., Schumaker, L. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012). doi:10.1007/978-3-642-27413-8_47
Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10593-2_13
Schulter, S., Leistner, C., Bischof, H.: Fast and accurate image upscalling with super-resolution forests. In: CVPR 2015, pp. 3791–3799 (2015)
Kopf, J., Cohen, M., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. ACM TOG 26(3), 96 (2007)
Yang, Q., Yang, R., Davis, J.: Spatial-depth super resolution for range images. In: CVPR (2007)
He, K., Sun, J., Tang, X.: Guided image filtering. In: ECCV, pp. 1–10 (2010)
Lu, J., Forsyth, D.: Sparse depth super resolution. In: CVPR 2015, pp. 2245–2253 (2015)
Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986). PAMI
Middlebury stereo database. http://vision.middlebury.edu/stereo/
Ferstl, D., Reinbacher, C., Ranftl, R.: Image guided depth upsampling using anisotropic total generalized variation. In: ICCV (2013). http://rvlab.icg.tugraz.at/project_page/project_tofusion/project_tofsuperresolution.html
Chan, D., Buisman, H., Theobalt, C.: A noise-aware filter for real-time depth upsampling. In: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications (2008)
Diebel, J., Thrun, S.: An application of markov random fields to range sensing. In: Proceedings of Advances in Neural Information Processing System (2005)
Park, J., Kim, H., Tai, W.Y.: High quality depth map upsampling for 3D-TOF cameras. In: ICCV (2011)
Acknowledgements
This work is supported in part by the NSFC-Guangdong Joint Foundation Key Project (U1201255) and project of NSFC 61371138, China.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Yuan, L., Jin, X., Yuan, C. (2016). Enhanced Joint Trilateral Up-sampling for Super-Resolution. In: Chen, E., Gong, Y., Tie, Y. (eds) Advances in Multimedia Information Processing - PCM 2016. PCM 2016. Lecture Notes in Computer Science(), vol 9917. Springer, Cham. https://doi.org/10.1007/978-3-319-48896-7_51
Download citation
DOI: https://doi.org/10.1007/978-3-319-48896-7_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48895-0
Online ISBN: 978-3-319-48896-7
eBook Packages: Computer ScienceComputer Science (R0)