Abstract
In this paper, we propose a temporal filtering algorithm to obtain the temporally consistent depth video. We apply a joint multilateral filter to depth video that is based on three Gaussian distributions: one spatial filter and two range filters for color and depth similarities. We also perform outlier reduction in the temporal domain using Gaussian-weighted least squares fitting method to deal with temporal displacement. Experimental results show that the proposed method improves depth quality by 1.22 dB on average, compared to other algorithms due to enhanced temporal consistency.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barfield, W., Weghorst, S.: The sense of presence within virtual environments: a conceptual framework. In: Salvendy, G., Smith, M. (eds.) Human Computer Interaction: Software and Hardware Interfaces, pp. 699–704. Elsevier, Amsterdam (1993)
Freeman, J., Avons, S.E.: Focus group exploration of presence through advanced broadcast services. SPIE Hum. Vis. Electron. Imaging 3959, 530–539 (2000)
Fehn, C.: Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3-D TV. SPIE Conf. Stereosc. Disp. Virtual Real. Syst. 5291, 93–104 (2004)
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1297–1304 (2011)
Tao, H., Sawhney, H.S., Kumar, R.: Dynamic depth recovery from multiple synchronized video streams. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 118–124 (2001)
Larsen, E.S., Mordohai, P., Pollefeys, M., Fuchs, H.: Temporally consistent reconstruction from multiple video streams using enhanced belief propagation. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)
Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. In: SIGGRAPH, pp. 96–100 (2007)
Lai, P., Tian, D., Lopez, P.: Depth map processing with iterative joint multilateral filtering. In: Picture Coding Symposium, pp. 9–12 (2010)
Yang, Q., Wang, L., Ahuja, N.: A constant-space belief propagation algorithm for stereo matching. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1458–1465 (2010)
Choi, J., Min, D., Sohn, K.: 2D-plus-depth based resolution and frame-rate up-conversion technique for depth video. In: IEEE Transactions on Consumer Electronics, pp. 2489–2497 (2010)
Tanimoto, M., Fujii, T., Suzuki, K.: Reference software of depth estimation and view synthesis for FTV/3DV. In: ISO/IEC JTC1/SC29/WG11, M15836 (2008)
Telea, A.: An image inpainting technique based on the fast marching method. J. Gr. Tools 9(1), 25–36 (2004)
Hannuksela, M., Rusanovskyy, D.: Extension of existing 3DV test set toward synthetic 3D video content. In: ISO/IEC JTC1/SC29/WG11, M19221 (2011)
Bruls, F., Gunnewiek, R.K., Walle, P.: Philips response to new call for 3DV test material: arrive book & mobile. In: ISO/IEC JTC1/SC29/WG11, M16419 (2011)
Acknowledgments
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. 2013-067321)).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lee, S., Ho, YS. Temporally consistent depth video filter using temporal outlier reduction. SIViP 9, 1401–1408 (2015). https://doi.org/10.1007/s11760-014-0642-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-014-0642-7