Skip to main content
SpringerLink
Log in

Real-Time Refinement of Kinect Depth Maps using Multi-Resolution Anisotropic Diffusion

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present a novel real-time algorithm to refine depth maps generated by low-cost commercial depth sensors like the Microsoft Kinect. The Kinect sensor falls under the category of RGB-D sensors that can generate a high resolution depth map and color image of a scene. They are relatively inexpensive and are commercially available off-the-shelf. However, owing to their low complexity, there are several artifacts that one encounters in the depth map like holes, mis-alignment between the depth map and color image and lack of sharp object boundaries in the depth map. This is a potential problem in applications that require the color image to be projected in 3-D using the depth map. Such applications depend heavily on the depth map and thus the quality of the depth map is of vital importance. In this paper, a novel multi-resolution anisotropic diffusion based algorithm is presented that accepts a Kinect generated depth map and color image and computes a dense depth map in which the holes have been filled and the edges of the objects are sharpened and aligned with the objects in the color image. The proposed algorithm also ensures that regions in the depth map where the depth is properly estimated are not filtered and ensures that the depth values in the final depth map are the same values that existed in the original depth map. Experimental results are provided to demonstrate the improvement in the quality of the depth map and also execution time results are provided to prove that the proposed method can be executed in real-time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Fehn C (2003) A 3D-TV system based on video plus depth information. In: Proceedings conference record of the thirty-seventh asilomar conference on signals, systems and computers, vol 2, pp 1529–1533

  2. Tian C, Lai P-L, Lopez P, Gomila C (2009) View synthesis techniques for 3D video. In: Proceedings of the SPIE, vol 7443, no 2, pp 74430T–74430T-11

  3. Scharstein D, Szeliski R, Zabih R (2001) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. In: Proceedings of the IEEE workshop on stereo and multi-baseline vision (SMBV 2001), pp 131–140

  4. Zhang Z (2012) Microsoft Kinect sensor and its effect. In: IEEE multimedia, vol 19, no 2, pp 4–10

  5. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Proceedings of the sixth international conference on computer vision, pp 839–846

  6. Kopf J, Cohen MF, Lichinski D, Uyttendaele M (2007) Joint bilateral upsampling. In: Proceedings of ACM transactions graph (SIGGRAPH ’07), vol 26, no 3, article 96

  7. Matyunin S, Vatolin D, Berdnikov Y, Smirnov M (2011) Temporal filtering for depth maps generated by Kinect depth camera. In: Proceedings of 3DTV conference: the true vision - capture, transactions and display of 3D video (3DTV-CON ’11), pp 1–4

  8. Berdnikov Y, Vatolin D (2011) Real-time depth map occlusion filling and scene background restoration for projected-pattern-based depth cameras. In: Proceedings of the 21st international conference on computer graphics and vision (GraphiCon ’11)

  9. Camplani M, Salgado L (2012) Efficient spatio temporal hole filling strategy for Kinect depth maps. In: Proceedings of IS & T/SPIE international conference on 3D image process and applications, vol 8290, pp 82900E1–10

  10. Miao C, Fu J, Lu Y, Li S, Chen CW (2012) Texture-assisted Kinect depth inpainting. In: Proceedings of the IEEE international symposium on circuits and systems (ISCAS ’12), pp 604–607

  11. Xu K, Zhou J, Wang Z (2012) A method of hole-filling for the depth map generated by Kinect with moving objects detection. In: Proceedings of the IEEE international symposium on broadband multimedia systems and broadcasting (BMSB ’12), pp 1–5

  12. Fu J, Wang S., Yan L, Shipeng Z, Zeng W (2012) Kinect-like depth denoising. In: Proceedinds of the IEEE international symposium on circuits and systems (ISCAS ’12), pp 512–515

  13. Milani S, Calvagno G (2011) Joint denoising and interpolation of depth maps for MS Kinect sensors. In: Proceedings of the IEEE international conference on acoustics, speech and signal processing (ICASSP ’12), pp 797–800

  14. Richardt C, Stoll C, Dodgson NA, Seidel H-P, Theobalt C (2012) Coherent spatiotemporal filtering, upsampling and rendering of RGBZ videos. In: Proceedings of eurgraphics

  15. Park J, Kim H, Tai Y-W, Brown MS, Kweon I (2011) High quality depth map upsampling for 3D-TOF cameras. In: Proceedings of the internationl conference on computer vision (ICCV ’11), pp 1623–1630

  16. Li Y, Sun L (2010) A novel upsampling scheme for depth map compression in 3DTV system. In: Proceedings of picture coding symposium (PCS ’10), pp 186–189

  17. Ekmekcioglu E, Mrak M, Worral S, Kondoz A (2009) Utilisation of edge adaptive upsampling in compression of depth map videos for enhanced free-viewpoint rendering. In: Proceedings of international conference on image processing (ICIP ’09), pp 733–736

  18. Ekmekcioglu E, Worral S, Kondoz A (2008) Bit-rate adaptive downsampling for the coding of multi-view video with depth information. In: Proceedings of the 3DTV conference: the true vision - capture, transmission and display of 3D video (3DTV-CON ’08), pp 137–140

  19. Schwarz S, Olsson R, Sjostrom M, Tourancheau S (2012) Adaptive depth filtering for HEVC 3D video coding. In: Proceedings of picture coding symposium (PCS ’12), pp 49–52

  20. Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. In: IEEE transactions on pattern analysis and machine intelligence, vol 12, pp 629–639

  21. Canny J (1986) A computational approach to edge detection. In: IEEE transactions pattern analysis and machine intelligence, vol 6, pp 679–698

Download references

Author information

Authors and Affiliations

  1. Illinois Institute of Technology, 3301 South Dearborn Street, Siegel Hall Suite 103, Chicago, IL, 60616, USA

    Krishna Rao Vijayanagar, Maziar Loghman & Joohee Kim

Authors
  1. Krishna Rao Vijayanagar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maziar Loghman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joohee Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joohee Kim.

Additional information

This work was supported by the Technology Development Program for Commercializing System Semiconductor funded By the Ministry of Trade, industry & Energy (MOTIE, Korea). (No. 10041126, Title: International Collaborative R&BD Project for System Semiconductor)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vijayanagar, K.R., Loghman, M. & Kim, J. Real-Time Refinement of Kinect Depth Maps using Multi-Resolution Anisotropic Diffusion. Mobile Netw Appl 19, 414–425 (2014). https://doi.org/10.1007/s11036-013-0458-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-013-0458-7

Keywords

Search

Navigation