Skip to main content
SpringerLink
Log in

Fusion of Time-of-Flight and Stereo for Disambiguation of Depth Measurements

  • Conference paper
Computer Vision – ACCV 2012 (ACCV 2012)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 7727))

Included in the following conference series:

Abstract

The complementary nature of time-of-flight and stereo has led to their fusion systems, providing high quality depth maps robustly against depth bias and random noise of the time-of-flight camera as well as the lack of scene texture. This paper shows that the fusion system is also effective for disambiguating time-of-flight depth measurements caused by phase wrapping, which records depth values that are much less than their actual values if the scene points are farther than a certain maximum range. To recover the unwrapped depth map, we build a Markov random field based on a constraint that an accurately unwrapped depth value should minimize the dissimilarity between its projections on the stereo images. The unwrapped depth map is then adapted to stereo matching, reducing the matching ambiguity and enhancing the depth quality in textureless regions. Through experiments we show that the proposed method extends the range use of the time-of-flight camera, delivering unambiguous depth maps of real scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. http://www.mesa-imaging.ch/prodview4k.php

  2. http://en.wikipedia.org/wiki/Time-of-flight_camera

  3. Lindner, M., Schiller, I., Kolb, A., Koch, R.: Time-of-flight sensor calibration for accurate range sensing. CVIU 114, 1318–1328 (2010)

    Google Scholar 

  4. Kim, Y.S., Kang, B., Lim, H., Choi, O., Lee, K., Kim, J.D.K., Kim, C.Y.: Parametric model-based noise reduction for ToF depth sensors. In: IS&T/SPIE EI (2012)

    Google Scholar 

  5. Göktürk, S.B., Yalcin, H., Bamji, C.: A time-of-flight depth sensor—system description, issues and solutions. In: CVPR Workshops (2004)

    Google Scholar 

  6. Choi, O., Lim, H., Kang, B., Kim, Y.S., Lee, K., Kim, J.D.K., Kim, C.Y.: Range unfolding for time-of-flight depth cameras. In: ICIP (2010)

    Google Scholar 

  7. Droeschel, D., Holz, D., Behnke, S.: Probabilistic phase unwrapping for time-of-flight cameras. In: Joint 41st International Symposium on Robotics and 6th German Conference on Robotics (2010)

    Google Scholar 

  8. Droeschel, D., Holz, D., Behnke, S.: Multifrequency phase unwrapping for time-of-flight cameras. In: IROS (2010)

    Google Scholar 

  9. McClure, S.H., Cree, M.J., Dorrington, A.A., Payne, A.D.: Resolving depth-measurement ambiguity with commercially available range imaging cameras. In: Image Processing: Machine Vision Applications III (2010)

    Google Scholar 

  10. Jongenelen, A.P.P., Carnegie, D.A., Payne, A.D., Dorrington, A.A.: Maximizing precision over extended unambiguous range for TOF range imaging systems. In: IEEE Instrumentation & Measurement Technology Conference (2010)

    Google Scholar 

  11. Kuhnert, K.D., Stommel, M.: Fusion of stereo-camera and PMD-camera data for real-time suited precise 3D environment reconstruction. In: IROS (2006)

    Google Scholar 

  12. Beder, C., Bartczak, B., Koch, R.: A Combined Approach for Estimating Patchlets from PMD Depth Images and Stereo Intensity Images. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 11–20. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  13. Lindner, M., Lambers, M., Kolb, A.: Data fusion and edge-enhanced distance refinement for 2D RGB and 3D range images. International Journal of Intelligent Systems Technologies and Applications 5, 344–354 (2008)

    Article  Google Scholar 

  14. Gudmundsson, S.Á., Aanæs, H., Larsen, R.: Fusion of stereo vision and time-of-flight imaging for improved 3D estimation. International Journal of Intelligent Systems Technologies and Applications 5, 425–433 (2008)

    Article  Google Scholar 

  15. Hahne, U., Alexa, M.: Combining time-of-flight depth and stereo images without accurate extrinsic calibration. International Journal of Intelligent Systems Technologies and Applications 5, 325–333 (2008)

    Article  Google Scholar 

  16. Netramai, C., Melnychuk, O., Joochim, C., Roth, H.: Combining PMD and stereo camera for motion estimation of a mobile robot. In: IFAC World Congress (2008)

    Google Scholar 

  17. Hahne, U., Alexa, M.: Depth Imaging by Combining Time-of-Flight and On-Demand Stereo. In: Kolb, A., Koch, R. (eds.) Dyn3D 2009. LNCS, vol. 5742, pp. 70–83. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  18. Kim, Y.M., Theobalt, C., Diebel, J., Kosecka, J., Miscusik, B., Thrun, S.: Multi-view image and ToF sensor fusion for dense 3D reconstruction. In: IEEE Workshop on 3-D Digital Imaging and Modeling (2009)

    Google Scholar 

  19. Zhu, J., Wang, L., Gao, J., Yang, R.: Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE T. PAMI 32, 899–909 (2010)

    Article  Google Scholar 

  20. Zhu, J., Wang, L., Yang, R., Davis, J.E., Pan, Z.: Reliability fusion of time-of-flight depth and stereo geometry for high quality depth maps. IEEE T. PAMI 33, 1400–1414 (2011)

    Article  Google Scholar 

  21. Hansard, M., Horaud, R., Amat, M., Lee, S.: Projective alignment of range and parallax data. In: CVPR (2011)

    Google Scholar 

  22. Choi, O., Lim, H., Kang, B., Kim, Y.S., Lee, K., Kim, J.D.K., Kim, C.Y.: Discrete and continuous optimizations for depth image super-resolution. In: IS&T/SPIE EI (2012)

    Google Scholar 

  23. http://msdn.microsoft.com/en-us/library/hh438998.aspx

  24. Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: Real-time dense surface mapping and tracking. In: ISMAR (2011)

    Google Scholar 

  25. http://www.ptgrey.com/products/bumblebee2/bumblebee2_stereo_camera.asp

  26. Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press (2003)

    Google Scholar 

  27. Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C++. Cambridge University Press (2002)

    Google Scholar 

  28. Birchfield, S., Tomasi, C.: A pixel dissimilarity measure that is insensitive to image sampling. IEEE T. PAMI 20, 401–406 (1998)

    Article  Google Scholar 

  29. Heo, Y.S., Lee, K.M., Lee, S.U.: Illumination and camera invariant stereo matching. In: CVPR (2008)

    Google Scholar 

  30. Yoon, K.J., Kweon, I.S.: Adaptive support-weight approach for correspondence search. IEEE T. PAMI 28, 650–656 (2006)

    Article  Google Scholar 

  31. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE T. PAMI 23, 1222–1239 (2001)

    Article  Google Scholar 

  32. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: ICCV (1998)

    Google Scholar 

  33. http://vision.middlebury.edu/stereo/eval/

  34. Buttgen, B., Lustenberger, F., Seitz, P.: Demodulation pixel based on static drift fields. IEEE Transactions on Electron Devices 53, 2741–2747 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Samsung Advanced Institute of Technology, Republic of Korea

    Ouk Choi & Seungkyu Lee

Authors
  1. Ouk Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seungkyu Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, 151-744, Seoul, Korea

    Kyoung Mu Lee

  2. Microsoft Research Asia, No. 5, Danling st., Haidian district, 100080, Beijing, P.R. China

    Yasuyuki Matsushita

  3. School of Interactive Computing, Georgia Institute of Technology, 801 Atlantic Drive, CCB 315, 30332, Atlanta, GA, USA

    James M. Rehg

  4. Institute of Automation, National Laboratory of Pattern Recognition, Chinese Academy of Sciences, Zhong Quan Cun East Road 95, Haidian District,, 100 190, Beijing, P.R. China

    Zhanyi Hu

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Choi, O., Lee, S. (2013). Fusion of Time-of-Flight and Stereo for Disambiguation of Depth Measurements. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds) Computer Vision – ACCV 2012. ACCV 2012. Lecture Notes in Computer Science, vol 7727. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37447-0_49

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-37447-0_49

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-37446-3

  • Online ISBN: 978-3-642-37447-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation