Skip to main content
SpringerLink
Log in

Video-rate calculation of depth from defocus on a FPGA

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Depth from defocus is a ranging method that provides depth estimation at every pixel in an image. It uses a pair of defocused images from a conventional monocular camera. To enable video-rate processing, which is important for industrial applications, an established algorithm was implemented on a field programmable gate array (FPGA). A bifurcating pipeline of 2-D filters provided the depth calculation. Multiplier and SRAM facilities on the FPGA were utilised efficiently by exploiting the filter symmetry. The coefficient and data bit-widths were limited to improve efficiency. The results compared favourably with the full width calculation. Range images were processed within 14 ms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Starck, J., Maki, A., Nobuhara, S., Hilton, A., Matsuyama, T.: The multiple-camera 3-D production studio. IEEE Trans. Circuits Syst. Video Technol. 19, 856–859 (2009)

    Article  Google Scholar 

  2. De Silva, D.V.S.X., Fernando, W.A.C., Yasakethu, S.L.P.: Object based coding of the depth maps for 3D video coding. IEEE Trans. Consum. Electron. 55, 1699–1706 (2009)

    Article  Google Scholar 

  3. Andreopoulos, A., Hasler, S., Wersing, H., Janssen, H., Tsotsos, J.K., Korner, E.: Active 3D object localization using a humanoid robot. IEEE Trans. Robot. 27, 47–64 (2011)

    Article  Google Scholar 

  4. De Cristóforis, P., Nitsche, M.A., Krajník, T., Mejail, M.: Real-time monocular image-based path detection: a GPU-based embedded solution for on-board execution on mobile robots. J. Real-time Image Process. 1–14 (2013). doi:10.1007/s11554-013-0356-z

  5. Taylor, R.H.: A perspective on medical robotics. Proc. IEEE 94, 1652–1664 (2006)

    Article  Google Scholar 

  6. Dandekar, O., Castro-Pareja, C., Shekhar, R.: FPGA-based real-time 3-D image preprocessing for image-guided medical interventions. J. Real-Time Image Process. 1, 285–301 (2007)

    Article  Google Scholar 

  7. Broggi, A., Cappalunga, A., Caraffi, C., Cattani, S., Ghidoni, S., Grisleri, P., Porta, P.P., Posterli, M., Zani, P.: TerraMax vision at the urban challenge 2007. IEEE Trans. Intell. Transp. Syst. 11, 194–205 (2010)

    Article  Google Scholar 

  8. Chang, K.I., Bowyer, K.W., Flynn, P.J.: An evaluation of multimodal 2D+3D face biometrics. IEEE Trans. Pattern Anal. Mach. Intell. 27, 619–624 (2005)

    Article  Google Scholar 

  9. Pentland, A.P.: A new sense for depth of field. IEEE Trans. Pattern Anal. Mach. Intell. 9, 523–531 (1987)

    Article  Google Scholar 

  10. Pentland, A., Darrell, T., Scherock, S., Girod, B.: Simple range cameras based on focal error. J Opt. Soc. Am. A 11(11), 2925–2934 (1994)

    Article  Google Scholar 

  11. Subbarao, M., Wei, T.C.: Depth from defocus and rapid auto focussing: a practical approach, pp. 773–776. Proc. IEEE Comput. Soc. Conf. Compu. Vis. Pattern Recognit., Champaign (1992)

    Google Scholar 

  12. Subbarao, M., Surya, G.: Depth from defocus: spatial domain approach. Int. J. Compu. Vis. 13(3), 271–294 (1994)

    Article  Google Scholar 

  13. Ens, J., Lawrence, P.: An investigation of methods for determining depth from defocus. IEEE Trans. Pattern Anal. Mach. Intell. 15, 97–108 (1993)

    Article  Google Scholar 

  14. Watanabe, M., Nayar, S.K.: Rational filters for passive depth from defocus. Int. J. Compu. Vis. 27(3), 203–225 (1998)

    Article  Google Scholar 

  15. Joseph-Raj, A.N., Staunton, R.C.: Rational filter design for depth from defocus. Pattern Recognit. 45, 198–207 (2012)

    Article  Google Scholar 

  16. Joseph-Raj, A.N.: Accurate depth from defocus estimation with video-rate implementation. Ph.D. dissertation, School of Eng., Univ. Warwick, Coventry (2009)

    Google Scholar 

  17. Rajagopalan, A.N., Chaudhuri, S.: Depth from defocus: a real aperture imaging approach. Springer, New York (1999)

    Google Scholar 

  18. Rajagopalan, A.N., Chaudhuri, S.: An MRF model-based approach to simultaneous recovery of depth and restoration from defocused images. IEEE Trans. Pattern Anal. Mach. Intell. 21, 577–589 (1999)

    Article  Google Scholar 

  19. Favaro, P., Soatto, S.: A geometric approach to shape from defocus. IEEE Trans. Pattern Anal. Mach. Intell. 27, 406–417 (2005)

    Article  Google Scholar 

  20. Favaro, P., Soatto, S.: Learning shape from defocus. In: Proc. 7th European Conf. Computer Vision, Copenhagen, Part 2 pp. 735–745 (2002)

  21. Deschenes, F., Ziou, D., Fuchs, P.: Improved estimation of defocus blur and spatial shifts in spatial domain: a homotopy-based approach. Pattern Recognit. 36(9), 2105–2125 (2003)

    Article  MATH  Google Scholar 

  22. Deschenes, F., Ziou, D., Fuchs, P.: A homotopy-based approach for computing defocus blur and affine transform simultaneously. Pattern Recognit. 41(7), 2263–2282 (2008)

    Article  MATH  Google Scholar 

  23. Watanabe, M., Nayar, S.K.: Minimal operator set for passive depth from defocus, pp. 431–438. IEEE Conference on CVPR, San Francisco (1996)

    Google Scholar 

  24. Nayar, S.K., Watanabe, M., Noguchi, M.: Real-time focus range sensor. IEEE Trans. Pattern Anal. Mach. Intell. 18(12), 1186–1198 (1996)

    Article  Google Scholar 

  25. Ghita, O., Whelan, P.F.: A video-rate sensor based on depth from defocus. Opt. Laser Technol. 33(3), 167–176 (2001)

    Article  Google Scholar 

  26. Leroy, J.V., Simon, T., Deschenes, F.: Real time monocular depth from defocus. In: 3rd International Conference on Image and Signal Processing, pp. 103–111 (2008)

  27. Favaro P.: Recovering thin structures via nonlocal-means regularization with application to depth from defocus. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1133–1140 (2010)

  28. Ben-Ari R., Raveh G.: Variational, depth from defocus in real-time. IEEE Int. Conf. Comput. Vis. 522–529 (2011)

  29. Zhou, C., Lin, S., Nayar, S.K.: Coded aperture pairs for depth from defocus and defocus deblurring. Int. J. Comput. Vis. 93(1), 53–72 (2011)

    Article  Google Scholar 

  30. Jin, S., Kim, D., Nguyen, T.T., Kim, D., Kim, M., Jeon, J.W.: Design and implementation of a pipelined datapath for high-speed face detection using FPGA. IEEE Trans. Ind. Informat. 8(1), 158–167 (2012)

    Article  Google Scholar 

  31. Monmasson, E., Idkhajine, L., Cirstea, M.N., Bahri, I., Tisan, A., Naouar, M.W.: FPGAs in industrial control applications. IEEE Trans. Ind. Informat. 7, 224–243 (2011)

    Article  Google Scholar 

  32. Xilinx, Virtex-II Pro/Virtex-II Pro X, Complete Data Sheet [Online]. Available: http://www.xilinx.com/support/documentation/user_guides/ug012.pdf. Accessed 14 Dec 2014

  33. Reddy, H.C., Khoo, I.H., Rajan, P.K.: 2-D symmetry: theory and filter design applications. IEEE Circuits Syst. Mag. 3(3), 4–33 (2003)

    Article  Google Scholar 

  34. Cheng, Chao, Parhi, K.K.: Hardware efficient fast parallel FIR filter structures based on iterated short convolution. IEEE Trans. Circuits Syst. I Regul. Pap. 51(8), 1492–1500 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  35. Bougas, P., Kalivas, P., Tsirikos, A., Pekmestzi, K.Z.: Pipelined array-based FIR filter folding. IEEE Trans. Circuits Syst. I Regul. Pap. 52(1), 108–118 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  36. Lim, Jae S.: Two dimensional signal and image processing, pp. 196–199. Prentice Hall Inc, New Jersey (1990)

    Google Scholar 

  37. Kung, H.T.: Why systolic architectures? IEEE Trans. Compu. 15, 37–46 (1982)

    Google Scholar 

  38. Kung, H.T., Leiserson, C.E., Mead, C.A., Conway, L.A.: Systolic arrays (for VLSI). Sparse Matrix Proc. 1978, 256 (1979)

    MathSciNet  Google Scholar 

  39. Hsia, S.-C.: Parallel VLSI design for a real-time video-impulse noise-reduction processor. IEEE Trans. Very Large Scale Integr. Syst. 11, 651–658 (2003)

    Article  Google Scholar 

  40. Claxton, C.D., Staunton, R.C.: Measurement of the point-spread function of a noisy imaging system. J. Optical Soc. Am. A 25, 159–170 (2008)

    Article  Google Scholar 

  41. Bonyadi, Y.: 3D TV from miniature camera systems. MSc. dissertation, School of Eng., Univ. Warwick, Coventry (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronics Engineering, VIT University, Vellore, Tamil Nadu, India

    Alex Noel Joseph Raj

  2. School of Engineering, University of Warwick, Coventry, CV4 7, UK

    Richard C. Staunton

Authors
  1. Alex Noel Joseph Raj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard C. Staunton
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alex Noel Joseph Raj.

Appendices

Appendix 1

See Table 4.

Table 4 A comparison of various depth estimation techniques
Full size table

Appendix 2: General triangular convolution

The following algorithm can be used to generate the partial equations used by the triangular method. For an m × m convolution where m is odd, there is eightfold symmetry, and the central pixel is I 0,0, then the pixels in the window will be indexed I −n,−n to I +n,+n , where n = (m −1)/2. The number of partial equations, coefficients and multipliers required \(a = \sum\nolimits_{i = 1}^{n + 1} i\). Let the partial results be P b and the coefficients C b where b is an integer from 1 to a. Then if n > 1:

As algorithm:

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Joseph Raj, A.N., Staunton, R.C. Video-rate calculation of depth from defocus on a FPGA. J Real-Time Image Proc 14, 469–480 (2018). https://doi.org/10.1007/s11554-014-0480-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-014-0480-4

Keywords

Search

Navigation