Skip to main content
SpringerLink
Log in

VLSI architecture for 4-D depth filtering

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

We propose the application of light field cameras and depth-selective 4-D IIR filtering to enable video surveillance, leveraging the post-capture depth-selective filtering enabled by computational photography. Novel ultralow-complexity differential-form depth-selective 4-D IIR filter algorithms and their corresponding architectures are proposed for processing 4-D light fields. Practical results are presented for real-world video sequences, and a CMOS VLSI implementation of the arithmetic processing elements is synthesized. The architecture shows \(86.66\), \(78.94\,\%\) reduction in multipliers and adders compared to direct-form structure and delivers 26 frames/s for light fields of size \(16\times 16\times 128\times 128\).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Adelson, E.H., Bergen, J.R.: The plenoptic function and the elements of early vision. Comput. Models Vis. Process. 1, 3–20 (1991)

    Google Scholar 

  2. Agathoklis, P., Bruton, L.T.: Practical-BIBO stability of N-dimensional discrete systems. Electron. Circuits Sys. IEEE Proc. G 130(6), 236–242 (1983)

    Article  Google Scholar 

  3. Bertschmann, R.K., Bartley, N.R., Bruton, L.T.: A 3-D integrator-differentiator double-loop (IDD) filter for raster-scan video processing. In: IEEE international symposium on circuits and systems, ISCAS’95, vol. 1, pp. 471–473 (1995)

  4. Bruton, L.T., Bartley, N.R.: Three-dimensional image processing using the concept of network resonance. IEEE Trans. Circuits Sys. 32(7), 664–672 (1985)

    Google Scholar 

  5. Dansereau, D.G., Bruton, L.T.: A 4D frequency-planar IIR filter and its application to light field processing. In: Circuits and systems, 2003. ISCAS ’03, vol. 4, pp. 476–479 (2003)

  6. Dansereau, D.G., Bruton, L.T.: A 4-D dual-fan filter bank for depth filtering in light fields. IEEE Trans. Signal Process. 55(2), 542–549 (2007)

    Article  MathSciNet  Google Scholar 

  7. Dansereau, D.G., Mahon, I., Pizarro, O., Williams, S.B.: Plenoptic flow: Closed-form visual odometry for light field cameras. In: Proceedings, 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 4455–4462. IEEE (2011)

  8. Harris, M.: Focusing on everything: light field cameras promise an imaging revolution. IEEE Spectr. 49, 44–50 (2012)

    Article  Google Scholar 

  9. Jarvis, R.A.: A perspective on range finding techniques for computer vision. Pattern Anal. Mach. Intell. IEEE Trans. PAMI 5, 122–139 (1983)

    Article  Google Scholar 

  10. Joshi, N., Avidan, S., Matusik, W., Kriegman, D.J.: Synthetic aperture tracking: tracking through occlusions. In: IEEE international conference on computer vision (ICCV), pp. 1–8. IEEE (2007)

  11. Lai, S.H., Fu, C.W., Chang, S.: A generalized depth estimation algorithm with a single image. Pattern Anal. Mach. Intell. IEEE Trans. 14, 405–411 (1992)

    Article  Google Scholar 

  12. Levoy, M., Hanrahan, P.: Light field rendering. In: Proceedings of the 23rd annual conference on computer graphics and interactive techniques, pp. 31–42. ACM, New York (1996)

  13. Lumsdaine, A., Georgiev, T.: Full resolution lightfield rendering. Technical report, Adobe Systems (2008)

  14. Madanayake, A., Bruton, L.T.: A fully-multiplexed first-order frequency-planar module for fan, beam, and cone plane-wave filters. IEEE Trans. Circuits Sys. II Express Briefs 53(8), 697–701 (2006)

    Article  Google Scholar 

  15. Madanayake, A., Wimalagunaratne, R., Dansereau, D.G., Bruton, L.T.: Design and FPGA-implementation of 1st-order 4D IIR frequency-hyperplanar digital filters. In: 2011 IEEE 54th international midwest symposium on circuits and systems (MWSCAS), pp. 1–4 (2011)

  16. Ng, R.: Fourier slice photography. ACM Trans. Graph. (TOG) 24, 735–744 (2005)

    Article  Google Scholar 

  17. Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P.: Light field photography with a hand-held plenoptic camera. Computer Science Technical Report CSTR 2 (2005)

  18. Ramamoorthy, P.A., Bruton, L.T.: Design of stable 2-dimensional discrete recursive filters. Electron. Lett. 12(25), 659–660 (1976)

    Article  Google Scholar 

  19. Smith, B.M., Zhang, L., Jin, H., Agarwala, A.: Light field video stabilization. In: 2009 IEEE 12th international conference on computer vision, pp. 341–348. IEEE (2010)

  20. The (New) Stanford Light Field Archive (2012) http://lightfield.stanford.edu/

  21. Vaish, V., Levoy, M., Szeliski, R., Zitnick, C., Kang, S.: Reconstructing occluded surfaces using synthetic apertures: Stereo, focus and robust measures. In: 2006 IEEE computer society conference on computer vision and pattern recognition, vol. 2, pp. 2331–2338 (2006)

  22. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.V., Antunez, E., Barth, A., Adams, A., Horowitz, M., Levoy, M.: High performance imaging using large camera arrays. ACM Trans. Graph. (TOG) 24, 765–776 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Akron, Akron, OH, USA

    Arjuna Madanayake & Randeel Wimalagunarathne

  2. Australian Centre for Field Robotics, School of Aerospace, Mechanical and Mechatronic Engineering, University of Sydney, Sydney, NSW, Australia

    Donald G. Dansereau

  3. Signal Processing Group, Departamento of Estatística, Universidade Federal de Pernambuco, Recife, PE, Brazil

    Renato J. Cintra

  4. Department of Electrical and Computer Engineering, University of Calgary, Calgary, AB, Canada

    Len T. Bruton

Authors
  1. Arjuna Madanayake
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Randeel Wimalagunarathne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Donald G. Dansereau
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Renato J. Cintra
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Len T. Bruton
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arjuna Madanayake.

Additional information

This work was supported by The University of Akron, Ohio, USA; CNPq and FACEPE, Brazil; and NSERC, Canada.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Madanayake, A., Wimalagunarathne, R., Dansereau, D.G. et al. VLSI architecture for 4-D depth filtering. SIViP 9, 809–818 (2015). https://doi.org/10.1007/s11760-013-0511-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-013-0511-9

Keywords

Search

Navigation