Skip to main content
SpringerLink
Log in

Reconfigurable hardware implementation of a phase-correlation stereoalgorithm

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper describes the implementation of a stereo-vision system using Field Programmable Gate Arrays (FPGAs). Reconfigurable hardware, including FPGAs, is an attractive platform for implementing vision algorithms due to its ability to exploit parallelism often found in these algorithms, and due to the speed with which applications can be developed as compared to hardware. The system outputs 8-bit, subpixel disparity estimates for 256× 360 pixel images at 30,fps. A local-weighted phase correlation algorithm for stereo disparity [Fleet, D. J.: {Int. Conf. Syst. Man Cybernetics 1:48–54 (1994)] is implemented. Despite the complexity of performing correlations on multiscale, multiorientation phase data, the system runs as much as 300 times faster in hardware than its software implementation. This paper describes the hardware platform used, the algorithm, and the issues encountered during its hardware implementation. Of particular interest is the implementation of multiscale, steerable filters, which are widely used in computer vision algorithms. Several trade-offs (reducing the number of filter orientations from three to two, using fixed-point computation, changing the location of one localized low-pass filter, and using L1 instead of L2 norms) were required to both fit the design into the available hardware and to achieve video-rate processing. Finally, results from the system are given both for synthetic data sets as well as several standard stereo-pair test images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Fleet, D.J.: Disparity from local weighted phase correlation. Int. Conf. Syst. Man Cybernetics 1, 48–54 (1994)

    Google Scholar 

  • Xilinx Inc.: Design tools center, cited May 3, 2005. [Online]. Available: http://www.xilinx.com/products/design_resources/design_tool/index.htm

  • Altera Corporation: DSP Builder, cited May 3, 2005 [Online]. Available: http://www.altera.com/products/software/products/dsp/dsp-builder.html

  • Fung, J., Mann, S.: Using multiple graphics cards as a general purpose parallel computer: Applications to computer vision. In: Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK, pp. 805–808 (2004) online: http://www.eyetap.org/papers/docs/procicpr2004/

  • Yang, R., Pollefeys, M.: Multi-resolution real-time stereo on commodity graphics hardware. In: Proceedings of the 2003 IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, Madison, Wisconsin, pp. 211–218 (2003) [Online]. Available: http://citeseer.ist.psu.edu/article/yang03multiresolution.html

  • van Ierssel, M.: TM-3 documentation (2002) [Online]. Available: http://www.eecg.toronto.edu/tm3/

  • van Ierssel, M., Galloway, D., Chow, P., Rose, J.: The Transmogrifier 3-a: Hardware and software for a 3 million gate rapid prototyping system. In: Micronet Annual Workshop (2001)

  • Xilinx Inc.: Xilinx data sheets (2002) [Online]. Available: http://direct.xilinx.com/bvdocs/publications/ds022-1.pdf

  • Faugeras, O., Hotz, B., Mathieu, H., Viéville, T., Zhang, Z., Fua, P., Théron, E., Moll, L., Berry, G., Vuillemin, J., Bertin, P., Proy, C.: Real time correlation-based stereo: Algorithm, implementations and applications. INRIA Sophia Antipolis, Tech. Rep. Research Report 2013 (1993) [Online]. Available: ftp://ftp.inria.fr/INRIA/publication/publi-ps-gz/RR/RR-2013.ps.gz

  • Hou, K.M., Belloum, A.: A reconfigurable and flexible parallel 3d vision system for a mobile robot. In: IEEE Workshop on Computer Architecture for Machine Perception New Orleans, Louisiana (1993)

  • Kanade, T., Yoshida, A., Oda, K., Kano, H., Tanaka, M.: A stereo machine for video-rate dense depth mapping and its new applications. In: Proceedings of the 15th IEEE Computer Vision & Pattern Recognition Conference, pp. 196–202. San Francisco (1996)

  • Woodfill, J., Herzen, B.V.: Real time stereo vision on the parts reconfigurable computer. In: 5th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, pp. 201–210 (1997)

  • Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Proceedings of the 3rd European Conference on Computer Vision, pp. 150–158 (1994) http://www.cs.cornell.edu/rdz/Papers/Archive/neccv.ps, http://www.cs.cornell.edu/rdz/Papers/Archive/nplt-journal.ps.gz

  • Burt, P.J.: A pyramid-based front-end processor for dynamic vision applications. In: Proceedings of the IEEE 90(7), 1188–1200 (2002)

  • van der Wal, G., Hansen, M., Piacentino, M.: The acadia vision processor.In: Proceedings of the Fifth IEEE International Workshop on Computer Architecture for Machine Perception, pp. 31–40. Padua, Italy (2000)

  • Hirschmüller, H., Innocent, P.R., Garibaldi, J.: Real-time correlation-based stereo vision with reduced border errors. Int. J. Comput. Vis. 47(1–3), 229–246 (2002), stereo, intensity correlation, MMX, fast

    Google Scholar 

  • Mühlmann, K., Maier, D., Hesser, J., Anner, R.M.: Calculating dense disparity maps from color stereo images, an efficient implementation. Int. J. Comput. Vis. 47(1–3), 79–88 (2002), stereo, intensity correlation, MMX, fast

    Google Scholar 

  • Mulligan, J., Isler, V., Daniilidis, K.: Trinocular stereo: A real-time algorithm and its evaluation. Int. J. Comput. Vis. 47(1–3), 51–61 (2002), stereo, trinocular, DFW-V500

    Google Scholar 

  • Sun, C.: Fast stereo matching using rectangular subregioning and 3d maximum-surface techniques. Int. J. Comput. Vis. 47(1–3), 99–117 (2002)

    Article  MATH  Google Scholar 

  • Veksler, O.: Stereo correspondence with compact windows via minimum ratio cycle. IEEE Trans. Pattern Anal. Machine Intell. 24(12), 1654–1660 (2002)

    Article  Google Scholar 

  • Burt, P., Adelson, E.: The laplacian pyramid as a compact image code. IEEE Trans. Commun. 31, 532–540 (1983)

    Article  Google Scholar 

  • Meerbergen, G.V., Vergauwen, M., Pollefeys, M., Gool, L.V.: A hierarchical symmetric stereo algorithm using dynamic programming. Int. J. Comput. Vis. 47(1–3), 275–285 (2002)

    Article  MATH  Google Scholar 

  • Birchfield, S., Tomasi, C.: Depth discontinuities by pixel-to-pixel stereo. Int. J. Comput. Vis. 35(3), 269–293 (1999)

    Article  Google Scholar 

  • Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1–3), 7–42 (2002)

    Article  MATH  Google Scholar 

  • Point Grey Research: Point Grey Research homepage. cited May 3, 2005 [Online]. Available: http://www.ptgrey.com/

  • Videre Design: SRI small vision system, cited May 3, 2005 [Online]. Available: http://www.videredesign.com/svs.htm

  • Konolige, K.: Small vision systems: Hardware and implentation. In: Proceedings of the Eighth International Symposium on Robotics Research (Robotics Research 8), Hayama, Japan, pp. 203–212 (1997) [Online].Available: citeseer.ist.psu.edu/konolige97small.html

  • Canesta Inc.: Canesta ep development kit. cited May 3, 2005. [Online]. Available: http://www.canesta.com/devkit.htm

  • Baker, H.H., Binford, T.O.: Depth from edges and intensity based stereo. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, pp. 631–636. Vancouver (1981)

  • Ayache, N., Faverjon, B.: Efficient registration of stereo images by matching graph descriptions of edge segments. Int. J. Comput. Vis. 1(2), 107–131 (1987)

    Article  Google Scholar 

  • Fleet, D.J., Jepson, A.D., Jenkin, M.R.M.: Phase-based disparity measurement. CVGIP: Image Understanding 53(2), 198–210 (1991)

    Article  MATH  Google Scholar 

  • Wang, J.J.: Image matching using the windowed fourier phase. Int. J. Comput. Vis. 11(3), 211–236 (1993), read

    Google Scholar 

  • Freeman, W.T., Adelson, E.H.: The design and use of steerable filters. IEEE Trans. Pattern Anal. Mach. Intell. 13(9), 891–906 (1991)

    Article  Google Scholar 

  • Trucco, E., Verri, A.: Introductory Techniques for 3-D Computer Vision. Prentice Hall (1998)

  • Chang, M.L., Hauck, S.: Precis: A design-time precision analysis tool. In: 10th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, pp. 229–283 (2002) [Online]. Available: http://www.ee.washington.edu/faculty/hauck/publications/PrecisFCCM2002

  • Darabiha, A.: Video-rate stereo vision on reconfigurable hardware. Master's thesis, Department of Electrical & Computer Engineering, University of Toronto (2003)

  • Andraka, R.: A survey of CORDIC algorithms for FPGAs. In:Proceedings of the 1998 ACM/SIGDA Sixth International Symposium on Field Programmable Gate Arrays (FPGA '98), Monterey, CA, pp. 191–200 (1998) [Online]. Available: http://www.andraka.com/files/crdcsrvy.pdf

  • Valls, J., Kuhlmann, M., Parhi, K.K.: Evaluation of CORDIC algorithms for FPGA design. J. VLSI Signal Process. 32, 207–222 (2002)

    Article  MATH  Google Scholar 

  • Fua, P.: A parallel stereo algorithm that produces dense depth maps and preserves image features. Mach. Vis. & Appl. 6(1), 35–49 (1993), INRIA Research Report 1369

    Google Scholar 

  • Gluckman, J., Nayar, S.K.: Rectified catadioptric stereo sensors. IEEE Trans. Pattern Anal. Machine Intell. 24(2), 224–236 (2002), stereo, mirros, rectified

    Google Scholar 

  • Fleet, D.J., Black, M.J., Jepson, A.D.: Motion feature detection using steerable flow fields. In: Proceedings of the 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 274–281. Santa Barbara (1998)

  • Fleet, D.J., Jepson, A.D.: Computation of component image velocity from local phase information. Int. J. Comput. Vis. 5(1), 77–104 (1990)

    Article  Google Scholar 

  • Jepson, A.D., Fleet, D.J., El-Maraghi, T.F.: Robust on-line appearance models for visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 415–422 Kauai, (2001) [Online]. Available: http://www.cs.toronto.edu/tem/cvpr01.pdf

  • Carneiro, G., Jepson, A.D.: Local phase-based features. In: Proceedings of the 2002 European Conference on Computer Vision, vol. 1, pp. 282–296. Copenhagen, Denmark (2002)

  • Masrani, D.K., MacLean, W.J.: Expanding disparity range in an FPGA stereo system while keeping resource utilization low. In: The First IEEE Workshop on Embedded Computer Vision, CVPR 2005 (2005)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Toronto, Toronto, M5S 1A1, Canada

    Ahmad Darabiha, W. James MacLean & Jonathan Rose

Authors
  1. Ahmad Darabiha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. W. James MacLean
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonathan Rose
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to W. James MacLean.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Darabiha, A., MacLean, W.J. & Rose, J. Reconfigurable hardware implementation of a phase-correlation stereoalgorithm. Machine Vision and Applications 17, 116–132 (2006). https://doi.org/10.1007/s00138-006-0018-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-006-0018-2

Keywords

Search

Navigation