Christian Zinner
ABSTRACT
This work introduces a real-time capable realization of an area-based stereo matching algorithm that is distributed on two embedded smart camera platforms. Combining common industrial smart cameras by this way enables real time stereo vision as a new application domain for these platforms. With the proposed method, the computational load can be shared among the two cameras equipped with a digital signal processor each. This results in an efficient processing of a computational intensive stereo matching algorithm--- the processing speed is significantly faster compared to a single chip solution. Beside that, various optimizations especially developed for digital signal processors additionally increase the performance. On input images of 450×375 and a disparity range of 60, the system achieves a stereo processing performance of 11.8 frames per second. The stereo matching quality is evaluated using the Middlebury stereo database where it is the only purely embedded algorithm.
- }}K. Ambrosch. Mapping Stereo Matching Algorithms to Hardware. PhD thesis, Vienna University of Technology, 2009.Google Scholar
- }}A. Banno and K. Ikeuchi. Disparity map refinement and 3d surface smoothing via directed anisotropic diffusion. In Proceedings of the IEEE 12th International Conference on Computer Vision Workshops, pages 1870--1877, 2009.Google ScholarCross Ref
- }}S. Birchfield and C. Tomasi. Depth discontinuities by pixel-to-pixel stereo. International Journal of Computer Vision, 35:3:269--293, 1996. Google ScholarDigital Library
- }}J. Y. Bouguet. Camera Calibration Toolbox for Matlab, 2008. http://www.vision.caltech.edu/bouguetj/calib_doc.Google Scholar
- }}G. Bradski and A. Kaehler. Learning OpenCV: Computer Vision with the OpenCV Library. O'Reilly, Cambridge, MA, 2008.Google Scholar
- }}M. Z. Brown, D. Burschka, and G. D. Hager. Advances in computational stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25:993--1008, 2003. Google ScholarDigital Library
- }}N. Chang, T.-M. Lin, T.-H. Tsai, Y.-C. Tseng, and T.-S. Chang. Real-time dsp implementation on local stereo matching. In Proc. IEEE International Conference on Multimedia and Expo, pages 2090--2093, 2--5 July 2007.Google ScholarCross Ref
- }}O. Faugeras, B. Hotz, H. Mathieu, T. Vieville, Z. Zhang, P. Fua, E. Theron, L. Moll, G. Berry, J. Vuillemin, P. Bertin, and C. Proy. Real-time correlation based stereo: algorithm implementations and applications. Technical Report 2013, INRIA, 1993.Google Scholar
- }}P. F. Felzenszwalb and D. P. Huttenlocher. Efficient belief propagation for early vision. Int. J. Comput. Vision, 70(1):41--54, 2006. Google ScholarDigital Library
- }}S. Forstmann, Y. Kanou, J. Ohya, S. Thuering, and A. Schmitt. Real-time stereo by using dynamic programming. In Conference on Computer Vision and Pattern Recognition Workshop, page 29. IEEE Computer Society, 2004. Google ScholarDigital Library
- }}H. Hirschmueller. Accurate and efficient stereo processing by semi-global matching and mutual information. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. Google ScholarDigital Library
- }}M. Humenberger, C. Zinner, and W. Kubinger. Performance evaluation of a census-based stereo matching algorithm on embedded and multi-core hardware. In Proceedings of the International Symposium on Image and Signal Processing and Analysis, 2009.Google ScholarCross Ref
- }}M. Humenberger, C. Zinner, M. Weber, W. Kubinger, and M. Vincze. A fast stereo matching algorithm suitable for embedded real-time systems. To appear in Journal on Computer Vision and Image Understanding, 2010. Google ScholarDigital Library
- }}T. Kanade, A. Yoshida, K. Oda, H. Kano, and M. Tanaka. A stereo machine for video-rate dense depth mapping and its new applications. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 196--202, 1996. Google ScholarDigital Library
- }}B. Khaleghi, S. Ahuja, and Q. Wu. An improved real-time miniaturized embedded stereo vision system (mesvs-ii). In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops CVPR Workshops 2008, pages 1--8, 23--28 June 2008.Google ScholarCross Ref
- }}V. Kolmogorov and R. Zabih. Multi-camera scene reconstruction via graph cuts. In Proceedings of the European Conference on Computer Vision, pages 82--96, 2002. Google ScholarDigital Library
- }}D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1--3):7--42, 2002. Google ScholarDigital Library
- }}M. Sonka, V. Hlavac, and R. Boyle. Image Processing, Analysis, and Machine Vision. Thomson-Engineering, 2 edition, 1999.Google Scholar
- }}Texas Instruments. TMS320C6414T, TMS320C6415T, TMS320C6416T Fixed-Point Digital Signal Processors, 2003. Lit. Number: SPRS226K.Google Scholar
- }}VidereDesign. Stereo-on-a-Chip Stereo Head User Manual 1.3. Videre Design, 2007. http://www.videredesign.com/vision/stoc.htm.Google Scholar
- }}Vision Components GmbH. VC4XXX Operating Manual, 2008. Document name: VC4XXX_HW.pdf.Google Scholar
- }}Vision Components GmbH. VCRT 5.0 Software Manual, 2008. Document name: VCRT5.pdf.Google Scholar
- }}J. Woodfill and B. V. Herzen. Real-time stereo vision on the parts reconfigurable computer. In IEEE Symposium on FPGAs for Custom Computing Machines, pages 242--250. IEEE Computer Society Press, 1997. Google ScholarDigital Library
- }}J. I. Woodfill, G. Gordon, D. Jurasek, T. Brown, and R. Buck. The Tyzx DeepSea G2 Vision System, A Taskable, Embedded Stereo Camera. In Proceedings of the 2006 Conference on Computer Vision and Pattern Recoginition Workshops, 2006. Google ScholarDigital Library
- }}Q. Yang, L. Wang, R. Yang, S. Wang, M. Liao, and D. Nister. Real-time global stereo matching using hierarchical belief propagation. In Proceedings of The British Machine Vision Conference, pages 989--998, 2006.Google ScholarCross Ref
- }}J. S. Yedidia, W. T. Freeman, and Y. Weiss. Understanding belief propagation and its generalizations. pages 239--269, 2003. Google ScholarDigital Library
- }}R. Zabih and J. Woodfill. Non-parametric local transforms for computing visual correspondence. In Proceedings of 3rd European Conf. Computer Vision, pages 151--158, Stockholm, 1994. Google ScholarDigital Library
- }}Z. Zhang. Flexible camera calibration by viewing a plane from unknown orientations. In Proc. Seventh IEEE International Conference on Computer Vision The, volume 1, pages 666--673, 20--27 Sept. 1999.Google ScholarCross Ref
- }}C. Zinner, M. Humenberger, K. Ambrosch, and W. Kubinger. An optimized software-based implementation of a census-based stereo matching algorithm. In Advances in Visual Computing, Lecture Notes in Computer Science, volume 5358, pages 216--227. Springer, 2008. Google ScholarDigital Library
- }}C. Zinner and W. Kubinger. ROS-DMA: A DMA double buffering method for embedded image processing with resource optimized slicing. In Proc. 12th IEEE Real-Time and Embedded Technology and Applications Symposium, pages 361--372, 04--07 April 2006. Google ScholarDigital Library
- }}C. Zinner, W. Kubinger, and R. Isaacs. Pfelib: A performance primitives library for embedded vision. EURASIP Journal on Embedded Systems, 2007(1):14, 1 2007. Google ScholarDigital Library
Index Terms
- Distributed real-time stereo matching on smart cameras
Recommendations
Stereo vision using two PTZ cameras
The research of traditional stereo vision is mainly based on static cameras. As PTZ (Pan-Tilt-Zoom) cameras are able to obtain multi-view-angle and multi-resolution information, they have received more and more concern in both research and real ...
Stereo with Oblique Cameras
Mosaics acquired by pushbroom cameras, stereo panoramas, omnivergent mosaics, and spherical mosaics can be viewed as images taken by non-central cameras, i.e. cameras that project along rays that do not all intersect at one point. It has been shown that ...
Stereo with Oblique Cameras
SMBV '01: Proceedings of the IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV'01)Mosaics acquired by pushbroom cameras, stereo panoramas,omnivergent mosaics, and spherical mosaics can beviewed as images taken by non-central cameras, i.e. camerasthat project along rays that do not all intersect at onepoint. It has been shown that in ...
Comments