Abstract
This paper describes an integrated vehicle control system with visual feedback. A general-purpose, low-level feature matching method, able to work in real time without any strict assumptions on the environment structure or camera parameters, generates low-level matching results, which are used as source of data for applications like mobile object tracking, among others. A generalized predictive path-tracking control approach keeps the vehicle on the trajectory defined by the moving target. In the low-level matching process, block-based features (windows) are selected and tracked along a stream of monocular images; least residual square error and similarity between clusters of features are used as constraints to select the right matching pair between multiple candidates. Real-time performance is achieved through optimized algorithms and a parallel DSP-based multiprocessor system implementation. Object detection and tracking is motion-based, and does not require a predefined model of the target. The integrated control system has been tested on the ROMEO-3R experimental vehicle.
Similar content being viewed by others
References
Amidi, O., Mesaki, Y., and Kanade, T.: Research on an autonomous vision-guided helicopter, Internal Report, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA, 1993.
Arsénio, A. and Santos-Victor, J.: Robust visual tracking by an active observer, in: Proc. of the IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems, Grenoble, 1997, pp. 1342–1347.
Aubert, D. and Thorpe, C.: Color image processing for navigation: Two road trackers, Internal Report CMU-RI-TR–90–09, Carnegie Mellon University, Pittsburgh, 1990.
Ayache, N.: Artificial Vision for Mobile Robots, The MIT Press, Cambridge, MA, 1991.
Bergen, J. R., Burt, P. J., Hingorani, R., and Peleg, S.: A three-frame algorithm for estimating two-component image motion, IEEE Trans. Pattern Analysis Machine Intelligence 14(9) (1992), 886–896.
Brown, C.: Gaze controls with interactions and delays, IEEE Trans. Systems Man Cybernetics 20(1) (1990), 518–527.
Clarke, D. W., Mohtadi, C., and Tuffs, P. S.: Generalized predictive control, Parts I and II, Automatica 23(2) (1987), 137–160.
Dias, J. et al.: Simulating pursuit with machine experiments with Robots and artificial vision, IEEE Trans. Robotics Automat. 14(1) (1998), 1–18.
Dickmanns, D.: Knowledge based real-time vision, in: Proc. of the 2nd IFAC Conf. on Intelligent Autonomous Vehicles, Espoo, 1995, pp. 13–18.
Evans, R.: 3D Computer Vision Techniques for Object Following and Obstacle Avoidance, Roke Manor Research Limited, Roke Manor, Romsey, Hampshire, UK, 1992.
Grimson, W. E. L.: Computational experiments with a feature based stereo algorithm, IEEE Trans. Pattern Analysis Machine Intelligence 7(1) (1985), 17–34.
Ferruz, J. and Ollero, A.: Estimació n de movimientos de vehículos en entornos no estructurados empleando una secuencia de imágenes, in: XVI Jornadas de Automática. IV Reunió n del grupo de Visió n Artificial, San Sebastián, 1995, pp. 93–98.
Ferruz, J. and Ollero, A.: Vehicle position estimation through visual tracking of features in a stream of images, in: Proc. of the IFAC Workshop on Intelligent Components for Autonomous and Semiautonomous Vehicles, Toulouse, 1995, pp. 40–50.
Ferruz, J.: Sistema para establecimiento de correspondencias en secuencias de imágenes. Aplicaciones en robó tica mó vil, Universidad de Sevilla, 1997.
Ferruz, J. and Ollero, A.: Autonomous mobile robot motion control in non-structured environments based on real-time video processing, in: Proc. of the IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems, Grenoble, 1997, pp. 725–731.
Ferruz, J. and Ollero, A.: Visual tracking of mobile objects. Applications in mobile robotics, in: World Automation Congress, Anchorage, 1998.
Ferruz, J. and Ollero, A.: Visual generalized predictive path tracking, in: 5th Internat.Workshop on Advanced Motion Control, Coimbra, 1998, pp. 159–164.
Hebert, M. and Krotkov, E.: Autonomous navigation in natural environments: Two approaches, in: Proc. of Computer Vision for Space Applications, Antibes, 1993.
Horn, B. K. P. and Schunck, B. G.: Determining optical flow, Artificial Intelligence 17 (1981), 185–203.
Horn, B. K. P.: Robot Vision, MIT Press, Cambridge, MA, 1987.
Jurie, F., Martinet P., and Gallice J., A global road scene analysis system for autonomous vehicles, in: Proc. of the 2nd IFAC Conf. on Intelligent Autonomous Vehicles, Espoo, 1995, pp. 19–24.
Kanade, T. and Okutomi, M.: A stereo matching algorithm with an adaptive window: Theory and experiment, Internal Report CMU-CS–90–120, Carnegie Mellon University, Pittsburgh, 1990.
Lee, J. W. and Kweon, I.: Vehicle segmentation using evidential reasoning, in: Proc. of the IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems, Grenoble, 1997, pp. 880–885.
Lucas, B. D. and Kanade, T.: An iterative image registration technique with an application to stereo vision, in: Proc. of the 7th Internat. Joint Conf. of Artificial Intelligence, 1981.
Marapane, S. B. and Triveldi, M. M.: Multi-Primitive Hierarchical (MPH) stereo analysis, IEEE Trans. Pattern Analysis Machine Intelligence 16(3) (1994), 227–240.
Marr, D. and Poggio, T.: Cooperative computation of stereo disparity, Science 194 (1976).
Medioni, G. and Nevatia, R.: Segment-based stereo matching, Computer Vision, Graphics, and Image Processing 31 (1985), 2–18.
Ollero, A. and Amidi, O.: Predictive path tracking of mobile robots. Application to the CMU NAVLAB, in: Proc. of the IEEE 5th Internat. Conf. on Advanced Robotics, Pisa, Italy, Vol. II, IEEE Press, 1991, pp. 1081–1086.
Ollero A., García-Cerezo, A., and Martínez, J. L.: Fuzzy supervisory path tracking of mobile robots, Control Engineering Practice (Pergamon) 2(2) (1994), 313–319.
Ollero, A., Ferruz, J., Heredia, G., Ló pez, F., and Nogales, C.: Romeo-3R: Aspectos hardware, in: XVII Jornadas de Automática, Santander, 1996.
Rehg and Witkin: Visual tracking with deformation models, in: Proc. of the IEEE Internat. Conf. on Robotics and Automation, Sacramento, CA, April 1991, pp. 844–850.
Rougeaux, S. and Kuniyoshi, Y.: Robust real-time tracking on an active vision head, in: Proc. of the IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems, Grenoble, 1997, pp. 873–879.
Shashua, A.: Projective structure from uncalibrated images: Structure from motion and recognition, IEEE Trans. Pattern Analysis Machine Intelligence 16(8) (1994), 778–790.
Tomasi, C.: Shape and motion from image streams: A factorization method, PhD Thesis, Carnegie Mellon University, 1991.
Wang, C., Prashanth B. B., and Prasanna, V. K.: High-performance computing for vision, Proc. IEEE 84(7) (1996), 931–946.
Zhang, Z., Deriche, R., Faugeras, O., and Luong, Q.T.: A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry, Rapport de Recherche No 2273, INRIA, 1994.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ferruz, J., Ollero, A. Real-Time Feature Matching in Image Sequences for Non-Structured Environments. Applications to Vehicle Guidance. Journal of Intelligent and Robotic Systems 28, 85–123 (2000). https://doi.org/10.1023/A:1008163332131
Issue Date:
DOI: https://doi.org/10.1023/A:1008163332131