Abstract
This paper presents a new approach for visual tracking and servoing in robotics. We introduce deformable active models as a powerful means for tracking a rigid or semi-rigid (possibly partially occluded) object in movement within the manipulator's workspace. Deformable models imitate, in real-time, the dynamic behavior of elastic structures. These computer-generated models are designed to capture the silhouette of objects with well-defined boundaries, in terms of image gradient. By means of an eye-in-hand robot arm configuration, the desired motion of the end-effector is computed with the objective of keeping the target's position and shape invariant with respect to the camera frame. Optimal estimation and control techniques (LQG regulator) have been successfully implemented in order to deal with noisy measurements provided by our vision sensor. Experimental results are presented for the tracking of a rigid or semi-rigid object. The experiments performed in a real-time environment show the effectiveness and robustness of the proposed method for servoing tasks based on visual feedback.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Papanikolopoulos, N. P.: Controlled active vision, PhD thesis, Department of Electrical and Computer Engineering, Carnegie Mellon University, August 1992.
Kass, B., Witkin, A., and Terzopoulos, D.: Snakes: Active contour models,Int. J. Computer Vision 1(4) (1987), 321–331.
Allen, P. K.: Real-time motion tracking using spatio-temporal filters, inProc. DARPA Image Understanding Workshop, 1989, pp. 695–701.
Allen, P. K., Yoshimi, B., and Timcenko, A.: Real-time visual servoing, inProc. IEEE Int. Conf. Robotics and Automation, April 1991, pp. 851–856.
Weiss, L. E., Sanderson, A. C., and Neuman, C. P.: Dynamic sensor-based control of robots with visual feedback,IEEE J. Robotics and Automation 3(5) (1987), 404–417.
Feddema, J. T., Lee, C. S. G., and Mitchell, O. R.: Weighted selection of image features for resolved rate visual feedback control,IEEE Trans. Robotics and Automation 7(1) (1991), 31–47.
Feddema, J. T. and Lee, C. S. G.: Adaptive image feature prediction and control for visual tracking with a hand-eye coordinated camera,IEEE Trans. Systems, Man and Cybernetics 20 (5) (1990), 1172–1183.
Jang, W. and Bien, Z.: Feature-based visual servoing of an eye-in-hand robot with improved tracking performance, inProc. IEEE Int. Conf. Robotics and Automation, April 1991, pp. 2254–2260.
Hashimoto, K., Elbine, T., and Kimura, H.: Dynamic visual feedback control for a hand-eye manipulator, inProc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 1992, pp. 1863–1866.
Chaumette, F., Rives, P., and Espiau, B.: Positioning of a robot with respect to an object, tracking it, and estimating its velocity by visual servoing, inProc. IEEE Int. Conf. Robotics and Automation, April 1991, pp. 2248–2253.
Castano, A. and Hutchinson, S.: Hybrid vision/position servo control of a robotic manipulator, inProc. IEEE Int. Conf. Robotics and Automation, May 1992, pp. 1264–1269.
Koivo, A.J. and Houshangi, N.: Real-time vision feedback for servoing of a robotic manipulator with self-tuning controller,IEEE Trans. Systems, Man and Cybernetics 21(1) (1991), 134–142.
Papanikolopoulos, N., Khosla, P. K., and Kanade, T.: Vision and control techniques for robotic visual tracking, inProc. IEEE Int. Conf. Robotics and Automation, 1991, pp. 857–864.
Papanikolopoulos, N. and Khosla, P. K.: Feature based robotic visual tracking of 3-D translational motion, inProc. 30th IEEE Conf. Control and Decision, December 1991, pp. 1877–1882.
Papanikolopoulos, N., Khosla, P. K., and Kanade, T.: Visual tracking of a moving target by a camera mounted on a robot: A combination of control and vision,IEEE Trans. Robotics and Automation 9(1) (1993), 14–35.
Papanikolopoulos, N. P. and Khosla, P. K.: Adaptive robotic visual tracking: Theory and experiments,IEEE Trans. Automatic Control 38(3) (1993), 429–445.
Hashimoto, K.:Visual Servoing—Real-Time Control of Robot Manipulators Based on Visual Sensory Feedback, World Scientific, Singapore, 1993.
Terzopoulos, D. and Metaxas, D.: Dynamic 3D models with local and global deformations: Deformable superquadrics,IEEE Trans. Pattern Analysis and Machine Intelligence 13(7) (1991), 703–714.
Rehg, J. A. and Witkin, A. P.: Visual tracking with deformation models, Technical Report CMU-CS-90-156, Carnegie Mellon University, School of Computer Science, August 1990.
Danielsson, P. E. and Seger, O.: Generalized and separable sobel operators, inMachine Vision for Three-Dimensional Scenes, Academic Press, New York, 1990, pp. 347–375.
Hestenes, D.:New Foundations for Classical Mechanics, Kluwer Academic Publishers, Dordrecht, 1990.
Thomson, W.:Theory of Vibration with Applications, Prentice-Hall, Englewood Cliffs, NJ, 1988.
Davies, E. R.:Machine Vision: Theory, Algorithms, Practicalities, Academic Press, New York, 1990.
Kalata, P. R.: The tracking index: A generalized parameter for α−β and α−β−γ target trackers,IEEE Trans. Aerospace and Electronic Systems 20(2) (1984), 174–182.
Maybeck, P.:Stochastic Models, Estimation, and Control, Academic Press, New York, 1982.
Anderson, B. D. and Moore, J. B.:Optimal Control—Linear Quadratic Methods, Prentice-Hall, Englewood Cliffs, NJ, 1990.
Stewart, D. B., Schmitz, D. E., and Khosla, P. K.: Implementing real-time robotic systems using Chimera-II, inProc. IEEE Int. Conf. Robotics and Automation, September 1990, pp. 598–603.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Couvignou, P.A., Papanikolopoulos, N.P., Sullivan, M. et al. The use of active deformable models in model-based robotic visual servoing. J Intell Robot Syst 17, 195–221 (1996). https://doi.org/10.1007/BF00776509
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00776509