Abstract
This chapter introduces visual servo control, using computer vision data in the servo loop to control the motion of a robot. We first describe the basic techniques that are by now well established in the field. We give a general overview of the formulation of the visual servo control problem, and describe the two archetypal visual servo control schemes: image-based and pose-based visual servo control. We then discuss performance and stability issues that pertain to these two schemes, motivating advanced techniques. Of the many advanced techniques that have been developed, we discuss two-and-a-half-dimensional (GlossaryTerm
2.5-D
), hybrid, partitioned, and switched approaches. Having covered a variety of control schemes, we deal with target tracking and controlling motion directly in the joint space and extensions to under-actuated ground and aerial robots. We conclude by describing applications of visual servoing in robotics.Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Abbreviations
- 2-D:
-
two-dimensional
- 2.5-D:
-
two-and-a-half-dimensional
- 3-D:
-
three-dimensional
- IBVS:
-
image-based visual servo control
- IMU:
-
inertial measurement unit
- LQG:
-
linear quadratic Gaussian
- MEMS:
-
microelectromechanical system
- PBVS:
-
pose-based visual servo control
- VS:
-
visual servo
References
L. Weiss, A. Sanderson, C. Neuman: Dynamic sensor-based control of robots with visual feedback, IEEE J. Robot. Autom. 3, 404–417 (1987)
S. Hutchinson, G. Hager, P. Corke: A tutorial on visual servo control, IEEE Trans. Robot. Autom. 12, 651–670 (1996)
B. Espiau, F. Chaumette, P. Rives: A new approach to visual servoing in robotics, IEEE Trans. Robot. Autom. 8, 313–326 (1992)
J. Feddema, O. Mitchell: Vision-guided servoing with feature-based trajectory generation, IEEE Trans. Robot. Autom. 5, 691–700 (1989)
D. Forsyth, J. Ponce: Computer Vision: A Modern Approach (Prentice Hall, Upper Saddle River 2003)
Y. Ma, S. Soatto, J. Kosecka, S. Sastry: An Invitation to 3-D Vision: From Images to Geometric Models (Springer, New York 2003)
P. Corke: Robotics, Vision and Control: Fundamental Algorithms in MATLAB (Springer, Berlin, Heidelberg 2011)
H. Michel, P. Rives: Singularities in the Determination of the Situation of a Robot Effector from the Perspective View of Three Points. Res. Rep. RR-1850 (INRIA 1993)
M. Fischler, R. Bolles: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Communications ACM 24, 381–395 (1981)
E. Malis: Improving vision-based control using efficient second-order minimization techniques, IEEE Int. Conf. Robot. Autom., New Orleans (2004) pp. 1843–1848
E. Marchand, F. Spindler, F. Chaumette: ViSP for visual servoing: A generic software platform with a wide class of robot control skills, IEEE Robot. Autom. Mag. 12(4), 40–52 (2005), https://team.inria.fr/lagadic/visp/visp.html
P. Corke, S. Hutchinson: A new partitioned approach to image-based visual servo control, IEEE Trans. Robot. Autom. 17, 507–515 (2001)
F. Chaumette: Potential problems of stability and convergence in image-based and position-based visual servoing, Lect. Note. Contr. Inform. Sci. 237, 66–78 (1998)
E. Malis: Visual servoing invariant to changes in camera intrinsic parameters, IEEE Trans. Robot. Autom. 20, 72–81 (2004)
A. Isidori: Nonlinear Control Systems, 3rd edn. (Springer, Berlin, Heidelberg 1995)
G. Hager, W. Chang, A. Morse: Robot feedback control based on stereo vision: Towards calibration-free hand-eye coordination, IEEE Control Syst. Mag. 15, 30–39 (1995)
M. Iwatsuki, N. Okiyama: A new formulation of visual servoing based on cylindrical coordinate system, IEEE Trans. Robot. Autom. 21, 266–273 (2005)
F. Chaumette, P. Rives, B. Espiau: Classification and realization of the different vision-based tasks, Robot. Autom. Syst. 7, 199–228 (1993)
A. Castano, S. Hutchinson: Visual compliance: Task directed visual servo control, IEEE Trans. Robot. Autom. 10, 334–342 (1994)
G. Hager: A modular system for robust positioning using feedback from stereo vision, IEEE Trans. Robot. Autom. 13, 582–595 (1997)
F. Chaumette: Image moments: A general and useful set of features for visual servoing, IEEE Trans. Robot. Autom. 20, 713–723 (2004)
O. Tahri, F. Chaumette: Point-based and region-based image moments for visual servoing of planar objects, IEEE Trans. Robot. 21, 1116–1127 (2005)
C. Geyer, K. Daniilidis: Catadioptric projective geometry, Int. J. Comput. Vis. 45(3), 223–243 (2001)
T. Hamel, R. Mahony: Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach, IEEE Trans Robot. 18(2), 187–198 (2002)
I. Suh: Visual servoing of robot manipulators by fuzzy membership function based neural networks. In: Visual Servoing, Robotics and Automated Systems, Vol. 7, ed. by K. Hashimoto (World Scientific, Singapore 1993) pp. 285–315
G. Wells, C. Venaille, C. Torras: Vision-based robot positioning using neural networks, Image Vis. Comput. 14, 75–732 (1996)
J.T. Lapresté, F. Jurie, F. Chaumette: An efficient method to compute the inverse jacobian matrix in visual servoing, IEEE Int. Conf. Robot. Autom., New Orleans (2004) pp. 727–732
K. Hosada, M. Asada: Versatile visual servoing without knowledge of true jacobian, IEEE/RSJ Int. Conf. Intell. Robots Syst., München (1994) pp. 186–193
M. Jägersand, O. Fuentes, R. Nelson: Experimental evaluation of uncalibrated visual servoing for precision manipulation, IEEE Int. Conf. Robot. Autom., Albuquerque (1997) pp. 2874–2880
J. Piepmeier, G.M. Murray, H. Lipkin: Uncalibrated dynamic visual servoing, IEEE Trans. Robot. Autom. 20, 143–147 (2004)
K. Deguchi: Direct interpretation of dynamic images and camera motion for visual servoing without image feature correspondence, J. Robot. Mechatron. 9(2), 104–110 (1997)
W. Wilson, C. Hulls, G. Bell: Relative end-effector control using cartesian position based visual servoing, IEEE Trans. Robot. Autom. 12, 684–696 (1996)
B. Thuilot, P. Martinet, L. Cordesses, J. Gallice: Position based visual servoing: Keeping the object in the field of vision, IEEE Int. Conf. Robot. Autom., Washington (2002) pp. 1624–1629
D. Dementhon, L. Davis: Model-based object pose in 25 lines of code, Int. J. Comput. Vis. 15, 123–141 (1995)
D. Lowe: Three-dimensional object recognition from single two-dimensional images, Artif. Intell. 31(3), 355–395 (1987)
E. Malis, F. Chaumette, S. Boudet: 2-1/2 D visual servoing, IEEE Trans. Robot. Autom. 15, 238–250 (1999)
E. Malis, F. Chaumette: Theoretical improvements in the stability analysis of a new class of model-free visual servoing methods, IEEE Trans. Robot. Autom. 18, 176–186 (2002)
J. Chen, D. Dawson, W. Dixon, A. Behal: Adaptive homography-based visual servo tracking for fixed camera-in-hand configurations, IEEE Trans. Control Syst. Technol. 13, 814–825 (2005)
G. Morel, T. Leibezeit, J. Szewczyk, S. Boudet, J. Pot: Explicit incorporation of 2-D constraints in vision-based control of robot manipulators, Lect. Note. Contr. Inform. Sci. 250, 99–108 (2000)
F. Chaumette, E. Malis: 2 1/2 D visual servoing: a possible solution to improve image-based and position-based visual servoings, IEEE Int. Conf. Robot. Autom., San Fransisco (2000) pp. 630–635
E. Cervera, A.D. Pobil, F. Berry, P. Martinet: Improving image-based visual servoing with three-dimensional features, Int. J. Robot. Res. 22, 821–840 (2004)
F. Schramm, G. Morel, A. Micaelli, A. Lottin: Extended 2-D visual servoing, IEEE Int. Conf. Robot. Autom., New Orleans (2004) pp. 267–273
N. Papanikolopoulos, P. Khosla, T. Kanade: Visual tracking of a moving target by a camera mounted on a robot: A combination of vision and control, IEEE Trans. Robot. Autom. 9, 14–35 (1993)
K. Hashimoto, H. Kimura: LQ optimal and nonlinear approaches to visual servoing, Robot. Autom. Syst. 7, 165–198 (1993)
B. Nelson, P. Khosla: Strategies for increasing the tracking region of an eye-in-hand system by singularity and joint limit avoidance, Int. J. Robot. Res. 14, 225–269 (1995)
B. Nelson, P. Khosla: Force and vision resolvability for assimilating disparate sensory feedback, IEEE Trans. Robot. Autom. 12, 714–731 (1996)
R. Sharma, S. Hutchinson: Motion perceptibility and its application to active vision-based servo control, IEEE Trans. Robot. Autom. 13, 607–617 (1997)
E. Marchand, F. Chaumette, A. Rizzo: Using the task function approach to avoid robot joint limits and kinematic singularities in visual servoing, IEEE/RSJ Int. Conf. Intell. Robots Syst., Osaka (1996) pp. 1083–1090
E. Marchand, G. Hager: Dynamic sensor planning in visual servoing, IEEE Int. Conf. Robot. Autom., Leuven (1998) pp. 1988–1993
N. Cowan, J. Weingarten, D. Koditschek: Visual servoing via navigation functions, IEEE Trans. Robot. Autom. 18, 521–533 (2002)
N. Gans, S. Hutchinson: An asymptotically stable switched system visual controller for eye in hand robots, IEEE/RSJ Int. Conf. Intell. Robots Syst., Las Vegas (2003) pp. 735–742
G. Chesi, K. Hashimoto, D. Prattichizio, A. Vicino: Keeping features in the field of view in eye-in-hand visual servoing: a switching approach, IEEE Trans. Robot. Autom. 20, 908–913 (2004)
K. Hosoda, K. Sakamato, M. Asada: Trajectory generation for obstacle avoidance of uncalibrated stereo visual servoing without 3-D reconstruction, IEEE/RSJ Int. Conf. Intell. Robots Syst. 3, Pittsburgh (1995) pp. 29–34
Y. Mezouar, F. Chaumette: Path planning for robust image-based control, IEEE Trans. Robot. Autom. 18, 534–549 (2002)
G. Chesi: Visual servoing path-planning via homogeneous forms and LMI optimizations, IEEE Trans. Robot. 25(2), 281–291 (2009)
L. Matthies, T. Kanade, R. Szeliski: Kalman filter-based algorithms for estimating depth from image sequences, Int. J. Comput. Vis. 3(3), 209–238 (1989)
C.E. Smith, N. Papanikolopoulos: Computation of shape through controlled active exploration, IEEE Int. Conf. Robot. Autom., San Diego (1994) pp. 2516–2521
A. De Luca, G. Oriolo, P. Robuffo Giordano: Feature depth observation for image-based visual servoing: Theory and experiments, Int. J. Robot. Res. 27(10), 1093–1116 (2008)
R. Basri, E. Rivlin, I. Shimshoni: Visual homing: Surfing on the epipoles, Int. J. Comput. Vis. 33, 117–137 (1999)
E. Malis, F. Chaumette, S. Boudet: 2 1/2 D visual servoing with respect to unknown objects through a new estimation scheme of camera displacement, Int. J. Comput. Vis. 37, 79–97 (2000)
O. Faugeras: Three-Dimensional Computer Vision: A Geometric Viewpoint (MIT Press, Cambridge 1993)
G. Silveira, E. Malis: Direct visual servoing: Vision-based estimation and control using only nonmetric information, IEEE Trans. Robot. 28(4), 974–980 (2012)
P. Corke, M. Goods: Controller design for high performance visual servoing, 12th World Congr. IFAC'93, Sydney (1993) pp. 395–398
F. Bensalah, F. Chaumette: Compensation of abrupt motion changes in target tracking by visual servoing, IEEE/RSJ Int. Conf. Intell. Robots Syst., Pittsburgh (1995) pp. 181–187
P. Allen, B. Yoshimi, A. Timcenko, P. Michelman: Automated tracking and grasping of a moving object with a robotic hand-eye system, IEEE Trans. Robot. Autom. 9, 152–165 (1993)
K. Hashimoto, H. Kimura: Visual servoing with non linear observer, IEEE Int. Conf. Robot. Autom., Nagoya (1995) pp. 484–489
A. Rizzi, D. Koditschek: An active visual estimator for dexterous manipulation, IEEE Trans. Robot. Autom. 12, 697–713 (1996)
R. Ginhoux, J. Gangloff, M. de Mathelin, L. Soler, M.A. Sanchez, J. Marescaux: Active filtering of physiological motion in robotized surgery using predictive control, IEEE Trans. Robot. 21, 67–79 (2005)
J. Gangloff, M. de Mathelin: Visual servoing of a 6-DOF manipulator for unknown 3-D profile following, IEEE Trans. Robot. Autom. 18, 511–520 (2002)
R. Tsai, R. Lenz: A new technique for fully autonomous efficient 3-D robotics hand-eye calibration, IEEE Trans. Robot. Autom. 5, 345–358 (1989)
N. Guenard, T. Hamel, R. Mahony: A practical visual servo control for an unmanned aerial vehicle, IEEE Trans. Robot. 24(2), 331–340 (2008)
G.L. Mariottini, G. Oriolo, D. Prattichizo: Image-based visual servoing for nonholonomic mobile robots using epipolar geometry, IEEE Trans. Robot. 23(1), 87–100 (2007)
G. Lopez-Nicolas, J.J. Guerrero, C. Sagues: Visual control through the trifocal tensor for nonholonomic robots, Robot. Auton. Syst. 58(2), 216–226 (2010)
P. Corke: Robotics, Vision and Control: Fundamental Algorithms in MATLAB, Springer Tracts in Advanced Robotics, Vol. 73 (Springer, Berlin, Heidelberg 2011)
A. Crétual, F. Chaumette: Visual servoing based on image motion, Int. J. Robot. Res. 20(11), 857–877 (2001)
C. Collewet, E. Marchand: Photometric visual servoing, IEEE Trans. Robot. 27(4), 828–834 (2011)
R. Mebarki, A. Krupa, F. Chaumette: 2D ultrasound probe complete guidance by visual servoing using image moments, IEEE Trans. Robot. 26(2), 296–306 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Video-References
Video-References
- :
-
IBVS on a 6 DOF robot arm (1); available from http://handbookofrobotics.org/view-chapter/34/videodetails/59
- :
-
IBVS on a 6 DOF robot arm (2); available from http://handbookofrobotics.org/view-chapter/34/videodetails/60
- :
-
IBVS on a 6 DOF robot arm (3); available from http://handbookofrobotics.org/view-chapter/34/videodetails/61
- :
-
PBVS on a 6 DOF robot arm (1); available from http://handbookofrobotics.org/view-chapter/34/videodetails/62
- :
-
PBVS on a 6 DOF robot arm (2); available from http://handbookofrobotics.org/view-chapter/34/videodetails/63
- :
-
2.5-D VS on a 6 DOF robot arm (1); available from http://handbookofrobotics.org/view-chapter/34/videodetails/64
- :
-
2.5-D VS on a 6 DOF robot arm (2); available from http://handbookofrobotics.org/view-chapter/34/videodetails/65
Rights and permissions
Copyright information
© 2016 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Chaumette, F., Hutchinson, S., Corke, P. (2016). Visual Servoing. In: Siciliano, B., Khatib, O. (eds) Springer Handbook of Robotics. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-319-32552-1_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-32552-1_34
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-32550-7
Online ISBN: 978-3-319-32552-1
eBook Packages: EngineeringEngineering (R0)