Skip to main content
SpringerLink
Log in

Telerobotic visual servoing

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

This paper addresses the problem of integrating the human operator with autonomous robotic visual tracking and servoing modules. A CCD camera is mounted on the end-effector of a robot and the task is to servo around a static or moving rigid target. In manual control mode, the human operator, with the help of a joystick and a monitor, commands robot motions in order to compensate for tracking errors. In shared control mode, the human operator and the autonomous visual tracking modules command motion along orthogonal sets of degrees of freedom. In autonomous control mode, the autonomous visual tracking modules are in full control of the servoing functions. Finally, in traded control mode, the control can be transferred from the autonomous visual modules to the human operator and vice versa. This paper presents an experimental setup where all these different schemes have been tested. Experimental results of all modes of operation are presented and the related issues are discussed. In certain degrees of freedom (DOF) the autonomous modules perform better than the human operator. On the other hand, the human operator can compensate fast for failures in tracking while the autonomous modules fail. Their failure is due to difficulties in encoding an efficient contingency plan.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A. K. Bejczy and Z. F. Szakaly, “An interactive manipulator control system,” Mini- and Microcomputers, vol. 5, no. 1, 1980.

  2. P. T. Boissiere and R. W. Harrigan, “Telerobotic control of conventional robot manipulators,” in Proc. of the IEEE Int. Conf. on Robotics and Automation, 1988, pp. 576–583.

  3. S. Hayati and S. T. Venkataraman, “Design and implementation of robot control system with traded and shared control capability,” in Proc. of the IEEE Int. Conf. on Robotics and Automation, 1989, pp. 1310–1315.

  4. V. Hayward, “Autonomous control issues in a telerobot,” in Proc. of the IEEE Int. Conf. on Systems, Man, and Cybernetics, 1988, pp. 122–125.

  5. H. Schneiderman, “Issues in a telerobotic system: control trading, control sharing, and safety during manual operation,” Master's thesis, Department of Electrical and Computer Engineering, Carnegie Mellon University. 1990.

  6. H. Schneiderman and P. K. Khosla, “Implementation of traded and shared control strategies for exploration using a tactile sensor,” in Proc. of the Fourth ANS Topical Meeting on Robotics and Remote Systems, 1991, pp. 217–226.

  7. T. B. Sheridan, “Human supervisory control of robot systems,” in Proc. of the IEEE Int. Conf. on Robotics and Automation, 1986, pp. 808–812.

  8. N. Papanikolopoulos, P. K. Khosla and T. Kanade, “Vision and control techniques for robotic visual tracking,” in Proc. of the IEEE Int. Conf. on Robotics and Automation, 1991, pp. 857–864.

  9. N. Papanikolopoulos, P. K. Khosla and T. Kanade, “Adaptive robotic visual tracking,” in Proc. of the American Control Conference, 1991, pp. 962–967.

  10. N. P. Papanikolopoulos and P. K. Khosla, “Feature based robotic visual tracking of 3-D translational motion,” in Proc. of the 30th IEEE CDC, Brighton, UK, 1991, pp. 1877–1882.

  11. P. K. Allen, “Real-time motion tracking using spatiotemporal filters,” in Proc. DARPA Image Understanding Workshop, 1989, pp. 695–701.

  12. E. D. Dickmanns, B. Mysliwetz and T. Christians, “An integrated spatio-temporal approach to automatic visual guidance of autonomous vehicles,” IEEE Trans. on Systems, Man, and Cybernetics, vol. 20, no. 6, pp. 1273–1284, 1990.

    Google Scholar 

  13. J. T. Feddema, C. S. G. Lee and O. R. Mitchell, “Weighted selection of image features for resolved rate visual feedback control,” IEEE Trans. Robotics and Automation, vol. 7, no. 1, pp. 31–47, 1991.

    Google Scholar 

  14. B. Bhanu, P. Symosek, J. Ming, W. Burger, H. Nasr and J. Kim, “Qualitative target motion detection and tracking,” in Proc. DARPA Image Understanding Workshop, 1989, pp. 370–398.

  15. V. H. L. Cheng and B. Sridhar, “Integration of active and passive sensors for obstacle avoidance,” IEEE Control Systems Magazine, vol. 10, no. 4, pp. 43–50, 1990.

    Google Scholar 

  16. L. Matthies, R. Szeliski and T. Kanade, “Kalman filter-based algorithms for estimating depth from image sequences,” Technical Report 88-1, Carnegie Mellon University, The Robotics Institute, 1988.

  17. N. P. Papanikolopoulos, B. Nelson and P. K. Khosla, “Monocular 3-D visual tracking of a moving target by an eye-in-hand robotic system,” Technical Report, Carnegie Mellon University, The Robotics Institute, 1991.

  18. G. C. Goodwin and K. S. Sin, Information and Systems Science Series. Volume 1: Adaptive filtering, prediction and control. Prentice-Hall, Inc.: Englewood Cliffs, NJ, 1984.

    Google Scholar 

  19. W. Forstner and A. Pertl, “Photogrammetric standard methods and digital image matching techniques for high precision surface measurements,” in Pattern Recognition in Practice II, edited by E. S. Gelsema and L. N. Kanal, Elsevier Science Publishers, pp. 57–72, 1986.

  20. D. J. Heeger, “Depth and flow from motion energy,” Science, vol. 86, pp. 657–663, 1986.

    Google Scholar 

  21. B. K. P. Horn and B. G. Schunck, “Determining optical flow,” Artificial Intelligence, vol. 17, pp. 185–204, 1981.

    Google Scholar 

  22. S. Ullman, The interpretation of visual motion. MIT Press, 1979.

  23. P. Anandan, “Measuring visual motion from image sequences,” Technical Report COINS-TR-87-21, COINS Department, University of Massachusetts, 1987.

  24. F. Chaumette, P. Rives and B. Espiau, “Positioning of a robot with respect to an object, tracking it and estimating its velocity by visual servoing,” in Proc. of the IEEE Int. Conf. on Robotics and Automation, 1991, pp. 2248–2253.

  25. F. L. Lewis, Optimal control. John Wiley & Sons: New York, 1986.

    Google Scholar 

  26. P. S. Maybeck, Stochastic models, estimation, and control. Academic Press: London, 1979.

    Google Scholar 

  27. D. B. Stewart, D. E. Schmitz and P. K. Khosla, “Implementing real-time robotic systems using CHIMERA II,” in Proc. of the 1990 IEEE Int. Conf. on Robotics and Automation, 1990, pp. 598–603.

  28. G. Hirzinger, “The space and telerobotic concepts of DFVLR ROTEX,” in Proc. of the IEEE Int. Conf. on Robotics and Automation, 1987, pp. 443–449.

  29. F. Chaumette and P. Rives, “Design and calibration of an experimental vision system based on a mobile camera mounted on a robot-arm,” Technical Report 994, INRIA, 1989.

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, The Robotics Institute, Carnegie Mellon University, 15213, Pittsburgh, PA

    P. K. Khosla & N. P. Papanikolopoulos

Authors
  1. P. K. Khosla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. N. P. Papanikolopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Khosla, P.K., Papanikolopoulos, N.P. Telerobotic visual servoing. Appl Intell 2, 127–153 (1992). https://doi.org/10.1007/BF00058759

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00058759

Key words

Search

Navigation