Skip to main content
SpringerLink
Log in
Book cover

Experimental Robotics IV pp 121–129Cite as

Task oriented model-driven visually servoed agents

  • Chapter 3 Autonomy Via Vision
  • Conference paper
  • First Online:

  • 1019 Accesses

Part of the book series: Lecture Notes in Control and Information Sciences ((LNCIS,volume 223))

Abstract

Most proposed visual servoing control strategies use image-based visual servoing techniques in which the visual servoing control loop is closed using 2D image plane coordinates rather than 3D world coordinates. These strategies do not allow for explicit representations of the 3D world. We propose a visual servoing framework that uses 3D environment models to describe the task. These models are augmented by sensor mappings that represent the visual sensors used by the system and are used to drive an image-based visually servoed agent to execute the task in the real world. This framework allows for the use of 3D reasoning within the environment model, while taking advantage of the benefits 2D image-based visual servoing provides. In this paper, we describe our proposed framework and present experimental results which show that the framework can be used to successfully perform visually servoed manipulation tasks.

This is a preview of subscription content, log in via an institution.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. L.E. Weiss, “Dynamic visual servo control of robots: an adaptive image-based approach,” Ph.D Thesis CMU-RI-TR-84-16, Pittsburgh, PA:The Robotics Institute Carnegie Mellon University, 1984.

    Google Scholar 

  2. B.J. Nelson and P.K. Khosla, “The Resolvability Ellipsoid for Visual Servoing,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR94), pp. 829–832, 1994.

    Google Scholar 

  3. B. Nelson and P.K. Khosla, “Integrating Sensor Placement and Visual Tracking Strategies,” in Experimental Robotics III: The Third International Symposium, Kyoto, October 28–30, 1993, eds. T. Yoshikawa and F. Miyazaki, Springer-Verlag, London, pp. 169–181. 1994.

    Google Scholar 

  4. N.P. Papanikolopoulos, Khosla, P.K. and Kanade, T., “Adaptive robotic visual tracking,” Proc. of the American Control Conference. Evanston, IL.:American Automation Control Council, pp. 962–967, 1991.

    Google Scholar 

  5. N.P. Papanikolopoulos, B. Nelson, and P.K. Khosla, “Full 3-d tracking using the controlled active vision paradigm,” Proc. 1992 IEEE Int. Symp. on Intelligent Control (ISIC-92), pp. 267–274, 1992.

    Google Scholar 

  6. P. Anandan, Measuring visual motion from image sequences. Tech. Rept. COINS-TR-87-21, Amherst, Mass.:University of Massachusetts COINS Department, 1987.

    Google Scholar 

  7. C. Tomasi and T. Kanade, “Detection and tracking of point features,” Tech. Rept. CMUCS-91-132. Pittsburgh:Carnegie Mellon University School of Computer Science, 1991.

    Google Scholar 

  8. E.D. Dickmanns, “Expectation-based Dynamic Scene Understanding,” in Active Vision, eds. A. Blake and A. Yuille, 303–335, The MIT Press, Cambridge, 1992.

    Google Scholar 

  9. Y. Roth and R. Jain, “Verification versus Discovery in Vision-Based Systems,” Technical Report CSE-TR-110-91, The University of Michigan, 1991.

    Google Scholar 

  10. U. Neisser, Cognition and Reality, W.H. Freeman and Co., New York, 1976.

    Google Scholar 

  11. R. Jain, “Environment Models and Information Assimilation,” Technical Report RJ 6866(65692), IBM-Yorktown Heights, 1989.

    Google Scholar 

  12. C. Fagerer, D. Dickmanns, and E.D. Dickmanns, “Visual Grasping with Long Delay Time of a Free Floating Object in Orbit,” Autonomous Robots, 1(1):53–68, 1994.

    Article  Google Scholar 

  13. G. Hirzinger, “ROTEX-the first space robot technology experiment,” Experimental Robotics III: The Third Int. Symp., Kyoto, Japan, Oct. 28–30, 1993, eds. T. Yoshikawa and F. Miyazaki, Springer-Verlag, pp.579–598, 1994.

    Google Scholar 

  14. D.B. Stewart, D.E. Schmitz, and P.K. Khosla, “The Chimera II real-time operating system for advanced sensor-based control systems,” IEEE Trans. Sys., Man Cyber. 22(6):1282–1295., 1992.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, University of Illinois at Chicago, Chicago, Illinois, USA

    Bradley J. Nelson

  2. Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA

    Pradeep K. Khosla

Authors
  1. Bradley J. Nelson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pradeep K. Khosla
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Oussama Khatib J. Kenneth Salisbury

Rights and permissions

Reprints and permissions

Copyright information

© 1997 Springer-Verlag London Limited

About this paper

Cite this paper

Nelson, B.J., Khosla, P.K. (1997). Task oriented model-driven visually servoed agents. In: Khatib, O., Salisbury, J.K. (eds) Experimental Robotics IV. Lecture Notes in Control and Information Sciences, vol 223. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0035203

Download citation

  • DOI: https://doi.org/10.1007/BFb0035203

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-76133-4

  • Online ISBN: 978-3-540-40942-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation