Abstract
Robotic manipulation systems that operate in unstructured environments must be responsive to feedback from sensors that are disparate in both location and modality. This paper describes a distributed framework for assimilating the disparate feedback provided by force and vision sensors, including active vision sensors, for robotic manipulation systems. The main components of the expectation-based framework include object schemas and port-based agents. Object schemas represent the manipulation task internally in terms of geometric models with attached sensor mappings. Object schemas are dynamically updated by sensor feedback, and thus provide an ability to perform three dimensional spatial reasoning during task execution. Because object schemas possess knowledge of sensor mappings, they are able to both select appropriate sensors and guide active sensors based on task characteristics. Port-based agents are the executors of reference inputs provided by object schemas and are defined in terms of encapsulated control strategies. Experimental results demonstrate the capabilities of the framework in two ways: the performance of manipulation tasks with active camera-lens systems, and the assimilation of force and vision sensory feedback.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allen, P.K. 1989. Real-time motion tracking using spatio-temporal filters. In Proc. Image Understanding Workshop, Morgan Kaufmann: San Mateo, CA, pp. 695-701.
Anandan, P. 1989. A Computational framework and an algorithm for the measurement of visual motion. Int. J. of Computer Vision, 2(3):283-310.
Dickmanns, E.D. 1992. Expectation-based dynamic scene understanding. In Active Vision, A. Blake and A. Yuille (Eds.), The MIT Press: Cambridge, pp. 303-335.
Espiau, B., Chaumette, F., and Rives, P. 1992. A new approach to visual servoing in robotics. IEEE Trans. on Robotics and Automation, 8(3):313-326.
Fagerer, C., Dickmanns, D., and Dickmanns, E.D. 1994. Visual grasping with long delay time of a free floating object in orbit. Autonomous Robots, 1(1):53-68.
Feddema, J.T. and Lee, C.S.G. 1990. Adaptive image feature prediction and control for visual tracking with a hand-eye coordinated camera. IEEE Trans. on Systems, Man, and Cybernetics, 20(5):1172-1183.
Hirzinger, G. 1994. ROTEX-the first space robot technology experiment. Experimental Robotics III: The Third Int. Symp., Kyoto, Japan, Oct. 28–30, 1993, T. Yoshikawa and F. Miyazaki (Eds.), Springer-Verlag, pp. 579-598.
Jain, R. 1989. Environment models and information assimilation. Technical Report RJ 6866(65692), IBM-Yorktown Heights.
Klema, V.C. and Laub, A.J. 1980. The singular value decomposition: its computation and some applications. IEEE Tran. on Automatic Control, 25(2):164-176.
Krotkov, E. 1987. Focusing. International Journal of Computer Vision 1, pp. 223-237.
Neisser, U. 1976. Cognition and Reality. W.H. Freeman and Co: New York.
Nelson, B. and Khosla, P.K. 1994a. Integrating sensor placement and visual tracking strategies. In Experimental Robotics III: The Third International Symposium, Kyoto, Oct. 28–30, 1993, T. Yoshikawa and F. Miyazaki (Eds.), Springer-Verlag, London, pp. 169-181.
Nelson, B.J. and Khosla, P.K. 1994b. The resolvability ellipsoid for visual servoing. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR94), pp. 829-832.
Nelson, B.J. and Khosla, P.K. 1996. Force and vision resolvability for assimilating disparate sensory feedback. IEEE Trans. on Robotics and Automation, 12(5):714-731.
Nelson, B.J., Morrow, J.D., and Khosla, P.K. 1995. Improved force control through visual servoing. In Proc. 1995 American Control Conference (ACC95), Seattle, June 21–23, pp. 380-386.
Nelson, B., Papanikolopoulos, N.P., and Khosla, P.K. 1993. Visual servoing for robotic assembly. Visual Servoing—Real-Time Control of Robot Manipulators Based on Visual Sensory Feedback, K. Hashimoto (Ed.), World Scientific: New Jersey, pp. 139-164.
Papanikolopoulos, N.P., Khosla, P.K., and Kanade, T. 1991. Adaptive robotic visual tracking. In Proc. of the American Control Conference, Evanston, IL: American Autom. Control Council, pp. 962-967.
Roth, Y. and Jain, R. 1991. Integrated architectures for autonomous systems. In Proc. of the SPIE—The Int. Society for Optical Engineering, vol. 1571, pp. 628-639.
Steenstrup, M., Arbib, M.A., and Manes, E.G. 1983. Port automata and the algebra of concurrent processes. J. of Computer and Sys. Sciences, 27(1):29-50
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Nelson, B.J., Khosla, P.K. An Expectation-Based Framework of Object Schemas and Port-Based Agents for Disparate Feedback Assimilation. Autonomous Robots 7, 159–173 (1999). https://doi.org/10.1023/A:1008962117806
Issue Date:
DOI: https://doi.org/10.1023/A:1008962117806