Abstract
This paper describes how virtual tools that represent real robot end-effectors are used in conjunction with a generalized conglomerate-of-spheres approach to collision avoidance in such a way that telerobotic trajectory planning can be accomplished using simple gesture phrases such as ‘put that there while avoiding that’. In this concept, an operator (or set of collaborators) need not train for cumbersome telemanipulation on several multiple-link robots, nor do robots need a priori knowledge of operator intent and exhaustive algorithms for evaluating every aspect of a detailed environment model. The human does what humans do best during task specification, while the robot does what machines do best during trajectory planning and execution.
Four telerobotic stages were implemented to demonstrate this strategic supervision concept that will facilitate collaborative control between humans and machines. In the first stage, virtual reality tools are selected from a ‘toolbox’ by the operator(s) and then these virtual tools are computationally interwoven into the live video scene with depth correlation. Each virtual tool is a graphic representation of a robot end-effector (gripper, cutter, or other robot tool) that carries tool-use attributes on how to perform a task. An operator uses an instrumented glove to virtually retrieve the disembodied tool, in the shared scene, and place it near objects and obstacles while giving key-point gesture directives, such as ‘cut there while avoiding that’. Collaborators on a network may alter the plan by changing tools or tool positioning to achieve preferred results from their own perspectives. When parties agree, from wherever they reside geographically, the robot(s) create and execute appropriate trajectories suitable to their own particular links and joints. Stage two generates standard joint-interpolated trajectories, and later creates potential field trajectories if necessary. Stage three tests for collisions with obstacles identified by the operator and modeled as conglomerates of spheres. Stage four involves automatic grasping (or cutting etc.) once the robot camera acquires a close-up view of the object during approach. In this paper particular emphasis is placed on the conglomerate-of-spheres approach to collision detection as integrated with the virtual tools concept for a Puma 560 robot by the Virtual Tools and Robotics Group in the Computer Integrated Manufacturing Laboratory at The Pennsylvania State University (Penn State).
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Sheridan, T. B. and Johannsen, G.: Monitoring Behavior and Supervisory Control, MIT Press, Cambridge, MA, 1992.
Sheridan, T. B.: Automation, Telerobotics and Human Supervisory Control, MIT Press, Cambridge, MA, 1992.
Kim, W. S. and Bejczy, A. K.: Graphics displays for operator aid in telemanipulation, in Proceedings of 1991 IEEE International Conference on Systems, Man, and Cybernetics 2 (1991) pp. 1059–1067.
Noyes, M. and Sheridan, T. B.: A novel predictor for telemanipulcation through a time delay, in Proc. of 20th Ann. Conf. on Man, Control, NASA/Ames Research Center, 1984.
Ince, I., Bryant, K., and Brooks, T.: Virtually and reality: a video/graphics environment for teleoperation, in Proceedings of 1991 IEEE International Conference on Systems, Man, and Cybernetics 1991.
Thomas, G., Barton, R., and Cannon, D.: Inferred advantage: Using Kogan's symmetric action principle to empirically assess alternatives in complex system development research, Engineering Design Journal 7 (1995), 241–252.
Cannon, D.: Point-and-direct telerobotics: Object level strategic supervisory control in unstructured interactive human-machine system environments, Stanford University Doctoral Dissertation, UMI Publishers, Ann Arbor, MI, 1992.
Wang, C. and Cannon, D.: A virtual end-effector pointing system in point-and-direct robotics for inspection of surface flaws using a neural network based skeleton transform, in Proceedings of the IEEE Robotics and Automation Conference, Atlanta, GA, 1993, pp. 784–795.
Wang, C. and Cannon, D.: Virtual reality based point-and-direct telerobotic inspection, IEEE Transactions on Robotics and Automation Journal (in press).
Lozano-Perez, T.: Spatial planning: A configuration space approach, IEEE Transactions on Computers 32(2) (1983), 108–120.
Brooks, R. A. and Lozano-Perez, T.: A subdivision algorithm in configuration space for findpath with rotation, IEEE Transactions on Systems, Man, and Cybernetics 15(2) (1985), 224–233.
Yu, Z. and Khalil, W.: Table look up for collision detection and safe operation of robots, IFAC/IFIP/IMACS Symposium, Vienna, Austria, 1986.
Hayward, V.: Fast collision detection scheme by recursive decomposition of a manipulator workspace, in Proceedings of IEEE International Conference on Robotics and Automation, San Francisco, CA, 1986.
Bonner, S. and Kelly, R. B.: A novel representation for planning 3-D collision-free paths, IEEE Transactions on System, Man, and Cybernetics 20(6) (1990), 1337–1351.
Del Pobil, A. P., Serna, M. A., and Llovet, J.: A new representation for collision avoidance and detection, IEEE International Conference on Robotics and Automation, Nice, France, 1992.
Lozano-Perez, T.: A simple motion-planning algorithm for general robot manipulators, IEEE Journal of Robotics and Automation RA-3(3) (1987), 224–238.
Shaffer, C. A. and Herb, G. M.: A real-time robot arm collision avoidance system, IEEE Transactions on Robotics and Automation 8(2) (1992), 149–160.
O'Rourke, J. and Badler, N.: Decomposition of three-dimensional objects into spheres, IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-1(3) (1979), 295–305.
Zapata, R., Coiffet, P., and Fournier, A.: Trajectory planning for a multi-arm robot in an assembly task, Digital Systems for Industrial Automation 2(2) (1984), 115–151.
Liu, Y., Arimoto, S., and Noborio, H.: A new solid model HSM and its application to interference detection between moving objects, Journal of Robotic Systems 8(1) (1991), 39–54.
Ma, H., Cannon, D., and Kumara, S.: A scheme integrating neural networks for real-time robotic collision detection, in Proceedings of the IEEE International Conference on Robotics and Automation, Nagoya, Japan, 1995, pp. 881–886.
Reilly, D. L., Cooper, L. N., and Elbaum, C.: A neural model for category learning, Biological Cybernetics 45 (1982), 35–41.
Reilly, D. L., Scofield, C., Elbaum, C., and Cooper, L. N.: Learning system architectures composed of multiple learning modules, IEEE First International Conference on Neural Networks, Washington, DC, 1987.
Nestor, Inc.: Learning System Based on Multiple Neural Networks, NDS Reference Manual, Nestor Inc., Providence, RI, 1988.
Groover, M. P., Weiss, M., Nagel, R. N., and Odrey, N. G.: Industrial Robotics Technology, Programming, and Applications, McGraw-Hill, New York, 1986.
Park, K. and Cannon, D.: Recognition and localization of a 3D polyhedral object using a neural network, in Proceedings of IEEE International Conference on Robotics and Automation, Minneapolis, MN, 1996.
Khatib, O.: Real-time obstacle avoidance for manipulators and mobile robots, International Journal of Robotics Research 5(1) (1986), 90–98.
Cannon, D. J., Graves, C., Lilly, K. W., and Bonaventura, C. S.: Virtual tools for interactive telerobotics: Potential fields and terrace following, 6th IFAC/IFIP/IFORS/IEA Symposium on Analysis, Design, and Evaluation of Man-Machine Systems, Cambridge, MA, 1995.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wang, C., Ma, H. & Cannon, D.J. Human-Machine Collaboration in Robotics: Integrating Virtual Tools with a Collision Avoidance Concept using Conglomerates of Spheres. Journal of Intelligent and Robotic Systems 18, 367–397 (1997). https://doi.org/10.1023/A:1007933917719
Issue Date:
DOI: https://doi.org/10.1023/A:1007933917719