Abstract
This research investigates a novel robot-programming approach that applies machine-vision techniques to generate a robot program automatically. The hand motions of a demonstrator are initially recorded as a long sequence of images using two CCD cameras. Machine-vision techniques are then used to recognize the hand motions in three-dimensional space including open, closed, grasp, release and move. The individual hand feature and its corresponding hand position in each sample image is translated to robot's manipulator-level instructions. Finally a robot plays back the task using the automatically generated program.
A robot can imitate the hand motions demonstrated by a human master using the proposed machine-vision approach. Compared with the traditional leadthrough and structural programming-language methods, the robot's user will not have to physically move the robot arm through the desired motion sequence and learn complicated robot-programming languages. The approach is currently focused on the classification of hand features and motions of a human arm and, therefore, is restricted to simple pick-and-place applications. Only one arm of the human master can be presented in the image scene, and the master must not wear long-sleeved clothes during demonstration to prevent false identification. Analysis and classification of hand motions in a long sequence of images are time-consuming. The automatic robot programming currently developed is performed off-line.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
ACL Advanced Control Language Reference Guide, Eshed Robotec Inc., Princeton, NJ, 1990.
Aggarwal, J. K. and Nandhakumar, N.: On the computation of motion from sequences of images — a review,Proc. IEEE 76(8) (1988), 917–935.
Akita, K.: Image sequence analyses of real world human motion,Pattern Recognition 17(1) (1984), 73–83.
Alvertos, N., et al.: Camera geometries for image matching in 3D machine vision,IEEE Trans. Pattern Anal. Mach. Intell. 11 (1989), 897–915.
Barnard, S. T. and Fischler, M. A.: Computational stereo,ACM Comput. Surveys 14 (1982), 553–572.
Derby, S.: Simulation motion elements of general purpose robot arm,Int. J. Robotic Res. 2(1) (1983).
Duelen, G., Bernhardt, R. and Schreck, G.: Use of CAD-data for the off-line programming of industrial robots, inRobotics 3, North-Holland, Amsterdam, 1987.
El-Zorkany, H. I. and Tondu, B.: Automatic sensor-based corrections in the programming of robot trajectories, in M. Briot (ed.),Proc. 6th Int. Conf. on Robot Vision and Sensory Controls, Paris, France, 1986.
Fioretti, S., Germani, A. and Leo, T.: Stereometry in very close-range stereophotogrammetry with non-metric cameras for human movement analyses,J. Biomech. 18 (1985), 831–842.
Freeman, H. and Davis, L. S.: A corner-finding algorithm for chain-coded curves,IEEE Trans. Computers C-26 (1977), 297–303.
Gonzalez, R. C. and Woods, R. E.:Digital Image Processing, Addison-Wesley, Reading, MA, 1992.
Groover, M. P., Weiss, M., Nagel, R. N. and Odrey, N. G.:Industrial Robotics, McGraw-Hill, New York, 1986.
Horn, B. K. P.:Robot Vision, MIT Press, Cambridge, MA, 1986.
Jain, R. and Nagel, H.-H.: On the analyses of accumulative difference pictures from image sequence of real world scenes,IEEE Trans. Pattern Anal. Mach. Intell. PAMI-1(2) (1979), 206–214.
Jayaramamurthy, S. N. and Jain, R.: An approach to the segmentation of textured dynamic scenes,Computer Vision, Graphics, Image Proc. 21 (1983), 239–261.
Langrana, N. A.: Spatial kinematic analysis of the upper extremity using a biplaner videotaping method,J. Biomech. Eng. 103 (1981), 11–17.
Latombe, J. C.: Automatic robot programming, inIFAC Artificial Intelligence, Leningrad, USSR, 1983.
Laugier, C. and Pertin-Troccaz, J.: SHARP: a system for automatic programming of manipulation robots, in Oliver Faugeras and George Giralt (eds),Robotics Research:3rd Int. Symp. MIT Press, Cambridge, MA, 1986.
Lee, J.: Computer-aided robot modeling, programming and simulation, inRobotics 3, North-Holland, Amsterdam, 1987.
Leung, M. K. and Yang, Y.-H.: Human body motion segmentation in a complex scene,Pattern Recognition 20(1) (1987), 55–64.
Leung, M. K. and Yang, Y.-H.: A region based approach for human body motion analysis,Pattern Recognition 20(3) (1987), 321–339.
Mecocci, A.: Moving object recognition and classification in external environments,Signal Processing 18 (1989), 183–194.
Mettala, E. G., Strong, D. B. and Egbelu, P. J.: Development of an artificial intelligence interface for robotic assembly in a flexible manufacturing system,Int. J. CIM 2(1) (1989), 48–58.
O'Rourke, J. and Badler, N. I.: Model-based image analysis of human motion using constraint propagation,IEEE Trans. Pattern Anal. Mach. Intell. PAMI-2(6) (1980), 522–536.
Rogers, D. F. and Adams, J. A.:Mathematical Elements for Computer Graphics, McGraw Hill, New York, 1990.
Safaee-Rad, R. and Shwedyk, E.: Three-dimensional measurement system for functional arm motion study,Med. Biol. Eng. Computing 28 (1990), 569–573.
Schalkoff, R. J.:Digital Image Processing and Computer Vision, Wiley, New York, 1989.
Sheu, P. C. and Kashyap, R. L.: Programming robot systems with knowledge,Robotic Computer Integrated Manufact. 3(3–4) (1988), 359–369.
Taylor, R. H.,et al.: AML: a manufacturing language,Int. J. Robotics Res. 1(3) (1982).
Tsai, D.-M. and Chen, Y.-H.: A fast histogram-clustering approach for multi-level thresholding,Pattern Recognition Lett. 13 (1992), 245–252.
Unimation Inc.,User's Guide to VAL: a Robot Programming and Control System, Unimation Inc., Danbury, CN, 1980.
Yachida, M., Asada, M. and Tsuji, S.: Automatic analysis of moving images,IEEE Trans. Pattern Anal. Mach. Intell. PAMI-3(1) (1981), 12–19.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Tsai, DM. A three-dimensional machine-vision approach for automatic robot programming. J Intell Robot Syst 12, 23–48 (1995). https://doi.org/10.1007/BF01258306
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF01258306