Abstract
Supervising robotic assembly of multi-functional objects by means of a computer vision system requires components to identify assembly operations and to recognize feasible assemblies of single objects. Thus, the structure of complex objects as well as their construction processes are of interest. If the results of both components should be consistent there have to be common models providing knowledge about the intended application. However, if the assembly system should handle not only exactly specified tasks it is rather impossible to model every possible assembly or action explicitly. The fusion of a flexible dynamic model for assemblies and a monitor for the construction process enables reliable and efficient learning and supervision. As an example, the construction of objects by aggregating wooden toy pieces is used. The system also integrates a natural speech dialog module, which provides the overall communication strategy and additionally supports decisions in the case of ambiguities and uncertainty.
This work has been supported by the German Research Foundation within the Collaborative Research Center ‘Situated Artificial Communicators’ and the Graduate Program ‘Task-oriented Communication’.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
M. Abu-Hamdan, A. Sherif-El-Gizawy: Computer-aided monitoring system for flexible assembly operations, Computers In Industry, Bd. 34, Nr. 1, 1997, S. 1–10.
R. Anantha, G. Kramer, R. Crawford: Assembly modelling by geometric constraint satisfaction, Computer-Aided Design, Bd. 28, Nr. 9, 1996, S. 707–722.
C. Bauckhage, F. Kummert, G. Sagerer: Modeling and Recognition of Assembled Objects, in Proc. IECON’98, 1998, S. 2051–2056.
C. Bauckhage, F. Kummert, G. Sagerer: Learning Assembly Sequence Plans Using Functional Models, in Proc. IEEE International Symposium on Assembly and Task Planning (ISATP’99), 1999, S. 1–7.
C. Bauckhage, S. Kronenberg, F. Kummert, G. Sagerer: Grammars and Discourse Theory to Describe and Recognize Mechanical Assemblies, in Advances in Pattern Recognition, Lecture Notes in Computer Science 1876, Springer-Verlag, 2000, S. 173–182.
C. Bauckhage: Haskell script ‘plansem.hls’, http://www.techfak.unibielefeld.de/~cbauckha/plansemantics/plansem.lhs, 2001.
M. J. Black, A. D. Jepson: A probabilistic framework for matching temporal trajectories: CONDENSATION-based recognition of gestures and expressions, Lecture Notes in Computer Science, Bd. 1406, 1998, S. 909–924.
H.-J. Boehme, A. Brakensiek, U.-D. Braumann, M. Krabbes, H.-M. Gross: Neural Architecture for Gesture-Based Human-Machine-Interaction, in I. Wachsmuth, M. Fröhlich (Hrsg.): Gesture and Sign Languge in Human-Computer Interaction, Springer, Bielefeld, 1998, S. 219–232.
H. Brandt-Pook, G. A. Fink, S. Wachsmuth, G. Sagerer: Integrated Recognition and Interpretation of Speech for a Construction Task Domain, in Proc. of the International Conference on Human Computer Interaction (HCI), Bd. 1, 1999, S. 550–554.
S. Chakrabarty, J. Wolter: A Structure-Oriented Approach to Assembly Sequence Planning, IEEE Transactions on Robotics and Automation, Bd. 13, Nr. 1, 1997, S. 14–29.
D. Comaniciu, P. Meer: Robust Analysis of Feature Space: Color Image Segmentation, in Proc. IEEE Conf. CVPR, Puerto Rico, 1997, S. 750–755.
M. Costa, L. Shapiro: Analysis of Scenes Containing Multiple Non-Polyhedral Objects, in C. Braccini, L. D. Floriani, G. Vernazza (Hrsg.): Image Analysis and Processing, Bd. 974 von Lecture Notes in Computer Science, Springer-Verlag, 1995.
T. De Fazio, S. Rhee, D. Whitney: Design-Specific Approach to Design fo Assembly (DFA) for Complex Mechanical Assemblies, IEEE Transactions on Robotics and Automation, Bd. 15, Nr. 5, 1999, S. 869–881.
O. Faugeras: Three-Dimensional Computer Vision, MIT Press, Cambridge, Mass., 1993.
G. A. Fink: Developing HMM-based Recognizers with ESMERALDA, in V. Matoušek, P. Mautner, J. Ocelíková, P. Sojka (Hrsg.): Lecture Notes in Artificial Intelligence, Bd. 1692, Springer, Berlin, 1999, S. 229–234.
J. Fritsch, F. Loemker, M. Wienecke, G. Sagerer: Erkennung von Konstruktionshandlungen aus Bildfolgen, in Mustererkennung 2000, 22. DAGMSymposium Kiel, Informatik aktuell, Springer-Verlag, 2000, S. 389–396.
G. Furnas, T. Landauer, L. Gomez, S. Dumais: The Vobabulary Problem in Human-System Communication, Communications of ACM, Bd. 30, Nr. 11, 1987.
G. Heidemann, F. Kummert, H. Ritter, G. Sagerer: A Hybrid Object Recognition Architecture, in C. von der Malsburg, W. von Seelen, J. Vorbrüggen, B. Sendho. (Hrsg.): Artificial Neural Networks-ICANN 96, 16.–19. July, Springer-Verlag, Berlin, 1996, S. 305–310.
G. Heidemann: Ein flexibel einsetzbares Objekterkennungssystem auf der Basis neuronaler Netze, Bd. 190 von Dissertationen zur Künstlichen Intelligenz, Infix, Sankt Augustin, 1998.
L. Homem de Mello, A. Sanderson: AND/OR Graph Representation of Assembly Plans, IEEE Transactions on Robotics and Automation, Bd. 6, Nr. 2, 1990, S. 188–199.
M. Hunke, A. Waibel: Face Locating and Tracking for Human-Computer Interaction, in Twenty-Eight Asilomar Conference on Signals, Systems & Computers, Monterey, California, Nov 1994.
K. Ikeuchi, T. Suehiro: Towards an Assembly Plan from Observation Part I: Task Recognition with Polyhedral Objects, IEEE Transactions on Robotics and Automation, Bd. 10, Nr. 3, 1994, S. 368–385.
M. Isard, A. Blake: Contour tracking by stochastic propagation of conditional density, Lecture Notes in Computer Science, Bd. 1064, 1996, S. 343–356.
R. Jackendo.: Languages of the Mind, The MIT Press, 1992.
B. Jung, M. Hoffhenke, I. Wachsmuth: Virtual Assembly with Construction Kits, in Proc. ASME Design for Engineering Technical Conferences, 1998.
F. Kummert, G. A. Fink, G. Sagerer, E. Braun: Hybrid object recognition in image sequences, in 14th ICPR, Bd. II, Brisbane, 1998, S. 1165–1170.
J. Lloyd, J. Beis, D. Pai, D. Lowe: Programming Contact Tasks Using a Reality-Based Virtual Environment Integrated with Vision, IEEE Transactions on Robotics and Automation, Bd. 15, Nr. 3, 1999, S. 423–434.
A. Ma\mann, S. Posch, G. Sagerer, D. Schlüter: Using Markov Random Fields for Contour-Based Grouping, in Proceedings International Conference on Image Processing, Bd. II, IEEE, 1997, S. 207–210.
B. Messmer, H. Bunke: A New Algorithm for Error-Tolerant Subgraph Isomorphism Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, Bd. 20, Nr. 5, 1998, S. 493–504.
J. Miura, K. Ikeuchi: Task Oriented Generation of Visual Sensing Strategies in Assembly Tasks, IEEE Transactions on Pattern Analysis and Machine Intelligence, Bd. 20, Nr. 2, 1998, S. 126–138.
A. Mukerjee: Neat vs Scruffy: A review of Computational Models for Spatial Expressions, in P. Olivier, K.-P. Gapp (Hrsg.): Representation and processing of spatial expressions, Lawrence Erlbaum Associates, 1997.
B. Parhami: Voting Algorithm, IEEE Trans. on Reliability, Bd. 43, Nr. 4, 1994, S. 617–629.
J. Pearl: Probabilstic reasoning in intelligent systems: networks of plausible inference., Morgan Kaufmann, 1989.
M. Rabemanantsoa, S. Pierre: An artificial intelligence approach for generating assembly sequences in CAD/CAM, Artificial Intelligence in Engineering, Bd. 10, 1996, S. 97–107.
G. Sagerer, C. Bauckhage, E. Braun, G. Heidemann, F. Kummert, H. Ritter, D. Schlüter: Integrating Recognition Paradigms in a Multiple-path Architecture, in International Conference on Advances in Pattern Recognition, Riode Janeiro, 2001, to appear.
J. Schürmann: Pattern classification: a unified view of statistical and neural approaches, Wiley, New York, 1996.
K. Siddiqi, A. Shokoufandeh, S. Dickinson: Shock Graphs and Shape Matching, Int. Journal of Computer Vision, Bd. 35, Nr. 1, 1999, S. 13–32.
G. Socher, G. Sagerer, P. Perona: Baysian Reasoning on Qualitative Descriptions from Images and Speech, in H. Buxton, A. Mukerjee (Hrsg.): ICCV’98 Workshop on Conceptual Description of Images, Bombay, India, 1998.
M. Sonka, V. Hlavac, R. Boyle: Image Processing, Analysis and Machine Vision, Chapman & Hall, London, 1993.
M. Störring, H. J. Andersen, E. Granum: Skin colour detection under changing lighting conditions, in H. Araújo, J. Dias (Hrsg.): SIRS’99 Proc. 7th Int. Symposium on Intelligent Robotic Systems, July 1999, S. 187–195.
J. Thomas, N. Nissanke: An algebra for modelling assembly tasks, Mathematics And Computers In Simulation, Bd. 41, 1996, S. 639–659.
L. Ungerleider, M. Mishkin: Two cortical visual systems, in Analysis of Visual Behaviour, The MIT Press, 1982, S. 549–586.
I. Wachsmuth, B. Jung: Dynamic Conceptualization in a Mechanical-Object Assembly Environment, Artificial Intelligence Review, Bd. 10, Nr. 3–4, 1996, S. 345–368.
S. Wachsmuth, G. A. Fink, G. Sagerer: Integration of Parsing and Incremental Speech Recognition, in Proceedings of the European Signal Processing Conference (EUSIPCO-98), Bd. 1, Rhodes, Sep. 1998, S. 371–375.
S. Wachsmuth, H. Brandt-Pook, F. Kummert, G. Socher, G. Sagerer: Integration of Vision and Speech Understanding using Bayesian Networks, Videre: A Journal of Computer Vision Research, Bd. 1, Nr. 4, 1999, S. 62–83.
S. Wachsmuth, G. A. Fink, F. Kummert, G. Sagerer: Using Speech in Visual Object Recognition, in Proc. of DAGM-2000, 2000.
G. Wei, I. K. Sethi, N. Dimitrova: Face Detection for Image Annotation, in Pattern Recognition Letters, special issue about “Pattern Recognition in Practice VI”, Vlieland, June 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sagerer, G. et al. (2002). Structure and Process: Learning of Visual Models and Construction Plans for Complex Objects. In: Hager, G.D., Christensen, H.I., Bunke, H., Klein, R. (eds) Sensor Based Intelligent Robots. Lecture Notes in Computer Science, vol 2238. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45993-6_18
Download citation
DOI: https://doi.org/10.1007/3-540-45993-6_18
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43399-6
Online ISBN: 978-3-540-45993-4
eBook Packages: Springer Book Archive