Skip to main content
SpringerLink
Log in

Structure and Process: Learning of Visual Models and Construction Plans for Complex Objects

  • Conference paper
  • First Online:
Sensor Based Intelligent Robots

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2238))

  • 546 Accesses

Abstract

Supervising robotic assembly of multi-functional objects by means of a computer vision system requires components to identify assembly operations and to recognize feasible assemblies of single objects. Thus, the structure of complex objects as well as their construction processes are of interest. If the results of both components should be consistent there have to be common models providing knowledge about the intended application. However, if the assembly system should handle not only exactly specified tasks it is rather impossible to model every possible assembly or action explicitly. The fusion of a flexible dynamic model for assemblies and a monitor for the construction process enables reliable and efficient learning and supervision. As an example, the construction of objects by aggregating wooden toy pieces is used. The system also integrates a natural speech dialog module, which provides the overall communication strategy and additionally supports decisions in the case of ambiguities and uncertainty.

This work has been supported by the German Research Foundation within the Collaborative Research Center ‘Situated Artificial Communicators’ and the Graduate Program ‘Task-oriented Communication’.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. M. Abu-Hamdan, A. Sherif-El-Gizawy: Computer-aided monitoring system for flexible assembly operations, Computers In Industry, Bd. 34, Nr. 1, 1997, S. 1–10.

    Article  Google Scholar 

  2. R. Anantha, G. Kramer, R. Crawford: Assembly modelling by geometric constraint satisfaction, Computer-Aided Design, Bd. 28, Nr. 9, 1996, S. 707–722.

    Article  Google Scholar 

  3. C. Bauckhage, F. Kummert, G. Sagerer: Modeling and Recognition of Assembled Objects, in Proc. IECON’98, 1998, S. 2051–2056.

    Google Scholar 

  4. C. Bauckhage, F. Kummert, G. Sagerer: Learning Assembly Sequence Plans Using Functional Models, in Proc. IEEE International Symposium on Assembly and Task Planning (ISATP’99), 1999, S. 1–7.

    Google Scholar 

  5. C. Bauckhage, S. Kronenberg, F. Kummert, G. Sagerer: Grammars and Discourse Theory to Describe and Recognize Mechanical Assemblies, in Advances in Pattern Recognition, Lecture Notes in Computer Science 1876, Springer-Verlag, 2000, S. 173–182.

    Chapter  Google Scholar 

  6. C. Bauckhage: Haskell script ‘plansem.hls’, http://www.techfak.unibielefeld.de/~cbauckha/plansemantics/plansem.lhs, 2001.

  7. M. J. Black, A. D. Jepson: A probabilistic framework for matching temporal trajectories: CONDENSATION-based recognition of gestures and expressions, Lecture Notes in Computer Science, Bd. 1406, 1998, S. 909–924.

    Google Scholar 

  8. H.-J. Boehme, A. Brakensiek, U.-D. Braumann, M. Krabbes, H.-M. Gross: Neural Architecture for Gesture-Based Human-Machine-Interaction, in I. Wachsmuth, M. Fröhlich (Hrsg.): Gesture and Sign Languge in Human-Computer Interaction, Springer, Bielefeld, 1998, S. 219–232.

    Chapter  Google Scholar 

  9. H. Brandt-Pook, G. A. Fink, S. Wachsmuth, G. Sagerer: Integrated Recognition and Interpretation of Speech for a Construction Task Domain, in Proc. of the International Conference on Human Computer Interaction (HCI), Bd. 1, 1999, S. 550–554.

    Google Scholar 

  10. S. Chakrabarty, J. Wolter: A Structure-Oriented Approach to Assembly Sequence Planning, IEEE Transactions on Robotics and Automation, Bd. 13, Nr. 1, 1997, S. 14–29.

    Article  Google Scholar 

  11. D. Comaniciu, P. Meer: Robust Analysis of Feature Space: Color Image Segmentation, in Proc. IEEE Conf. CVPR, Puerto Rico, 1997, S. 750–755.

    Google Scholar 

  12. M. Costa, L. Shapiro: Analysis of Scenes Containing Multiple Non-Polyhedral Objects, in C. Braccini, L. D. Floriani, G. Vernazza (Hrsg.): Image Analysis and Processing, Bd. 974 von Lecture Notes in Computer Science, Springer-Verlag, 1995.

    Google Scholar 

  13. T. De Fazio, S. Rhee, D. Whitney: Design-Specific Approach to Design fo Assembly (DFA) for Complex Mechanical Assemblies, IEEE Transactions on Robotics and Automation, Bd. 15, Nr. 5, 1999, S. 869–881.

    Article  Google Scholar 

  14. O. Faugeras: Three-Dimensional Computer Vision, MIT Press, Cambridge, Mass., 1993.

    Google Scholar 

  15. G. A. Fink: Developing HMM-based Recognizers with ESMERALDA, in V. Matoušek, P. Mautner, J. Ocelíková, P. Sojka (Hrsg.): Lecture Notes in Artificial Intelligence, Bd. 1692, Springer, Berlin, 1999, S. 229–234.

    Google Scholar 

  16. J. Fritsch, F. Loemker, M. Wienecke, G. Sagerer: Erkennung von Konstruktionshandlungen aus Bildfolgen, in Mustererkennung 2000, 22. DAGMSymposium Kiel, Informatik aktuell, Springer-Verlag, 2000, S. 389–396.

    Google Scholar 

  17. G. Furnas, T. Landauer, L. Gomez, S. Dumais: The Vobabulary Problem in Human-System Communication, Communications of ACM, Bd. 30, Nr. 11, 1987.

    Google Scholar 

  18. G. Heidemann, F. Kummert, H. Ritter, G. Sagerer: A Hybrid Object Recognition Architecture, in C. von der Malsburg, W. von Seelen, J. Vorbrüggen, B. Sendho. (Hrsg.): Artificial Neural Networks-ICANN 96, 16.–19. July, Springer-Verlag, Berlin, 1996, S. 305–310.

    Google Scholar 

  19. G. Heidemann: Ein flexibel einsetzbares Objekterkennungssystem auf der Basis neuronaler Netze, Bd. 190 von Dissertationen zur Künstlichen Intelligenz, Infix, Sankt Augustin, 1998.

    Google Scholar 

  20. L. Homem de Mello, A. Sanderson: AND/OR Graph Representation of Assembly Plans, IEEE Transactions on Robotics and Automation, Bd. 6, Nr. 2, 1990, S. 188–199.

    Article  Google Scholar 

  21. M. Hunke, A. Waibel: Face Locating and Tracking for Human-Computer Interaction, in Twenty-Eight Asilomar Conference on Signals, Systems & Computers, Monterey, California, Nov 1994.

    Google Scholar 

  22. K. Ikeuchi, T. Suehiro: Towards an Assembly Plan from Observation Part I: Task Recognition with Polyhedral Objects, IEEE Transactions on Robotics and Automation, Bd. 10, Nr. 3, 1994, S. 368–385.

    Article  Google Scholar 

  23. M. Isard, A. Blake: Contour tracking by stochastic propagation of conditional density, Lecture Notes in Computer Science, Bd. 1064, 1996, S. 343–356.

    Google Scholar 

  24. R. Jackendo.: Languages of the Mind, The MIT Press, 1992.

    Google Scholar 

  25. B. Jung, M. Hoffhenke, I. Wachsmuth: Virtual Assembly with Construction Kits, in Proc. ASME Design for Engineering Technical Conferences, 1998.

    Google Scholar 

  26. F. Kummert, G. A. Fink, G. Sagerer, E. Braun: Hybrid object recognition in image sequences, in 14th ICPR, Bd. II, Brisbane, 1998, S. 1165–1170.

    Google Scholar 

  27. J. Lloyd, J. Beis, D. Pai, D. Lowe: Programming Contact Tasks Using a Reality-Based Virtual Environment Integrated with Vision, IEEE Transactions on Robotics and Automation, Bd. 15, Nr. 3, 1999, S. 423–434.

    Article  Google Scholar 

  28. A. Ma\mann, S. Posch, G. Sagerer, D. Schlüter: Using Markov Random Fields for Contour-Based Grouping, in Proceedings International Conference on Image Processing, Bd. II, IEEE, 1997, S. 207–210.

    Google Scholar 

  29. B. Messmer, H. Bunke: A New Algorithm for Error-Tolerant Subgraph Isomorphism Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, Bd. 20, Nr. 5, 1998, S. 493–504.

    Article  Google Scholar 

  30. J. Miura, K. Ikeuchi: Task Oriented Generation of Visual Sensing Strategies in Assembly Tasks, IEEE Transactions on Pattern Analysis and Machine Intelligence, Bd. 20, Nr. 2, 1998, S. 126–138.

    Article  Google Scholar 

  31. A. Mukerjee: Neat vs Scruffy: A review of Computational Models for Spatial Expressions, in P. Olivier, K.-P. Gapp (Hrsg.): Representation and processing of spatial expressions, Lawrence Erlbaum Associates, 1997.

    Google Scholar 

  32. B. Parhami: Voting Algorithm, IEEE Trans. on Reliability, Bd. 43, Nr. 4, 1994, S. 617–629.

    Article  Google Scholar 

  33. J. Pearl: Probabilstic reasoning in intelligent systems: networks of plausible inference., Morgan Kaufmann, 1989.

    Google Scholar 

  34. M. Rabemanantsoa, S. Pierre: An artificial intelligence approach for generating assembly sequences in CAD/CAM, Artificial Intelligence in Engineering, Bd. 10, 1996, S. 97–107.

    Article  Google Scholar 

  35. G. Sagerer, C. Bauckhage, E. Braun, G. Heidemann, F. Kummert, H. Ritter, D. Schlüter: Integrating Recognition Paradigms in a Multiple-path Architecture, in International Conference on Advances in Pattern Recognition, Riode Janeiro, 2001, to appear.

    Google Scholar 

  36. J. Schürmann: Pattern classification: a unified view of statistical and neural approaches, Wiley, New York, 1996.

    Google Scholar 

  37. K. Siddiqi, A. Shokoufandeh, S. Dickinson: Shock Graphs and Shape Matching, Int. Journal of Computer Vision, Bd. 35, Nr. 1, 1999, S. 13–32.

    Article  Google Scholar 

  38. G. Socher, G. Sagerer, P. Perona: Baysian Reasoning on Qualitative Descriptions from Images and Speech, in H. Buxton, A. Mukerjee (Hrsg.): ICCV’98 Workshop on Conceptual Description of Images, Bombay, India, 1998.

    Google Scholar 

  39. M. Sonka, V. Hlavac, R. Boyle: Image Processing, Analysis and Machine Vision, Chapman & Hall, London, 1993.

    Google Scholar 

  40. M. Störring, H. J. Andersen, E. Granum: Skin colour detection under changing lighting conditions, in H. Araújo, J. Dias (Hrsg.): SIRS’99 Proc. 7th Int. Symposium on Intelligent Robotic Systems, July 1999, S. 187–195.

    Google Scholar 

  41. J. Thomas, N. Nissanke: An algebra for modelling assembly tasks, Mathematics And Computers In Simulation, Bd. 41, 1996, S. 639–659.

    Article  Google Scholar 

  42. L. Ungerleider, M. Mishkin: Two cortical visual systems, in Analysis of Visual Behaviour, The MIT Press, 1982, S. 549–586.

    Google Scholar 

  43. I. Wachsmuth, B. Jung: Dynamic Conceptualization in a Mechanical-Object Assembly Environment, Artificial Intelligence Review, Bd. 10, Nr. 3–4, 1996, S. 345–368.

    Article  Google Scholar 

  44. S. Wachsmuth, G. A. Fink, G. Sagerer: Integration of Parsing and Incremental Speech Recognition, in Proceedings of the European Signal Processing Conference (EUSIPCO-98), Bd. 1, Rhodes, Sep. 1998, S. 371–375.

    Google Scholar 

  45. S. Wachsmuth, H. Brandt-Pook, F. Kummert, G. Socher, G. Sagerer: Integration of Vision and Speech Understanding using Bayesian Networks, Videre: A Journal of Computer Vision Research, Bd. 1, Nr. 4, 1999, S. 62–83.

    Google Scholar 

  46. S. Wachsmuth, G. A. Fink, F. Kummert, G. Sagerer: Using Speech in Visual Object Recognition, in Proc. of DAGM-2000, 2000.

    Google Scholar 

  47. G. Wei, I. K. Sethi, N. Dimitrova: Face Detection for Image Annotation, in Pattern Recognition Letters, special issue about “Pattern Recognition in Practice VI”, Vlieland, June 1999.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Technology, Bielefeld University, Germany

    G. Sagerer, C. Bauckhage, E. Braun, J. Fritsch, F. Kummert, F. Lömker & S. Wachsmuth

Authors
  1. G. Sagerer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. Bauckhage
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. E. Braun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. Fritsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. F. Kummert
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. F. Lömker
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. S. Wachsmuth
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, Johns Hopkins University, Baltimore

    Gregory D. Hager

  2. Center for Autonomous Systems, CVAP, Royal Institute of Technology, Sweden

    Henrik Iskov Christensen

  3. Department of Computer Science, University of Bern, German

    Horst Bunke

  4. Institute for Computer Science I, University of Bonn, USA

    Rolf Klein

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Sagerer, G. et al. (2002). Structure and Process: Learning of Visual Models and Construction Plans for Complex Objects. In: Hager, G.D., Christensen, H.I., Bunke, H., Klein, R. (eds) Sensor Based Intelligent Robots. Lecture Notes in Computer Science, vol 2238. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45993-6_18

Download citation

  • DOI: https://doi.org/10.1007/3-540-45993-6_18

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43399-6

  • Online ISBN: 978-3-540-45993-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation