Skip to main content
SpringerLink
Log in

Human knowledge acquisition from 3D interaction in virtual environments

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Recently, virtual environment (VE) based design and planning, which is a kind of interaction intensive computing, has shown great potential. Most users of such systems are specialists or technicians and their 3D interactions with the system embed a great deal of knowledge and skills. Thus, how to integrate human knowledge and skills into VE is a great challenge to researchers in the human computer interaction field. This paper proposes a method for acquiring user knowledge from VE. The main ideas and work include, abstracting the interactive process and formalizing the interactive semantics, as well as providing a semantic model of an interactive information repository, from which user interactive processes can be retrieved and all kinds of application logics can be established. A virtual assembly planning system is introduced as an example of a practical application of this method. Experiments show that the related models can well capture user knowledge and retrieve the interaction process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Yuan X B, Yang S X. Virtual assembly with biologically inspired intelligence. IEEE Trans Syst Man Cybern C, 2003, 33: 159–167

    Article  Google Scholar 

  2. Smith S S F, Smith G, Liao X. Automatic stable assembly sequence generation and evaluation. J Manufact Syst, 2001, 20: 225–235

    Article  Google Scholar 

  3. Prendinger H, Ishizuka M. Symmetric multimodality revisited: Unveiling users’ physiological activity. IEEE Trans Indust Electr, 2007, 54: 692–698

    Article  Google Scholar 

  4. Zhang J W, Knoll A. A two-arm situated artificial communicator for human-robot cooperative assembly. IEEE Trans Indust Electr, 2003, 50: 651–658

    Article  Google Scholar 

  5. Rodriguez A, Basanez L, Celaya E. A relational positioning methodology for robot task specification and execution. IEEE Trans Robot, 2008, 24: 600–611

    Article  Google Scholar 

  6. Chueh M, Au Yeung Y L W, Lei K P C, et al. Following controller for autonomous mobile robots using behavioral cues. IEEE Trans Indust Electr, 2008, 55: 3124–3132

    Article  Google Scholar 

  7. Ogawara K, Takamatsu J, Kimura H, et al. Extraction of essential interactions through multiple observations of human demonstrations. IEEE Trans Indust Electr, 2003, 50: 667–675

    Article  Google Scholar 

  8. Hsieh F S. Analysis of flexible assembly processes based on structural decomposition of Petri nets. IEEE Trans Syst Man Cybern A, 2007, 37: 792–803

    Article  Google Scholar 

  9. Wu N Q, Zhou C M, Li Z W. Resource-oriented Petri net for deadlock avoidance in flexible assembly systems. IEEE Trans Syst Man Cybern A, 2008, 38: 56–69

    Article  Google Scholar 

  10. Cao T H, Sanderson A C. Task decomposition and analysis robotic assembly task plans using Petri nets. IEEE Trans Indust Electr, 1994, 41: 620–630

    Article  Google Scholar 

  11. Thomas J P, Nissanke N, Baker K D. Hierarchical Petri net framework for the representation and analysis of assembly. IEEE Trans Robot Autom, 1996, 12: 268–279

    Article  Google Scholar 

  12. Zha X F, Lim S Y E. Assembly/disassembly task planning and simulation using expert Petri nets. Int J Product Res, 2000, 38: 3639–3676

    Article  MATH  Google Scholar 

  13. McCarragher B J. Petri net modelling for robotic assembly and trajectory planning. IEEE Trans Indust Electr, 1994, 41: 631–640

    Article  Google Scholar 

  14. Seow K T, Devanathan R. A temporal framework for assembly sequence representation and analysis. IEEE Trans Robot Autom, 1994, 10: 220–229

    Article  Google Scholar 

  15. Kim S H, Imai H, Sankai Y. Interactive task generation for humanoid based on human motion strategy. In: Proceedings of the 2004 IEEE International Workshop on Robot and Human Interactive Communication, Hurashiki, Okayama, 2004. 485–490

  16. Faria D R, Dias J. 3D hand trajectory segmentation by curvatures and hand orientation for classification through a probabilistic approach. In: The 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, USA, 2009. 1284–1289

  17. Richter C, Fischer M, Irlinger F, et al. A spatial path specification system for mechanism development. In: International Conference of Human System Interaction 2009, Catania, Italy, 2009. 236–241

  18. Hyodo S, Ohnishi K. A method for motion abstraction based on haptic information directionality and an application to haptic motion display system. IEEE Trans Indust Electr, 2009, 56: 1356–1363

    Article  Google Scholar 

  19. Li L, Wesley M A. AUTOPASS: an automatic programming system for computer controlled mechanical assembly. IBM J Res Dev, 1979, 21: 321–333

    Google Scholar 

  20. Eastman C M. The design of assemblies. Society of Automotive Engineers, Technical Paper Series, 1981, 0148-7191/81/0223-0197

  21. Lee K, Gossard D C. A hierarchical data structure for representing assemblies: part 1. Comput Aid Des, 1985, 17: 15–19

    Article  Google Scholar 

  22. Homem de Mello L S, Sanderson A C. AND/OR graph representation of assembly plans. IEEE Trans Robot Autom, 1990, 6: 188–199

    Article  Google Scholar 

  23. Gu T L, Xu Z B, Yang Z F. Symbolic OBDD representations for mechanical assembly sequences. Comput Aid Des, 2008, 40: 409–520

    Article  Google Scholar 

  24. Horvath L, Rudas I J, Jezernik K. Towards content oriented integration of product and robot system models. In: International Symposium on Logistics and Industrial Informatics, 2007. 7–12

  25. Ciurana J, Garcia-Romeu M L, Ferrer I, et al. A model for integrating process planning and production planning and control in machining processes. Robot Comput-Integrat Manufact, 2008, 24: 532–544

    Article  Google Scholar 

  26. De Lit P, Danloy J, Delchambre A, et al. An assembly-oriented product family representation for integrated Design. IEEE Trans Robot Autom, 2003, 19: 75–88

    Article  Google Scholar 

  27. Wang H, Dong X, Duan G H, et al. Assembly planning based on semantic modeling approach. Comput Indust, 2007, 58: 227–239

    Article  Google Scholar 

  28. Banerjee A, Banerjee P, DeFanti T, et al. A behavioral layer architecture for telecollaborative virtual manufacturing operations. IEEE Trans Robot Autom, 2000, 16: 218–227

    Article  Google Scholar 

  29. Tang C S. Temporal Logic Programming and Software Engineering (in Chinese). Beijing: Science Press, 1999

    Google Scholar 

  30. Bedny G, Karwowski W, Bedny M. The principle of unity of cognition and behavior: implications of activity theory for the study of human work. Int J Cognit Ergon, 2001, 5: 401–420

    Article  Google Scholar 

  31. Kulyk O, Kosara R, Urquiza J, et al. Human-centered aspects. In: Kerren A, et al. eds. Human-centered visualization environments, LNCS 4417. Berlin: Springer-Verlag, 2007. 13–75

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Beijing Laboratory of Intelligent Information Technology, School of Computer Science, Beijing Institute of Technology, Beijing, 100081, China

    Cheng Cheng, Ru Jiang & XueMei Dong

Authors
  1. Cheng Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ru Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. XueMei Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cheng Cheng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cheng, C., Jiang, R. & Dong, X. Human knowledge acquisition from 3D interaction in virtual environments. Sci. China Inf. Sci. 55, 1528–1540 (2012). https://doi.org/10.1007/s11432-011-4436-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-011-4436-z

Keywords

Search

Navigation