Skip to main content

Advertisement

SpringerLink
Log in

A comprehensive survey of AR/MR-based co-design in manufacturing

  • Original Article
  • Published:
Engineering with Computers Aims and scope Submit manuscript

Abstract

For more than 2 decades, Augmented Reality (AR)/Mixed Reality (MR) has received an increasing amount of attention by researchers and practitioners in the manufacturing community, because it has applications in many fields, such as product design, training, maintenance, assembly, and other manufacturing operations. However, to the best of our knowledge, there has been no comprehensive review of AR-based co-design in manufacturing. This paper presents a comprehensive survey of existing research, projects, and technical characteristics between 1990 and 2017 in the domain of co-design based on AR technology. Among these papers, more than 90% of them were published between 2000 and 2017, and these recent relevant works are discussed at length. The paper provides a comprehensive academic roadmap and useful insight into the state-of-the-art of AR-based co-design systems and developments in manufacturing for future researchers all over the world. This work will be useful to researchers who plan to utilize AR as a tool for design research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. http://www.google.com/glass/start.

  2. https://www.microsoft.com.

  3. https://www.daqri.com.

  4. https://zspace.com.

  5. http://www.metavision.com.

  6. https://tortoisesvn.net.

  7. https://developer.vuforia.com.

  8. https://www.opengl.org.

  9. http://trac.openscenegraph.org/projects/osg.

  10. https://www.opencascade.com.

  11. http://artoolkit.org/.

  12. http://www.empathiccomputing.org/.

  13. https://www.empatica.com/e4-wristband.

  14. https://www.leapmotion.com/.

  15. https://developer.microsoft.com/en-us/windows/kinect.

  16. https://www.softkinetic.com/.

  17. https://pupil-labs.com/pupil/.

  18. https://www.tobii.com/.

  19. http://www.eyetracking.com/.

References

  1. Chandrasegaran SK, Ramani K, Sriram RD, Horváth I, Bernard A, Harik RF et al (2013) The evolution, challenges, and future of knowledge representation in product design systems. Comput Aided Des 45(2):204–228

    Google Scholar 

  2. Wang X, Dunston PS (2011) Comparative effectiveness of mixed reality-based virtual environments in collaborative design. IEEE Trans Syst Man Cybern Part C (Appl Rev) 41(3):284–296

    Google Scholar 

  3. Gu N, Mi JK, Maher ML (2011) Technological advancements in synchronous collaboration: the effect of 3d virtual worlds and tangible user interfaces on architectural design. Autom Constr 20(3):270–278

    Google Scholar 

  4. Karakaya AF, Demirkan H (2015) Collaborative digital environments to enhance the creativity of designers. Comput Hum Behav 42:176–186

    Google Scholar 

  5. Mitchell V, Ross T, May A, Sims R, Parker C (2016) Empirical investigation of the impact of using co-design methods when generating proposals for sustainable travel solutions. Codesign 12(4):205–220

    Google Scholar 

  6. Tseng MM, Du X (1998) Design by customers for mass customization products. CIRP Ann Manuf Technol 47(1):103–106

    Google Scholar 

  7. Porter ME, Heppelmann JE (2017) Why every organization needs an augmented reality strategy. Harv Bus Rev 95(6):46–57

    Google Scholar 

  8. Mohr P, Mandl D, Tatzgern M, Veas E, Schmalstieg D, Kalkofen D. (2017) Retargeting video tutorials showing tools with surface contact to augmented reality. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, pp 6547–6558

  9. Lukosch S, Billinghurst M, Alem L, Kiyokawa K (2015) Collaboration in Augmented Reality. Comput Support Cooperative Work 24:515

    Google Scholar 

  10. See ZS, Billinghurst M, Rengganaten V, Soo S (2016) Medical learning murmurs simulation with mobile audible augmented reality. In: SIGGRAPH ASIA 2016 mobile graphics and interactive applications, ACM, p. 4

  11. Wang X, Ong SK, Nee AYC (2016) A comprehensive survey of augmented reality assembly research. Adv Manuf 4(1):1–22

    Google Scholar 

  12. Henderson S, Feiner S (2011) Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Trans Visual Comput Graph 17(10):1355

    Google Scholar 

  13. Shen Y, Ong SK, Nee AYC (2010) Augmented reality for collaborative product design and development. Des Stud 31(2):118–145

    Google Scholar 

  14. Ong SK, Shen Y (2009) A mixed reality environment for collaborative product design and development. CIRP Ann 58(1):139–142

    Google Scholar 

  15. Nee AY, Ong SK, Chryssolouris G, Mourtzis D (2012) Augmented reality applications in design and manufacturing. CIRP Ann Manuf Technol 61(2):657–679

    Google Scholar 

  16. Ong SK, Yuan ML, Nee AYC (2008) Augmented reality applications in manufacturing: a survey. Int J Prod Res 46(10):2707–2742

    MATH  Google Scholar 

  17. Huang TC, Chen CC, Chou YW (2016) Animating eco-education: to see, feel, and discover in an augmented reality-based experiential learning environment. Comput Educ 96:72–82

    Google Scholar 

  18. Wu HK, Lee WY, Chang HY, Liang JC (2013) Current status, opportunities and challenges of augmented reality in education. Comput Educ 62:41–49

    Google Scholar 

  19. Arroyave-Tobón S, Osorio-Gómez G, Cardona-McCormick JF (2015) Air-modelling: a tool for gesture-based solid modelling in context during early design stages in AR environments. Comput Ind 66:73–81

    Google Scholar 

  20. Millette A, McGuffin MJ (2016) DualCAD: integrating augmented reality with a desktop GUI and smartphone interaction. In: Mixed and augmented reality (ISMAR-Adjunct), 2016 IEEE International Symposium, IEEE, pp 21–26

  21. Uva AE, Cristiano S, Fiorentino M, Monno G (2010) Distributed design review using tangible augmented technical drawings. Comput Aided Des 42(5):364–372

    Google Scholar 

  22. Januszka M, Moczulski W (2011) Augmented reality system for aiding engineering design process of machinery systems. J Syst Sci Syst Eng 20(3):294

    Google Scholar 

  23. Wang X, Love PE, Kim MJ, Wang W (2014) Mutual awareness in collaborative design: an augmented reality integrated telepresence system. Comput Ind 65(2):314–324

    Google Scholar 

  24. Chen Y, Feng P, Lin Z (2005) A genetics-based approach for the principle conceptual design of mechanical products. Int J Adv Manuf Technol 27(3–4):225–233

    Google Scholar 

  25. Wang L, Shen W, Xie H, Neelamkavil J, Pardasani A (2002) Collaborative conceptual design—state of the art and future trends. Comput Aided Des 34(13):981–996

    Google Scholar 

  26. Li WD, Ong SK, Fuh JYH, Wong YS, Lu YQ, Nee AYC (2004) Feature-based design in a distributed and collaborative environment. Comput Aided Des 36(9):775–797

    Google Scholar 

  27. Chu CH, Cheng CY, Wu CW (2006) Applications of the web-based collaborative visualization in distributed product development. Comput Ind 57(3):272–282

    Google Scholar 

  28. Yin X, Wu Y, Liu Q, Wang S (2015) A web-based, collaborative modeling, simulation, and parallel computing environment for electromechanical systems. Adv Mech Eng 7(3):1–6

    Google Scholar 

  29. Qiang L, Zhang YF, Nee AYC (2001) A distributive and collaborative concurrent product design system through the www/internet. Int J Adv Manuf Technol 17(5):315–322

    Google Scholar 

  30. Fan H, Liu Y, Hu B, Ye X (2016) Multidomain model integration for online collaborative system design and detailed design of complex mechatronic systems. IEEE Trans Autom Sci Eng 13(2):709–728

    Google Scholar 

  31. Wang K, Takahashi A (2012) Semantic web based innovative design knowledge modeling for collaborative design. Expert Syst Appl 39(5):5616–5624

    Google Scholar 

  32. Monostori L, Váncza J, Kumara SRT (2006) Agent-based systems for manufacturing. CIRP Ann Manuf Technol 55(2):697–720

    Google Scholar 

  33. Wang JX, Tang MX, Song LN, Jiang SQ (2009) Design and implementation of an agent-based collaborative product design system. Comput Ind 60(7):520–535

    Google Scholar 

  34. Shen W, Maturana F, Norrie DH (2000) Metamorph ii: an agent-based architecture for distributed intelligent design and manufacturing. J Intell Manuf 11(3):237–251

    MATH  Google Scholar 

  35. Chen Y, Liu ZL, Xie YB (2014) A multi-agent-based approach for conceptual design synthesis of multi-disciplinary systems. Int J Prod Res 52(6):1681–1694

    Google Scholar 

  36. Shen W, Norrie DH, Barthés JPA (2000) Multi-agent systems for concurrent intelligent design and manufacturing, vol 47, no 11. CRC Press, Boca Raton, pp 1253–1254

    Google Scholar 

  37. Wu D, Rosen DW, Wang L, Schaefer D (2015) Cloud-based design and manufacturing: a new paradigm in digital manufacturing and design innovation. Comput Aided Des 59:1

    Google Scholar 

  38. Schaefer D (2014) Cloud-based design and manufacturing (CBDM). Springer, Berlin

    Google Scholar 

  39. Li WD, Lu WF, Fuh JYH, Wong YS (2005) Collaborative computer-aided design-research and development status. Comput Aided Des 37(9):931–940

    Google Scholar 

  40. Fuh JYH, Li WD (2005) Advances in collaborative cad: the-state-of-the-art. Comput Aided Des 37(5):571–581

    Google Scholar 

  41. Shen W, Wang L (2003) Web-based and agent-based approaches for collaborative product design: an overview. Int J Comput Appl Technol 16(2–3):103–112

    Google Scholar 

  42. French D, Jensen G, Walker SS, Madsen P, Red E (2013) Emerging design methods and tools in collaborative product development. J Comput Inf Sci Eng 13(3):031001

    Google Scholar 

  43. Huang GQ (2002) Web-based support for collaborative product design review. Comput Ind 48(1):71–88

    Google Scholar 

  44. Ren L, Zhang L, Tao F, Zhang XL, Zhang Y, Zhang Y (2012) A methodology towards virtualisation-based high performance simulation platform supporting multidisciplinary design of complex products. Enterp Inf Syst 6(3):267–290

    Google Scholar 

  45. Gao S, Wan H, Peng Q (2000) An approach to solid modeling in a semi-immersive virtual environment. Comput Graph 24(2):191–202

    Google Scholar 

  46. Kiyokawa K, Billinghurst M, Hayes SE, Gupta A, Sannohe Y, Kato H (2002) Communication behaviors of co-located users in collaborative AR interfaces. In: mixed and augmented reality, 2002. ISMAR 2002. Proceedings. International symposium on IEEE, pp 139–148

  47. Rekimoto J (1996) Transvision: a hand-held augmented reality system for collaborative design. Proc Virtual Syst Multimed 96:18–20

    Google Scholar 

  48. Masai K, Kunze K, Billinghurst M (2016) Empathy glasses. In: Proceedings of the 2016 CHI Conference Extended abstracts on human factors in computing systems. ACM, pp 1257–1263

  49. Kouprie M, Visser FS (2009) A framework for empathy in design: stepping into and out of the user’s life. J Eng Des 20(5):437–448

    Google Scholar 

  50. Yuan S, Dong H (2014) Empathy building through co-design. In: International conference on universal access in human-computer interaction. Springer, Cham, pp 85–91

  51. Schmalstieg D, Fuhrmann A, Gervautz M (1998) “studierstube”: an environment for collaboration in augmented reality. Virtual Real 3(1):37–48

    Google Scholar 

  52. Blanding R, Turkiyyah GECAD (2007) A prototype screen-based VR solid modeling environment incorporating tangible deformable models. Comput Aided Des Appl 4(5):595–605

    Google Scholar 

  53. Jezernik A, Hren G (2003) A solution to integrate computer-aided design (CAD) and virtual reality (VR) databases in design and manufacturing processes. Int J Adv Manuf Technol 22(11–12):768–774

    Google Scholar 

  54. Ma W, Zhong Y, Tso SK, Zhou T (2004) A hierarchically structured and constraint-based data model for intuitive and precise solid modeling in a virtual reality environment. Comput Aided Des 36(10):903–928

    Google Scholar 

  55. Zhong Y, Ma W, Shirinzadeh B (2005) A methodology for solid modelling in a virtual reality environment. Robot Comput Integr Manuf 21(6):528–549

    Google Scholar 

  56. Wang QH, Li JR, Wu BL, Zhang XM (2010) Live parametric design modifications in cad-linked virtual environment. Int J Adv Manuf Technol 50(9–12):859–869

    Google Scholar 

  57. Kan HY, Duffy VG, Su CJ (2001) An internet virtual reality collaborative environment for effective product design. Comput Ind 45(2):197–213

    Google Scholar 

  58. Mahdjoub M, Monticolo D, Gomes S, Sagot JC (2010) A collaborative design for usability approach supported by virtual reality and a multi-agent system embedded in a plm environment. Comput Aided Des 42(5):402–413

    Google Scholar 

  59. Ishii H (1997) Tangible bits: towards seamless interfaces between people, bits and atoms. In: ACM Sigchi conference on human factors in computing systems. ACM, pp 234–241

  60. Dunston P, Wang X, Billinghurst M, Hampson B (2003) Mixed reality benefits for design perception. In: 19th International Symposium on automation and robotics construction (ISARC 2002), pp 191–196

  61. Zhou F, Duh BL, Billinghurst M (2008) Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR. In: IEEE/ACM International Symposium on mixed and augmented reality. IEEE, pp 193–202

  62. Piya C, Vinayak, Chandrasegaran S, Elmqvist N, Ramani K (2017) Co-3Deator: ateam-first collaborative 3D design ideation tool. In: Chi Conference on human factors in computing systems. ACM, pp 6581–6592

  63. Kiyokawa K, Iwasa H, Takemura H, Yokoya N (1998) Collaborative immersive workspace through. In: Proceedings of SPIE International Symposium on intelligent systems and advanced manufacturing, Vol. 3517, pp 2–13

  64. Billinghurst M, Kato H (2002) Collaborative augmented reality. Commun ACM 45(7):64–70

    Google Scholar 

  65. Tait M, Billinghurst M (2015) The effect of view independence in a collaborative AR system. Comput Support Coop Work 24(6):563–589

    Google Scholar 

  66. Tan CSS, Luyten K, Jan VDB, Schöning J, Coninx K (2014) The role of physiological cues during remote collaboration. Presence 23(1):90–107

    Google Scholar 

  67. Xolocotzin Eligio U, Ainsworth SE, Crook CK (2012) Emotion understanding and performance during computer-supported collaboration. Comput Hum Behav 28:2046

    Google Scholar 

  68. Lu CY, Shpitalni M, Gadh R (1999) Virtual and augmented reality technologies for product realization. CIRP Ann Manuf Technol 48(2):471–495

    Google Scholar 

  69. Wang X, Gu N, Marchant D (2008) An empirical study on designers’ perceptions of augmented reality within an architectural firm. Electron J Inf Technol Constr 13:536–552

    Google Scholar 

  70. Li JR, Tang C, Wang QH (2015) A proxy approach to integrate heterogeneous CAD resources for distributed collaborative design. Int J Comput Integr Manuf 28(6):593–606

    Google Scholar 

  71. Shen Y, Ong SK, Nee AYC (2006) A Framework for Multiple-View Product Representation Using Augmented Reality. In: International Conference on cyberworlds. IEEE Computer Society, pp 157–164

  72. Shen Y, Ong SK, Nee AYC (2008) Product information visualization and augmentation in collaborative design. Comput Aided Des 40(9):963–974

    Google Scholar 

  73. Xia H, Herscher S, Perlin K, Wigdor D (2018) Spacetime: enabling fluid individual and collaborative editing in virtual reality. In: The 31st Annual ACM Symposium on user interface software and technology. ACM, pp 853–866

  74. Vinayak, Ramanujan D, Piya C, Ramani K (2016) MobiSweep: exploring spatial design ideation using a smartphone as a hand-held reference plane. Proceedings of the TEI'16: tenth international conference on tangible, embedded, and embodied interaction. ACM, pp 12–20

  75. Huo K, Vinayak, Ramani K (2017) Window-Shaping: 3D design ideation by creating on, borrowing from, and looking at the physical world. In: Eleventh International Conference on Tangible, 2017

  76. Sang HY, Ke H, Ramani K (2016) TMotion: embedded 3D mobile input using magnetic sensing technique. In: Tei 16: Tenth International Conference on Tangible, 2016

  77. Zhao Z, Badam S K, Chandrasegaran S, Park D G, Elmqvist N, Kisselburgh L, Ramani K (2014) skWiki:a multimedia sketching system for collaborative creativity. In: Acm Conference on Human Factors in Computing Systems, 2014

  78. Dey A, Billinghurst M, Lindeman RW, Ii JES (2017) A systematic review of usability studies in augmented reality between 2005 and 2014. In: IEEE International Symposium on Mixed and Augmented Reality, IEEE, pp 49–50

  79. Ahlers KH, Kramer A, Breen DE, Chevalier PY, Crampton C, Rose E, Tuceryan M, Whitaker RT, Greer D (1995) Distributed augmented reality for collaborative design applications. Comput Graph Forum 14(3):3–14

    Google Scholar 

  80. Kiyokawa K, Takemura H, Katayama Y, Iwasa H, Yokoya N (1996) Vlego: a simple two-handed modeling environment based on toy blocks. Proceedings of the ACM symposium on virtual reality software and technology. ACM, pp 27–34

  81. Kiyokawa K, Takemura H, Yokoya N (1999) Seamlessdesign: a face-to-face collaborative virtual/augmented environment for rapid prototyping of geometrically constrained 3-d objects. In: Multimedia computing and systems, 1999. IEEE International Conference on (Vol. 2, pp 447–453), IEEE

  82. Schumann H, Burtescu S, Siering F (1998) Applying augmented reality techniques in the field of interactive collaborative design. 3D structure from multiple images of large-scale environments. Springer, Berlin, pp 290–303

    Google Scholar 

  83. Schmalstieg D, Fuhrmann A, Hesina G, Gervautz M, Purgathofer W (2002) The studierstube augmented reality project. Presence Teleoperators Virtual Environ 11(1):33–54

    Google Scholar 

  84. Grønbæk K, Mogensen P, Rbk P (2001) Interactive room support for complex and distributed design projects. INTERACT, pp 407–414

  85. Gupta K, Lee GA, Billinghurst M (2016) Do you see what i see? the effect of gaze tracking on task space remote collaboration. IEEE Trans Visual Comput Graphics 22(11):2413–2422

    Google Scholar 

  86. Piumsomboon T, Lee Y, Lee GA, Dey A, Billinghurst M (2017) Empathic mixed reality: sharing what you feel and interacting with what you see. In: International Symposium on Ubiquitous Virtual Reality, IEEE, pp 38–41

  87. Lee Y, Masai K, Kai K, Sugimoto M, Billinghurst M (2017) A remote collaboration system with empathy glasses. In: IEEE International Symposium on mixed and augmented reality, IEEE, pp 342–343

  88. Gao L, Bai H, Lee G, Billinghurst M (2016) An oriented point-cloud view for MR remote collaboration. In: SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications. ACM, p 8

  89. Regenbrecht HT, Wagner M, Baratoff G (2002) Magicmeeting: a collaborative tangible augmented reality system. Virtual Real 6(3):151–166

    Google Scholar 

  90. Wagner MT, Wagner MT (2002) Interaction in a collaborative augmented reality environment. In: CHI ‘02 Extended Abstracts on human factors in computing systems, ACM, pp 504–505

  91. Klinker G, Dutoit AH, Bauer M, Bayer J, Novak V, Matzke D (2002) Fata Morgana “A Presentation System for Product Design”. In: International Symposium on mixed and augmented reality, IEEE Computer Society, pp 76

  92. Liu P, Georganas ND, Boulanger P (2002) Designing real-time vision based augmented reality environments for 3D collaborative applications. Des Real, IEEE

    Google Scholar 

  93. Verlinden JC (2003) Development of a flexible augmented prototyping system. UNION Agency–Science Press, pp 1–8

  94. Hirakawa M, Koike S (2004) A collaborative augmented reality system using transparent display. In: IEEE Sixth International Symposium on multimedia software engineering, IEEE Computer Society, pp 410–416

  95. Penn A, Mottram C, Schieck AFG, Wittkämper M, Störring M, Romell O, Strothmann A, Aish F (2004) Recent advances in design and decision support systems in architecture and urban planning. Springer, Dordrecht, pp 213–231

    Google Scholar 

  96. Broll W, Lindt I, Ohlenburg J, Wittkamper M, Yuan C, Novotny T, Schieck AFG, Mottram C, Strothmann A (2004) Arthur: a collaborative augmented environment for architectural design and urban planning. J Virtual Real Broadcast 1(1):1–10

    Google Scholar 

  97. Yuan ML, Ong SK, Nee AYC (2004) The virtual interaction panel: an easy control tool in augmented reality systems. Comput Anim Virtual Worlds 15(3–4):425–432

    Google Scholar 

  98. Shen Y, Ong SK, Nee AYC (2008) Collaborative design in 3D space. In: ACM SIGGRAPH International Conference on virtual-reality continuum and its applications in industry, ACM, p 29

  99. Maher ML, Mi JK (2005) Do tangible user interfaces impact spatial cognition in collaborative design?. In: International Conference on cooperative design, visualization, and engineering (vol 3675), Springer, pp 30–41

  100. Kim MJ, Maher ML (2005) Comparison of designers using a tangible user interface and a graphical user interface and the impact on spatial cognition. In: International Workshop on human behaviour in designing, pp 81–94

  101. Lee W, Park J (2006) Augmented foam: touchable and graspable augmented reality for product design simulation. Bull Jpn Soc Sci Des 52(6):17–26

    Google Scholar 

  102. Sidharta R, Oliver J, Sannier A (2006) Augmented reality tangible interface for distributed design review. In: computer graphics, imaging and visualisation, 2006 International Conference on IEEE, pp 464–470

  103. Sakong K, Nam TJ (2006) Supporting telepresence by visual and physical cues in distributed 3D collaborative design environments. In: CHI’06 extended abstracts on human factors in computing systems, ACM, pp 1283–1288

  104. Wang X, Chen R (2009) An experimental study on collaborative effectiveness of augmented reality potentials in urban design. CoDesign 5(4):229–244

    Google Scholar 

  105. Wang X, Dunston PS (2006) Potential of augmented reality as an assistant viewer for computer-aided drawing. J Comput Civil Eng 20(6):437–441

    Google Scholar 

  106. Wang X, Dunston PS (2008) User perspectives on mixed reality tabletop visualization for face-to-face collaborative design review. Autom Constr 17(4):399–412

    Google Scholar 

  107. Wang X (2007) Exploring an innovative collaborative design space through mixed reality boundaries. In: Computer supported cooperative work in design, 2007. CSCWD 2007. 11th International Conference on IEEE, pp 264–269

  108. Seichter H (2007) Augmented reality and tangible interfaces in collaborative urban design. In: computer-aided architectural design futures (CAADFutures) 2007, Springer, Dordrecht, pp 3–16

  109. Santos P, Stork A, Gierlinger T, Pagani A, Paloc C, Barandarian I, Jiménez JM (2007) Improve: an innovative application for collaborative mobile mixed reality design review. Int J Interactive Des Manuf 1(2):115–126

    Google Scholar 

  110. Santos P, Stork A, Gierlinger T, Pagani A, Araújo B, Jota R, Conti G (2007) Improve: collaborative design review in mobile mixed reality. In: International Conference on Virtual Reality. Springer, Berlin, pp 543–553

  111. Chastine J, Nagel K, Zhu Y, Hudachek-Buswell M (2008) Studies on the effectiveness of virtual pointers in collaborative augmented reality. In: 3D User Interfaces, 2008. 3DUI 2008. IEEE Symposium on IEEE, pp 117–124

  112. Lee JY, Rhee GW, Park H (2009) AR/RP-based tangible interactions for collaborative design evaluation of digital products. Int J Adv Manuf Technol 45(7–8):649–665

    Google Scholar 

  113. Hammad A, Wang H, Mudur SP (2009) Distributed augmented reality for visualizing collaborative construction tasks. J Comput Civil Eng 23(6):418–427

    Google Scholar 

  114. Dolinsky M (2010) Interactive augmented reality system for product design review. IS&T/SPIE Electron Imaging 7525:75250H–75250H

    Google Scholar 

  115. Jota R, Araújo BRD, Bruno LC, Pereira JM, Jorge JA (2010) Immiview: a multi-user solution for design review in real-time. J Real Time Image Process 5(2):91–107

    Google Scholar 

  116. Januszka M, Moczulski W (2010) Augmented reality for machinery systems design and development. New world situation: new directions in concurrent engineering. Springer, London

    Google Scholar 

  117. Januszka M, Moczulski W (2007) Machinery designing aided by augmented reality technology. Comput Assist Mech Eng Sci 14(14):621–630

    Google Scholar 

  118. Januszka M, Moczulski W (2006) Collaborative augmented reality in CAD design. Mach Dyn Probl 30(3):124–131

    Google Scholar 

  119. Moczulski W, Panfil W, Januszka M, Mikulski G (2007) Applications of augmented reality in machinery design, maintenance and diagnostics. Recent Advances in Mechatronics. Springer, Berlin, Heidelberg, pp 52–56

    Google Scholar 

  120. Thomas BH, Itzstein GSV, Vernik R, Porter S, Marner MR, Smith RT et al. (2011) Spatial augmented reality support for design of complex physical environments. In: IEEE International Conference on pervasive computing and communications workshops, IEEE, pp 588–593

  121. Poppe E, Brown R, Johnson D, Recker J (2012) Preliminary evaluation of an augmented reality collaborative process modelling system. In: International Conference on cyberworlds. IEEE Computer Society, pp 77–84

  122. Poppe E, Brown R, Johnson D, Recker J (2011) A prototype augmented reality collaborative process modelling tool. In: Demo Track of the Nineth Conference on Business Process Management 2011, Clermont-Ferrand, France, August. DBLP

  123. Ko CH, Chang TC (2012) Evaluation and student perception of augmented reality-based design collaboration[J]. Management 6(6):6

    Google Scholar 

  124. Gauglitz S, Nuernberger B, Turk M (2014) World-stabilized annotations and virtual scene navigation for remote collaboration. In: Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, pp 449–459

  125. Gauglitz S, Lee C, Turk M (2012) Integrating the physical environment into mobile remote collaboration. In: International Conference on human-computer interaction with mobile devices and services. ACM, pp 241–250

  126. Sodhi RS, Jones BR, Forsyth D, Maciocci G, Maciocci G (2013) BeThere: 3D mobile collaboration with spatial input. In: Sigchi Conference on human factors in computing systems. ACM, pp 179–188

  127. Gurevich P, Lanir J, Cohen B (2015) Design and implementation of teleadvisor: a projection-based augmented reality system for remote collaboration. Comput Support Coop Work 24(6):527–562

    Google Scholar 

  128. Gurevich P, Lanir J, Cohen B, Ran S (2012) TeleAdvisor: a versatile augmented reality tool for remote assistance. In: Sigchi Conference on human factors in computing systems. ACM, pp 619–622

  129. Gül LF, Halıcı M, Uzun C, Esengün M (2016) Understanding the impact of mobile augmented reality on co-design cognition and co-modelling. In: International Conference on cooperative design, visualization and engineering. Springer International Publishing, pp 362–370

  130. Gül LF (2017) Studying gesture-based interaction on a mobile augmented reality application for co-design activity. J Multimodal User Interfaces 12(2):109–124

    Google Scholar 

  131. Grandi JG (2017) Design of collaborative 3D user interfaces for virtual and augmented reality. In: virtual reality (VR), 2017 IEEE. IEEE, pp 419–420

  132. Grasset R, Lamb P, Billinghurst M (2005) Evaluation of mixed-space collaboration. In: Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE Computer Society, pp 90–99

  133. Ohshima T, Satoh K, Yamamoto H, Tamura H (1998) AR2 Hockey: a case study of collaborative augmented reality. In: Virtual reality annual international symposium, 1998. Proceedings., IEEE 1998. IEEE, pp 268–275

  134. Wang Y, Zhang S, Yang S, He W, Bai X, Zeng Y (2017) A LINE-MOD-based markerless tracking approachfor AR applications. Int J Adv Manuf Technol 89(5–8):1699–1707

    Google Scholar 

  135. Agrawala M, Beers AC, McDowall I, Fröhlich B, Bolas M, Hanrahan P (1997) The two-user Responsive Workbench: support for collaboration through individual views of a shared space. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., pp 327–332

  136. Yu D, Jin JS, Luo S, Lai W, Huang Q (2009) A useful visualization technique: a literature review for augmented reality and its application, limitation and future direction. Visual information communication. Springer, Boston, MA, pp 311–337

    Google Scholar 

  137. Papagiannakis G, Singh G, Magnenat-Thalmann N (2008) A survey of mobile and wireless technologies for augmented reality systems. Comput Anim Virtual Worlds 19(1):3–22

    Google Scholar 

  138. Gutwin C, Greenberg S (1998) Effects of awareness support on groupware usability. In: Sigchi Conference on Human Factors in Computing Systems (vol 98, pp 511–518). ACM Press/Addison-Wesley Publishing Co

  139. Szalavári Z, Gervautz M (2010) The personal interaction panel—a two-handed interface for augmented reality. Comput Graph Forum 16(3):C335–C346

    Google Scholar 

  140. Budhiraja R, Lee GA, Billinghurst M (2013) Interaction techniques for HMD-HHD hybrid AR systems. IEEE International Symposium on mixed and augmented reality (vol 8783, pp 243–244). IEEE

  141. Billinghurst M (2017) The coming age of empathic computing. http://www.slideshare.net/marknb00/the-coming-age-of-empathic-computing

  142. Hancock JT, Gee K, Ciaccio K, Lin MH (2008) I’m sad you’re sad: emotional contagion in CMC. In: ACM Conference on computer supported cooperative work. ACM, pp 295–298

  143. Barsade SG (2002) The ripple effect: emotional contagion and its influence on group behavior. Adm Sci Q 47(4):644–675

    Google Scholar 

  144. Cheshin A, Rafaeli A, Bos N (2011) Anger and happiness in virtual teams: emotional influences of text and behavior on others’ affect in the absence of non-verbal cues. Organ Behav Hum Decis Process 116(1):2–16

    Google Scholar 

  145. Höllerer T, Feiner S, Terauchi T, Rashid G, Hallaway D (1999) Exploring mars: developing indoor and outdoor user interfaces to a mobile augmented reality system. Comput Graph 23(6):779–785

    Google Scholar 

  146. Billinghurst M, Thomas BH (2011) Mobile collaborative augmented reality. In: recent trends of mobile collaborative augmented reality systems. Springer, New York, pp 1–19

  147. Jeon S, Hwang J, Kim GJ, Billinghurst M (2010) Interaction with large ubiquitous displays using camera-equipped mobile phones. Pers Ubiquitous Comput 14(2):83–94

    Google Scholar 

  148. Gül LF, Halici SM (2016) Collaborative design with mobile augmented reality. In: 34th eCAADe Conference Proceedings. Oulu, Finland, pp 493–500

  149. Billinghurst M, Grasset R, Looser J (2005) Designing augmented reality interfaces. ACM Siggraph Comput Graph 39(1):17–22

    Google Scholar 

  150. Ishii H, Lakatos D, Bonanni L, Labrune JB (2012) Radical atoms: beyond tangible bits, toward transformable materials. Interactions 19(1):38–51

    Google Scholar 

  151. Purdy TG, Choi YM (2014) Enhancing augmented reality for use in product design. In: CHI’14 Extended Abstracts on human factors in computing systems. ACM, pp 1303–1308

  152. Wang X (2008) Exploring augmented reality benefits in collaborative conceptualization. In: Computer supported cooperative work in design, 2008. CSCWD 2008. 12th International Conference on IEEE, pp 699–704

  153. Xing NL (2013) Augmented reality 3D design space. Doctoral dissertation

  154. Ng LX, Ong SK, Nee AYC (2015) Conceptual design using functional 3D models in augmented reality. Int J Interact Des Manuf 9(2):115–133

    Google Scholar 

  155. Safin S, Delfosse V, Leclercq P (2010) Mixed-reality prototypes to support early creative design. The Engineering of Mixed Reality Systems. Springer, London, pp 419–445

    Google Scholar 

  156. Fuge M, Yumer ME, Orbay G, Kara LB (2012) Conceptual design and modification of freeform surfaces using dual shape representations in augmented reality environments. Comput Aided Des 44(10):1020–1032

    Google Scholar 

  157. Turk M (2014) Multimodal interaction: a review. Pattern Recogn Lett 36:189–195

    Google Scholar 

  158. Deng S, Jiang N, Chang J, Guo S, Zhang JJ (2017) Understanding the impact of multimodal interaction using gaze informed mid-air gesture control in 3D virtual objects manipulation. Int J Hum Comput Stud 105:68–80

    Google Scholar 

  159. Quek F, McNeill D, Bryll R, Duncan S, Ma XF, Kirbas C, Ansari R (2002) Multimodal human discourse: gesture and speech. ACM Trans Comput Hum Interact 9(3):171–193

    Google Scholar 

  160. Shim J, Yang Y, Kang N, Seo J, Han TD (2016) Gesture-based interactive augmented reality content authoring system using HMD. Virtual Real 20(1):57–69

    Google Scholar 

  161. Henry P, Krainin M, Herbst E, Ren X, Fox D (2010) RGB-D mapping: using depth cameras for dense 3D modeling of indoor environments. In: In the 12th International Symposium on Experimental Robotics (ISER)

  162. Ismail AW, Billinghurst M, Sunar MS (2015) Vision-based technique and issues for multimodal interaction in augmented reality. In: Proceedings of the 8th International Symposium on Visual Information Communication and Interaction. ACM, pp 75–82

  163. Lee JY, Park HM, Lee SH, Shin SH, Kim TE, Choi JS (2014) Design and implementation of an augmented reality system using gaze interaction. Multimed Tools Appl 68(2):265–280

    Google Scholar 

  164. Krafka K, Khosla A, Kellnhofer P, Kannan H, Bhandarkar S, Matusik W, Torralba A (2016) Eye tracking for everyone. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2176–2184

  165. Chatterjee I, Xiao R, Harrison C (2015) Gaze + gesture: expressive, precise and targeted free-space interactions. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ACM, pp 131–138

  166. Song J, Cho S, Baek SY, Lee K, Bang H (2014) GaFinC: gaze and Finger Control interface for 3D model manipulation in CAD application. Comput Aided Des 46:239–245

    Google Scholar 

  167. Wu D, Rosen DW, Schaefer D (2014) Cloud-based design and manufacturing: status and promise. Cloud-Based Design and Manufacturing (CBDM). Springer, Cham, pp 1–24

    Google Scholar 

  168. Wang X, Dunston PS (2013) Tangible mixed reality for remote design review: a study understanding user perception and acceptance. Vis Eng 1(1):1–15

    Google Scholar 

  169. Huang JM, Ong SK, Nee AYC (2016) Visualization and interaction of finite element analysis in augmented reality. Comput Aided Des 84:1–14

    Google Scholar 

  170. Huang JM, Ong SK, Nee AYC (2015) Real-time finite element structural analysis in augmented reality. Adv Eng Softw 87(2):43–56

    Google Scholar 

  171. Burdea GC (1999) Invited review: the synergy between virtual reality and robotics. IEEE Trans Robot Autom 15(3):400–410

    Google Scholar 

  172. Seth A, Vance JM, Oliver JH (2011) Virtual reality for assembly methods prototyping: a review. Virtual Real 15(1):5–20

    Google Scholar 

  173. Bordegoni M, Ferrise F (2013) Designing interaction with consumer products in a multisensory virtual reality environment. Virtual Phys Prototyp 8(1):51–64

    Google Scholar 

  174. Wang P, Zhang S, Bai X, Billinghurst M, He W, Sun M, Chen Y, Lv H, Ji H (2019) 2.5DHANDS: a gesture-based MR remote collaborative platform. Int J Adv Manuf Technol 2019:1–15

    Google Scholar 

  175. Wang P, Zhang S, Bai X, Billinghurst M, He W, Zhang L, Du J, Wang S (2018) Do you know what i mean? an mr-based collaborative platform. In: IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2018

Download references

Acknowledgements

This research was sponsored by the civil aircraft special project (MJZ-2017-G73). Besides, we would like to appreciate the anonymous reviewers for their constructive suggestions for enhancing this paper. Specifically, Mark Billinghurst has checked the English of an early version, and his constructive comments are gratefully acknowledged which have helped the author to improve the paper. Moreover, we thank Dechuan Han and Mengmeng Sun for filtering and downloading papers. We also thank other schoolfellows, including Jinping Chen, Qian Cheng, Li Zhang, Yongxing Chen, Zhuang Chang, Yue Wang, and Xiaokun Zhang for their invaluable discussion, criticism, and advice for further improving this paper.

Author information

Authors and Affiliations

  1. Key Laboratory of Contemporary Designing and Integrated Manufacturing Technology, Ministry of Education, Cyber-Physical Interaction Lab, Northwestern Polytechnical University, Xi’an, 710072, China

    Peng Wang, Shusheng Zhang, Mark Billinghurst, Xiaoliang Bai, Weiping He, Shuxia Wang, Mengmeng Sun & Xu Zhang

  2. Empathic Computing Lab, University of South Australia, Mawson Lakes, Australia

    Mark Billinghurst

Authors
  1. Peng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shusheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark Billinghurst
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoliang Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weiping He
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shuxia Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mengmeng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shusheng Zhang or Xiaoliang Bai.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, P., Zhang, S., Billinghurst, M. et al. A comprehensive survey of AR/MR-based co-design in manufacturing. Engineering with Computers 36, 1715–1738 (2020). https://doi.org/10.1007/s00366-019-00792-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00366-019-00792-3

Keywords

Search

Navigation