skip to main content
10.1145/3369457.3370919acmotherconferencesArticle/Chapter ViewAbstractPublication PagesozchiConference Proceedingsconference-collections
extended-abstract

Handling Work Complexity with AR/Deep Learning

Published: 10 January 2020 Publication History

Abstract

Complexity is a fundamental part of product design and manufacturing today, owing to increased demands for customization and advances in digital design techniques. Assembling and repairing such an enormous variety of components means that workers are cognitively challenged, take longer to search for the relevant information and are prone to making mistakes. Although in recent years deep learning approaches to object recognition have seen rapid advances, the combined potential of deep learning and augmented reality in the industrial domain remains relatively under explored. In this paper we introduce AR-ProMO, a combined hardware/software solution that provides a generalizable assistance system for identifying mistakes during product assembly and repair.

References

[1]
Sebastian Büttner, Markus Funk, Oliver Sand, and Carsten Röcker. 2016. Using Head-Mounted Displays and In-Situ Projection for Assistive Systems: A Comparison. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments. ACM, 44.
[2]
Sebastian Büttner, Henrik Mucha, Markus Funk, Thomas Kosch, Mario Aehnelt, Sebastian Robert, and Carsten Röcker. 2017. The Design Space of Augmented and Virtual Reality Applications for Assistive Environments in Manufacturing: A Visual Approach. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments. ACM, 433--440.
[3]
Sebastian Büttner, Oliver Sand, and Carsten Röcker. 2015. Extending the Design Space in Industrial Manufacturing Through Mobile Projection. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. ACM, 1130--1133.
[4]
Hitesh Dhiman, Sascha Martinez, Volker Paelke, and Carsten Röcker. 2018. Head-Mounted Displays in Industrial AR-Applications: Ready for Prime Time?. In International Conference on HCI in Business, Government, and Organizations. Springer, 67--78.
[5]
Hitesh Dhiman and Carsten Röcker. 2019. Worker Assistance in Smart Production Environments Using Pervasive Technologies. In 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, 95--100.
[6]
Pierre Fite-Georgel. 2011. Is there a reality in industrial augmented reality?. In 2011 10th ieee international symposium on mixed and augmented reality. IEEE, 201--210.
[7]
Markus Funk, Andreas Bächler, Liane Bächler, Oliver Korn, Christoph Krieger, Thomas Heidenreich, and Albrecht Schmidt. 2015. Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments. ACM, 1.
[8]
Markus Funk, Sven Mayer, and Albrecht Schmidt. 2015. Using in-situ projection to support cognitively impaired workers at the workplace. In Proceedings of the 17th international ACM SIGACCESS conference on Computers & accessibility. ACM, 185--192.
[9]
Mika Hakkarainen, Charles Woodward, and Mark Billinghurst. 2008. Augmented assembly using a mobile phone. In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE Computer Society, 167--168.
[10]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[11]
Mario Heinz, Sebastian Büttner, and Carsten Röcker. 2019. Exploring training modes for industrial augmented reality learning. In Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments. ACM, 398--401.
[12]
Mario Heinz, Hitesh Dhiman, and Carsten Röcker. 2018. A Multi-device Assistive System for Industrial Maintenance Operations. In International Cross-Domain Conference for Machine Learning and Knowledge Extraction. Springer, 239--247.
[13]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.
[14]
Farhad Malik. 2019. What Are Hidden Layers? Important Topic To Understand When Working On Machine Learning Models. https://medium.com/fintechexplained/what-are-hidden-layers-4f54f7328263. [Online; accessed 6-August-2019].
[15]
MathWorks. 2019. Object Detection. https://www.mathworks.com/discovery/object-detection.html. [Online; accessed 6-August-2019].
[16]
Microsoft. 2019. HoloLens 2 Mixed Reality is Ready for Business. https://www.microsoft.com/en-us/hololens/. [Online; accessed 6-August-2019].
[17]
Michael A Nielsen. 2015. Neural networks and deep learning. Vol. 25. Determination press San Francisco, CA, USA:.
[18]
Volker Paelke, Carsten Röcker, Nils Koch, Holger Flatt, and Sebastian Büttner. 2015. User interfaces for cyber-physical systems. at-Automatisierungstechnik 63, 10 (2015), 833--843.
[19]
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 779--788.
[20]
Joseph Redmon and Ali Farhadi. 2017. YOLO9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7263--7271.
[21]
Joseph Redmon and Ali Farhadi. 2018. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018).
[22]
Sebastian Robert, Sebastian Büttner, Carsten Röcker, and Andreas Holzinger. 2016. Reasoning Under Uncertainty: Towards Collaborative Interactive Machine Learning. In Machine Learning for Health Informatics. Springer International Publishing, 357--376.
[23]
Bjorn Schwerdtfeger and Gudrun Klinker. 2008. Supporting order picking with augmented reality. In 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, 91--94.
[24]
Björn Schwerdtfeger, Daniel Pustka, Andreas Hofhauser, and Gudrun Klinker. 2008. Using laser projectors for augmented reality. In Proceedings of the 2008 ACM symposium on Virtual reality software and technology. Citeseer, 134--137.
[25]
Bjorn Schwerdtfeger, Rupert Reif, Willibald A Gunthner, Gudrun Klinker, Daniel Hamacher, Lutz Schega, Irina Bockelmann, Fabian Doil, and Johannes Tumler. 2009. Pick-by-Vision: A first stress test. In 2009 8th IEEE International Symposium on Mixed and Augmented Reality. IEEE, 115--124.
[26]
Richard Szeliski. 2010. Computer vision: algorithms and applications. Springer Science & Business Media.
[27]
Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative effectiveness of augmented reality in object assembly. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 73--80.
[28]
Paul Viola, Michael Jones, et al. 2001. Robust real-time object detection. International journal of computer vision 4, 34--47 (2001), 4.
[29]
Stefan Werrlich, Austino Daniel, Alexandra Ginger, Phuc-Anh Nguyen, and Gunther Notni. 2018. Comparing HMD-based and Paper-based Training. In 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 134--142.

Cited By

View all
  • (2024)Implementation of Augmented Reality in Smart Engineering Manufacturing: Literature ReviewMobile Networks and Applications10.1007/s11036-023-02121-x29:1(119-132)Online publication date: 1-Feb-2024
  • (2024)Concept of Mixed Reality Application Design for Technical Solutions8th EAI International Conference on Management of Manufacturing Systems10.1007/978-3-031-53161-3_10(137-149)Online publication date: 24-Mar-2024
  • (2022)Advancements in Vocational Training Through Mobile Assistance SystemsHuman-Technology Interaction10.1007/978-3-030-99235-4_6(151-172)Online publication date: 14-Dec-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
OzCHI '19: Proceedings of the 31st Australian Conference on Human-Computer-Interaction
December 2019
631 pages
ISBN:9781450376969
DOI:10.1145/3369457
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

In-Cooperation

  • HFESA: Human Factors and Ergonomics Society of Australia Inc.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 January 2020

Check for updates

Author Tags

  1. Augmented Reality
  2. Deep Learning

Qualifiers

  • Extended-abstract
  • Research
  • Refereed limited

Conference

OZCHI'19
OZCHI'19: 31ST AUSTRALIAN CONFERENCE ON HUMAN-COMPUTER-INTERACTION
December 2 - 5, 2019
WA, Fremantle, Australia

Acceptance Rates

Overall Acceptance Rate 362 of 729 submissions, 50%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)13
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Implementation of Augmented Reality in Smart Engineering Manufacturing: Literature ReviewMobile Networks and Applications10.1007/s11036-023-02121-x29:1(119-132)Online publication date: 1-Feb-2024
  • (2024)Concept of Mixed Reality Application Design for Technical Solutions8th EAI International Conference on Management of Manufacturing Systems10.1007/978-3-031-53161-3_10(137-149)Online publication date: 24-Mar-2024
  • (2022)Advancements in Vocational Training Through Mobile Assistance SystemsHuman-Technology Interaction10.1007/978-3-030-99235-4_6(151-172)Online publication date: 14-Dec-2022
  • (2020)Middleware for providing activity-driven assistance in cyber-physical production systems☆Journal of Computational Design and Engineering10.1093/jcde/qwaa0888:1(428-451)Online publication date: 26-Dec-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media