research-article

AVIKOM: towards a mobile audiovisual cognitive assistance system for modern manufacturing and logistics

Authors:

Alexander Neumann,

Benjamin Strenge,

Janne C. Uhlich,

Katharina D. Schlicher,

Günter W. Maier,

Lars Schalkwijk,

Joachim Waßmuth,

Thomas SchackAuthors Info & Claims

PETRA '20: Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments

Article No.: 1, Pages 1 - 8

https://doi.org/10.1145/3389189.3389191

Published: 30 June 2020 Publication History

Abstract

This paper introduces the novel Augmented Reality (AR) assistance system AVIKOM, a joint endeavour of three research groups together with four small and medium-sized enterprises (SME) as well as network partners and a diaconal institution. In particular, we investigate how AR-enabled assistance systems can be tailored to individual requirements of workers with diverse cognitive and physical capabilities for today's real-world industrial applications in the areas of (manual) assembly, logistics and operation of industrial machinery. We combine best practices from the domains of artificial intelligence, machine learning, user experience engineering, ethics research, and cognitive science with state-of-the-art insights for multi-modal system development to create a cognitive action assistance system that recognizes and adapts to individual users in various situational contexts, such as picking and training.

Proven work and organizational psychology methods and worth-related evaluations will accompany the system introduction into working environments. Using user- and worth-centred system design and change management strategies (e.g. information and participation) right from the beginning of such a technological development facilitates proper involvement of future users in the development process. This can lead to better congruence of technology features with workers' requirements and positively shape future users' attitudes towards the system.

Supplementary Material

MP4 File (a1-neumann.mp4)

Download
78.32 MB

References

[1]

Aashish Agarwal and Torsten Zesch. 2019. German End-to-end Speech Recognition based on DeepSpeech. In Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019): Long Papers. German Society for Computational Linguistics & Language Technology, Erlangen, Germany, 111--119.

[2]

Dominik Bentler, Lisa Mlekus, Agnieszka Paruzel, Michael Bansmann, Marc Foullois, Sascha Jenderny, and Lars Woeste. 2019. Einführung von Augmented Reality in der Produktentstehung. Technische Realisierung und Change-Management als Erfolgsfaktor für den Veränderungsprozess.

[3]

Bielefeld University of Applied Sciences. 2018. Besserer Lärmschutz und Steigerung der Produktivität. https://www.fh-bielefeld.de/presse/pressemitteilungen/besserer-laermschutz-und-steigerung-der-produktivitaet

[4]

Jonas Blattgerste, Patrick Renner, Benjamin Strenge, and Thies Pfeiffer. 2018. In-Situ Instructions Exceed Side-by-Side Instructions in Augmented Reality Assisted Assembly. In Proceedings of the 11th ACM International Conference on PErvasive Technologies Related to Assistive. ACM Press, New York, New York, USA, 133--140.

Digital Library

[5]

Tom Bocklisch, Joey Faulkner, Nick Pawlowski, and Alan Nichol. 2017. Rasa: Open Source Language Understanding and Dialogue Management. http://arxiv.org/pdf/1712.05181v2

[6]

British Museum. 2015. A gift for Athena: Digital self-led session. https://www.britishmuseum.org/learn/schools/ages-7-11/ancient-greece/digital-self-led-session-gift-athena

[7]

Gilbert Cockton. 2008. Designing worth---connecting preferred means to desired ends. interactions 15, 4 (2008), 54.

Digital Library

[8]

Gilbert Cockton, Sari Kujala, Piia Nurkka, and Taneli Hölttä. 2009. Supporting Worth Mapping with Sentence Completion. In Human-Computer Interaction - INTERACT 2009, Tom Gross, Jan Gulliksen, Paula Kotzé, Lars Oestreicher, Philippe Palanque, Raquel Oliveira Prates, and Marco Winckler (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 566--581.

Digital Library

[9]

Fred D. Davis. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13, 3 (1989), 319.

Digital Library

[10]

Digi-Capital. 2019. For AR/VR 2.0 to live, AR/VR 1.0 must die | Digi Capital. https://www.digi-capital.com/news/2019/01/for-ar-vr-2-0-to-live-ar-vr-1-0-must-die/

[11]

Alistair Douglas Norman Edwards. 2011. Auditory Display in Assistive Technology. In The Sonification Handbook, Thomas Hermann, Andy Hunt, and John Neuhoff (Eds.). Logos Publishing House, Berlin, Germany, 431--453.

[12]

J. Edworthy, S. Loxley, and I. Dennis. 1991. Improving auditory warning design: Relationship between warning sound parameters and perceived urgency. Human Factors 33, 2 (1991), 205--231.

[13]

Kai Essig, Benjamin Strenge, and Thomas Schack. 2016. ADAMAAS - Towards Smart Glasses for Mobile and Personalized Action Assistance. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments. ACM Press, New York, New York, USA, 1--4.

Digital Library

[14]

Lars Fromme, Joachim Waßmuth, and Dominik Wehmeier. 2015. Simulation of Acoustical Transfer Paths for Active Noise Control. In Proceedings of the Comsol Conference. Grenoble, France.

[15]

Jesse Glover. 2018. Unity 2018 Augmented Reality Projects: Build Four Immersive and Fun AR Applications Using ARKit, ARCore, and Vuforia. Packt Publishing Ltd, Birmingham.

[16]

Google. 2020. Glass - Discover Glass Enterprise Edition. https://www.google.com/glass

[17]

Eric Griffith. 2019. The Biggest Tech Product Flops of the 2010s. https://www.pcmag.com/feature/371549/the-biggest-tech-product-flops-of-the-2010s

[18]

gRPC. 2020. gRPC - A High-Performance Open-Source Universal RPC Framework. https://grpc.io/

[19]

Anne Guillaume. 2011. Intelligent auditory alarms. In The Sonification Handbook, Thomas Hermann, Andy Hunt, and John Neuhoff (Eds.). Logos Publishing House, Berlin, Germany, 493--508.

[20]

Alan B. Johnston and Daniel C. Burnett. 2012. WebRTC: APIs and RTCWEB protocols of the HTML5 real-time web (1. ed., paperback ed. ed.). Digital Codex LLC, St. Louis, Mo.

[21]

Dennis Kaupmann, Tobias Lehmann, and Joachim Waßmuth. 2015. Methodische Entwicklung kostengünstiger Störschallkompensationssysteme. In VDI/VDE Mechatronik.

[22]

John Koetsier. 2019. 10-Gram Smartglasses Kit: Bosch Smartglasses System Is Tiny, Light And Privacy-Safe. https://www.forbes.com/sites/johnkoetsier/2019/12/10/10-gram-smartglasses-kit-bosch-smartglasses-system-is-tiny-light-and-privacy-safe/

[23]

Leap Motion. 2018. Project North Star is Now Open Source. http://blog.leapmotion.com/north-star-open-source/

[24]

A. Linden and Fenn J. 2003. Understanding Gartner's Hype Cycles. Strategic Analysis Report R-20-1971 (2003), 12.

[25]

Paul Luff, Christian Heath, Hideaki Kuzuoka, Jon Hindmarsh, Keiichi Yamazaki, and Shinya Oyama. 2003. Fractured Ecologies: Creating Environments for Collaboration. Human-Computer Interaction 18, 1--2 (2003), 51--84.

Digital Library

[26]

M. McGee-Lennon, M. Wolters, and T. McBryan. 2007. Audio Reminders in the Home Environment. In Proceedings of International Conference on Auditory Displays (ICAD). 437--444.

[27]

Microsoft. 2019. Spectator View. https://github.com/microsoft/MixedReality-SpectatorView

[28]

Lisa Mlekus and Günter W. Maier. 2017. Digitalisierung der Arbeitswelt: Ergebnisse einer Unternehmensumfrage zum Stand der Transformation. In Wissenschaftsforum intelligente technische Systeme (WInTeSys). Vol. 369. Heinz Nixdorf Institut, Paderborn University, Paderborn, Germany, 141--153.

[29]

Lisa Mlekus and Günter W. Maier. (submitted). Not everyone benefits from technological advancements: Associations with competency requirements and well-being in two occupations. ((submitted)).

[30]

Alexander Neumann and Thomas Hermann. 2013. Interactive Sonification of Collaborative AR-based Planning Tasks for Enhancing Joint Attention. In Proceedings of the 19th International Conference on Auditory Displays, P. Strumiłło, M. Bujacz, and M. Popielata (Eds.). 49--55.

[31]

Sharon K. Parker, Frederick P. Morgeson, and Gary Johns. 2017. One hundred years of work design research: Looking back and looking forward. The Journal of applied psychology 102, 3 (2017), 403--420.

[32]

Agnieszka Paruzel, Dominik Bentler, Katharina D. Schlicher, Wolfgang Nettelstroth, and Günter W. Maier. 2020. Employees First, Technology Second. Zeitschrift für Arbeits- und Organisationspsychologie A&O 64, 1 (2020), 46--57.

[33]

Felix Richter. 2018. The (Yet Untapped) Potential of Augmented Reality. https://www.statista.com/chart/15310/

[34]

L. Rosenberg. [n.d.]. The Use of Virtual Fixtures as Perceptual Overlays to Enhance Operator Performance in Remote Environments. http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA292450&Location=U2&doc=GetTRDoc.pdf

[35]

Valerio Santangelo and Charles Spence. 2007. Multisensory cues capture spatial attention regardless of perceptual load. Journal of experimental psychology. Human perception and performance 33, 6 (2007), 1311--1321.

[36]

Thomas Schack. 2012. Measuring mental representations. In Measurement in sport and exercise psychology, G. Tenenbaum, Eklund R. C., and Kamata A (Eds.). Champaign, IL: Human Kinetics, 203--214.

[37]

Katharina D. Schlicher, Dominik Bentler, Agnieszka Paruzel, and Günter W. Maier. (in press). Arbeit4.0@Hettich: Berufliche Handlungskompetenz in der Umsetzung des Auftragsdurchlaufs von morgen. In Gestaltung digitalisierter Arbeitswelten. Handlungsfelder und Praxisbeispiele zur Umsetzung digitalisierter Arbeit, R. Dumitrescu (Ed.).

[38]

Katharina D. Schlicher, Agnieszka Paruzel, Barbara Steinmann, and Günter W. Maier. 2017. Change Management für die Einführung digitaler Arbeitswelten. In Handbuch Gestaltung digitaler und vernetzter Arbeitswelten, Günter W. Maier, Gregor Engels, and Eckhard Steffen (Eds.). Springer Reference Psychologie, Vol. 21. Springer, Berlin and Heidelberg, 1--36.

[39]

Marc Schröder and Jürgen Trouvain. 2003. The German Text-to-Speech Synthesis System MARY: A Tool for Research, Development and Teaching. International Journal of Speech Technology 6, 4 (2003), 365--377.

[40]

K. L. Shapiro, J. E. Raymond, and K. M. Arnell. 1997. The attentional blink. Trends in Cognitive Sciences 1, 8 (1997), 291--296.

[41]

Benjamin Strenge and Thomas Schack. 2019. AWOSE - A Process Model for Incorporating Ethical Analyses in Agile Systems Engineering. Science and engineering ethics (2019), 20.

[42]

Benjamin Strenge, Ludwig Vogel, and Thomas Schack. 2018. Individualized cognitive assistance by smart glasses for manual assembly processes in industry. In Technische Unterstützungssysteme, die die Menschen wirklich wollen, Robert Weidner and Athanasios Karafillidis (Eds.). Helmut-Schmidt-Universität, Hamburg, Deutschland, 399--407. https://pub.uni-bielefeld.de/record/2932587

[43]

Benjamin Strenge, Ludwig Vogel, and Thomas Schack. 2019. Computational assessment of long-term memory structures from SDA-M related to action sequences. PLOS ONE 14, 2 (2019), e0212414.

[44]

Paul Vickers. 2011. Sonification for Process Monitoring. In The Sonification Handbook, Thomas Hermann, Andy Hunt, and John Neuhoff (Eds.). Logos Publishing House, Berlin, Germany, 455--491.

[45]

Vuzix. 2020. Compare Vuzix Smart Glasses. https://www.vuzix.com/products/compare-vuzix-smart-glasses

[46]

Joachim Waßmuth. 2015. Simulationsgestützter Entwurf von Active Noise Control-Systemen für Kabinenanwendungen. In Proceedings of ASIM. 246--253.

[47]

S. Wojewoda and S. Hastie. 2015. Standish Group 2015 Chaos Report - Q&A with Jennifer Lynch [Web log post]. https://www.infoq.com/articles/standish-chaos-2015

[48]

Robert Woll, Thomas Damerau, Kevin Wrasse, and Rainer Stark. 2011. Augmented reality in a serious game for manual assembly processes. In 2011 IEEE International Symposium on Mixed and Augmented Reality - Arts, Media, and Humanities. IEEE Computer Society, Washington, DC, USA, 37--39.

Digital Library

[49]

Y. Zhang, T. Fernando, H. Xiao, and A. R. L. Travis. 2006. Evaluation of Auditory and Visual Feedback on Task Performance in a Virtual Assembly Environment. Presence 15, 6 (2006), 613--626.

Digital Library

Cited By

Tokkonen TTokkonen HKinnula AKinnula MKuutti K(2023)Dimensions of Influence in Trucking: Beyond Work CommunityProceedings of the 11th International Conference on Communities and Technologies10.1145/3593743.3593769(133-143)Online publication date: 29-May-2023
https://dl.acm.org/doi/10.1145/3593743.3593769
Schulte JKato-Beiderwieden AMaier G(2023)Welche Kompetenzen benötigen Mitarbeitende für den Einsatz von Augmented Reality in Logistik und Produktion?What competencies do employees need for the use of augmented reality in logistics and production?Gruppe. Interaktion. Organisation. Zeitschrift für Angewandte Organisationspsychologie (GIO)10.1007/s11612-023-00701-954:3(301-310)Online publication date: 15-Aug-2023
https://doi.org/10.1007/s11612-023-00701-9
Neumann AStrenge BSchalkwijk LSchulte JKato-Beiderwieden AEssig KMaier GWaßmuth JSchack T(2023)Kognitive Augmented-Reality-Assistenzsysteme in KMU – organisationale und technische Ansätze für eine individuelle ArbeitsunterstützungDigitalisierung der Arbeitswelt im Mittelstand 310.1007/978-3-662-67024-8_8(291-331)Online publication date: 4-Oct-2023
https://doi.org/10.1007/978-3-662-67024-8_8
Show More Cited By

Index Terms

AVIKOM: towards a mobile audiovisual cognitive assistance system for modern manufacturing and logistics

Recommendations

ADAMAAS: Towards Smart Glasses for Mobile and Personalized Action Assistance
PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

In this paper, we describe the assistive system ADAMAAS (Adaptive and Mobile Action Assistance) introducing a new advanced smartglasses technology. The aim of ADAMAAS is to move from stationary status diagnosis systems to a mobile and adaptive action ...
Comparative Study of HMD-based Virtual and Augmented Realities for Immersive Museums: User Acceptance, Medium, and Learning
Technologies like Head Mounted Display (HMD)-based Virtual Reality (VR) and Augmented Reality (AR) have made HMD-based immersive museums possible. To investigate the user acceptance, medium, and interaction experience for HMD-based immersive museums, an ...
Augmented Winter Ski with AR HMD
AH '16: Proceedings of the 7th Augmented Human International Conference 2016

At time of writing, several affordable Head-Mounted Displays (HMD) are going to be released to the mass market, most of them for Virtual Reality (VR with Oculus Rift, Samsung Gear...) but also for indoor Augmented Reality (AR) with Hololens. We have ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PETRA '20: Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments

June 2020

574 pages

ISBN:9781450377737

DOI:10.1145/3389189

Conference Chair:
Fillia Makedon
University of Texas at Arlington

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

NSF: National Science Foundation
CSE@UTA: Department of Computer Science and Engineering, The University of Texas at Arlington
NCRS: Demokritos National Center for Scientific Research

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Bundesministerium für Bildung und Forschung

Conference

PETRA '20

Sponsor:

NSF
CSE@UTA
NCRS

PETRA '20: The 13th PErvasive Technologies Related to Assistive Environments Conference

June 30 - July 3, 2020

Corfu, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
283
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)4

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tokkonen TTokkonen HKinnula AKinnula MKuutti K(2023)Dimensions of Influence in Trucking: Beyond Work CommunityProceedings of the 11th International Conference on Communities and Technologies10.1145/3593743.3593769(133-143)Online publication date: 29-May-2023
https://dl.acm.org/doi/10.1145/3593743.3593769
Schulte JKato-Beiderwieden AMaier G(2023)Welche Kompetenzen benötigen Mitarbeitende für den Einsatz von Augmented Reality in Logistik und Produktion?What competencies do employees need for the use of augmented reality in logistics and production?Gruppe. Interaktion. Organisation. Zeitschrift für Angewandte Organisationspsychologie (GIO)10.1007/s11612-023-00701-954:3(301-310)Online publication date: 15-Aug-2023
https://doi.org/10.1007/s11612-023-00701-9
Neumann AStrenge BSchalkwijk LSchulte JKato-Beiderwieden AEssig KMaier GWaßmuth JSchack T(2023)Kognitive Augmented-Reality-Assistenzsysteme in KMU – organisationale und technische Ansätze für eine individuelle ArbeitsunterstützungDigitalisierung der Arbeitswelt im Mittelstand 310.1007/978-3-662-67024-8_8(291-331)Online publication date: 4-Oct-2023
https://doi.org/10.1007/978-3-662-67024-8_8
Heinz-Jakobs MOestreich HWrede SRocker C(2022)User Expectations Regarding Design Dimensions of Adapative Assistance Systems2022 15th International Conference on Human System Interaction (HSI)10.1109/HSI55341.2022.9869509(1-7)Online publication date: 28-Jul-2022
https://doi.org/10.1109/HSI55341.2022.9869509
Neumann AStrenge BSchalkwijk LEssig KSchack T(2021)Facilitating Workers’ Task Proficiency with Subtle Decay of Contextual AR-Based Assistance Derived from Unconscious Memory StructuresInformation10.3390/info1201001712:1(17)Online publication date: 4-Jan-2021
https://doi.org/10.3390/info12010017
Dörr SSchmidt-Huber MMaier G(2021)Das LEaD-Kompetenzmodell – wirksam Führen im Kontext der digitalen TransformationThe LEaD competence model: Leading effectively in the context of digital transformationGruppe. Interaktion. Organisation. Zeitschrift für Angewandte Organisationspsychologie (GIO)10.1007/s11612-021-00582-wOnline publication date: 10-Jun-2021
https://doi.org/10.1007/s11612-021-00582-w
Santana RRossi GMéndez GRodríguez ACajas V(2021)Smart Glasses User Experience in STEM Students: A Systematic Mapping StudyTrends and Applications in Information Systems and Technologies10.1007/978-3-030-72657-7_44(455-467)Online publication date: 23-Apr-2021
https://doi.org/10.1007/978-3-030-72657-7_44

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten