research-article

Comparing Tactile, Auditory, and Visual Assembly Error-Feedback for Workers with Cognitive Impairments

Authors:

Romina Kettner,

Albrecht SchmidtAuthors Info & Claims

ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility

Pages 53 - 60

https://doi.org/10.1145/2982142.2982157

Published: 23 October 2016 Publication History

Abstract

More and more industrial manufacturing companies are outsourcing assembly tasks to sheltered work organizations where cognitively impaired workers are employed. To facilitate these assembly tasks assistive systems have been introduced to provide cognitive assistance. While previous work found that these assistive systems have a great impact on the workers' performance in giving assembly instructions, these systems are further capable of detecting errors and notifying the worker of an assembly error. However, the topic of how assembly errors are presented to cognitively impaired workers has not been analyzed scientifically. In this paper, we close this gap by comparing tactile, auditory, and visual error feedback in a user study with 16 cognitively impaired workers. The results reveal that visual error feedback leads to a significantly faster assembly time compared to tactile error feedback. Further, we discuss design implications for providing error feedback for workers with cognitive impairments.

References

[1]

United nations: United nations convention on the rights of persons with disabilities. http://www.un.org/disabilities/convention/conventionfull.shtml, 2008. {last accessed 02-May-2016}.

[2]

M. Akamatsu, I. S. MacKenzie, and T. Hasbroucq. A comparison of tactile, auditory, and visual feedback in a pointing task using a mouse-type device. Ergonomics, 38(4):816--827, 1995.

[3]

A. Bannat, F. Wallhoff, G. Rigoll, F. Friesdorf, H. Bubb, S. Stork, H. Müller, A. Schubö, M. Wiesbeck, and M. F. Zäh. Towards optimal worker assistance: a framework for adaptive selection and presentation of assembly instructions. In Proceedings of the 1st international workshop on cognition for technical systems, Cotesys, 2008.

[4]

H. Behrendt, M. Funk, and O. Korn. Ethical implications regarding assistive technology at workplaces. In Collective Agency and Cooperation in Natural and Artificial Systems, pages 109--130. Springer, 2015.

[5]

D. Bial, D. Kern, F. Alt, and A. Schmidt. Enhancing outdoor navigation systems through vibrotactile feedback. In CHI'11 Extended Abstracts on Human Factors in Computing Systems, pages 1273--1278. ACM, 2011.

Digital Library

[6]

S. Büttner, M. Funk, O. Sand, and C. Röcker. Using head-mounted displays and in-situ projection for assistive systems -- a comparison. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, New York, NY, USA, 2016. ACM.

Digital Library

[7]

S. Büttner, O. Sand, and C. Röcker. Extending the design space in industrial manufacturing through mobile projection. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pages 1130--1133. ACM, 2015.

Digital Library

[8]

A. Butz, M. Schneider, and M. Spassova. Searchlight--a lightweight search function for pervasive environments. In Pervasive Computing, pages 351--356. Springer, 2004.

[9]

A. J. Cuvo, P. K. Davis, M. F. O'Reilly, B. M. Mooney, and R. Crowley. Promoting stimulus control with textual prompts and performance feedback for persons with mild disabilities. Journal of applied behavior analysis, 25(2):477--489, 1992.

[10]

M. Funk, A. Bächler, L. Bächler, O. Korn, C. Krieger, T. Heidenreich, and A. Schmidt. Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments, page 1. ACM, 2015.

Digital Library

[11]

M. Funk, J. Heusler, E. Akcay, K. Weiland, and A. Schmidt. Haptic, auditory, or visual? towards optimal error feedback at manual assembly workplaces. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, New York, NY, USA, 2016. ACM.

Digital Library

[12]

M. Funk, T. Kosch, S. W. Greenwald, and A. Schmidt. A benchmark for interactive augmented reality instructions for assembly tasks. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia, pages 253--257. ACM, 2015.

Digital Library

[13]

M. Funk, S. Mayer, and A. Schmidt. Using in-situ projection to support cognitively impaired workers at the workplace. In Proceedings of the 17th international ACM SIGACCESS conference on Computers & accessibility, 2015.

Digital Library

[14]

O. Korn, M. Funk, S. Abele, T. Hörz, and A. Schmidt. Context-aware assistive systems at the workplace: analyzing the effects of projection and gamification. In Proceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments, page 38. ACM, 2014.

Digital Library

[15]

O. Korn, M. Funk, and A. Schmidt. Assistive systems for the workplace: Towards context-aware assistance. Assistive Technologies for Physical and Cognitive Disabilities, pages 121--133, 2015.

[16]

O. Korn, A. Schmidt, and T. Hörz. The potentials of in-situ-projection for augmented workplaces in production: a study with impaired persons. In CHI'13 Extended Abstracts on Human Factors in Computing Systems, pages 979--984. ACM, 2013.

Digital Library

[17]

M. Löchtefeld, S. Gehring, J. Schöning, and A. Krüger. Shelftorchlight: Augmenting a shelf using a camera projector unit. In Conference on Pervasive Computing, volume 10. Citeseer, 2010.

[18]

B. Petzold, M. F. Zaeh, B. Faerber, B. Deml, H. Egermeier, J. Schilp, and S. Clarke. A study on visual, auditory, and haptic feedback for assembly tasks. Presence: teleoperators and virtual environments, 13(1):16--21, 2004.

Digital Library

[19]

C. Pinhanez. The everywhere displays projector: A device to create ubiquitous graphical interfaces. In Ubicomp 2001: Ubiquitous Computing, pages 315--331. Springer, 2001.

Digital Library

[20]

M. Rauterberg and E. Styger. Positive effects of sound feedback during the operation of a plant simulator. In Human-Computer Interaction, pages 35--44. Springer, 1994.

Digital Library

[21]

P. Richard, G. Burdea, D. Gomez, and P. Coiffet. A comparison of haptic, visual and auditive force feedback for deformable virtual objects. In Proceedings of the Internation Conference on Automation Technology (ICAT), pages 49--62, 1994.

[22]

S. Rüther, T. Hermann, M. Mracek, S. Kopp, and J. Steil. An assistance system for guiding workers in central sterilization supply departments. In Proceedings of the 6th International Conference on PErvasive Technologies Related to Assistive Environments, page 3. ACM, 2013.

Digital Library

[23]

A. L. Sauer, A. Parks, and P. C. Heyn. Assistive technology effects on the employment outcomes for people with cognitive disabilities: a systematic review. Disability and Rehabilitation: Assistive Technology, 5(6):377--391, 2010.

[24]

A. Tang, C. Owen, F. Biocca, and W. Mou. Comparative effectiveness of augmented reality in object assembly. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 73--80. ACM, 2003.

Digital Library

[25]

P. Wellner. The digitaldesk calculator: tangible manipulation on a desk top display. In Proceedings of the 4th annual ACM symposium on User interface software and technology, pages 27--33. ACM, 1991.

Digital Library

Cited By

Dominiak JWysokińska ABentvelzen MRomanowski AWoźniak P(2025)A design space for reconfigurable interactive desktop environments (RIDEs)Behaviour & Information Technology10.1080/0144929X.2024.2447920(1-26)Online publication date: 16-Feb-2025
https://doi.org/10.1080/0144929X.2024.2447920
Rojas MBalderas DMaldonado JPonce PLopez-Bernal DMolina A(2024)Lack of verified Inclusive Technology for Workers with disabilities in industry 4.0: a systematic reviewInternational Journal of Sustainable Engineering10.1080/19397038.2024.232871117:1(190-210)Online publication date: 14-Mar-2024
https://doi.org/10.1080/19397038.2024.2328711
Bonello AFrancalanza ERefalo P(2023)Smart and Sustainable Human-Centred Workstations for Operators with Disability in the Age of Industry 5.0: A Systematic ReviewSustainability10.3390/su1601028116:1(281)Online publication date: 28-Dec-2023
https://doi.org/10.3390/su16010281
Show More Cited By

Index Terms

Comparing Tactile, Auditory, and Visual Assembly Error-Feedback for Workers with Cognitive Impairments
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Mixed / augmented reality
2. Human-centered computing
  1. Accessibility
    1. Empirical studies in accessibility

Recommendations

Comparing projected in-situ feedback at the manual assembly workplace with impaired workers
PETRA '15: Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments

With projectors and depth cameras getting cheaper, assistive systems in industrial manufacturing are becoming increasingly ubiquitous. As these systems are able to continuously provide feedback using in-situ projection, they are perfectly suited for ...
Using In-Situ Projection to Support Cognitively Impaired Workers at the Workplace
ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility

Today's working society tries to integrate more and more impaired workers into everyday working processes. One major scenario for integrating impaired workers is in the assembly of products. However, the tasks that are being assigned to cognitively ...
Haptic, Auditory, or Visual?: Towards Optimal Error Feedback at Manual Assembly Workplaces
PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

We are observing a trend that more and more manual assembly workplaces are equipped with sensor technology to assist workers during complex work tasks. These assistive systems mostly use visual feedback for providing assembly instructions or hinting at ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility

October 2016

362 pages

ISBN:9781450341240

DOI:10.1145/2982142

General Chair:
Jinjuan Heidi Feng
Towson University, USA
,
Program Chair:
Matt Huenerfauth
Rochester Institute of Technology, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGACCESS: ACM Special Interest Group on Accessible Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Bundesministerium für Wirtschaft und Energie

Conference

ASSETS '16

Sponsor:

SIGACCESS

ASSETS '16: The 18th International ACM SIGACCESS Conference on Computers and Accessibility

October 23 - 26, 2016

Nevada, Reno, USA

Acceptance Rates

ASSETS '16 Paper Acceptance Rate 24 of 95 submissions, 25%;

Overall Acceptance Rate 436 of 1,556 submissions, 28%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

48
Total Citations
View Citations
560
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)2

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dominiak JWysokińska ABentvelzen MRomanowski AWoźniak P(2025)A design space for reconfigurable interactive desktop environments (RIDEs)Behaviour & Information Technology10.1080/0144929X.2024.2447920(1-26)Online publication date: 16-Feb-2025
https://doi.org/10.1080/0144929X.2024.2447920
Rojas MBalderas DMaldonado JPonce PLopez-Bernal DMolina A(2024)Lack of verified Inclusive Technology for Workers with disabilities in industry 4.0: a systematic reviewInternational Journal of Sustainable Engineering10.1080/19397038.2024.232871117:1(190-210)Online publication date: 14-Mar-2024
https://doi.org/10.1080/19397038.2024.2328711
Bonello AFrancalanza ERefalo P(2023)Smart and Sustainable Human-Centred Workstations for Operators with Disability in the Age of Industry 5.0: A Systematic ReviewSustainability10.3390/su1601028116:1(281)Online publication date: 28-Dec-2023
https://doi.org/10.3390/su16010281
Drolshagen SPfingsthorn MHein A(2023)Context-Aware Robotic Assistive System: Robotic Pointing Gesture-Based Assistance for People with Disabilities in Sheltered WorkshopsRobotics10.3390/robotics1205013212:5(132)Online publication date: 27-Sep-2023
https://doi.org/10.3390/robotics12050132
Koushik VKane S(2023)Ability + Motivation: Understanding Factors that Influence People with Cognitive Disabilities in Regularly Practicing Daily ActivitiesProceedings of the 20th International Web for All Conference10.1145/3587281.3587295(122-133)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3587281.3587295
Zigart TKormann-Hainzl GLovasz-Bukvova HHölzl MMoser TSchlund S(2023)From lab to industry: lessons learned from the evaluation of augmented and virtual reality use cases in the Austrian manufacturing industryProduction & Manufacturing Research10.1080/21693277.2023.228634511:1Online publication date: 27-Nov-2023
https://doi.org/10.1080/21693277.2023.2286345
Kosch TGrote TSchmidt AWoźniak P(2022)Engagement, not Dependence: Ethically Designing Assistive Systems for Users with Cognitive ImpairmentsNordic Human-Computer Interaction Conference10.1145/3546155.3546662(1-13)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3546155.3546662
Jost MLuxenburger AKnoch SAlexandersson JMakedon F(2022)PARTAS: A Personalizable Augmented Reality Based Task Adaption System for Workers with Cognitive DisabilitiesProceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3529190.3529208(159-168)Online publication date: 29-Jun-2022
https://dl.acm.org/doi/10.1145/3529190.3529208
Colley MKränzle TRukzio E(2022)Accessibility-Related Publication Distribution in HCI Based on a Meta-AnalysisExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519701(1-28)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519701
Zigart TSchlund S(2022)Ready for Industrial Use? A User Study of Spatial Augmented Reality in Industrial Assembly2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00022(60-65)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00022
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten