skip to main content
10.1145/2700648.2809853acmconferencesArticle/Chapter ViewAbstractPublication PagesassetsConference Proceedingsconference-collections
research-article

Using In-Situ Projection to Support Cognitively Impaired Workers at the Workplace

Published: 26 October 2015 Publication History

Abstract

Today's working society tries to integrate more and more impaired workers into everyday working processes. One major scenario for integrating impaired workers is in the assembly of products. However, the tasks that are being assigned to cognitively impaired workers are easy tasks that consist of only a small number of assembly steps. For tasks with a higher number of steps, cognitively impaired workers need instructions to help them with assembly. Although supervisors provide general support and assist new workers while learning new assembly steps, sheltered work organizations often provide additional printed pictorial instructions that actively guide the workers. To further improve continuous instructions, we built a system that uses in-situ projection and a depth camera to provide context-sensitive instructions. To explore the effects of in-situ instructions, we compared them to state-of-the-art pictorial instructions in a user study with 15 cognitively impaired workers at a sheltered work organization. The results show that using in-situ instructions, cognitively impaired workers can assemble more complex products up to 3 times faster and with up to 50% less errors. Further, the workers liked the in-situ instructions provided by our assistive system and would use it for everyday assembly.

References

[1]
United nations: United nations convention on the rights of persons with disabilities. http://www.un.org/disabilities/convention/conventionfull.shtml, 2008. {Online; accessed 02-May-2015}.
[2]
Bannat, A., Wallhoff, F., Rigoll, G., Friesdorf, F., Bubb, H., Stork, S., Müller, H., Schubö, A., Wiesbeck, M., and Z\"ah, M. Towards optimal worker assistance: a framework for adaptive selection and presentation of assembly instructions. In Proceedings of the 1st international workshop on cognition for technical systems, Cotesys (2008).
[3]
Caudell, T. P., and Mizell, D. W. Augmented reality: An application of heads-up display technology to manual manufacturing processes. In System Sciences, 1992. Proceedings of the Twenty-Fifth Hawaii International Conference on, vol. 2, IEEE (1992), 659--669.
[4]
Echtler, F., Sturm, F., Kindermann, K., Klinker, G., Stilla, J., Trilk, J., and Najafi, H. The intelligent welding gun: Augmented reality for experimental vehicle construction. In Virtual and augmented reality applications in manufacturing. Springer, 2004, 333--360.
[5]
Fite-Georgel, P. Is there a reality in industrial augmented reality-- In Mixed and Augmented Reality (ISMAR), 2011 10th IEEE International Symposium on, IEEE (2011), 201--210.
[6]
Funk, M., Bachler, A., Bachler, L., Korn, O., Krieger, C., Heidenreich, T., and Schmidt, A. Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In Proceedings of the 8th International Conference on PErvasive Technologies Related to Assistive Environments, ACM (2015).
[7]
Funk, M., and Schmidt, A. Cognitive assistance in the workplace. Pervasive Computing, IEEE 14, 3 (2015), 53--55.
[8]
Hardy, J., and Alexander, J. Toolkit support for interactive projected displays. In Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia, ACM (2012), 42.
[9]
Korn, O., Schmidt, A., and Hörz, T. Assistive systems in production environments: exploring motion recognition and gamification. In Proceedings of the 5th International Conference on PErvasive Technologies Related to Assistive Environments, ACM (2012).
[10]
Korn, O., Schmidt, A., and Hörz, T. The potentials of in-situ-projection for augmented workplaces in production: a study with impaired persons. In CHI'13 Extended Abstracts on Human Factors in Computing Systems, ACM (2013), 979--984.
[11]
Navab, N. Developing killer apps for industrial augmented reality. Computer Graphics and Applications, IEEE 24, 3 (2004), 16--20.
[12]
Pierce, K. L., and Schreibman, L. Teaching daily living skills to children with autism in unsupervised settings through pictorial self-management. Journal of applied behavior analysis 27, 3 (1994), 471--481.
[13]
Pinhanez, C. The everywhere displays projector: A device to create ubiquitous graphical interfaces. In Ubicomp 2001: Ubiquitous Computing, Springer (2001), 315--331.
[14]
Raskar, R., Welch, G., and Fuchs, H. Spatially augmented reality. In First IEEE Workshop on Augmented Reality (IWAR'98), Citeseer (1998), 11--20.
[15]
Rüther, S., Hermann, T., Mracek, M., Kopp, S., and Steil, J. An assistance system for guiding workers in central sterilization supply departments. In Proceedings of the 6th International Conference on PErvasive Technologies Related to Assistive Environments, ACM (2013), 3.
[16]
Sakata, N., Kurata, T., and Kuzuoka, H. Visual assist with a laser pointer and wearable display for remote collaboration. In Proceedings of 2nd international conference on Collaboration Technologies (2006).
[17]
Sauer, A. L., Parks, A., and Heyn, P. C. Assistive technology effects on the employment outcomes for people with cognitive disabilities: a systematic review. Disability & Rehabilitation: Assistive Technology 5, 6 (2010), 377--391.
[18]
Sutherland, I. E. A head-mounted three dimensional display. In Proceedings of the December 9--11, 1968, fall joint computer conference, part I, ACM (1968), 757--764.
[19]
Tang, A., Owen, C., Biocca, F., and Mou, W. Comparative effectiveness of augmented reality in object assembly. In Proceedings of the SIGCHI conference on Human factors in computing systems, ACM (2003), 73--80.
[20]
Zheng, X. S., Foucault, C., Matos da Silva, P., Dasari, S., Yang, T., and Goose, S. Eye-wearable technology for machine maintenance: Effects of display position and hands-free operation. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM (2015), 2125--2134.
[21]
Zhou, J., Lee, I., Thomas, B., Menassa, R., Farrant, A., and Sansome, A. Applying spatial augmented reality to facilitate in-situ support for automotive spot welding inspection. In Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry, ACM (2011), 195--200.

Cited By

View all
  • (2024)"Uh, This One?": Leveraging Behavioral Signals for Detecting Confusion during Physical TasksProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685727(194-203)Online publication date: 4-Nov-2024
  • (2024)"It Is Easy Using My Apps:" Understanding Technology Use and Needs of Adults with Down SyndromeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642950(1-17)Online publication date: 11-May-2024
  • (2024): Visualization of AI-Assisted Task Guidance in ARIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332739630:1(1313-1323)Online publication date: 1-Jan-2024
  • Show More Cited By

Index Terms

  1. Using In-Situ Projection to Support Cognitively Impaired Workers at the Workplace

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility
    October 2015
    466 pages
    ISBN:9781450334006
    DOI:10.1145/2700648
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 26 October 2015

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. assistance for impaired workers
    2. assistive system
    3. augmented reality
    4. in-situ projection

    Qualifiers

    • Research-article

    Funding Sources

    • Bundesministerium fur Wirtschaft und Energie

    Conference

    ASSETS '15
    Sponsor:

    Acceptance Rates

    ASSETS '15 Paper Acceptance Rate 30 of 127 submissions, 24%;
    Overall Acceptance Rate 436 of 1,556 submissions, 28%

    Upcoming Conference

    ASSETS '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)59
    • Downloads (Last 6 weeks)7
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)"Uh, This One?": Leveraging Behavioral Signals for Detecting Confusion during Physical TasksProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685727(194-203)Online publication date: 4-Nov-2024
    • (2024)"It Is Easy Using My Apps:" Understanding Technology Use and Needs of Adults with Down SyndromeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642950(1-17)Online publication date: 11-May-2024
    • (2024): Visualization of AI-Assisted Task Guidance in ARIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332739630:1(1313-1323)Online publication date: 1-Jan-2024
    • (2024)Lack of verified Inclusive Technology for Workers with disabilities in industry 4.0: a systematic reviewInternational Journal of Sustainable Engineering10.1080/19397038.2024.232871117:1(190-210)Online publication date: 14-Mar-2024
    • (2024)The realities of achieving a Smart, Sustainable, and Inclusive shopfloor in the age of Industry 5.0.Procedia Computer Science10.1016/j.procs.2024.02.059232:C(2406-2415)Online publication date: 1-Jan-2024
    • (2024)Assessing user performance in augmented reality assembly guidance for industry 4.0 operatorsComputers in Industry10.1016/j.compind.2024.104085157:COnline publication date: 1-May-2024
    • (2024)Potentials of an informational assembly assistance system for persons with cognitive disabilities—Results of a laboratory studyPotenziale eines informatorischen Montageassistenzsystems für Personen mit kognitiven Unterstützungsbedarfen – Ergebnisse einer LaborstudieZeitschrift für Arbeitswissenschaft10.1007/s41449-024-00414-978:2(240-253)Online publication date: 11-Apr-2024
    • (2024)An externally guided spatial augmented reality assembly assistance system in the aviation manufacturing industryThe International Journal of Advanced Manufacturing Technology10.1007/s00170-024-13906-z133:9-10(4893-4906)Online publication date: 26-Jun-2024
    • (2024)An Examination of the Limited Adoption of Personalized Work Instructions in Assembly to Accommodate Individual Worker’s NeedsAdvances in Production Management Systems. Production Management Systems for Volatile, Uncertain, Complex, and Ambiguous Environments10.1007/978-3-031-65894-5_23(321-335)Online publication date: 6-Sep-2024
    • (2023)Smart and Sustainable Human-Centred Workstations for Operators with Disability in the Age of Industry 5.0: A Systematic ReviewSustainability10.3390/su1601028116:1(281)Online publication date: 28-Dec-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media