Abstract
This paper presents a prototype of an intelligent assistive system for workers in stationary manual assembly using projection-based augmented reality (AR) and intelligent hand tracking. By using depth cameras, the system can track the hands of the user and makes the user aware of wrong picking actions or errors in the assembly process. The system automatically adapts the digital projection-based overlay according to the current work situation. The main research contribution of our work is the presentation of a novel hand-tracking algorithm. In addition, we present the results of an user study of the system that shows the challenges and opportunities of our system and the hand-tracking algorithm in particular. We assume that our results will inform the future design of assistive systems in manual assembly.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bannat, A., Wallhoff, F., Rigoll, G., Friesdorf, F., Bubb, H., Stork, S., Müller, H., Schubö, A., Wiesbeck, M., Zäh, M.F., et al.: Towards optimal worker assistance: a framework for adaptive selection and presentation of assembly instructions. In: Proceedings of the 1st international Workshop on Cognition for Technical Systems, CoTeSys (2008)
Billinghurst, M., Hakkarainen, M., Woodward, C.: Augmented assembly using a mobile phone. In: Proceedings of the 7th International Conference on Mobile and Ubiquitous Multimedia, pp. 84–87. ACM (2008)
Büttner, S., Funk, M., Sand, O., Röcker, C.: Using head-mounted displays and in-situ projection for assistive systems-a comparison. In: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, vol. 8. ACM (2016)
Büttner, S., Sand, O., Röcker, C.: Extending the design space in industrial manufacturing through mobile projection. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pp. 1130–1133. ACM (2015)
Dipietro, L., Sabatini, A.M., Dario, P.: A survey of glove-based systems and their applications. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 38(4), 461–482 (2008)
Funk, M., Bächler, A., Bächler, L., Korn, O., Krieger, C., Heidenreich, T., Schmidt, A.: Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In: Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments, p. 1. ACM (2015)
Funk, M., Mayer, S., Schmidt, A.: Using in-situ projection to support cognitively impaired workers at the workplace. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 185–192. ACM (2015)
Grubert, J., Hamacher, D., Mecke, R., Böckelmann, I., Schega, L., Huckauf, A., Urbina, M., Schenk, M., doil, F., Tümler, J.: Extended investigations of user-related issues in mobile industrial AR. In: 2010 9th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 229–230. IEEE (2010)
Guo, A., Raghu, S., Xie, X., Ismail, S., Luo, X., Simoneau, J., Gilliland, S., Baumann, H., Southern, C., Starner, T.: A comparison of order picking assisted by head-up display (HUD), cart-mounted display (CMD), light, and paper pick list. In: Proceedings of the 2014 ACM International Symposium on Wearable Computers, pp. 71–78. ACM (2014)
Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv. Psychol. 52, 139–183 (1988)
Kirch, M., Poenicke, O.: Using the RFID wristband for automatic identification in manual processes-the RFID wristband in the automotive industry. In: 2014 European Conference on Smart Objects, Systems and Technologies (Smart SysTech), pp. 1–7. IEEE (2014)
Lasi, H., Fettke, P., Kemper, H.G., Feld, T., Hoffmann, M.: Industry 4.0. business and information. Syst. Eng. 6(4), 239 (2014)
Paelke, V., Röcker, C., Koch, N., Flatt, H., Büttner, S.: User interfaces for cyber-physical systems. at-Automatisierungstechnik 63(10), 833–843 (2015)
Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. I-195. IEEE (2003)
Schwerdtfeger, B., Klinker, G.: Supporting order picking with augmented reality. In: Proceedings of the 7th IEEE/ACM international Symposium on Mixed and Augmented Reality, pp. 91–94. IEEE Computer Society (2008)
Schwerdtfeger, B., Reif, R., Gunthner, W.A., Klinker, G., Hamacher, D., Schega, L., Bockelmann, I., doil, F., Tumler, J.: Pick-by-vision: a first stress test. In: 8th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2009, pp. 115–124. IEEE (2009)
Tang, A., Owen, C., Biocca, F., Mou, W.: Comparative effectiveness of augmented reality in object assembly. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 73–80. ACM (2003)
Zheng, X.S., Foucault, C., Matos da Silva, P., Dasari, S., Yang, T., Goose, S.: Eye-wearable technology for machine maintenance: effects of display position and hands-free operation. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 2125–2134. ACM (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Büttner, S., Sand, O., Röcker, C. (2017). Exploring Design Opportunities for Intelligent Worker Assistance: A New Approach Using Projetion-Based AR and a Novel Hand-Tracking Algorithm. In: Braun, A., Wichert, R., Maña, A. (eds) Ambient Intelligence. AmI 2017. Lecture Notes in Computer Science(), vol 10217. Springer, Cham. https://doi.org/10.1007/978-3-319-56997-0_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-56997-0_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56996-3
Online ISBN: 978-3-319-56997-0
eBook Packages: Computer ScienceComputer Science (R0)
