Abstract
With the continuous emergence and vigorous development of various new technologies in the information age, human-computer interaction is also changing constantly. People are increasingly connected with the virtual world, so it is necessary to study more natural ways of human-computer interaction. Projection interaction is one of the most important interaction methods. The purpose of this paper is to study how to overcome the misoperation problem caused by foreign objects (interferers) in the environment of projection interaction (Fig. 1c). The method (named after FDGR) adopted is to use the fingertip detection algorithm based on user habits in image processing and the gesture recognition algorithm based on Leap Motion to complete the accurate and convenient natural human-computer interaction. Through user experiments, the experiences of completing the fixed assembly task in the mode of projection (P), gesture (G) and FDGR (PG) were compared, and the data was processed and analyzed. The experimental results showed that FDGR could effectively overcome the misoperation caused by foreign objects (interferers) and the non-intuitiveness of simple gestures, and effectively improved the user's operating experience.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Hongjian, C.: Research on gesture recognition method based on computer vision technology. In: 2020 International Conference on Computer Information and Big Data Applications (CIBDA), pp. 358–362. IEEE, Guiyang (2020)
Fengjun, Z.: A survey on human-computer interaction in virtual reality. Sci. Sinica Inf. 46(12), 1711–1736 (2016)
Jin, H.: An overview of human-computer interaction in mixed reality. J. Comput. Aided Des. Graph. 28(06), 869–880 (2016)
Granit, L.: A sliding window approach to natural hand gesture recognition using a custom data glove. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI), pp. 81–90 (2016)
Munir, O.: Hand gesture recognition based on computer vision: a review of techniques. J. Imaging 6(8), 73–102 (2020)
Ivan, P.: The go-go interaction technique: non-linear mapping for direct manipulation in VR. In: Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology, pp. 79–80. Association for Computing Machinery, Seattle, Washington (1996)
Doshi, A., Smith, R.T., Thomas, B.H., Bouras, C.: Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. Int. J. Adv. Manuf. Technol. 89(5–8), 1279–1293 (2016). https://doi.org/10.1007/s00170-016-9164-5
Jianlong, Z.: Applying spatial augmented reality to facilitate in-situ support for automotive spot welding inspection. In: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry, pp. 195–200 (2011)
Priyanka, P.: Development of hand gesture recognition system using machine learning. J. Amb. Intell. Hum. Comput. 12, 6793–6800 (2020)
Kuen-Meau, C., Ming-Jen, W.: Using the Interactive Design of Gesture Recognition in Augmented Reality. Trans Tech Publications Ltd., Durnten-Zurich (2013)
Huidong, B.: Using 3D hand gestures and touch input for wearable AR interaction. In: CHI, pp. 1321–1326. ACM, Toronto (2014)
Mark, B.: Hands in space gesture interaction with augmented-reality interfaces. IEEE Comput. Graph. Appl. 34(1), 77–81 (2014)
Kim, M., Lee, J.Y.: Touch and hand gesture-based interactions for directly manipulating 3D virtual objects in mobile augmented reality. Multimedia Tools Appl. 75(23), 16529–16550 (2016). https://doi.org/10.1007/s11042-016-3355-9
Acknowledgement
This work is partly support by National Key R&D Program of China (Grant No. 2019YFB1703800), Natural Science Basic Research Plan in Shaanxi Province of China (Grant No. 2016JM6054), the Programme of Introducing Talents of Discipline to Universities (111 Project), China (Grant No. B13044).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Cao, Z., He, W., Wang, S., Zhang, J., Wei, B., Li, J. (2021). Research on Projection Interaction Based on Gesture Recognition. In: Stephanidis, C., Antona, M., Ntoa, S. (eds) HCI International 2021 - Late Breaking Posters. HCII 2021. Communications in Computer and Information Science, vol 1498. Springer, Cham. https://doi.org/10.1007/978-3-030-90176-9_40
Download citation
DOI: https://doi.org/10.1007/978-3-030-90176-9_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90175-2
Online ISBN: 978-3-030-90176-9
eBook Packages: Computer ScienceComputer Science (R0)