Skip to main content
SpringerLink
Log in

A Comparative Study of Hand-Gesture Recognition Devices for Games

  • Conference paper
  • First Online:
Human-Computer Interaction. Multimodal and Natural Interaction (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12182))

Included in the following conference series:

Abstract

Gesture recognition devices provide a new means for natural human-computer interaction. However, when selecting these devices to be used in games, designers might find it challenging to decide which gesture recognition device will work best. In the present research, we compare three vision-based, hand-gesture devices: Leap Motion, Microsoft’s Kinect, and Intel’s RealSense. The comparison provides game designers with an understanding of the main factors to consider when selecting these devices and how to design games that use them. We developed a simple hand-gesture-based game to evaluate performance, cognitive demand, comfort, and player experience of using these gesture devices. We found that participants preferred and performed much better using Leap Motion and Kinect compared to using RealSense. Leap Motion also outperformed or was equivalent to Kinect. These findings were supported by players’ accounts of their experiences using these gesture devices. Based on these findings, we discuss how such devices can be used by game designers and provide them with a set of design cautions that provide insights into the design of gesture-based games.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In the NASA-TLX, the performance score is reverse-coded so that, like the other scores, a higher score is worse, thus, a high score on performance indicates worse perceived performance.

  2. 2.

    https://www.leapmotion.com.

  3. 3.

    https://downloadcenter.intel.com/product/92255.

  4. 4.

    https://developer.microsoft.com/en-us/windows/kinect.

References

  1. Ahmed, M.A., Zaidan, B.B., Zaidan, A.A., Salih, M.M., Lakulu, M.M.B.: A review on systems-based sensory gloves for sign language recognition state of the art between 2007 and 2017. Sensors (Basel, Switzerland) 18(7) (2018). http://europepmc.org/articles/PMC6069389

  2. Ahmed, S.F., Ali, S.M.B., Qureshi, S.S.M.: Electronic speaking glove for speechless patients, a tongue to a dumb. In: 2010 IEEE Conference on Sustainable Utilization and Development in Engineering and Technology, pp. 56–60, November 2010

    Google Scholar 

  3. Alharthi, S.A., Alsaedi, O., Toups Dugas, P.O., Tanenbaum, T.J., Hammer, J.: Playing to wait: a taxonomy of idle games. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, pp. 621:1–621:15. ACM, New York (2018). https://doi.org/10.1145/3173574.3174195

  4. Alimanova, M., et al.: Gamification of hand rehabilitation process using virtual reality tools: using leap motion for hand rehabilitation. In: 2017 First IEEE International Conference on Robotic Computing (IRC), pp. 336–339, April 2017

    Google Scholar 

  5. Bachynskyi, M., Palmas, G., Oulasvirta, A., Steimle, J., Weinkauf, T.: Performance and ergonomics of touch surfaces: a comparative study using biomechanical simulation. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 2015, pp. 1817–1826. ACM, New York (2015). https://doi.org/10.1145/2702123.2702607

  6. Bayliss, P.: Beings in the game-world: characters, avatars, and players. In: Proceedings of the 4th Australasian Conference on Interactive Entertainment, IE 2007, pp. 4:1–4:6. RMIT University, Melbourne (2007). http://dl.acm.org/citation.cfm?id=1367956.1367960

  7. Boulabiar, M.-I., Coppin, G., Poirier, F.: The issues of 3D hand gesture and posture recognition using the kinect. In: Kurosu, M. (ed.) HCI 2014. LNCS, vol. 8511, pp. 205–214. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07230-2_20

    Chapter  Google Scholar 

  8. Cabreira, A.T., Hwang, F.: An analysis of mid-air gestures used across three platforms. In: Proceedings of the 2015 British HCI Conference, British HCI 2015, pp. 257–258. ACM, New York (2015). http://libezp.nmsu.edu:4009/10.1145/2783446.2783599

  9. Carvalho, D., Bessa, M., Magalhães, L., Carrapatoso, E.: Performance evaluation of gesture-based interaction between different age groups using Fitts’ law. In: Proceedings of the XVI International Conference on Human Computer Interaction, Interacción 2015, pp. 5:1–5:7. ACM, New York (2015). http://libezp.nmsu.edu:4009/10.1145/2829875.2829920

  10. Chan, E., Seyed, T., Stuerzlinger, W., Yang, X.D., Maurer, F.: User elicitation on single-hand microgestures. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI 2016, pp. 3403–3414. ACM, New York (2016). https://doi.org/10.1145/2858036.2858589

  11. Cheng, J., Bian, W., Tao, D.: Locally regularized sliced inverse regression based 3d hand gesture recognition on a dance robot. Inf. Sci. 221, 274–283 (2013)

    Article  Google Scholar 

  12. Cook, H., Nguyen, Q.V., Simoff, S.: Enabling finger-gesture interaction with kinect. In: Proceedings of the 8th International Symposium on Visual Information Communication and Interaction, VINCI 2015, pp. 152–153. ACM, New York (2015). http://libezp.nmsu.edu:2763/10.1145/2801040.2801060

  13. Doliotis, P., Stefan, A., McMurrough, C., Eckhard, D., Athitsos, V.: Comparing gesture recognition accuracy using color and depth information. In: Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA 2011, pp. 20:1–20:7. ACM, New York (2011). http://libezp.nmsu.edu:2763/10.1145/2141622.2141647

  14. Erol, A., Bebis, G., Nicolescu, M., Boyle, R.D., Twombly, X.: Vision-based hand pose estimation: a review. Comput. Vis. Image Underst. 108(1), 52–73 (2007). Special Issue on Vision for Human-Computer Interaction. http://www.sciencedirect.com/science/article/pii/S1077314206002281

  15. Freeman, W.T., Ichi Tanaka, K., Ohta, J., Kyuma, K.: Computer vision for computer games. In: FG (1996)

    Google Scholar 

  16. Gardner, A., Duncan, C.A., Selmic, R., Kanno, J.: Real-time classification of dynamic hand gestures from marker-based position data. In: Proceedings of the Companion Publication of the 2013 International Conference on Intelligent User Interfaces Companion, IUI 2013 Companion, pp. 13–16. ACM, New York (2013). http://libezp.nmsu.edu:2763/10.1145/2451176.2451181

  17. Gutwin, C., Greenberg, S.: A descriptive framework of workspace awareness for real-time groupware. Comput. Support. Coop. Work (CSCW) 11(3), 411–446 (2002). https://doi.org/10.1023/A:1021271517844

  18. Hart, S.G.: Nasa-task load index (NASA-TLX); 20 years later. Proc. Hum. Fact. Ergon. Soc. Annu. Meet. 50(9), 904–908 (2006). https://doi.org/10.1177/154193120605000909

  19. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv. Psychol. 52, 139–183 (1988). Human Mental Workload. http://www.sciencedirect.com/science/article/pii/S0166411508623869

  20. Hincapié-Ramos, J.D., Guo, X., Moghadasian, P., Irani, P.: Consumed endurance: a metric to quantify arm fatigue of mid-air interactions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2014, pp. 1063–1072. ACM, New York (2014). http://libezp.nmsu.edu:2763/10.1145/2556288.2557130

  21. Intel RealSense Technology: Intel RealSense, March 2016. https://software.intel.com/en-us/articles/intel-realsense-data-ranges

  22. Khundam, C.: First person movement control with palm normal and hand gesture interaction in virtual reality. In: 2015 12th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 325–330, July 2015

    Google Scholar 

  23. Labs, T.: Myo gesture control armband (2015). https://support.getmyo.com/hc/en-us/articles/203398347-Getting-started-with-your-Myo-armband

  24. LaViola Jr., J.J.: Context aware 3D gesture recognition for games and virtual reality. In: ACM SIGGRAPH 2015 Courses, SIGGRAPH 2015, pp. 10:1–10:61. ACM, New York (2015). https://doi.org/10.1145/2776880.2792711

  25. Leap Motion: Detection Utilities, March 2016. https://developer-archive.leapmotion.com/documentation/v2/unity/unity/Unity_DetectionUtilities.html

  26. Lee, P.W., Wang, H.Y., Tung, Y.C., Lin, J.W., Valstar, A.: Transection: hand-based interaction for playing a game within a virtual reality game. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA 2015, pp. 73–76. ACM, New York (2015). http://libezp.nmsu.edu:2763/10.1145/2702613.2728655

  27. Lee, S., Park, K., Lee, J., Kim, K.: User study of VR basic controller and data glove as hand gesture inputs in VR games. In: 2017 International Symposium on Ubiquitous Virtual Reality (ISUVR), pp. 1–3, June 2017

    Google Scholar 

  28. Liu, M., Nancel, M., Vogel, D.: Gunslinger: subtle arms-down mid-air interaction. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, UIST 2015, pp. 63–71. ACM, New York (2015). http://libezp.nmsu.edu:2763/10.1145/2807442.2807489

  29. Lu, W., Tong, Z., Chu, J.: Dynamic hand gesture recognition with leap motion controller. IEEE Signal Process. Lett. 23(9), 1188–1192 (2016)

    Article  Google Scholar 

  30. Lv, Z., Halawani, A., Feng, S., Ur Réhman, S., Li, H.: Touch-less interactive augmented reality game on vision-based wearable device. Pers. Ubiquitous Comput. 19(3–4), 551–567 (2015). https://doi.org/10.1007/s00779-015-0844-1

  31. MacKenzie, I.S.: Fitts’ law as a research and design tool in human-computer interaction. Hum. Comput. Interact. 7(1), 91–139 (1992). https://doi.org/10.1207/s15327051hci0701_3

    Article  Google Scholar 

  32. Matthies, D.J.C., Müller, F., Anthes, C., Kranzlmüller, D.: Shoesolesense: proof of concept for a wearable foot interface for virtual and real environments. In: Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology, VRST 2013, pp. 93–96. ACM, New York (2013). http://libezp.nmsu.edu:4009/10.1145/2503713.2503740

  33. Morasso, P.: Spatial control of arm movements. Exp. Brain Res. 42(2), 223–227 (1981). https://doi.org/10.1007/BF00236911

    Article  Google Scholar 

  34. Moroney, W.F., Biers, D.W., Eggemeier, F.T., Mitchell, J.A.: A comparison of two scoring procedures with the NASA task load index in a simulated flight task. In: Proceedings of the IEEE 1992 National Aerospace and Electronics Conference, NAECON 1992, pp. 734–740. IEEE (1992)

    Google Scholar 

  35. Moser, C., Tscheligi, M.: Physics-based gaming: exploring touch vs. mid-air gesture input. In: Proceedings of the 14th International Conference on Interaction Design and Children, IDC 2015, pp. 291–294. ACM, New York (2015). http://libezp.nmsu.edu:4009/10.1145/2771839.2771899

  36. Mueller, F.F., Byrne, R., Andres, J., Patibanda, R.: Experiencing the body as play. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, pp. 210:1–210:13. ACM, New York (2018). https://doi.org/10.1145/3173574.3173784

  37. Napier, J., Napier, J.R., Tuttle, R.H.: Hands, vol. 9. Princeton University Press, Princeton (1993)

    Book  Google Scholar 

  38. Norman, D.: The Design of Everyday Things: Revised and Expanded Edition. Basic Books, Revised 2013 edn. (2013)

    Google Scholar 

  39. Paavilainen, J., Hamari, J., Stenros, J., Kinnunen, J.: Social network games: players’ perspectives. Simul. Gaming 44(6), 794–820 (2013). https://doi.org/10.1177/1046878113514808

    Article  Google Scholar 

  40. Peixoto, P., Carreira, J.: A natural hand gesture human computer interface using contour signatures (2005)

    Google Scholar 

  41. Pino, A., Tzemis, E., Ioannou, N., Kouroupetroglou, G.: Using kinect for 2D and 3D pointing tasks: performance evaluation. In: Kurosu, M. (ed.) HCI 2013. LNCS, vol. 8007, pp. 358–367. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39330-3_38

    Chapter  Google Scholar 

  42. Rautaray, S.S., Agrawal, A.: Interaction with virtual game through hand gesture recognition. In: 2011 International Conference on Multimedia, Signal Processing and Communication Technologies, pp. 244–247, December 2011

    Google Scholar 

  43. Rautaray, S.S., Agrawal, A.: Vision based hand gesture recognition for human computer interaction: a survey. Artif. Intell. Rev. 43(1), 1–54 (2015). https://doi.org/10.1007/s10462-012-9356-9

    Article  Google Scholar 

  44. Reifinger, S., Wallhoff, F., Ablassmeier, M., Poitschke, T., Rigoll, G.: Static and dynamic hand-gesture recognition for augmented reality applications. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 728–737. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73110-8_79

    Chapter  Google Scholar 

  45. Sambrooks, L., Wilkinson, B.: Comparison of gestural, touch, and mouse interaction with Fitts’ law. In: Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration, OzCHI 2013, pp. 119–122. ACM, New York (2013). http://libezp.nmsu.edu:4009/10.1145/2541016.2541066

  46. Seixas, M.C.B., Cardoso, J.C.S., Dias, M.T.G.: The leap motion movement for 2D pointing tasks: characterisation and comparison to other devices. In: 2015 International Conference on Pervasive and Embedded Computing and Communication Systems (PECCS), pp. 15–24, February 2015

    Google Scholar 

  47. Spanogianopoulos, S., Sirlantzis, K., Mentzelopoulos, M., Protopsaltis, A.: Human computer interaction using gestures for mobile devices and serious games: a review. In: 2014 International Conference on Interactive Mobile Communication Technologies and Learning, IMCL 2014, pp. 310–314, November 2014

    Google Scholar 

  48. Sridhar, S., Feit, A.M., Theobalt, C., Oulasvirta, A.: Investigating the dexterity of multi-finger input for mid-air text entry. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 2015, pp. 3643–3652. ACM, New York (2015). https://doi.org/10.1145/2702123.2702136

  49. Staretu, I., Moldovan, C.: Leap motion device used to control a real anthropomorphic gripper. Int. J. Adv. Rob. Syst. 13(3), 113 (2016). https://doi.org/10.5772/63973

    Article  Google Scholar 

  50. Svoboda, J., Bronstein, M.M., Drahansky, M.: Contactless biometric hand geometry recognition using a low-cost 3D camera. In: 2015 International Conference on Biometrics (ICB), pp. 452–457, May 2015

    Google Scholar 

  51. Tabor, A., Bateman, S., Scheme, E., Flatla, D.R., Gerling, K.: Designing game-based myoelectric prosthesis training. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI 2017, pp. 1352–1363. ACM, New York (2017). https://doi.org/10.1145/3025453.3025676

  52. Tabor, A., Kienzle, A., Smith, C., Watson, A., Wuertz, J., Hanna, D.: The falling of momo: a myo-electric controlled game to support research in prosthesis training. In: Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, CHI PLAY Companion 2016, pp. 71–77. ACM, New York (2016). https://doi.org/10.1145/2968120.2971806

  53. Tognazzini, B.: First principles of interaction design. Interaction design solutions for the real world, AskTog (2003)

    Google Scholar 

  54. Ververidis, D., Karavarsamis, S., Nikolopoulos, S., Kompatsiaris, I.: Pottery gestures style comparison by exploiting myo sensor and forearm anatomy. In: Proceedings of the 3rd International Symposium on Movement and Computing, MOCO 2016, pp. 3:1–3:8. ACM, New York (2016). http://libezp.nmsu.edu:4009/10.1145/2948910.2948924

  55. Vokorokos, L., Mihal’ov, J., Chovancová, E.: Motion sensors: gesticulation efficiency across multiple platforms. In: 2016 IEEE 20th Jubilee International Conference on Intelligent Engineering Systems (INES), pp. 293–298, June 2016

    Google Scholar 

  56. Vrellis, I., Moutsioulis, A., Mikropoulos, T.A.: Primary school students’ attitude towards gesture based interaction: a comparison between microsoft kinect and mouse. In: 2014 IEEE 14th International Conference on Advanced Learning Technologies, pp. 678–682, July 2014

    Google Scholar 

  57. Wachs, J.P., Kölsch, M., Stern, H., Edan, Y.: Vision-based hand-gesture applications. Commun. ACM 54(2), 60–71 (2011). http://libezp.nmsu.edu:2763/10.1145/1897816.1897838

    Article  Google Scholar 

  58. Weichert, F., Bachmann, D., Rudak, B., Fisseler, D.: Analysis of the accuracy and robustness of the leap motion controller. Sensors 13(5), 6380–6393 (2013). http://www.mdpi.com/1424-8220/13/5/6380

  59. Wuertz, J., et al.: A design framework for awareness cues in distributed multiplayer games. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, pp. 243:1–243:14. ACM, New York (2018). https://doi.org/10.1145/3173574.3173817

  60. Yao, Y., Chiu, P.T., Fu, W.T.: A gestural interface for practicing children’s spatial skills. In: Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion, IUI 2017 Companion, pp. 43–47. ACM, New York (2017). http://libezp.nmsu.edu:2763/10.1145/3030024.3038265

  61. Yuan, B., Folmer, E., Harris, F.C.: Game accessibility: a survey. Univers. Access Inform. Soc. 10(1), 81–100 (2011). https://doi.org/10.1007/s10209-010-0189-5

  62. Zhang, K., Zhai, Y., Leong, H.W., Wang, S.: An interaction educational computer game framework using hand gesture recognition. In: Proceedings of the 4th International Conference on Internet Multimedia Computing and Service, ICIMCS 2012, pp. 219–222. ACM, New York (2012). http://libezp.nmsu.edu:2763/10.1145/2382336.2382398

  63. Zhang, X., Chen, X., Wang, W.H., Yang, J.H., Lantz, V., Wang, K.Q.: Hand gesture recognition and virtual game control based on 3D accelerometer and EMG sensors. In: Proceedings of the 14th International Conference on Intelligent User Interfaces, IUI 2009, pp. 401–406. ACM, New York (2009). http://libezp.nmsu.edu:2763/10.1145/1502650.1502708

  64. Zhu, Y., Yuan, B.: Real-time hand gesture recognition with kinect for playing racing video games. In: 2014 International Joint Conference on Neural Networks (IJCNN), pp. 3240–3246, July 2014

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Play and Interactive Experiences for Learning Lab, Computer Science Department, New Mexico State University, Las Cruces, NM, USA

    Ahmed S. Khalaf, Sultan A. Alharthi & Phoebe O. Toups Dugas

  2. Electrical Engineering Department, New Mexico State University, Las Cruces, NM, USA

    Ali Alshehri

  3. Psychology Department, New Mexico State University, Las Cruces, NM, USA

    Igor Dolgov

Authors
  1. Ahmed S. Khalaf
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sultan A. Alharthi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ali Alshehri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Igor Dolgov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Phoebe O. Toups Dugas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed S. Khalaf .

Editor information

Editors and Affiliations

  1. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khalaf, A.S., Alharthi, S.A., Alshehri, A., Dolgov, I., Toups Dugas, P.O. (2020). A Comparative Study of Hand-Gesture Recognition Devices for Games. In: Kurosu, M. (eds) Human-Computer Interaction. Multimodal and Natural Interaction. HCII 2020. Lecture Notes in Computer Science(), vol 12182. Springer, Cham. https://doi.org/10.1007/978-3-030-49062-1_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-49062-1_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-49061-4

  • Online ISBN: 978-3-030-49062-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation