Abstract
Recognition of hand gestures (hand signals) is an active research area for human computer interaction with many possible applications. Automatic machine vision-based hand gesture interfaces for real-time applications require fast and extremely robust human, pose and hand detection, and gesture recognition. Attempting to recognize gestures performed by official referees in sports (such as basketball game) video places tough requirements on the image segmentation techniques. Here we propose an image segmentation technique based on the histogram of oriented gradients and local binary pattern (LBP) features, which allow recognizing the signals of basketball referee from recorded game videos and achieved an accuracy of 95.6% using LBP features and support vector machine for classification. Our results are relevant for real-time analysis of basketball game.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bagheri MA, Montazer GA, Escalera S (2012) Error correcting output codes for multiclass classification: application to two image vision problems. In: 16th CSI international symposium on artificial intelligence and signal processing (AISP 2012), pp 508–513. https://doi.org/10.1109/AISP.2012.6313800
Basketball New Zealand (2016) Introduction to refereeing basketball. Retrieved from http://websites.sportstg.com/get_file.cgi?id=609417
Belgacem S, Chatelain C, Paquet T (2017) Gesture sequence recognition with one shot learned CRF/HMM hybrid model. Image Vis Comput 61:12–21. https://doi.org/10.1016/j.imavis.2017.02.003
Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8:679–698. https://doi.org/10.1109/TPAMI.1986.4767851
Chambers GS, Venkatesh S, West GAW, Bui HH (2004) Segmentation of intentional human gestures for sports video annotation. In: 10th International multimedia modelling conference. Brisbane, pp 124–129. https://doi.org/10.1109/MULMM.2004.1264976
Chen L, Wang F, Deng H, Ji K (2013) A survey on hand gesture recognition. In: 2013 International conference on computer sciences and applications, pp 313–316. https://doi.org/10.1109/CSA.2013.79
Cheng H, Yang L, Liu Z (2016) Survey on 3D hand gesture recognition. IEEE Trans Circuits Syst Video Technol 26(9):1659–1673. https://doi.org/10.1109/TCSVT.2015.2469551
Collignon H, Sultan N, Santander C (2011) The sports market. A.T. Kearney, Inc. Retrieved from http://www.atkearney.no/paper/-/asset_publisher/dVxv4Hz2h8bS/content/the-sports-market/10192
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 1, pp 886–893. https://doi.org/10.1109/CVPR.2005.177
Fernando B, Gavves E, Jose Oramas M, Ghodrati A, Tuytelaars T (2016) Rank pooling for action recognition. IEEE Trans Pattern Anal Mach Intell 39(4):773–787. https://doi.org/10.1109/TPAMI.2016.2558148
Gabryel M, Damasevicius R (2017) The image classification with different types of image features. In: Artificial intelligence and soft computing. ICAISC 2017. Lecture Notes in Computer Science, vol 10245, pp 497–506. https://doi.org/10.1007/978-3-319-59063-9_44
Geetha M, Kaimal MR (2018) A 3D stroke based representation of sign language signs using key maximum curvature points and 3D chain codes. Multimed Tools Appl 77(6):7097–7130. https://doi.org/10.1007/s11042-017-4624-y
Guo F, Cai Z, Tang J (2011) Chinese traffic police gesture recognition in complex scene. In: IEEE 10th international conference trust, security and privacy in computing and communications (TrustCom), pp 1505–1511. https://doi.org/10.1109/TrustCom.2011.208
Guyon I, Athitsos V, Jangyodsuk P, Hamner B, Escalante HJ (2012) ChaLearn gesture challenge: design and first results. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops, pp 1–6. https://doi.org/10.1109/CVPRW.2012.6239178
Hari R, Wilscy M (2014) Event detection in cricket videos using intensity projection profile of Umpire gestures. In: 2014 Annual IEEE India conference (INDICON), pp 1–6. https://doi.org/10.1109/INDICON.2014.7030519
Ho TK (1995) Random decision forests. In: 3rd international conference on document analysis and recognition, ICDAR ’95, vol 1. IEEE Computer Society, Washington, DC, USA, pp 278–282
Hu Z, Hu Y, Liu J, Wu B, Han D, Kurfess T (2018) 3D separable convolutional neural network for dynamic hand gesture recognition. Neurocomputing 318:151–161. https://doi.org/10.1016/j.neucom.2018.08.042
Joshi A, Monnier C, Betke M, Sclaroff S (2017) Comparing random forest approaches to segmenting and classifying gestures. Image Vis Comput 58:86–95. https://doi.org/10.1016/j.imavis.2016.06.001
Kirsch R (1971) Computer determination of the constituent structure of biological images. Comput Biomed Res 4:315–328
Kumar P, Gauba H, Roy PP, Dogra DP (2017) Coupled HMM-based multi-sensor data fusion for sign language recognition. Pattern Recogn Lett 86:1–8. https://doi.org/10.1016/j.patrec.2016.12.004
Le QK, Pham CH, Le TH (2012) Road traffic control gesture recognition using depth images. IEEK Trans Smart Process Comput 1:1–7
Li F-F, Fergus R, Perona P (2006) One-shot learning of object categories. IEEE Trans Pattern Anal Mach Intell 28(4):594–611. https://doi.org/10.1109/TPAMI.2006.79
Liang B, Zheng L (2013) Gesture recognition from one example using depth images. Lect Notes Softw Eng 1(4):339. https://doi.org/10.7763/LNSE.2013.V1.73
Liu H, Wang L (2018) Gesture recognition for human-robot collaboration: A review. Int J Ind Ergon 68:355–367. https://doi.org/10.1016/j.ergon.2017.02.004
Ma C, Zhang Y, Wang A, Wang Y, Chen G (2018) Traffic command gesture recognition for virtual urban scenes based on a spatiotemporal convolution neural network. ISPRS Int J Geo Inf 7:37. https://doi.org/10.3390/ijgi7010037
Mahmud M, Hasan K, Al-Tariq A, Kabir H, Mottalib MA (2018) Recognition of symbolic gestures using depth information. Adv Human Comput Interact. https://doi.org/10.1155/2018/1069823(Article ID: 1069823)
Malukas U, Maskeliunas R, Damasevicius R, Wozniak M (2018) Real time path finding for assisted living using deep learning. J UCS 24(4):475–487
McDonald P (2016, December 31) FIBA signals—basketball referee education. Retrieved from https://www.youtube.com/watch?v=k1yNcWsvu84
Nanjappan V, Liang H-, Lu F, Papangelis K, Yue Y, Man KL (2018) User-elicited dual-hand interactions for manipulating 3D objects in virtual reality environments. Human Centric Comput Inf Sci. https://doi.org/10.1186/s13673-018-0154-5
Neiva DH, Zanchettin C (2018) Gesture recognition: a review focusing on sign language in a mobile context. Expert Syst Appl 103:159–183. https://doi.org/10.1016/j.eswa.2018.01.051
Pecev P, Rackovic M, Ivkovic M (2016) A system for deductive prediction and analysis of movement of basketball referees. Multimed Tools Appl 75(23):16389–16416, 2016
Petraitis T, Maskeliunas R, Damasevicius R, Polap D, Wozniak W, Gabryel M (2017) Environment recognition based on images using bag-of-words. In: 9th international joint conference on computational intelligence, IJCCI 2017. Funchal, Madeira, pp 166–176. https://doi.org/10.5220/0006585601660176(1–3 November 2017)
Pietikäinen M, Hadid A, Zhao G, Ahonen T (2011) Computer vision using local binary patterns. Springer, New York. https://doi.org/10.1007/978-0-85729-748-8
Polap D, Wozniak M (2017) Detection of important features from images using heuristic approach. In: 23rd International conference on information and software technologies, ICIST 2017, Communications in computer and information science, vol 756, pp 432–441. https://doi.org/10.1007/978-3-319-67642-5_36
Prewitt JMS (1970) Object enhancement and extraction. In: Lipkin BS, Rosenfeld A (eds) Picture analysis and psychopictorics. Academic Press, New York, pp 75–149
Quesada L, López G, Guerrero L (2017) Automatic recognition of the american sign language fingerspelling alphabet to assist people living with speech or hearing impairments. J Ambient Intell Humaniz Comput 8(4):625–635. https://doi.org/10.1007/s12652-017-0475-7
Rossi M, Benatti S, Farella E, Benini L (2015) Hybrid EMG classifier based on HMM and SVM for hand gesture recognition in prosthetics. In: IEEE international conference on industrial technology, pp 1700–1705. https://doi.org/10.1109/ICIT.2015.7125342
Rothe R, Guillaumin M, Van Gool L (2015) Non-maximum suppression for object detection by passing messages between windows. In: Computer vision—ACCV 2014, pp 290–306. Springer International Publishing, New York. https://doi.org/10.1007/978-3-319-16865-4_19
Sathya R, Geetha MK (2015) Framework for traffic personnel gesture recognition. Procedia Comput Sci 46:1700–1707. https://doi.org/10.1016/j.procs.2015.02.113
Shih H-C (2017) A survey on content-aware video analysis for sports. IEEE Trans Circuits Syst Video Technol 28(5):1212–1231. https://doi.org/10.1109/TCSVT.2017.2655624
Singh M, Mandal M, Basu A (2005) Visual gesture recognition for ground air traffic control using the Radon transform. In: IEEE/RSJ international conference on intelligent robots and systems, pp 2586–2591. https://doi.org/10.1109/IROS.2005.1545408
Singha J, Roy A, Laskar RH (2018) Dynamic hand gesture recognition using vision-based approach for human–computer interaction. Neural Comput Appl 29(4):1129–1141. https://doi.org/10.1007/s00521-016-2525-z
Sobel I (1970) Camera models and perception. Ph.D. Thesis, Stanford University, Stanford, CA, USA, 1970
Thabet E, Khalid F, Sulaiman PS, Yaakob R (2018) Fast marching method and modified features fusion in enhanced dynamic hand gesture segmentation and detection method under complicated background. J Ambient Intell Humaniz Comput 9(3):755–769. https://doi.org/10.1007/s12652-017-0512-6
Tian TP, Sclaroff S (2005) Handsignals recognition from video using 3D motion capture data. In: Seventh IEEE workshops on application of computer vision, WACV/MOTIONS ‘05, vol 1, pp 189–194. https://doi.org/10.1109/ACVMOT.2005.57
Trigueiros P, Ribeiro F, Reis LP (2014) Vision based referee sign language recognition system for the RoboCup MSL league. In: RoboCup 2013: robot world cup XVII. LNCS, vol 8371, pp 360–372. https://doi.org/10.1007/978-3-662-44468-9_32
Verma R, Dev A (2009) Vision based hand gesture recognition using finite state machines and fuzzy logic. In: 2009 international conference on ultra modern telecommunications & workshops. St. Petersburg, pp 1–6. https://doi.org/10.1109/ICUMT.2009.5345425
Yeh CW, Pan TY, Hu MC (2017) A sensor-based official basketball referee signals recognition system using deep belief networks. In: 23rd international conference on multimedia modeling—MMM 2017, Reykjavik, Iceland, 2017, Part I. LNCS, vol 10132, pp 565–575. https://doi.org/10.1007/978-3-319-51811-4_46
Žemgulys J, Raudonis V, Maskeliūnas R, Damaševičius R (2018) Recognition of basketball referee signals from videos using histogram of oriented gradients (HOG) and support vector machine (SVM). Procedia Comput Sci 130:953–960. https://doi.org/10.1016/j.procs.2018.04.095
Zhang Y, Yao Y, Luo Y (2015) An improved HMM/SVM dynamic hand gesture recognition algorithm. In: Lee B, Su Y, Gu M, Yuan X, Jaque D (eds) AOPC 2015: advanced display technology; and Micro/Nano optical imaging technologies and applications. Proceedings of SPIE—the international society for optical engineering, vol 9672. https://doi.org/10.1117/12.2197328
Zhang Z, Tian Z, Zhou M (2018) HandSense: Smart multimodal hand gesture recognition based on deep neural networks. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-018-0989-7
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Žemgulys, J., Raudonis, V., Maskeliūnas, R. et al. Recognition of basketball referee signals from real-time videos. J Ambient Intell Human Comput 11, 979–991 (2020). https://doi.org/10.1007/s12652-019-01209-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-019-01209-1