Skip to main content
SpringerLink
Log in

Analysis of tennis games using TrackNet-based neural network and applying morphological operations to the match videos

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Computer vision plays a crucial role in current technological development, understanding a scene from the properties of 2D images. This research line becomes valuable in sports applications, where the scenario can be challenging to take technical decisions only from the observation. This work aims to develop a system based on computer vision for analyzing tennis games. The implemented method captures videos during the game through cameras installed on the court. Machine learning methods and morphological operations will be used over the images to locate the ball position, the court lines and the players location. In addition, the algorithm determines the moment the ball bounces during the game and analyzes whether it occurred in or out of the field. These data are available to players and judges through an Android application, allowing all processed data to be accessed from mobile devices, providing the results quickly and accessible to the user. From the results obtained, the system demonstrated robustness and reliability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://www.youtube.com/watch?v=EnPIQdtNbx8.

  2. https://www.youtube.com/watch?v=Yil9PpIr1DA.

  3. https://github.com/nayaramsr/Ball_Field_Player_Detection/tree/main/Dataset_player.

  4. https://bit.ly/Tennis_App.

  5. https://github.com/nayaramsr/Ball_Field_Player_Detection.git.

References

  1. Archana, M., Geetha, M.K.: Object detection and tracking based on trajectory in broadcast tennis video. Procedia Comput. Sci. 58, 225–232 (2015)

    Article  Google Scholar 

  2. Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision in C++ with the OpenCV Library. O ’Reilly Media Inc, Sebastopol (2012)

    Google Scholar 

  3. Cipolla, R., Farinella, G.M., Battiato, S.: Machine Learning for Computer Vision. Springer, Berlin (2012)

    MATH  Google Scholar 

  4. Daniel, G., Chen, M.: Video Visualization. IEEE (2003)

  5. Direkoglu, C., Sah, M., O’Connor, N.E.: Player detection in field sports. Mach. Vision Appl. 29(2), 187–206 (2018)

    Article  Google Scholar 

  6. Farhat, M., Khalfallah, A., Bouhlel, M.S.: A new model based approach for tennis court tracking in real time. Int. J. Signal Imaging Syst. Eng. 11, 9–19 (2018)

    Article  Google Scholar 

  7. Fazio, M., Fisher, K., Fujinami, T.: Tennis Ball Tracking: 3-D Trajectory Estimation Using Smartphone Videos. Department of Electrical Engineering, Stanford University (2018)

  8. Gomez-Gonzalez, S., Nemmour, Y., Schölkopf, B., Peters, J.: Reliable real-time ball tracking for robot table tennis. Robotics 8, 90 (2019)

    Article  Google Scholar 

  9. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT (2016)

  10. Huang, Y.C., Liao, I.N., Chen, C.H., İk, T.U., Peng, W.C.: Tracknet: a deep learning network for tracking high-speed and tiny objects in sports applications. In: 2019 16th IEEE International Conference on Advanced Video (AVSS), pp. 1–8. IEEE (2019)

  11. Kaehler, A., Bradski, G.: Learning OpenCV 3. O’Reilly, Sebastopol (2016)

    Google Scholar 

  12. Lehuger, A., Duffner, S., Garcia, C.: A Robust Method for Automatic Player Detection in Sport Videos. Orange Labs 4 (2007)

  13. Lin, H.I., Yu, Z., Huang, Y.C.: Ball tracking and trajectory prediction for table-tennis robots. Sensors 20, 333 (2020)

    Article  Google Scholar 

  14. Lu, K., Chen, J., Little, J.J., He, H.: Light cascaded convolutional neural networks for accurate player detection. arXiv:1709.10230 (2017)

  15. Mahtani, A., Sanchez, L., Fernandez, E., Martinez, A., Joseph, L.: ROS Programming: Building Powerful Robots. Packt (2018)

  16. Manafifard, M., Ebadi, H., Moghaddam, H.A.: A survey on player tracking in soccer videos. Comput. Vis. Image Underst. 159, 19–46 (2017)

    Article  Google Scholar 

  17. Mao, J.: Tracking a tennis ball using image processing techniques. University Online Library (2006)

  18. Messelodi, S., Modena, C.M., Ropele, V., Marcon, S., Sgro, M.: A low-cost computer vision system for real-time tennis analysis. In: International Conference on Image Analysis and Processing (2019)

  19. Mora, S.V.: Computer vision and machine learning for in-play tennis analysis. Ph.D. dissertation, University of London (2017)

  20. Mukai, R., Asano, T., Hara, H.: Analysis and evaluation of tennis plays by computer vision. In: 2011 IEEE International Conference on Mechatronics and Automation, pp. 784–788. IEEE (2011)

  21. Nishani, E., Çiço, B.: Computer vision approaches based on deep learning and neural networks. In: 2017 6th Mediterranean Conference on Embedded Computing (MECO), pp. 1–4. IEEE (2017)

  22. Otani, M., Nakashima, Y., Sato, T., Yokoya, N.: Video summarization using textual descriptions for authoring video blogs. Multimed. Tools Appl. 76, 12097–12115 (2017)

    Article  Google Scholar 

  23. Owens, N., Harris, C., Stennett, C.: Hawk-eye tennis system. In: 2003 International Conference on Visual Information Engineering VIE 2003, pp. 182–185. IET (2003)

  24. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv:1804.02767 (2018)

  25. Renò, V., Mosca, N., Marani, R., Nitti, M., D’Orazio, T., Stella, E.: Convolutional neural networks based ball detection in tennis games. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (2018)

  26. Silveira, G.: Direct 3-d tracking for central omnidirectional cameras under general lighting variations. J. Control Autom. Electr. Syst. 24, 129–138 (2013)

    Article  Google Scholar 

  27. Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. arXiv:1406.2199 (2014)

  28. Teachabarikiti, K., Chalidabhongse, T.H., Thammano, A.: Players tracking and ball detection for an automatic tennis video annotation. In: 2010 11th International Conference on Control Automation Robotics & Vision, pp. 2461–2494. IEEE (2010)

  29. Thomas, G., Gade, R., Moeslund, T.B., Carr, P., Hilton, A.: Computer vision for sports: current applications and research topics. Comput. Vis. Image Underst. 159, 3–18 (2017)

    Article  Google Scholar 

  30. Tian, B., Zhang, D., Zhang, C.: High-speed tiny tennis ball detection based on deep convolutional neural networks. In: 2020 IEEE 14th International Conference on Anti-Falling, Security, and Identification (ASID), pp. 30–33 (2020). https://doi.org/10.1109/ASID50160.2020.9271695

  31. Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3d convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4489–4497 (2015)

  32. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, pp. I–I (2001). https://doi.org/10.1109/CVPR.2001.990517

  33. Volna, E., Kotyrba, M.: Vision system for licence plate recognition based on neural networks. In: 13th International Conference on Hybrid Intelligent Systems (HIS 2013), pp. 140–143. IEEE (2013)

  34. Yan, F., Kittler, J., Windridge, D., Christmas, W., Mikolajczyk, K., Cox, S., Huang, Q.: Automatic annotation of tennis games: an integration of audio, vision, and learning. Image Vis. Comput. 32, 896–903 (2014)

    Article  Google Scholar 

  35. Yang, H., Shao, L., Zheng, F., Wang, L., Song, Z.: Recent advances and trends in visual tracking: a review. Neurocomputing 74, 3823–3831 (2011)

    Article  Google Scholar 

  36. Zhang, Z., Xu, D., Tan, M.: Visual measurement and prediction of ball trajectory for table tennis robot. IEEE Trans. Instrum. Meas. 59, 3195–3205 (2010)

    Article  Google Scholar 

  37. Zheng, F., Luo, S., Song, K., Yan, C.W., Wang, M.C.: Improved lane line detection algorithm based on Hough transform. Pattern Recognit. Image Anal. 28, 254–260 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank UFJF, CEFET-RJ, FAPERJ, and FAPEMIG for research support.

Author information

Authors and Affiliations

  1. Federal University of Juiz de Fora (UFJF), Juiz de Fora, Brazil

    Nayara M. S. Rocha, Iago Z. Biundini, Aurelio G. Melo & André L. M. Marcato

  2. Federal Center for Technological Education of Rio de Janeiro, Rio de Janeiro, Brazil

    Milena F. Pinto

Authors
  1. Nayara M. S. Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Milena F. Pinto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Iago Z. Biundini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aurelio G. Melo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. André L. M. Marcato
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nayara M. S. Rocha.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rocha, N.M.S., Pinto, M.F., Biundini, I.Z. et al. Analysis of tennis games using TrackNet-based neural network and applying morphological operations to the match videos. SIViP 17, 1133–1141 (2023). https://doi.org/10.1007/s11760-022-02320-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02320-1

Keywords

Search

Navigation