skip to main content
10.1145/3568444.3570592acmotherconferencesArticle/Chapter ViewAbstractPublication PagesmumConference Proceedingsconference-collections
poster

Video Annotation Tool using Human Pose Estimation for Sports Training

Published: 29 December 2022 Publication History

Abstract

This paper presents and discusses the integration of human pose estimation techniques into an existing web-based multimodal video annotation tool, applying it to the sports context, where basketball is the first case study. The relevance of video analysis extends across many fields of work (e.g., professional sports, education). In sports, systematic, detailed analysis using videos of players and teams is vital to evaluate many aspects of both training and competition. MotionNotes annotation tool now combines human pose and motion information with existing traditional annotation mechanisms (e.g., text and drawings annotations), allowing users to add further details to their annotation work. The paper reports feedback from a pilot study based on a participatory workshop involving people with relevant competitive experience in basketball. Based on this use case feedback, we conclude with an outlook of future iterations for our video annotation tool.

References

[1]
Matija Buric, Marina Ivasic-Kos, and Miran Pobar. 2019. Player tracking in sports videos. In Proceedings of the International Conference on Cloud Computing Technology and Science, CloudCom, 334–340. https://doi.org/10.1109/CloudCom.2019.00058
[2]
Diogo Cabral, João Valente, João Silva, Urândia Aragão, Carla Fernandes, and Nuno Correia. 2011. A creation-tool for contemporary dance using multimodal video annotation. In Proceedings of the 19th ACM international conference on Multimedia - MM ’11, 905. https://doi.org/10.1145/2072298.2071899
[3]
Zhe Cao, Tomas Simon, Shih En Wei, and Yaser Sheikh. 2017. Realtime multi-person 2D pose estimation using part affinity fields. In Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. https://doi.org/10.1109/CVPR.2017.143
[4]
Yu Chen, Chunhua Shen, Xiu Shen Wei, Lingqiao Liu, and Jian Yang. 2017. Adversarial PoseNet: A Structure-Aware Convolutional Network for Human Pose Estimation. In Proceedings of the IEEE International Conference on Computer Vision. https://doi.org/10.1109/ICCV.2017.137
[5]
Gisela Miranda Difini, Marcio Garcia Martins, and Jorge Luis Victória Barbosa. 2021. Human Pose Estimation for Training Assistance: A Systematic Literature Review. ACM International Conference Proceeding Series: 189–196. https://doi.org/10.1145/3470482.3479633
[6]
Tong Gao, Mira Dontcheva, Eytan Adar, Zhicheng Liu, and Karrie Karahalios. 2015. Datatone: Managing ambiguity in natural language interfaces for data visualization. In UIST 2015 - Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology. https://doi.org/10.1145/2807442.2807478
[7]
R Goldman, R Pea, B Barron, and SJ Derry. 2014. Video Research in the Learning Sciences. Routledge. https://doi.org/10.4324/9780203877258
[8]
Renshu Gu, Gaoang Wang, Zhongyu Jiang, and Jenq Neng Hwang. 2020. Multi-Person Hierarchical 3D Pose Estimation in Natural Videos. IEEE Transactions on Circuits and Systems for Video Technology 30, 11: 4245–4257. https://doi.org/10.1109/TCSVT.2019.2953678
[9]
Joachim Gudmundsson and Michael Horton. 2017. Spatio-Temporal Analysis of Team Sports r r r. 50, 2: 1–34.
[10]
Yanming Guo, Yu Liu, Ard Oerlemans, Songyang Lao, Song Wu, and Michael S. Lew. 2016. Deep learning for visual understanding: A review. Neurocomputing. https://doi.org/10.1016/j.neucom.2015.09.116
[11]
Wei Hao. 2021. Auxiliary basketball training system based on big data. World Automation Congress Proceedings 2021-August: 61–64. https://doi.org/10.23919/WAC50355.2021.9559536
[12]
Ryunosuke Kurose, Masaki Hayashi, Takeo Ishii, and Yoshimitsu Aoki. 2018. Player pose analysis in tennis video based on pose estimation. 2018 International Workshop on Advanced Image Technology, IWAIT 2018: 1–4. https://doi.org/10.1109/IWAIT.2018.8369762
[13]
Scott de Lahunta and Florian Jenett. 2016. Making digital choreographic objects interrelate. In Performing the Digital. transcript Verlag, 63–80. https://doi.org/10.14361/9783839433553-003
[14]
Catherine Nguoi Chui Lam and Hadina Habil. 2021. The Use of Video Annotation in Education: A Review. Asian Journal of University Education 17, 4: 84–94. https://doi.org/10.24191/ajue.v17i4.16208
[15]
Yi Shan Lan, Shih Wei Sun, Huang Chia Shih, Kai Lung Hua, and Pao Chi Chang. 2018. O-Shooting: An Orientation-based Basketball Shooting Mixed Reality Game Based on Environment 3D Scanning and Object Positioning. In 2018 IEEE 7th Global Conference on Consumer Electronics, GCCE 2018, 109–113. https://doi.org/10.1109/GCCE.2018.8574481
[16]
Narelle Lemon, Meg Colasante, Karen Corneille, and Kathy Douglas. 2013. Video annotation for collaborative connections to learning: Case studies from an Australian higher education context. Cutting-Edge Technologies in Higher Education 6, PARTF: 181–214. https://doi.org/10.1108/S2044-9968(2013)000006F010
[17]
Yung Che Li, Ching Tang Chang, Chin Chang Cheng, and Yu Len Huang. 2021. Baseball Swing Pose Estimation Using OpenPose. 2021 IEEE International Conference on Robotics, Automation and Artificial Intelligence, RAAI 2021: 6–9. https://doi.org/10.1109/RAAI52226.2021.9507807
[18]
Engin Mendi, Hélio B. Clemente, and Coskun Bayrak. 2013. Sports video summarization based on motion analysis. Computers and Electrical Engineering 39, 3: 790–796. https://doi.org/10.1016/j.compeleceng.2012.11.020
[19]
Cai Ning. 2019. Design and research of motion video image analysis system in sports training. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-019-7374-1
[20]
Katerina El Raheb, Theofilos Mailis, Vladislav Ryzhikov, Nicolas Papapetrou, and Yannis Ioannidis. 2017. BalOnSe: Temporal Aspects of Dance Movement and Its Ontological Representation. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 49–64. https://doi.org/10.1007/978-3-319-58451-5_4
[21]
Evan F. Risko, Tom Foulsham, Shane Dawson, and Alan Kingstone. 2013. The collaborative lecture annotation system (CLAS): A new TOOL for distributed learning. IEEE Transactions on Learning Technologies 6, 1: 4–13. https://doi.org/10.1109/TLT.2012.15
[22]
Anna Rizzo, Katerina El Raheb, Sarah Whatley, Rosa Maria Cisneros, Massimiliano Zanoni, Antonio Camurri, Vladimir Viro, Jean Marc Matos, Stefano Piana, Michele Buccoli, Amalia Markatzi, Pablo Palacio, Oshri Even Zohar, Augusto Sarti, Yannis Ioannidis, and Edwin Morley Fletcher. 2018. WhoLoDancE: Whole-body interaction learning for dance education. In CEUR Workshop Proceedings, 41–50.
[23]
Rui Rodrigues, Rui Madeira, and Nuno Correia. 2021. Studying Natural User Interfaces for Smart Video Annotation towards Ubiquitous Environments; Studying Natural User Interfaces for Smart Video Annotation towards Ubiquitous Environments. 11. https://doi.org/10.1145/3490632.3490672
[24]
Huang Chia Shih. 2018. A Survey of Content-Aware Video Analysis for Sports. IEEE Transactions on Circuits and Systems for Video Technology 28, 5: 1212–1231. https://doi.org/10.1109/TCSVT.2017.2655624
[25]
João Silva, Diogo Cabral, Carla Fernandes, and Nuno Correia. 2012. Real-time annotation of video objects on tablet computers. In Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia - MUM ’12, 1. https://doi.org/10.1145/2406367.2406391
[26]
Vikash Singh, Celine Latulipe, Erin Carroll, and Danielle Lottridge. 2011. The choreographer's notebook. In Proceedings of the 8th ACM conference on Creativity and cognition - C&C ’11, 197. https://doi.org/10.1145/2069618.2069653
[27]
Dave Towey, David Foster, Filippo Gilardi, Paul Martin, Andrew White, Yiru Jiang, Yichen Pan, and Yu Qu. 2017. Students as partners in a multi-media note-taking app development: Best practices. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017, 334–335. https://doi.org/10.1109/ICSE-C.2017.58
[28]
Jianbo Wang, Kai Qiu, Houwen Peng, Jianlong Fu, and Jianke Zhu. 2019. AI Coach: Deep Human Pose Estimation and Analysis for Personalized Athletic Training Assistance. 374–382. https://doi.org/10.1145/3343031.3350910
[29]
Zhiwen Wang, Pengtao Wang, Lianyuan Jiang, Bowen Tang, Canlong Zhang, and Zhenghuan Hu. 2018. Analysis of influencing factors of shooting rate based on trajectory prediction of the basketball. Proceedings - 2017 14th Web Information Systems and Applications Conference, WISA 2017 2018-Janua: 176–180. https://doi.org/10.1109/WISA.2017.18
[30]
Peter Wittenburg, Hennie Brugman, Albert Russel, Alex Klassmann, and Han Sloetjes. 2006. ELAN: A professional framework for multimodality research. In Proceedings of the 5th International Conference on Language Resources and Evaluation, LREC 2006, 1556–1559.

Cited By

View all
  • (2023)Joint graph convolution networks and transformer for human pose estimation in sports technique analysisJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2023.10181935:10(101819)Online publication date: Dec-2023
  • (2023)Sample-Based Human Movement Detection for Interactive Videos Applied to Performing ArtsHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42286-7_32(567-587)Online publication date: 28-Aug-2023

Index Terms

  1. Video Annotation Tool using Human Pose Estimation for Sports Training

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    MUM '22: Proceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia
    November 2022
    315 pages
    ISBN:9781450398206
    DOI:10.1145/3568444
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 29 December 2022

    Check for updates

    Author Tags

    1. Human Pose and Motion
    2. People Tracking
    3. Performing Sports
    4. Video Analysis
    5. Video Annotation

    Qualifiers

    • Poster
    • Research
    • Refereed limited

    Conference

    MUM 2022

    Acceptance Rates

    Overall Acceptance Rate 190 of 465 submissions, 41%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)49
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 27 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Joint graph convolution networks and transformer for human pose estimation in sports technique analysisJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2023.10181935:10(101819)Online publication date: Dec-2023
    • (2023)Sample-Based Human Movement Detection for Interactive Videos Applied to Performing ArtsHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42286-7_32(567-587)Online publication date: 28-Aug-2023

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media