Skip to main content
SpringerLink
Log in

Hybrid Methods for Real-Time Video Sequence Identification of Human Soccer Referee Signals

  • Conference paper
  • First Online:
RoboCup 2023: Robot World Cup XXVI (RoboCup 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14140))

Included in the following conference series:

  • 98 Accesses

Abstract

We evaluate several real-time machine learning and hybrid symbolic algorithms to identify body-gestures of Human Soccer Referees as part of the 2022 RoboCup Soccer Standard Platform League (SPL) Visual Referee Challenge. This challenge, in part, encourages development of software for effective human-robot interactions as the Nao must interpret and respond in real-time to human referee’s gestures, so RoboCup can progress towards it’s 2050 goal. We train and evaluate the real-time accuracy and speed of two Deep Convolutional Neural Networks architectures, OpenPose and Stacked HourGlass, against our novel hybrid machine-learning and symbolic approach on the resource constrained SoftBank Nao V6. Our results show that there remains an important trade-off between the single-image accuracy of Deep Networks, and the execution speed of hybrid when evaluating across a video sequence. Our hybrid method achieved a strong result in 2022 SPL “Visual Referee Challenge”. Our work has been motivated by the requirements for methods in human-robot interaction, where the human perspective of a robot’s behaviour strongly influences the evaluation criteria of software.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The exception to this in 2022 is the use of a whistle for Kick-off, followed by a electronic kick-off signal sent after a 15 s delay.

References

  1. Alpaydin, E.: Introduction to Machine Learning. MIT Press, Cambridge (2014)

    Google Scholar 

  2. Andriluka, M., Pishchulin, L., Gehler, P., Schiele, B.: 2D human pose estimation: new benchmark and state of the art analysis. In: IEEE Conference on Computer Vision and Pattern Recognition (2014)

    Google Scholar 

  3. Anjomshoae, S., Najjar, A.: Explainable agents and robots: results from a systematic. In: 18th International Conference on Autonomous Agents and Multiagent Systems, pp. 1078–1088. Montreal, Canada (2019)

    Google Scholar 

  4. Cao, Z., Hidalgo Martinez, G., Simon, T., Wei, S., Sheikh, Y.A.: Openpose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 7291–7299 (2019)

    Google Scholar 

  5. Committee, R.T.: Robocup standard platform league (NAO) rule book. Pdf, May 2022. https://spl.robocup.org/wp-content/uploads/SPL-Rules-master.pdf

  6. Cruz, N., Lobos-Tsunekawa, K., Ruiz-del Solar, J.: Using convolutional neural networks in robots with limited computational resources: detecting NAO robots while playing soccer. In: Akiyama, H., Obst, O., Sammut, C., Tonidandel, F. (eds.) RoboCup 2017: Robot World Cup XXI. RoboCup 2017. LNCS, vol. 11175, pp. 19–30. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00308-1_2

  7. Graaf, M.M.A.D., Allouch, S.B.: Exploring influencing variables for the acceptance of social robots. Robot. Auton. Syst. 61(12), 1476–1486 (2013)

    Google Scholar 

  8. Jocher, G.: Yolov5 (v6.2) (2022). https://github.com/ultralytics/yolov5

  9. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  10. Leite, I., Martinho, C., Paiva, A.: Social robots for long-term interaction: a survey. Int. J. Soc. Robot. 5(2), 291–308 (2013)

    Article  Google Scholar 

  11. Lin, T., et al.: Microsoft COCO: common objects in context. CoRR (2014)

    Google Scholar 

  12. Menashe, J., et al., Stone, P.: Fast and precise black and white ball detection for robocup soccer. In: Akiyama, H., Obst, O., Sammut, C., Tonidandel, F. (eds.) RoboCup 2017: Robot World Cup XXI. RoboCup 2017. LNCS, vol. 11175, pp. 45–58. Springer, Cham (2017). https://doi.org/10.1007/978-3-030-00308-1_4

  13. Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision – ECCV 2016. ECCV 2016. LNCS, vol. 9912, pp. 483–499 Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29

  14. Poppinga, B., Laue, T.: Jet-net: real-time object detection for mobile robots. In: Chalup, S., Niemueller, T., Suthakorn, J., Williams, M.A. (eds.) RoboCup 2019: Robot World Cup XXIII. RoboCup 2019. LNCS, vol. 11531, pp. 227–240. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35699-6_18

  15. Robotics, S.: Nao - user guide - aldebaran 2.8.7.4 documentation (2022)

    Google Scholar 

  16. Röfer, T., et al.: B-human 2019-complex team play under natural lighting conditions. In: Chalup, S., Niemueller, T., Suthakorn, J., Williams, M.A. (eds.) RoboCup 2019: Robot World Cup XXIII. RoboCup 2019. LNCS, vol. 11531, pp. 646–657. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35699-6_52

  17. Sammut, C., Farid, R., Wicaksono, H., Wiley, T.: Logic-Based Robotics. In: Human-Like Machine Intelligence, pp. 465–486. Oxford University Press, Oxford (2021)

    Google Scholar 

  18. Sheh, R.K.: Different XAI for different HRI. In: AAAI Fall Symposium Series, pp. 114–117. AAAI (2017)

    Google Scholar 

  19. Szemenyei, M., Estivill-Castro, V.: Real-time scene understanding using deep neural networks for RoboCup SPL. In: Holz, D., Genter, K., Saad, M., von Stryk, O. (eds.) RoboCup 2018: Robot World Cup XXII. RoboCup 2018. LNCS, vol. 11374, pp. 96–108. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-27544-0_8

  20. Toshev, A., Szegedy, C.: Deeppose: human pose estimation via deep neural networks. CoRR abs/1312.4659, 1653–1660 (2013)

    Google Scholar 

  21. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, p. I (2001)

    Google Scholar 

  22. Wicaksono, H., Sammut, C., Sheh, R.K.: Towards explainable tool creation by a robot. In: IJCAI-17 Workshop on Explainable AI, pp. 63–67. Melbourne (2017)

    Google Scholar 

Download references

Acknowledgements

We acknowledge the members of our 2022 RoboCup team, RedBackBots, who contributed to the data collection to support this work.

Author information

Authors and Affiliations

  1. School of Computing Technologies, RMIT University, Melbourne, Australia

    Prawal Lohani & Timothy Wiley

Authors
  1. Prawal Lohani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timothy Wiley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Timothy Wiley .

Editor information

Editors and Affiliations

  1. ENIB/Naval Group Pacific, Plouzané, France

    Cédric Buche

  2. University of Naples Federico II, Naples, Italy

    Alessandra Rossi

  3. Bahia State University, Salvador, Brazil

    Marco Simões

  4. University of Miami, Coral Gables, FL, USA

    Ubbo Visser

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lohani, P., Wiley, T. (2024). Hybrid Methods for Real-Time Video Sequence Identification of Human Soccer Referee Signals. In: Buche, C., Rossi, A., Simões, M., Visser, U. (eds) RoboCup 2023: Robot World Cup XXVI. RoboCup 2023. Lecture Notes in Computer Science(), vol 14140. Springer, Cham. https://doi.org/10.1007/978-3-031-55015-7_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-55015-7_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-55014-0

  • Online ISBN: 978-3-031-55015-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation