Abstract
Registering a soccer field image into the unified standard template model image can provide preconditions for semantic analysis of soccer videos. In order to complete the task of soccer field registration, the soccer field marker lines need to be detected, which is a challenging problem. In order to solve the problem, we propose a soccer field registration framework based on geometric constraint and deep learning method. We construct a multi-task learning network to realize marker lines detection, homography matrix calculation and soccer field registration. Firstly, the input image is preprocessed to remove the background and occlusion areas and extract the marker lines to obtain the edge map of the soccer field; then, we extract the features on the edge map and on the standard template model image to calculate the homography matrix. In this paper, a two-stage deep training network is proposed. The first stage mainly completes the soccer field marker lines detection and the initial calculation of the homography matrix. The second stage mainly completes the optimization of the homography matrix using geometric constraint, which can provide more accurate homography matrix calculation. We propose to integrate the geometric constraint of the marker lines into the multi-task learning network, in which structural loss is constructed to import prior information such as the shape and direction of the marker lines of the soccer field. We evaluate our method on the World Cup dataset to show its performance against the state-of-the-art methods.
Supported by the Open Projects Program of National Laboratory of Pattern Recognition (No.202100009), the Fundamental Research Funds for Central Universities (No. 2021TD006).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cuevas, C., Quilon, D., Garcia, N.: Automatic soccer field of play registration. Pattern Recogn. 103, 107278 (2020)
Bu, J., Lao, S., Bai, L.: Automatic line mark recognition and its application in camera calibration in soccer video. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2011)
Dong, H., Prasad, D.K., Chen, I.-M.: Accurate detection of ellipses with false detection control at video rates using a gradient analysis. Pattern Recogn. 81, 112–130 (2018)
Hess, R., Fern, A.: Improved video registration using non distinctive local image features. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2007). https://doi.org/10.1109/CVPR.2007.382989
Lu, W.-L., Ting, J.-A., Little, J.J., Murphy, K.P.: Learning to track and identify players from broadcast sports videos. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 35(7), 1704–1716 (2013)
Mukhopadhyay, P., Chaudhuri, B.B.: A survey of Hough transform. Pattern Recogn. 48(3), 993–1010 (2015)
Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 20(2), 91–110 (2004)
Pajdla, J.M.C.U.: Robust wide-baseline stereo from maximally stable extremal regions. Image Vis. Comput. 22(10), 761–767 (2004)
Brachmann, E., et al.: DSAC-Differentiable RANSAC for camera localization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2492–2500 (2017). https://doi.org/10.1109/CVPR.2017.267
Bochkovskiy, A., Wang, C.-Y., Liao, H.: YOLOv4: optimal speed and accuracy of object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Qin, Z., Wang, H., Li, X.: Ultra fast structure-aware deep lane detection. In: European Conference on Computer Vision (ECCV), pp. 1–16 (2020)
Pan, X., Shi, J., Luo, P., Wang, X., Tang, X.: Spatial as deep: spatial CNN for traffic scene understanding. In: AAAI Conference on Artificial Intelligence (2018)
Citraro, L.: Real-time camera pose estimation for sports fields. Mach. Vis. Appl. 31(16), 1–12 (2020)
Chen, J., Little, J.J.: Sports camera calibration via synthetic data. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Homayounfar, N., Fidler, S., Urtasun, R.: Sports field localization via deep structured models. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4012–4020 (2017). https://doi.org/10.1109/CVPR.2017.427
Sharma, R.A., Bhat, B., Gandhi, V., Jawahar, C.V.: Automated top view registration of broadcast soccer videos. In: IEEE Winter Conference on Applications of Computer Vision, pp. 305–313, (2018). https://doi.org/10.1109/WACV.2018.00040
Wei, J., Camilo, J., Higuera, G., Angles, B., Javan, W.S.M., Yi, K.M.: Optimizing through learned errors for accurate sports field registration. In: IEEE Winter Conference on Applications of Computer Vision (WACV) (2020). https://doi.org/10.1109/WACV45572.2020.9093581
Gupta, A., Little, J.J., Woodham, R.: Using line and ellipse features for rectification of broadcast hockey video. In: Canadian Conference on Computer and Robot Vision, pp. 32–39 (2011). https://doi.org/10.1109/CRV.2011.12
Puwein, J., Ziegler, R., Vogel, J., Pollefeys, M.: Robust multi-view camera calibration for wide-baseline camera networks. In: IEEE Winter Conference on Applications of Computer Vision, pp. 321–328 (2011). https://doi.org/10.1109/WACV.2011.5711521
Detone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: self-supervised interest point detection and description. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop on Deep Learning for Visual SLAM (2018). https://doi.org/10.1109/CVPRW.2018.00060
Yi, K.M., Trulls, E., Lepetit, V., Fua, P.: LIFT: learned invariant feature transform. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 467–483. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_28
Verdie, Y., Yi, K.M., Fua, P., Lepetit, V.: TILDE: a temporally invariant learned detector. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5279–5288 (2015). https://doi.org/10.1109/CVPR.2015.7299165
Yan, Q., Xu, Y., Yang, X., Nguyen, T.: HEASK: robust homography estimation based on appearance similarity and keypoint correspondences. Pattern Recogn. 47(1), 368–387 (2014)
DeTone, D., Malisiewicz, T., Rabinovich, A.: Deep image homography estimation. In: RSS Workshop on Limits and Potentials of Deep Learning in Robotics (2016)
Nguyen, T., Chen, S.W., Shivakumar, S.S., Taylor, C.J., Kumar, V.: Unsupervised deep homography: a fast and robust homography estimation model. IEEE Robot. Autom. Lett. 3(3), 2346–2353 (2018). https://doi.org/10.1109/LRA.2018.2809549
Sha, L., Hobbs, J., Felsen, P., Wei, X., Lucey, P., Ganguly, S.: End-to-end camera calibration for broadcast videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13624–13633 (2020). https://doi.org/10.1109/CVPR42600.2020.01364
Zhang, J., Xu, Y., Ni, B., Duan, Z.: Geometric constrained joint lane segmentation and lane boundary detection. In: European Conference on Computer Vision (ECCV), pp. 486–502 (2018)
Kendall, A., Grimes, M., Cipolla, R.: PoseNet: a convolutional network for real-time 6-DOF camera relocalization. In: International Conference on Computer Vision (ICCV), pp. 2938–2946 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, P., Li, J., Zong, S., Zhang, K. (2021). Soccer Field Registration Based on Geometric Constraint and Deep Learning Method. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13020. Springer, Cham. https://doi.org/10.1007/978-3-030-88007-1_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-88007-1_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88006-4
Online ISBN: 978-3-030-88007-1
eBook Packages: Computer ScienceComputer Science (R0)