Abstract
In this paper we present an adaptation module for feature matching based Semi-automatic Video Object Segmentation methods (SVOS). Most current solutions to adapt SVOS methods during inference are slow or inefficient. Feature matching based methods use affinity between a set of reference and query features to segment a target in the current frame based on a reference. We propose an adaptation module working solely with the user supplied mask in the first frame of a video. Our adaptation of the matching module provides more reliable information to the model for segmentation in all the video frames and does not significantly increase inference time. The evaluation on both OVIS and DAVIS 17 datasets shows a significant improvement on the segmentation (respectively \(+2.9\%\) and \(+1\%\) of the Jaccard index). This demonstrates that our adaptation of the feature space provides a better matching between query and reference features.
Supported by France Canada Research Fund.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bhat, G., et al.: Learning what to learn for video object segmentation. In: Proceedings of the CVF European Conference on Computer Vision (2020)
Caelles, S., Maninis, K.K., Pont-Tuset, J., Leal-Taixé, L., Cremers, D., Gool, L.V.: One-shot video object segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2017)
Chen, X., Li, Z., Yuan, Y., Yu, G., Shen, J., Qi, D.: State-aware tracker for real-time video object segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)
Cheng, H.K., Schwing, A.G.: Xmem: long-term video object segmentation with an Atkinson-Shiffrin memory model. In: Proceedings of the CVF European Conference on Computer Vision (2022)
Gao, M., Zheng, F., Yu, J.J., Shan, C., Ding, G., Han, J.: Deep learning for video object segmentation: a review. Artif. Intell. Rev. (2022)
Hu, Y.T., Huang, J.B., Schwing, A.: Videomatch: matching based video object segmentation. In: Proceedings of the CVF European Conference on Computer Vision (2018)
Khoreva, A., Benenson, R., Ilg, E., Brox, T., Schiele, B.: Lucid data dreaming for video object segmentation. Int. J. Comput. Vis. 127(9), 1175–1197 (2019)
Li, X., Ma, C., Wu, B., He, Z., Yang, M.H.: Target-aware deep tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)
Lin, T.Y., et al.: Microsoft coco: common objects in context. In: Proceedings of the CVF European Conference on Computer Vision (2014)
Maninis, K.K., et al.: Video object segmentation without temporal information. IEEE Trans. Pattern Anal. Mach. Intell. (2018)
Oh, S.W., Lee, J.Y., Sunkavalli, K., Kim, S.J.: Fast video object segmentation by reference-guided mask propagation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2018)
Oh, S.W., Lee, J.Y., Xu, N., Kim, S.J.: Video object segmentation using space-time memory networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)
Paul, V., Leibe, B.: Online adaptation of convolutional neural networks for video object segmentation. In: Proceedings of the British Machine Vision Conference (2017)
Pont-Tuset, J., Perazzi, F., Caelles, S., Arbeláez, P., Sorkine-Hornung, A., Gool, L.V.: The 2017 Davis challenge on video object segmentation. In: arXiv preprint arXiv:1704.00675 (2017)
Qi, J., et al.: Occluded video instance segmentation: a benchmark. Int. J. Comput. Vis. (2022)
Robinson, A., Lawin, F.J., Danelljan, M., Khan, F.S., Felsberg, M.: Learning fast and robust target models for video object segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)
Tim, M., Leal-Taixé, L.: Make one-shot video object segmentation efficient again. In: Proceedings of Advances in Neural Information Processing Systems (2020)
Wang, H., Jiang, X., Ren, H., Hu, Y., Bai, S.: Swiftnet: real-time video object segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021)
Xu, N., et al.: Youtube-vos: sequence-to-sequence video object segmentation. In: Proceedings of the CVF European Conference on Computer Vision (2018)
Yuxi, L., Ning, X., Jinlong, P., John, S., Weiyao, L.: Delving into the cyclic mechanism in semi-supervised video object segmentation. In: Proceedings of Advances in Neural Information Processing Systems (2020)
Zhang, Y., Li, L., Wang, W., Xie, R., Song, L., Zhang, W.: Boosting video object segmentation via space-time correspondence learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023)
Zhou, T., Li, J., Li, X., Shao, L.: Target-aware object discovery and association for unsupervised video multi-object segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dubuisson, I., Muselet, D., Ducottet, C., Lang, J. (2023). Fast Context Adaptation for Video Object Segmentation. In: Tsapatsoulis, N., et al. Computer Analysis of Images and Patterns. CAIP 2023. Lecture Notes in Computer Science, vol 14184. Springer, Cham. https://doi.org/10.1007/978-3-031-44237-7_26
Download citation
DOI: https://doi.org/10.1007/978-3-031-44237-7_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44236-0
Online ISBN: 978-3-031-44237-7
eBook Packages: Computer ScienceComputer Science (R0)