Abstract
Motion segmentation plays an important role in many applications including autonomous driving, computer vision and robotics. Previous works mainly focus on segmenting objects from seen videos. In this paper, we present a novel approach based on pixel distribution learning for motion segmentation in unseen videos. In particular, optical flow is extracted from consecutive frames to describe motion information. We then randomly permute these modified motion features, which are used as input of a convolution neural network. The random permutation process forces the network to learn the pixels’ distributions rather than local pattern information. Consequently, the proposed approach has a favorable generalization capacity and can be applied for unseen videos. In contrast to previous approaches based on deep learning, the training videos and testing videos of our proposed approach are completely different. Experiments based on videos from the KITTI-MOD dataset demonstrates that the proposed approach achieves promising results and shows potential for better motion segmentation on unseen videos.
Z. Lu and Y. Chen—Equally contributed to this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bideau, P., Learned-Miller, E.: It’s moving! a probabilistic model for causal motion segmentation in moving camera videos. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 433–449. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_26
Dosovitskiy, A., et al.: FlowNet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2758–2766 (2015)
Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)
Hui, T.W., Tang, X., Loy, C.C.: LiteFlowNet: a lightweight convolutional neural network for optical flow estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8981–8989 (2018)
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: FlowNet 2.0: evolution of optical flow estimation with deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2462–2470 (2017)
Jiang, M., Crookes, D.: Video object motion segmentation for intelligent visual surveillance. In: International Machine Vision and Image Processing Conference (IMVIP 2007), pp. 202–202 (2007). https://doi.org/10.1109/IMVIP.2007.7
Keuper, M., Tang, S., Andres, B., Brox, T., Schiele, B.: Motion segmentation & multiple object tracking by correlation co-clustering. IEEE Trans. Pattern Anal. Mach. Intell. 42(1), 140–153 (2018)
Kottinger, J., Almagor, S., Lahijanian, M.: Maps-x: explainable multi-robot motion planning via segmentation. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 7994–8000. IEEE (2021)
Lin, J.F.S., Joukov, V., Kulić, D.: Classification-based segmentation for rehabilitation exercise monitoring. J. Rehabil. Assistive Technol. Eng. 5, 2055668318761523 (2018)
Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision. Vancouver (1981)
Mohamed, E., et al.: Monocular instance motion segmentation for autonomous driving: KITTI instancemotseg dataset and multi-task baseline. In: 2021 IEEE Intelligent Vehicles Symposium (IV), pp. 114–121. IEEE (2021)
Ochs, P., Brox, T.: Object segmentation in video: a hierarchical variational approach for turning point trajectories into dense regions. In: 2011 International Conference on Computer Vision, pp. 1583–1590. IEEE (2011)
Perazzi, F., Pont-Tuset, J., McWilliams, B., Gool, L.V., Gross, M., Sorkine-Hornung, A.: A benchmark dataset and evaluation methodology for video object segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Ranjan, A., Black, M.J.: Optical flow estimation using a spatial pyramid network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4161–4170 (2017)
Siam, M., Mahgoub, H., Zahran, M., Yogamani, S., Jagersand, M., El-Sallab, A.: ModNet: moving object detection network with motion and appearance for autonomous driving. arXiv preprint arXiv:1709.04821 (2017)
Sun, D., Yang, X., Liu, M.Y., Kautz, J.: PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8934–8943 (2018)
Teed, Z., Deng, J.: RAFT: recurrent all-pairs field transforms for optical flow. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 402–419. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_24
Vertens, J., Valada, A., Burgard, W.: SMSnet: semantic motion segmentation using deep convolutional neural networks. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 582–589. IEEE (2017)
Wong, K.Y., Spetsakis, M.E.: Tracking based motion segmentation under relaxed statistical assumptions. Comput. Vis. Image Underst. 101(1), 45–64 (2006)
Wu, X., Ma, J., Sun, Y., Zhao, C., Basu, A.: Multi-scale deep pixel distribution learning for concrete crack detection. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 6577–6583. IEEE (2021)
Zelkowitz, M.: Advances in Computers. Elsevier (2002)
Zhao, C., Basu, A.: Pixel distribution learning for vessel segmentation under multiple scales. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 2717–2721. IEEE (2021)
Zhao, C., Cham, T.L., Ren, X., Cai, J., Zhu, H.: Background subtraction based on deep pixel distribution learning. In: 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2018)
Acknowledgements
The authors would like to thank the reviewers and the following colleagues for helpful comments and discussions: Anup Basu and Chuqing Fu.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lu, Z., Chen, Y., Zhao, C. (2022). Motion Segmentation Based on Pixel Distribution Learning on Unseen Videos. In: Berretti, S., Su, GM. (eds) Smart Multimedia. ICSM 2022. Lecture Notes in Computer Science, vol 13497. Springer, Cham. https://doi.org/10.1007/978-3-031-22061-6_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-22061-6_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-22060-9
Online ISBN: 978-3-031-22061-6
eBook Packages: Computer ScienceComputer Science (R0)