Skip to main content
SpringerLink
Log in

MotionRFCN: Motion Segmentation Using Consecutive Dense Depth Maps

  • Conference paper
  • First Online:
PRICAI 2019: Trends in Artificial Intelligence (PRICAI 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11671))

Included in the following conference series:

Abstract

It is important to enable autonomous robots to detect or segment moving objects in dynamic scenes as they must perform collision-free navigation. Motion segmentation from a moving platform is challenging due to the dual motion caused by the background and the moving objects. Existing approaches for motion segmentation either have long multistage pipelines which are inefficient for real-time application or utilize optical flow which is sensitive to environment. In this paper, this challenging task is tackled by constructing spatiotemporal features from two consecutive dense depth maps. Depth maps can be generated either by LiDaR scans data or stereo vision algorithms. The core of the proposed approach is a fully convolutional network with inserted Gated-Recurrent-Units, denoted as MotionRFCN. We also create a publicly available dataset (KITTI-MoSeg) which contains more than 2000 frames with motion annotations. Qualitative and quantitative evaluation of MotionRFCN are presented to demonstrate its state-of-the-art performance on the KITTI dataset. The basic MotionRFCN can run in real time and segment moving objects whether the platform is stationary or moving. To the best of our knowledge, the proposed method is the first to implement motion segmentation with only dense depth maps inputs.

Supported in part by the Natural Science Foundation of China under Grant U1613218 and 61722309.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Xiao, J., Shah, M.: Motion layer extraction in the presence of occlusion using graph cut. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1644–1659 (2005)

    Article  Google Scholar 

  2. Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: Computer Vision and Pattern Recognition (2009)

    Google Scholar 

  3. Kundu, A., Krishna, K.M., Sivaswamy, J.: Moving object detection by multi-view geometric techniques from a single camera mounted robot. In: International Conference on Intelligent Robots and Systems (2009)

    Google Scholar 

  4. Sheikh, Y., Javed, O., Kanade, T.: Background subtraction for freely moving cameras. In: International Conference on Computer Vision (2009)

    Google Scholar 

  5. Lenz, P., Ziegler, J., Geiger, A., Roser, M.: Sparse scene flow segmentation for moving object detection in urban environments. In: Intelligent Vehicles Symposium (2011)

    Google Scholar 

  6. Namdev, R.K., Kundu, A., Krishna, K.M., Jawahar, C.V.: Motion segmentation of multiple objects from a freely moving monocular camera. In: International Conference on Robotics and Automation (2012)

    Google Scholar 

  7. Narayana, M., Hanson, A., Learned-Miller, E.: Coherent motion segmentation in moving camera videos using optical flow orientations. In: International Conference on Computer Vision (2013)

    Google Scholar 

  8. Romero-Cano, V., Nieto, J.I.: Stereo-based motion detection and tracking from a moving platform. In: Intelligent Vehicles Symposium (2013)

    Google Scholar 

  9. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32, 1231–1237 (2013)

    Article  Google Scholar 

  10. Reddy, N.D., Singhal, P., Krishna, K.M.: Semantic motion segmentation using dense CRF formulation. In: Proceedings of the Indian Conference on Computer Vision Graphics and Image Processing (2014)

    Google Scholar 

  11. Lin, T.H., Wang, C.C.: Deep learning of spatio-temporal features with geometric-based moving point detection for motion segmentation. In: International Conference on Robotics and Automation (2014)

    Google Scholar 

  12. Ochs, P., Malik, J., Brox, T.: Segmentation of moving objects by long term video analysis. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1187–1200 (2014)

    Article  Google Scholar 

  13. Fragkiadaki, K., Arbelaez, P., Felsen, P., Malik, J.: Learning to segment moving objects in videos. In: Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  14. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  15. Ballas, N., Yao, L., Pal, C., Courville, A.: Delving deeper into convolutional networks for learning video representations. arXiv preprint arXiv:1511.06432 (2015)

  16. Qiu, F., Yang, Y., Li, H., Fu, M., Wang, S.: Semantic motion segmentation for urban dynamic scene understanding. In: International Conference on Automation Science and Engineering (2016)

    Google Scholar 

  17. Kao, J.Y., Tian, D., Mansour, H., Vetro, A., Ortega, A.: Moving object segmentation using depth and optical flow in car driving sequences. In: International Conference on Image Processing (2016)

    Google Scholar 

  18. Drayer, B., Brox, T.: Object detection, tracking, and motion segmentation for object-level video segmentation. arxiv:1608.03066 (2016)

  19. Tourani, S., Krishna, K.M.: Using in-frame shear constraints for monocular motion segmentation of rigid bodies. J. Intell. Robotic Syst. 82(2), 237–255 (2016)

    Article  Google Scholar 

  20. Chen, T., Lu, S.: Object-level motion detection from moving cameras. IEEE Trans. Circuits Syst. Video Technol. 27, 2333–2343 (2016)

    Article  Google Scholar 

  21. Bideau, P., Learned-Miller, E.: It’s moving! a probabilistic model for causal motion segmentation in moving camera videos. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 433–449. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_26

    Chapter  Google Scholar 

  22. Postica, G., Romanoni, A., Matteucci, M.: Robust moving objects detection in lidar data exploiting visual cues. In: International Conference on Intelligent Robots and Systems (2016)

    Google Scholar 

  23. Dewan, A., Caselitz, T., Tipaldi, G.D., Burgard, W.: Motion-based detection and tracking in 3D LiDAR scans. In: International Conference on Robotics and Automation (2016)

    Google Scholar 

  24. Haque, N., Reddy, N.D., Krishna, K.M.: Joint semantic and motion segmentation for dynamic scenes using deep convolutional networks. In: International Conference on Computer Vision Theory and Applications (2017)

    Google Scholar 

  25. Vertens, J., Valada, A., Burgard, W.: SMSnet: semantic motion segmentation using deep convolutional neural networks. In: International Conference on Intelligent Robots and Systems (2017)

    Google Scholar 

  26. Wehrwein, S., Szeliski, R.: Video segmentation with background motion models. In: British Machine Vision Conference (2017)

    Google Scholar 

  27. Tokmakov, P., Alahari, K., Schmid, C.: Learning motion patterns in videos. In: Computer Vision and Pattern Recognition (2017)

    Google Scholar 

  28. Haque, N., Reddy, N.D., Krishna, M.: Temporal semantic motion segmentation using spatio temporal optimization. In: Pelillo, M., Hancock, E. (eds.) EMMCVPR 2017. LNCS, vol. 10746, pp. 93–108. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78199-0_7

    Chapter  Google Scholar 

  29. Siam, M., Mahgoub, H., Zahran, M., Yogamani, S., Jagersand, M.: Motion and appearance based moving object detection network for autonomous driving. arXiv:1709.04821v2 (2017)

  30. He, K.M., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: International Conference on Computer Vision (2017)

    Google Scholar 

  31. Sun, D., Yang, X., Liu, M., Kautz, J.: PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  32. Cordts, M., Omaran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R.: The cityscapes dataset for semantic urban scene understanding. In: Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  33. Zheng, L., Wang, H., Chen, W.: A fast 3D object recognition pipeline in cluttered and occluded scenes. In: Huang, Y.A., Wu, H., Liu, H., Yin, Z. (eds.) ICIRA 2017. LNCS (LNAI), vol. 10463, pp. 588–598. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65292-4_51

    Chapter  Google Scholar 

  34. Yang, B., Wang, H., Chen, W., Liang, Y.: Vision-based automatic hair follicular unit separation. In: Liu, H., Kubota, N., Zhu, X., Dillmann, R., Zhou, D. (eds.) ICIRA 2015. LNCS (LNAI), vol. 9246, pp. 273–284. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22873-0_24

    Chapter  Google Scholar 

  35. Xu, L., Wang, H., Chen, W., Wang, J.: Light intensity optimization in trajectory planning of inspection robot. In: Liu, H., Kubota, N., Zhu, X., Dillmann, R., Zhou, D. (eds.) ICIRA 2015. LNCS (LNAI), vol. 9245, pp. 297–308. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22876-1_26

    Chapter  Google Scholar 

  36. Wang, H., Lai, Y., Chen, W.: The time optimal trajectory planning with limitation of operating task for the Tokamak inspecting manipulator. Fusion Eng. Des. 113, 57–65 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Automation, Shanghai Jiao Tong University, Shanghai, China

    Yiling Liu & Hesheng Wang

Authors
  1. Yiling Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hesheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hesheng Wang .

Editor information

Editors and Affiliations

  1. Department of Computing, Macquarie University, Sydney, NSW, Australia

    Abhaya C. Nayak

  2. RIKEN Center for Integrative Medical Sciences, Yokohama, Japan

    Alok Sharma

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, Y., Wang, H. (2019). MotionRFCN: Motion Segmentation Using Consecutive Dense Depth Maps. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11671. Springer, Cham. https://doi.org/10.1007/978-3-030-29911-8_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29911-8_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29910-1

  • Online ISBN: 978-3-030-29911-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation