Abstract
Resolving depth from monocular RGB image has been a long-standing task in computer vision and robotics. Recently, deep learning based methods has become a popular algorithm on depth estimation. Most existing learning based methods take image-pair as input and utilize feature matching across frames to resolve depth. However, two-frame methods require sufficient and static camera motion to reach optimal performance, while camera motion is usually uncontrollable in most application scenarios. In this paper we propose a recurrent neural network based depth estimation network. With the ability of taking multiple images as input, recurrent neural network will decide by itself which image to reference during estimation. We train a u-net like network architecture which utilizes convolutional LSTM in the encoder. We demonstrate our proposed method with the TUM RGB-D dataset, where our proposed method shows the ability of estimating depth with various sequence lengths as input.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Pizzoli, M., Forster, C., Scaramuzza, D.: REMODE: probabilistic, monocular dense reconstruction in real time. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 2609–2616, May 2014
Zhou, T., Brown, M., Snavely, N., Lowe, D.G.: Unsupervised learning of depth and ego-motion from video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1851–1858, June 2017
Gwn Lore, K., Reddy, K., Giering, M., Bernal, E.A.: Generative adversarial networks for depth map estimation from RGB video. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1177–1185 (2018)
Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., Brox, T.: Demon: depth and motion network for learning monocular stereo. In: Proceedings of IEEE Conference on computer vision and pattern recognition, vol. 5, p. 6, Jul 2017
Wang, K., Shen, S.: MVDepthNet: real-time multiview depth estimation neural network. In: Proceedings of International Conference on 3D Vision (2018)
Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems, pp. 802–810 (2015)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241, October 2015
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: Proceedings of IEEE International Conference on Intelligent Robots and Systems, pp. 573–580, Oct 2012
Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. In: Advances in Neural Information Processing Systems, pp. 2366–2374 (2014)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361 (2012)
Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Proceedings of European Conference on Computer Vision (2012)
Acknowledgments
The authors would like to thank the Ministry of Science and Technology, Taiwan, R.O.C. for financially supporting this research under grants MOST 107-2218-E-003-003-, MOST 107-2218-E-110-004-, MOST 105-2221-E-110-094-MY3 and MOST 106-2221-E-110-083-MY2.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Yeh, CH., Huang, YP., Lin, CY., Lin, MH. (2020). Learning Depth from Monocular Sequence with Convolutional LSTM Network. In: Barolli, L., Nishino, H., Enokido, T., Takizawa, M. (eds) Advances in Networked-based Information Systems. NBiS - 2019 2019. Advances in Intelligent Systems and Computing, vol 1036. Springer, Cham. https://doi.org/10.1007/978-3-030-29029-0_48
Download citation
DOI: https://doi.org/10.1007/978-3-030-29029-0_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29028-3
Online ISBN: 978-3-030-29029-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)