Abstract
In this paper, we study depth reconstruction via RGB-based, Sparse-Depth, and RGBd approaches. We showed that combination of RGB and Sparse Depth approach in RGBd scenario provides the best results. We also proved that the models performance can be further tuned via proper selection of architecture blocks and number of depth points guiding RGB-to-depth reconstruction. We also provide real-time architecture for depth estimation that is on par with state-of-the-art real-time depth reconstruction methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. arXiv preprint arXiv:1406.2283 (2014)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361. IEEE (2012)
Karsch, K., Liu, C., Kang, S.B.: Depth extraction from video using non-parametric sampling. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 775–788. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33715-4_56
Korinevskaya, A., Makarov, I.: Fast depth map super-resolution using deep neural network. In: 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 117–122. IEEE (2018)
Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., Navab, N.: Deeper depth prediction with fully convolutional residual networks. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 239–248. IEEE (2016)
Li, Q., et al.: Deep learning based monocular depth prediction: Datasets, methods and applications. arXiv preprint arXiv:2011.04123 (2020)
Liu, F., Shen, C., Lin, G.: Deep convolutional neural fields for depth estimation from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5162–5170 (2015)
Ma, F., Karaman, S.: Sparse-to-dense: depth prediction from sparse depth samples and a single image. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 4796–4803. IEEE (2018)
Makarov, I., Aliev, V., Gerasimova, O.: Semi-dense depth interpolation using deep convolutional neural networks. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1407–1415. MM ’17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3123266.3123360
Makarov, I., Aliev, V., Gerasimova, O., Polyakov, P.: Depth map interpolation using perceptual loss. In: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), pp. 93–94. IEEE (2017)
Makarov, I., Korinevskaya, A., Aliev, V.: Fast semi-dense depth map estimation. In: Proceedings of the 2018 ACM Workshop on Multimedia for Real Estate Tech, pp. 18–21 (2018)
Makarov, I., Korinevskaya, A., Aliev, V.: Sparse depth map interpolation using deep convolutional neural networks. In: 2018 41st International Conference on Telecommunications and Signal Processing (TSP), pp. 1–5. IEEE (2018)
Makarov, I., Korinevskaya, A., Aliev, V.: Super-resolution of interpolated downsampled semi-dense depth map. In: Proceedings of the 23rd International ACM Conference on 3D Web Technology, pp. 1–2 (2018)
Makarov, I., et al.: On reproducing semi-dense depth map reconstruction using deep convolutional neural networks with perceptual loss. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 1080–1084 (2019)
Maslov, D., Makarov, I.: Online supervised attention-based recurrent depth estimation from monocular video. Peer J. Comput. Sci. 6, e317 (2020)
Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: Orb-slam: a versatile and accurate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)
Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)
Saxena, A., Chung, S.H., Ng, A.Y., et al.: Learning depth from single monocular images. NIPS 18, 1–8 (2005)
Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33715-4_54
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Maslov, D., Makarov, I. (2021). Fast Depth Reconstruction Using Deep Convolutional Neural Networks. In: Rojas, I., Joya, G., Català , A. (eds) Advances in Computational Intelligence. IWANN 2021. Lecture Notes in Computer Science(), vol 12861. Springer, Cham. https://doi.org/10.1007/978-3-030-85030-2_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-85030-2_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85029-6
Online ISBN: 978-3-030-85030-2
eBook Packages: Computer ScienceComputer Science (R0)