Skip to main content
SpringerLink
Log in

Deep Two-Stage LiDAR Depth Completion

  • Conference paper
  • First Online:
Computer Vision and Image Processing (CVIP 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1568))

Included in the following conference series:

  • 832 Accesses

Abstract

LiDAR depth completion aims at accurately estimating dense depth maps from sparse and noisy LiDAR depth scans, often with the aid of the color image. However, most of the existing deep learning-based LiDAR depth completion approaches focus on learning one-stage networks with computationally intensive RGB-D fusion strategies to compensate for the prediction errors. To eliminate such drawbacks, we have explored a simple yet effective two-stage learning framework where the former stage generates a coarse dense output which is processed in the latter stage to produce a fine dense depth map. The refined dense depth map is obtained at the output of the second stage by employing iterative feedback mechanism that removes any ambiguity associated with a single feed-forward network. Our two-stage learning mechanism allows for simple RGB-D fusion operations devoid of high computational overload. Experiments conducted on the KITTI depth completion benchmark validate the efficacy of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chen,, Y., Yang, B., Liang, M., Urtasun, R.: Learning joint 2D–3D representations for depth completion. In: IEEE International Conference on Computer Vision, pp. 10023–10032 (2019)

    Google Scholar 

  2. Cheng, X., Wang, P., Guan, C., Yang, R.: CSPN++: learning context and resource aware convolutional spatial propagation networks for depth completion. In: AAAI Conference on Artificial Intelligence. vol. 34, pp. 10615–10622 (2020)

    Google Scholar 

  3. Cheng, X., Wang, P., Yang, R.: Learning depth with convolutional spatial propagation network. IEEE Trans. Pattern Anal. Mach. Intell. 42(10), 2361–2379 (2019)

    Article  Google Scholar 

  4. Chodosh, N., Wang, C., Lucey, S.: Deep convolutional compressed sensing for LiDAR depth completion. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11361, pp. 499–513. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20887-5_31

    Chapter  Google Scholar 

  5. Eldesokey, A., Felsberg, M., Khan, F.S.: Propagating confidences through CNNs for sparse data regression. In: British Machine Vision Conference, p. 14 (2018)

    Google Scholar 

  6. Eldesokey, A., Felsberg, M., Khan, F.S.: Confidence propagation through CNNs for guided sparse depth regression. IEEE Trans. Pattern Anal. Mach. Intell. 42(10), 2423–2436 (2019)

    Article  Google Scholar 

  7. Ferstl, D., Reinbacher, C., Ranftl, R., Rüther, M., Bischof, H.: Image guided depth upsampling using anisotropic total generalized variation. In: IEEE International Conference on Computer Vision, pp. 993–1000 (2013)

    Google Scholar 

  8. Hambarde, P., Dudhane, A., Patil, P.W., Murala, S., Dhall, A.: Depth estimation from single image and semantic prior. In: IEEE International Conference on Image Processing, pp. 1441–1445 (2020)

    Google Scholar 

  9. Hambarde, P., Murala, S.: S2DNET: depth estimation from single image and sparse samples. IEEE Trans. Comput. Imaging 6, 806–817 (2020)

    Article  Google Scholar 

  10. Huang, Z., Fan, J., Cheng, S., Yi, S., Wang, X., Li, H.: HMS-NET: hierarchical multi-scale sparsity-invariant network for sparse depth completion. IEEE Trans. Image Process. 29, 3429–3441 (2019)

    Article  Google Scholar 

  11. Imran, S., Liu, X., Morris, D.: Depth completion with twin surface extrapolation at occlusion boundaries. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2583–2592 (2021)

    Google Scholar 

  12. Imran, S., Long, Y., Liu, X., Morris, D.: Depth coefficients for depth completion. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 12438–12447 (2019)

    Google Scholar 

  13. Khan, M.F.F., Troncoso Aldas, N.D., Kumar, A., Advani, S., Narayanan, V.: Sparse to dense depth completion using a generative adversarial network with intelligent sampling strategies. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 5528–5536 (2021)

    Google Scholar 

  14. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)

    Google Scholar 

  15. Knutsson, H., Westin, C.F.: Normalized and differential convolution. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 515–523 (1993)

    Google Scholar 

  16. Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., Navab, N.: Deeper depth prediction with fully convolutional residual networks. In: International Conference on 3D Vision, pp. 239–248 (2016)

    Google Scholar 

  17. Lee, B.U., Lee, K., Kweon, I.S.: Depth completion using plane-residual representation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 13916–13925 (2021)

    Google Scholar 

  18. Liu, L., Song, X., Lyu, X., Diao, J., Wang, M., Liu, Y., Zhang, L.: FCFR-Net: feature fusion based coarse-to-fine residual learning for depth completion. In: AAAI Conference on Artificial Intelligence, vol. 35, pp. 2136–2144 (2021)

    Google Scholar 

  19. Ma, F., Cavalheiro, G.V., Karaman, S.: Self-supervised sparse-to-dense: self-supervised depth completion from lidar and monocular camera. In: International Conference on Robotics and Automation, pp. 3288–3295 (2019)

    Google Scholar 

  20. Ma, F., Karaman, S.: Sparse-to-dense: depth prediction from sparse depth samples and a single image. In: IEEE International Conference on Robotics and Automation, pp. 1–8 (2018)

    Google Scholar 

  21. Qiu, J., et al.: DeepLiDAR: deep surface normal guided depth prediction for outdoor scene from sparse LiDAR data and single color image. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3313–3322 (2019)

    Google Scholar 

  22. Shivakumar, S.S., Nguyen, T., Miller, I.D., Chen, S.W., Kumar, V., Taylor, C.J.: DFuseNet: deep fusion of RGB and sparse depth information for image guided dense depth completion. In: IEEE Intelligent Transportation Systems Conference, pp. 13–20 (2019)

    Google Scholar 

  23. Tsuji, Y., Chishiro, H., Kato, S.: Non-guided depth completion with adversarial networks. In: International Conference on Intelligent Transportation Systems. pp. 1109–1114 (2018)

    Google Scholar 

  24. Uhrig, J., Schneider, N., Schneider, L., Franke, U., Brox, T., Geiger, A.: Sparsity invariant CNNs. In: International Conference on 3D Vision, pp. 11–20 (2017)

    Google Scholar 

  25. Van Gansbeke, W., Neven, D., De Brabandere, B., Van Gool, L.: Sparse and noisy LiDAR completion with RGB guidance and uncertainty. In: International Conference on Machine Vision Applications, pp. 1–6 (2019)

    Google Scholar 

  26. Xu, Y., Zhu, X., Shi, J., Zhang, G., Bao, H., Li, H.: Depth completion from sparse LiDAR data with depth-normal constraints. In: IEEE International Conference on Computer Vision, pp. 2811–2820 (2019)

    Google Scholar 

  27. Zhang, Y., Funkhouser, T.A.: Deep depth completion of a single RGB-D image. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 175–185 (2018)

    Google Scholar 

  28. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2472–2481 (2018)

    Google Scholar 

  29. Zhou, T., Brown, M., Snavely, N., Lowe, D.G.: Unsupervised learning of depth and ego-motion from video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1851–1858 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Technology, Kharagpur, Kharagpur, India

    Moushumi Medhi & Rajiv Ranjan Sahay

Authors
  1. Moushumi Medhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajiv Ranjan Sahay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Moushumi Medhi .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Roorkee, Roorkee, India

    Balasubramanian Raman

  2. Indian Institute of Technology Ropar, Ropar, India

    Subrahmanyam Murala

  3. Jadavpur University, Kolkata, India

    Ananda Chowdhury

  4. Indian Institute of Technology Ropar, Ropar, India

    Abhinav Dhall

  5. Indian Institute of Technology Ropar, Ropar, India

    Puneet Goyal

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Medhi, M., Sahay, R.R. (2022). Deep Two-Stage LiDAR Depth Completion. In: Raman, B., Murala, S., Chowdhury, A., Dhall, A., Goyal, P. (eds) Computer Vision and Image Processing. CVIP 2021. Communications in Computer and Information Science, vol 1568. Springer, Cham. https://doi.org/10.1007/978-3-031-11349-9_44

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-11349-9_44

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-11348-2

  • Online ISBN: 978-3-031-11349-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation