Skip to main content
SpringerLink
Log in

Efficient Unsupervised Monocular Depth Estimation with Inter-Frame Depth Interpolation

  • Conference paper
  • First Online:
Book cover Image and Graphics (ICIG 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12890))

Included in the following conference series:

  • 2198 Accesses

Abstract

To alleviate the need of expensive depth annotations, some existing works resort to unsupervised learning methods for depth estimation using monocular videos. To improve the accuracy of the prediction with the relationship of the inter-frame, we present a new unsupervised learning algorithm for monocular depth estimation. In contrast to most existing works, our method predicts depth maps for both target and source frames in each iteration. An inter-frame depth interpolation module is designed, which learns to infer the depth map of the target frame based on those of the source frames. A temporal consistency loss is also proposed to penalize the discrepancy of the predicted depth maps across frames, which not only enforces the coherence of video depth prediction but also provides supervision to source view depth estimation. Similar to the idea of the mutual learning, our method takes full advantages of both target and source views for depth estimation learning. Each module of our proposed method is differentiable, allowing end-to-end training of the whole system. Experiments on the popular KITTI dataset have been conducted and shown that our method performs favorably against state-of-the-art approaches.

Supported by the National Natural Science Foundation of China 61771088.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bian, J.W., et al.: Unsupervised scale-consistent depth and ego-motion learning from monocular video. In: Thirty-Third Conference on Neural Information Processing Systems (NeurIPS), pp. 35–45 (2019)

    Google Scholar 

  2. Casser, V., Pirk, S., Mahjourian, R., Angelova, A.: Depth prediction without the sensors: leveraging structure for unsupervised learning from monocular videos. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8001–8008 (2019)

    Google Scholar 

  3. Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. In: Advances in Neural Information Processing Systems, vol. 27, pp. 2366–2374 (2014)

    Google Scholar 

  4. Garg, R., B.G., V.K., Carneiro, G., Reid, I.: Unsupervised CNN for single view depth estimation: geometry to the rescue. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 740–756. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_45

    Chapter  Google Scholar 

  5. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the kitti dataset. Int. J. Robot. Res. (IJRR) 32(11), 1231–1237 (2013)

    Article  Google Scholar 

  6. Godard, C., Mac Aodha, O., Brostow, G.J.: Unsupervised monocular depth estimation with left-right consistency. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6602–6611 (2017)

    Google Scholar 

  7. Godard, C., Oisin, M.A., Firman, M., Brostow, G.J.: Digging into self-supervised monocular depth estimation. In: The IEEE International Conference on Computer Vision (ICCV), pp. 3827–3837 (2019)

    Google Scholar 

  8. Klodt, M., Vedaldi, A.: Supervising the new with the old: learning SFM from SFM. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 713–728. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_43

    Chapter  Google Scholar 

  9. Liu, F., Shen, C., Lin, G., Reid, I.: Learning depth from single monocular images using deep convolutional neural fields. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2024–2039 (2015)

    Article  Google Scholar 

  10. Mahjourian, R., Wicke, M., Angelova, A.: Unsupervised learning of depth and ego-motion from monocular video using 3D geometric constraints. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5667–5675 (2018)

    Google Scholar 

  11. Mayer, N., et al.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4040–4048 (2016)

    Google Scholar 

  12. Mehta, I., Sakurikar, P., Narayanan, P.: Structured adversarial training for unsupervised monocular depth estimation. In: 2018 International Conference on 3D Vision (3DV), pp. 314–323 (2018)

    Google Scholar 

  13. Poggi, M., Tosi, F., Mattoccia, S.: Learning monocular depth estimation with unsupervised trinocular assumptions. In: 2018 International Conference on 3D Vision (3DV), pp. 324–333 (2018)

    Google Scholar 

  14. Ranjan, A., et al.: Competitive collaboration: joint unsupervised learning of depth, camera motion, optical flow and motion segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12232–12241 (2019)

    Google Scholar 

  15. Saxena, A., Sun, M., Ng, A.Y.: Make3D: learning 3D scene structure from a single still image. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 824–840 (2008)

    Article  Google Scholar 

  16. Wang, C., Miguel Buenaposada, J., Zhu, R., Lucey, S.: Learning depth from monocular videos using direct methods. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2022–2030 (2018)

    Google Scholar 

  17. Yang, Z., Wang, P., Wang, Y., Xu, W., Nevatia, R.: LEGO: learning edge with geometry all at once by watching videos. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 225–234 (2018)

    Google Scholar 

  18. Yang, Z., Wang, P., Xu, W., Zhao, L., Nevatia, R.: Unsupervised learning of geometry from videos with edge-aware depth-normal consistency. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pp. 7493–7500 (2018)

    Google Scholar 

  19. Yin, Z., Shi, J.: GeoNet: unsupervised learning of dense depth, optical flow and camera pose. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1983–1992 (2018)

    Google Scholar 

  20. Zhan, H., Garg, R., Saroj Weerasekera, C., Li, K., Agarwal, H., Reid, I.: Unsupervised learning of monocular depth estimation and visual odometry with deep feature reconstruction. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 340–349 (2018)

    Google Scholar 

  21. Zhang, Y., Xiang, T., Hospedales, T.M., Lu, H.: Deep mutual learning. In: CVPR, pp. 4320–4328 (2018)

    Google Scholar 

  22. Zhou, T., Brown, M., Snavely, N., Lowe, D.G.: Unsupervised learning of depth and ego-motion from video. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6612–6619 (2017)

    Google Scholar 

  23. Zou, Y., Luo, Z., Huang, J.-B.: DF-Net: unsupervised joint learning of depth and flow using cross-task consistency. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11209, pp. 38–55. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01228-1_3

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dalian University of Technology, No. 2 Linggong Road, Ganjingzi District, Dalian, Liaoning, China

    Min Zhang & Jianhua Li

Authors
  1. Min Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianhua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianhua Li .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Yuxin Peng

  2. Tsinghua University, Beijing, China

    Shi-Min Hu

  3. Tampere University, Tampere, Finland

    Moncef Gabbouj

  4. Zhejiang University, Hangzhou, China

    Kun Zhou

  5. Technion – Israel Institute of Technology, Haifa, Israel

    Michael Elad

  6. Tsinghua University, Beijing, China

    Kun Xu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, M., Li, J. (2021). Efficient Unsupervised Monocular Depth Estimation with Inter-Frame Depth Interpolation. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12890. Springer, Cham. https://doi.org/10.1007/978-3-030-87361-5_59

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87361-5_59

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87360-8

  • Online ISBN: 978-3-030-87361-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation