Skip to main content

Advertisement

Springer Nature Link
Log in

Apply Fuzzy Mask to Improve Monocular Depth Estimation

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

A fuzzy mask applied to pixel-wise dissimilarity weighting is proposed to improve the monocular depth estimation in this study. The parameters in the monocular depth estimation model are learned unsupervised through the image reconstruction of binocular images. The significant reconstructed dissimilarity, which is challenging to reduce, always occurs at pixels outside the binocular overlap. The fuzzy mask is designed based on the binocular overlap to adjust the weight of the dissimilarity for each pixel. More than 68% of pixels with significant dissimilarity outside binocular overlap are suppressed with weights less than 0.5. The model with the proposed fuzzy mask would focus on learning the depth estimation for pixels within binocular overlap. Experiments on the KITTI dataset show that the inference of the fuzzy mask only increases the training time of the model by less than 1%, while the number of pixels whose depth is accurately estimated enhances, and the monocular depth estimation also improves.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The datasets analyzed during the current study are available in the KITTI repository at https://www.cvlibs.net/datasets/kitti/, and all data generated during this study are included in this published article.

References

  1. Luo, X., Huang, J.-B., Szeliski, R., Matzen, K., Kopf, J.: Consistent video depth estimation. ACM Trans. Graph. 39(4), 71–1 (2020)

    Article  Google Scholar 

  2. Chen, H.-C.: Monocular vision-based obstacle detection and avoidance for a multicopter. IEEE Access 7, 167869–167883 (2019)

    Article  Google Scholar 

  3. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 3354-3361 (2012)

  4. Xue, F., Zhuo, G., Huang, Z., Fu, W., Wu, Z., Ang, M.H.: Toward hierarchical self-supervised monocular absolute depth estimation for autonomous driving applications. In: Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2330-2337 (2020)

  5. Miclea, V.-C., Nedevschi, S.: Monocular depth estimation with improved long-range accuracy for UAV environment perception. IEEE Trans. Geosci. Remote Sens. 60, 1–15 (2021)

    Article  Google Scholar 

  6. Yin, W., Zhang, J., Wang, O., et al.: Learning to recover 3D scene shape from a single image. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 204-213 (2021)

  7. Godard, C., Mac Aodha, O., Brostow, G.J.: Unsupervised monocular depth estimation with left-right consistency. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 6602-6611 (2017)

  8. Godard, C., Mac Aodha, O., Firman, M., Brostow, G.J.: Digging into self-supervised monocular depth estimation. In: Proc. IEEE/CVF Int. Conf. Comput. Vis., 3827-3837 (2019)

  9. Watson, J., Mac Aodha, O., Prisacariu, V., Brostow, G., Firman, M.: The temporal opportunist: Self-supervised multi-frame monocular depth. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 1164-1174 (2021)

  10. Shu, C., Yu, K., Duan, Z., Yang, K.: Feature-metric loss for self-supervised learning of depth and egomotion. In: Proc. 16th Eur. Conf. Comput. Vis., 572-588 (2020)

  11. Gordon, A., Li, H., Jonschkowski, R., Angelova, A.: Depth from videos in the wild: Unsupervised monocular depth learning from unknown cameras. In: Proc. IEEE/CVF Int. Conf. Comput. Vis., 8976-8985 (2019)

  12. Watson, J., Firman, M., Brostow, G.J., Turmukhambetov, D.: Self-supervised monocular depth hints. In: Proc. IEEE/CVF Int. Conf. Comput. Vis., 2162-2171 (2019)

  13. Bian, J., Li, Z., Wang, N., et al.: Unsupervised scale-consistent depth and ego-motion learning from monocular video. Adv. Neural Inf. Process. Syst. 32, 35–45 (2019)

    Google Scholar 

  14. Ranjan, A., Jampani, V., Balles, L., et al.: Competitive collaboration: Joint unsupervised learning of depth, camera motion, optical flow and motion segmentation. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 12232-12241 (2019)

  15. Klingner, M., Termöhlen, J.-A., Mikolajczyk, J., Fingscheidt, T.: Self-supervised monocular depth estimation: Solving the dynamic object problem by semantic guidance. In: Proc. 16th Eur. Conf. Comput. Vis., 582-600 (2020)

  16. Li, X., Hou, Y., Wu, Q., Wang, P., Li, W.: DVONet: unsupervised monocular depth estimation and visual odometry. In: Proc. IEEE Vis. Commun. Image Process., 1-4 (2019)

  17. Sun, Q., Tang, Y., Zhang, C., Zhao, C., Qian, F., Kurths, J.: Unsupervised estimation of monocular depth and VO in dynamic environments via hybrid masks. IEEE Trans. Neural Networks Learn. Syst. 33(5), 2023–2033 (2021)

    Article  Google Scholar 

  18. Alamoodi, A.H., Albahri, O.S., Zaidan, A.A., et al.: New extension of fuzzy-weighted zero-inconsistency and fuzzy decision by opinion score method based on cubic Pythagorean fuzzy environment: a benchmarking case study of sign language recognition systems. Int. J. Fuzzy Syst. 24(4), 1909–1926 (2022)

    Article  Google Scholar 

  19. Daradkeh, Y.I., Tvoroshenko, I., Gorokhovatskyi, V., Latiff, L.A., Ahmad, N.: Development of effective methods for structural image recognition using the principles of data granulation and apparatus of fuzzy logic. IEEE Access 9, 13417–13428 (2021)

    Article  Google Scholar 

  20. Jiang, Y., Peng, X., Xue, M., Wang, C., Qi, H.: An underwater human-robot interaction using hand gestures for fuzzy control. Int. J. Fuzzy Syst. 23, 1879–1889 (2021)

    Article  Google Scholar 

  21. Yang, T., Sun, N., Fang, Y.: Adaptive fuzzy control for a class of MIMO underactuated systems with plant uncertainties and actuator deadzones: Design and experiments. IEEE Trans. Cybern. 52(8), 8213–8226 (2022)

    Article  Google Scholar 

  22. Hu, M., Zhong, Y., Xie, S., Lv, H., Lv, Z.: Fuzzy system based medical image processing for brain disease prediction. Front. Neurosci. 15, 714318 (2021)

    Article  Google Scholar 

  23. Sadiq, M., Masood, S., Pal, O.: FD-YOLOv5: a fuzzy image enhancement based robust object detection model for safety helmet detection. Int. J. Fuzzy Syst. 24(5), 2600–2616 (2022)

    Article  Google Scholar 

  24. Hsu, M.-J., Chien, Y.-H., Wang, W.-Y., Hsu, C.-C.: A convolutional fuzzy neural network architecture for object classification with small training database. Int. J. Fuzzy Syst. 22(1), 1–10 (2020)

    Article  Google Scholar 

  25. Le, T.-L., Huynh, T.-T., Lin, L.-Y., Lin, C.-M., Chao, F.: A k-means interval type-2 fuzzy neural network for medical diagnosis. Int. J. Fuzzy Syst. 21(7), 2258–2269 (2019)

    Article  Google Scholar 

  26. Shang, H., Lu, D., Zhou, Q.: Early warning of enterprise finance risk of big data mining in internet of things based on fuzzy association rules. Neural Comput. Appl. 33(9), 3901–3909 (2021)

    Article  Google Scholar 

  27. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Preprint at arXiv:1409.1556 (2014)

  28. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 770-778 (2016)

  29. Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017)

    Article  Google Scholar 

  30. Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. Adv. Neural Inf. Process. Syst. 28, 2017–2025 (2015)

    Google Scholar 

  31. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  32. Zimmermann, H.-J.: Fuzzy Set Theory-and Its Applications. Springer Science & Business Media (2011)

  33. Abadi, M., Agarwal, A., Barham, P., et al.: TensorFlow: Large-scale machine learning on heterogeneous distributed systems. Preprint at arXiv:1603.04467 (2016)

  34. Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. Adv. Neural Inf. Process. Syst. 27 (2014)

  35. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. Preprint at arXiv:1412.6980 (2014)

  36. Menze, M., Geiger, A.: Object scene flow for autonomous vehicles. In: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 3061-3070 (2015)

Download references

Acknowledgements

This study is supported in part by the Ministry of Science and Technology of Taiwan, ROC, under Grants MOST 111-2221-E008-107.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Central University, No. 300, Zhongda Rd., Zhongli District, Taoyuan City, 320317, Taiwan, ROC

    Hsuan Chen & Wen-June Wang

  2. Department of Mechanical Engineering, National Central University, No. 300, Zhongda Rd., Zhongli District, Taoyuan City, 320317, Taiwan, ROC

    Hsiang-Chieh Chen

  3. Department of Electrical Engineering, National Kaohsiung University of Science and Technology, No. 415, Jiangong Rd., Sanmin Dist., Kaohsiung City, 807618, Taiwan, ROC

    Chung-Hsun Sun

Authors
  1. Hsuan Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Hsiang-Chieh Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Chung-Hsun Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Wen-June Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Chung-Hsun Sun.

Ethics declarations

Conflict of interest

All authors declare no conflict of interest.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, H., Chen, HC., Sun, CH. et al. Apply Fuzzy Mask to Improve Monocular Depth Estimation. Int. J. Fuzzy Syst. 26, 1143–1157 (2024). https://doi.org/10.1007/s40815-023-01657-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-023-01657-0

Keywords