Skip to main content
SpringerLink
Log in

Enhanced data fusion of ultrasonic and stereo vision in real-time obstacle detection

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In this research, the accuracy and speed of obstacle detection in data fusion of ultrasonic and stereo vision have been improved. The smoothness assumption has been used in such a way that the responses are significantly improved without increasing calculation. In addition, with the development of the proposed method to run on the graphics card, the cross-checking process has been done without the need to change the reference image and without more calculation of the cost function. The results of this study show that the proposed method improved the quality of the responses compared to the previous study, and the obstacle detection rate in intelligent vehicles has increased to 41 pairs of frames per second. This processing rate is 477.40 times faster than the usual local stereo method and 33.77% faster than the previous study on the data fusion of ultrasonic and stereo vision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Wang, H., Patil, S.V., Aziz, H.A., Young, S.: Modeling and control using stochastic distribution control theory for intersection traffic flow. IEEE Transact. Intell. Transport. Syst. 23, 1885–1898 (2020)

  2. Wang, H., Hu, P., Wang, H.: A genetic timing scheduling model for urban traffic signal control. Inf. Sci. 576, 475–483 (2021)

  3. Wang, Z.: Review of real-time three-dimensional shape measurement techniques. Measurement 156, 107624 (2020)

    Article  Google Scholar 

  4. Wang, H., Zhang, X.: Real-time vehicle detection and tracking using 3d lidar. Asian J. Control. 24, 1459–1469 (2021)

  5. Ye, D.H., Li, J., Chen, Q., Wachs, J., Bouman, C.: Deep learning for moving object detection and tracking from a single camera in unmanned aerial vehicles (uavs). Electronic Imaging 2018(10), 466–1 (2018)

    Google Scholar 

  6. Fan, L., Zhang, T., Du, W.: Optical-flow-based framework to boost video object detection performance with object enhancement. Expert Syst. Appl. 170, 114544 (2021)

    Article  Google Scholar 

  7. Lu, S., Luo, Z., Gao, F., Liu, M., Chang, K., Piao, C.: A fast and robust lane detection method based on semantic segmentation and optical flow estimation. Sensors 21(2), 400 (2021)

    Article  Google Scholar 

  8. Yu, L., Pan, B.: Single-camera high-speed stereo-digital image correlation for full-field vibration measurement. Mech. Syst. Signal Process. 94, 374–383 (2017). https://doi.org/10.1016/j.ymssp.2017.03.008

    Article  Google Scholar 

  9. Durand-Texte, T., Simonetto, E., Durand, S., Melon, M., Moulet, M.-H.: Vibration measurement using a pseudo-stereo system, target tracking and vision methods. Mech. Syst. Signal Process. 118, 30–40 (2019). https://doi.org/10.1016/j.ymssp.2018.08.049

    Article  Google Scholar 

  10. Gorjup, D., Slavič, J., Babnik, A., Boltežar, M.: Still-camera multiview spectral optical flow imaging for 3d operating-deflection-shape identification. Mech. Syst. Signal Process. 152, 107456 (2021). https://doi.org/10.1016/j.ymssp.2020.107456

    Article  Google Scholar 

  11. Kim, W.-S., Lee, D.-H., Kim, Y.-J., Kim, T., Lee, W.-S., Choi, C.-H.: Stereo-vision-based crop height estimation for agricultural robots. Comput. Electron. Agric. 181, 105937 (2021)

    Article  Google Scholar 

  12. Wu, B., Wang, L., Liu, X., Wang, L., Xu, K.: Closed-loop pose control and automated suturing of continuum surgical manipulators with customized wrist markers under stereo vision. IEEE Robot. Automat. Lett. 6(4), 7137–7144 (2021)

    Article  Google Scholar 

  13. Han, C., Liu, W., Jin, L., Jiang, S., Li, H.: Adaptive weight based sparse block aggregation algorithm for stereo matching. In: 2019 9th International Conference on Information Science and Technology (ICIST), pp. 408–412 (2019). IEEE

  14. Qiao, W., Xu, Y., Zhang, C., Xu, Z., Huang, J., Xie, P., Lu, J.: An improved adaptive window stereo matching algorithm. J. Phys. 1634, 012066 (2020). (IOP Publishing)

    Google Scholar 

  15. Wu, W., Zhu, H., Yu, S., Shi, J.: Stereo matching with fusing adaptive support weights. IEEE Access 7, 61960–61974 (2019)

    Article  Google Scholar 

  16. Ye, X., Yan, B., Liu, B., Wang, H., Qi, S., Chen, D., Wang, P., Wang, K., Sang, X.: Improved real-time three-dimensional stereo matching with local consistency. Image Vis. Comput. 124, 104509 (2022). https://doi.org/10.1016/j.imavis.2022.104509

    Article  Google Scholar 

  17. Woodford, O., Torr, P., Reid, I., Fitzgibbon, A.: Global stereo reconstruction under second-order smoothness priors. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), 2115–2128 (2009)

    Article  Google Scholar 

  18. Bleyer, M., Gelautz, M.: Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions. Signal Process. 22(2), 127–143 (2007)

    Google Scholar 

  19. Wang, H.-q., Wu, M., Zhang, Y.-b., Zhang, L.: Effective stereo matching using reliable points based graph cut. In: 2013 Visual Communications and Image Processing (VCIP), pp. 1–6 (2013). IEEE

  20. Lu, B., Sun, L., Yu, L., Dong, X.: An improved graph cut algorithm in stereo matching. Displays 69, 102052 (2021)

    Article  Google Scholar 

  21. Hirschmuller, H.: Stereo vision in structured environments by consistent semi-global matching. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 2, pp. 2386–2393 (2006). IEEE

  22. Toledo, J., Lauer, M., Stiller, C.: Real-time stereo semi-global matching for video processing using previous incremental information. J. Real-Time Image Proc. 19(1), 205–216 (2022)

    Article  Google Scholar 

  23. Zbontar, J., LeCun, Y.: Computing the stereo matching cost with a convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1592–1599 (2015)

  24. Hamid, M.S., Abd Manap, N., Hamzah, R.A., Kadmin, A.F.: Stereo matching algorithm based on deep learning: a survey. J. King Saud Univ-Comput. Inform. Sci. 34, 1663–1673 (2020)

  25. Premaratne, P., Safaei, F.: Feature based stereo correspondence using moment invariant. In: 2008 4th International Conference on Information and Automation for Sustainability, pp. 104–108 (2008). IEEE

  26. Lee, C., Lim, Y.-C., Kwon, S., Lee, J.: Stereo vision-based vehicle detection using a road feature and disparity histogram. Opt. Eng. 50(2), 027004 (2011)

    Article  Google Scholar 

  27. Xue, T., Owens, A., Scharstein, D., Goesele, M., Szeliski, R.: Multi-frame stereo matching with edges, planes, and superpixels. Image Vis. Comput. 91, 103771 (2019). https://doi.org/10.1016/j.imavis.2019.05.006

    Article  Google Scholar 

  28. Bertozzi, M., Broggi, A., Fascioli, A., Nichele, S.: Stereo vision-based vehicle detection. In: Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No. 00TH8511), pp. 39–44 (2000). IEEE

  29. Li, Y., Wang, Z.: Rgb line pattern-based stereo vision matching for single-shot 3-d measurement. IEEE Trans. Instrum. Meas. 70, 1–13 (2020)

    Google Scholar 

  30. Shuang, Y., Wang, Z.: Active stereo vision three-dimensional reconstruction by rgb dot pattern projection and ray intersection. Measurement 167, 108195 (2021)

    Article  Google Scholar 

  31. Gholami, F., Khanmirza, E., Riahi, M.: Real-time obstacle detection by stereo vision and ultrasonic data fusion. Measurement. 190, 110718 (2022)

  32. Li, J., Wu, J., You, Y., Jeon, G.: Parallel binocular stereo-vision-based gpu accelerated pedestrian detection and distance computation. J. Real-Time Image Proc. 17(3), 447–457 (2020)

    Article  Google Scholar 

  33. Yang, S., Scherer, S.A., Yi, X., Zell, A.: Multi-camera visual slam for autonomous navigation of micro aerial vehicles. Robot. Auton. Syst. 93, 116–134 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Iran University of Science and Technology, Narmak, 1684613114, Tehran, Iran

    Farshad Gholami, Esmaeel Khanmirza & Mohammad Riahi

Authors
  1. Farshad Gholami
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Esmaeel Khanmirza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Riahi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

FG: wrote the main manuscript text and code Dr. EK: supervisor and contributor Dr. MR: supervisor and contributor.

Corresponding author

Correspondence to Esmaeel Khanmirza.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gholami, F., Khanmirza, E. & Riahi, M. Enhanced data fusion of ultrasonic and stereo vision in real-time obstacle detection. J Real-Time Image Proc 20, 63 (2023). https://doi.org/10.1007/s11554-023-01314-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-023-01314-7

Keywords

Search

Navigation