Skip to main content
SpringerLink
Log in

Optimization for image stereo-matching using deep reinforcement learning in rule constraints and parallax estimation

  • S.I.: Applications and Techniques in Cyber Intelligence (ATCI2022)
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Stereo-matching is a hot topic in the field of visual image research, to address the low image-matching accuracy of traditional algorithms. In this paper, an optimization for image stereo-matching algorithm using deep reinforcement learning (DRL) is proposed in rule constraints and parallax estimation. First, the image edge pixel constraint rules are established, and the image sample blocks are adjusted. Second, the image parallax estimation is performed by computing geometric constraint and pixel parallax probability in rule constraints, and a DRL structure is designed. Finally, the DRL analysis is performed iteratively by the convolutional neural networks feature extraction, agent training decision, and reward value accumulation, and stereo-matching images are output. Experiments show that the image structural similarity of the proposed algorithm is high, and the correct matching rate is more than 95%. The images have good interpretability, and the stereo-matching effect is good.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The data used to support the findings of this study are available from the corresponding author upon request.

References

  1. Zhang H, Li H, Wang Z et al (2020) Geometry and context guided refinement for stereo matching. IET Image Proc 14(12):2652–2659

    Article  Google Scholar 

  2. Deng Y, Xiao J, Zhou SZ et al (2021) Detail preserving coarse-to-fine matching for stereo matching and optical flow. IEEE Trans Image Process 3(99):1–10

    Google Scholar 

  3. Zhu C, Chang YZ (2020) Stereo matching for infrared images using guided filtering weighted by exponential moving average. IET Image Proc 14(5):830–837

    Article  Google Scholar 

  4. Jin Y, Zhao H, Bu P (2021) Spatial milree filter for cost aggregation in stereo matching. IET Image Proc 15(10):56–66

    Article  Google Scholar 

  5. Kautz E, Ma W, Jana S et al (2020) An image-driven machine learning approach to kinetic modeling of a discontinuous precipitation reaction. Mater Charact 166:110379

    Article  Google Scholar 

  6. Chatterjee J, Bhattacharyya R, Maulik A et al (2021) Application of Machine Learning on sequential de-convolution and convolution techniques for analysis of images from nuclear track detectors (NTDs). Radiat Meas 144(11):106581

    Article  Google Scholar 

  7. Ziatdinov M, Jesse S, Sumpter BG et al (2020) Tracking atomic structure evolution during directed electron beam induced Si-atom motion in graphene via deep machine learning. Nanotechnology 32(3):1–6

    Google Scholar 

  8. Koktzoglou I, Huang R, Ankenbrandt WJ et al (2021) Super-resolution head and neck MRA using deep machine learning. Magn Reson Med 86(1):335–345

    Article  Google Scholar 

  9. Nguyen HT, Barat C, Ducottet C (2014) Approximate image-matching using strings of bag-of-visual words representation. Int Conf Comput Vision Theory Appl (VISAPP) 2014:345–353

    Google Scholar 

  10. Wang L, Li Y, Huang J, Lazebnik S (2019) Learning two-branch neural networks for image-text matching tasks. IEEE Trans Pattern Anal Mach Intell 41(2):394–407

    Article  Google Scholar 

  11. Cui S, Ma A, Wan Y, Zhong Y, Luo B, Xu M (2022) Cross-modality image matching network with modality-invariant feature representation for airborne-ground thermal infrared and visible datasets. IEEE Trans Geosci Remote Sens 60:1–14

    Google Scholar 

  12. Zheng X, Tao Y, Zhang R et al (2021) TimNet: a text-image-matching network integrating multi-stage feature extraction with multi-scale metrics. Neurocomputing 465(20):540–548

    Article  Google Scholar 

  13. Hossein-Nejad Z, Agahi H, Mahmoodzadeh A (2021) Image-matching based on the adaptive redundant keypoint elimination method in the SIFT algorithm. Pattern Anal Appl 24(2):669–683

    Article  Google Scholar 

  14. Wei H, Meng L (2023) An accurate stereo matching method based on color segments and edges. Pattern Recogn 133:108996

    Article  Google Scholar 

  15. Zhou RG, Liu DQ (2019) Quantum image edge extraction based on improved sobel operator. Int J Theor Phys 58(9):1–17

    Article  MathSciNet  MATH  Google Scholar 

  16. Ren U, Nobukawa T, Miura M et al (2020) Dense parallax image acquisition method using single-pixel imaging for integral photography. Opt Lett 45(1):25–30

    Article  Google Scholar 

  17. Yan XA, QHW B, Hui RA et al (2019) Optical arbitrary-depth refocusing for large-depth scene in integral imaging display based on reprojected parallax image. Optics Commun 433:209–214

    Article  Google Scholar 

  18. Jiao L, Qian W, Qian C et al (2014) An efficient stereo matching method based on Bayesian theory. Laser Optoelectron Progress 51(10):69–76

    Google Scholar 

  19. Li J, Wang J, Wang X et al (2020) Evaluation and comparison of the results of the NET-VISA seismic event association method based on Bayesian theory. Nat Hazards 171(1):1521–1539

    Google Scholar 

  20. Manchella K, Umrawal AK, Aggarwal V (2021) FlexPool: a distributed model-free deep reinforcement learning algorithm for joint passengers and goods transportation. IEEE Trans Intell Transp Syst 3(99):1–13

    Google Scholar 

  21. Xie Y, Zhao X (2021) Sloshing suppression with active controlled baffles through deep reinforcement learning–expert demonstrations–behavior cloning process. Phys Fluids 33(1):017115

    Article  Google Scholar 

  22. Vereng SS, Dong TN, Hamre G (2021) Dynamic positioning using deep reinforcement learning. Ocean Eng 235(3):109433

    Article  Google Scholar 

  23. Msfba B, Sika B (2021) Deep reinforcement learning control of hydraulic fracturing. Comput Chem Eng 154(4):107489

    Google Scholar 

  24. Gao Y, Mosalam KM (2020) PEER hub imagenet: a large-scale multiattribute benchmark data set of structural images. J Struct Eng 146(10):04020198

    Article  Google Scholar 

  25. Ibrahem H, Salem A, Kang HS (2021) Real-time weakly supervised object detection using center-of-features localization. IEEE Access 23(9):1–10

    Google Scholar 

Download references

Funding

This work was supported by Natural Science Foundation of Heilongjiang Province of China under grant number LH2021F040.

Author information

Authors and Affiliations

  1. College of Physical Education and Training, Harbin Sport University, Harbin, 150008, China

    Jie Ren, Fuyu Guan, Xueyan Li & Jie Cao

  2. Department of Information Engineering, Heilongjiang International University, Harbin, 150025, China

    Xiaofeng Li

Authors
  1. Jie Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fuyu Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xueyan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaofeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaofeng Li.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest. No conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, J., Guan, F., Li, X. et al. Optimization for image stereo-matching using deep reinforcement learning in rule constraints and parallax estimation. Neural Comput & Applic 35, 24701–24711 (2023). https://doi.org/10.1007/s00521-023-08227-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08227-3

Keywords

Search

Navigation