Skip to main content
SpringerLink
Log in

SPMVP: Spatial PatchMatch Stereo with Virtual Pixel Aggregation

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10636))

Included in the following conference series:

Abstract

Stereo matching is one of the critical problems in the field of computer vision and it has been widely applied to 3D Reconstruction, Image Refocusing and etc. Recently proposed PatchMatch (PM) stereo algorithm effectively overcomes the limitation of integer-value within the support window but it is still inferior in twofold: (1) view propagation of PM stereo algorithm generally yields underwhelming particle propagation; (2) it still suffers from a coarse performance in textureless regions. To mitigate these weaknesses, a Spatial-PM stereo algorithm without view propagation is proposed for improving the original one at first. Then a virtual pixel based cost aggregation framework with two sped-up strategies is proposed for tackling the problem of textureless mismatching. Jointing the two incremental improvements, we name the novel one as Spatial PatchMatch Stereo with Virtual Pixel Aggregation (SPMVP). Experiments show that SPMVP achieves superior results than other four challenging PM based stereo algorithms both in integer & subpixel level accuracy on all 31 Middlebury stereo pairs; and also performs better on Microsoft i2i stereo videos.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002). Springer

    Article  MATH  Google Scholar 

  2. Gong, M., Yang, R., Wang, L., et al.: A performance study on different cost aggregation approaches used in real-time stereo matching. Int. J. Comput. Vis. 75, 283–296 (2007). Springer

    Article  Google Scholar 

  3. Tombari, F., Mattoccia, S., Stefano, L., et al.: Classification and evaluation of cost aggregation methods for stereo correspondence. In: IEEE International Conference on Computer Vision Pattern Recognition, pp. 1–8 (2008)

    Google Scholar 

  4. Wang, L., Yang, R., Gong, M., et al.: Real-time stereo using approximated joint bilateral filtering and dynamic programming. J. Real-Time Image Proc. 9, 447–461 (2014). Springer

    Article  Google Scholar 

  5. Taniai, T., Matsushita, Y., Naemura, T.: Graph cut based continuous stereo matching using locally shared labels. In: IEEE International Conference on Computer Vision Pattern Recognition, pp. 1613–1620 (2014)

    Google Scholar 

  6. Yang, Q., Wang, L., Ahuja, N.: A constant space belief propagation algorithm for stereo matching. In: IEEE International Conference on Computer Vision Pattern Recognition, pp. 1458–1465 (2010)

    Google Scholar 

  7. Yoon, K.J., Kweon, I.S.: Adaptive support-weight approach for correspondence search. IEEE Trans. Pattern Anal. 28, 650–656 (2006)

    Article  Google Scholar 

  8. Hosni, A., Bleyer, M., Gelautz, M., et al.: Local stereo matching using geodesic support weights. In: IEEE International Conference Image Processing, pp. 2093–2096 (2009)

    Google Scholar 

  9. Hu, W., Zhang, K., Sun, L., et al.: Virtual support window for adaptive-weight stereo matching. In: IEEE Visual Communications and Image Processing, pp. 1–4 (2011)

    Google Scholar 

  10. Min, D., Lu, J., Do, M.N.: A revisit to cost aggregation in stereo matching: how far can we reduce its computational redundancy? In: IEEE International Conference on Computer Vision, pp. 1567–1574 (2011)

    Google Scholar 

  11. Min, D., Lu, J., Do, M.N.: Joint histogram based cost aggregation for stereo matching. IEEE Trans. Pattern Anal. 35, 2539–2545 (2013). IEEE

    Article  Google Scholar 

  12. Bleyer, M., Breiteneder, C.: Stereo matching—state-of-the-art and research challenges. In: Farinella, G., Battiato, S., Cipolla, R. (eds.) Advanced Topics in Computer Vision. Advances in Computer Vision and Pattern Recognition. Springer, London (2013). doi:10.1007/978-1-4471-5520-1_6

    Google Scholar 

  13. Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28(3), 24:1–24:11 (2009)

    Article  Google Scholar 

  14. Bleyer, M., Rhemann, C., Rother, C.: PatchMatch stereo - stereo matching with slanted support windows. In: British Machine Vision Conference (2011)

    Google Scholar 

  15. Besse, F., Rother, C., Fitzgibbon A., et al.: PMBP: PatchMatch belief propagation for correspondence field estimation. In: British Machine Vision Conference (2012)

    Google Scholar 

  16. Heise, P., Klose, S., Jensen, B., et al.: PM-Huber: PatchMatch with huber regularization for stereo matching. In: IEEE International Conference on Computer Vision, pp. 2360–2367 (2013)

    Google Scholar 

  17. Achanta, R., Shaji, A., Smith, K., et al.: SLIC Superpixels Compared to State-of-The-Art Superpixels Methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012). IEEE

    Article  Google Scholar 

  18. Zhang, C., Li, Z., Cai, R., et al.: As-rigid-as-possible stereo under second order smoothness priors. In: European Conference on Computer Vision, pp. 112–126 (2014)

    Google Scholar 

  19. Zhang, K., Fang, Y., Min, D., et al.: Cross-scale cost aggregation for stereo matching. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1590–1597 (2014)

    Google Scholar 

  20. Zhang, K., Fang, Y., Min, D., et al.: Cross-scale cost aggregation for stereo matching. In: IEEE Transactions on Circuits and Systems for Video Technology (2016)

    Google Scholar 

  21. Lu, J., Yang, H., Min D., et al.: PatchMatch filter: efficient edge aware filtering meets randomized search for fast correspondence field estimation. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1854–1861 (2013)

    Google Scholar 

  22. Li, Y., Min, D., Brown, M.S., et al.: SPM-BP: sped-up PatchMatch belief propagation for continuous MRFs. In: IEEE International Conference on Computer Vision, pp. 4006–4014 (2015)

    Google Scholar 

  23. Lu, L., Shi, K., Min, D., et al.: Cross-based local multipoint filtering. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 430–437 (2012)

    Google Scholar 

  24. Christo, R., Hosni, A., Bleyer, M., et al.: Fast cost-volume filtering for visual correspondence and beyond. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 3017–3024 (2011)

    Google Scholar 

  25. Hosni, A., Christo, R., Bleyer, M., et al.: Fast cost-volume filtering for visual correspondence and beyond. IEEE Trans. Pattern Anal. 32, 504–511 (2013)

    Article  Google Scholar 

  26. Middlebury Online Stereo Evaluation (2016). http://vision.middlebury.edu/stereo/data/

  27. Microsoft i2i Stereo Videos (2016). http://research.microsoft.com/enus/projects/i2i/data.aspx

Download references

Acknowledgements

This research has been supported by National Natural Science Foundation of China (U1509207, 61325019, 61472278, 61403281 and 61572357), Key project of Natural Science Foundation of Tianjin (14JCZDJC31700).

Author information

Authors and Affiliations

  1. Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology, Tianjin, China

    Peng Yao, Hua Zhang, Yanbing Xue & Shengyong Chen

  2. Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, China

    Peng Yao, Hua Zhang, Yanbing Xue & Shengyong Chen

Authors
  1. Peng Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hua Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanbing Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shengyong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hua Zhang .

Editor information

Editors and Affiliations

  1. Guangdong University of Technology, Guangzhou, China

    Derong Liu

  2. Guangdong University of Technology, Guangzhou, China

    Shengli Xie

  3. South China University of Technology, Guangzhou, China

    Yuanqing Li

  4. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Dongbin Zhao

  5. King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia

    El-Sayed M. El-Alfy

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Yao, P., Zhang, H., Xue, Y., Chen, S. (2017). SPMVP: Spatial PatchMatch Stereo with Virtual Pixel Aggregation. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10636. Springer, Cham. https://doi.org/10.1007/978-3-319-70090-8_54

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-70090-8_54

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-70089-2

  • Online ISBN: 978-3-319-70090-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation