Skip to main content

Learning Multi-dimensional Parallax Prior for Stereo Image Super-Resolution

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1517))

Included in the following conference series:

  • 2071 Accesses

Abstract

Recent years have witnessed great success in image super-resolution (SR). However, how to effectively exploit stereo information for the SR purpose is still challenging. This paper focuses on proposing a general solution to stereo image SR. We propose a novel module named Parallax Multi-Dimensional Attention (PMDA) that could not only be seamlessly integrated into most of existing SISR networks but also explore cross-view information from stereo images. Specifically, a pair of stereo images are fed into two identical SISR networks. The extracted middle features are transferred into PMDA to capture the inherent correlation within stereo image pairs. Finally, the internal-view and cross-view information is mixed by SISR network to generate the final output. We also introduce Self Multi-Dimensional Attention (SMDA) to effectively improve the feature representation capacity of a single image. Based on PMDA and SMDA, we design a stereo image SR model named Progressive Attention Stereo SR (PASR), which illustrates the flexibility of PMDA and performance-boosting guided by PMDA and SMDA. Extensive experiments show its superiority in the aspects of visual quality and quantitative comparison.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Dai, F., Chen, X., Ma, Y., Jin, G., Zhao, Q.: Wide range depth estimation from binocular light field camera. In: BMVC, p. 107 (2018)

    Google Scholar 

  2. Dai, T., Cai, J., Zhang, Y., Xia, S., Zhang, L.: Second-order attention network for single image super-resolution. In: CVPR, pp. 11065–11074 (2019)

    Google Scholar 

  3. Gao, S., Cheng, M., Zhao, K., Zhang, X., Yang, M., Torr, P.H.S.: Res2Net: a new multi-scale backbone architecture. IEEE Trans. Pattern Anal. Mach. Intell. 43(2), 652–662 (2021)

    Article  Google Scholar 

  4. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: CVPR, pp. 3354–3361 (2012)

    Google Scholar 

  5. Hui, Z., Gao, X., Yang, Y., Wang, X.: Lightweight image super-resolution with information multi-distillation network. In: ACM MM, pp. 2024–2032 (2019)

    Google Scholar 

  6. Jeon, D.S., Baek, S., Choi, I., Kim, M.H.: Enhancing the spatial resolution of stereo images using a parallax prior. In: CVPR, pp. 1721–1730 (2018)

    Google Scholar 

  7. Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: CVPR, pp. 105–114 (2017)

    Google Scholar 

  8. Mei, Y., Fan, Y., Zhou, Y., Huang, L., Huang, T.S., Shi, H.: Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining. In: CVPR, pp. 5689–5698 (2020)

    Google Scholar 

  9. Menze, M., Geiger, A.: Object scene flow for autonomous vehicles. In: CVPR, pp. 3061–3070 (2015)

    Google Scholar 

  10. Scharstein, D., et al.: High-resolution stereo datasets with subpixel-accurate ground truth. In: Jiang, X., Hornegger, J., Koch, R. (eds.) GCPR 2014. LNCS, vol. 8753, pp. 31–42. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11752-2_3

    Chapter  Google Scholar 

  11. Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: CVPR, pp. 1874–1883 (2016)

    Google Scholar 

  12. Song, W., Choi, S., Jeong, S., Sohn, K.: Stereoscopic image super-resolution with stereo consistent feature. In: AAAI, pp. 12031–12038 (2020)

    Google Scholar 

  13. Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)

    Google Scholar 

  14. Wang, L., et al.: Learning parallax attention for stereo image super-resolution. In: CVPR, pp. 12250–12259 (2019)

    Google Scholar 

  15. Xie, W., Zhang, J., Lu, Z., Cao, M., Zhao, Y.: Non-local nested residual attention network for stereo image super-resolution. In: ICASSP, pp. 2643–2647 (2020)

    Google Scholar 

  16. Yan, B., Ma, C., Bare, B., Tan, W., Hoi, S.C.H.: Disparity-aware domain adaptation in stereo image restoration. In: CVPR, pp. 13176–13184 (2020)

    Google Scholar 

  17. Ying, X., Wang, Y., Wang, L., Sheng, W., An, W., Guo, Y.: A stereo attention module for stereo image super-resolution. IEEE Signal Process. Lett. 27, 496–500 (2020)

    Article  Google Scholar 

  18. Zhang, T., Gu, Y., Huang, X., Tu, E., Yang, J.: Stereo endoscopic image super-resolution using disparity-constrained parallel attention. In: ICLR Workshop (2020)

    Google Scholar 

  19. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 294–310. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_18

    Chapter  Google Scholar 

Download references

Acknowledgments

This work is supported by Sichuan Science and Technology Program (No. 2021YFS0007).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jie Shao .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, C., Zhang, D., Jiang, C., Xie, N., Shao, J. (2021). Learning Multi-dimensional Parallax Prior for Stereo Image Super-Resolution. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Communications in Computer and Information Science, vol 1517. Springer, Cham. https://doi.org/10.1007/978-3-030-92310-5_83

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-92310-5_83

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-92309-9

  • Online ISBN: 978-3-030-92310-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics