Abstract
Recent years have witnessed great success in image super-resolution (SR). However, how to effectively exploit stereo information for the SR purpose is still challenging. This paper focuses on proposing a general solution to stereo image SR. We propose a novel module named Parallax Multi-Dimensional Attention (PMDA) that could not only be seamlessly integrated into most of existing SISR networks but also explore cross-view information from stereo images. Specifically, a pair of stereo images are fed into two identical SISR networks. The extracted middle features are transferred into PMDA to capture the inherent correlation within stereo image pairs. Finally, the internal-view and cross-view information is mixed by SISR network to generate the final output. We also introduce Self Multi-Dimensional Attention (SMDA) to effectively improve the feature representation capacity of a single image. Based on PMDA and SMDA, we design a stereo image SR model named Progressive Attention Stereo SR (PASR), which illustrates the flexibility of PMDA and performance-boosting guided by PMDA and SMDA. Extensive experiments show its superiority in the aspects of visual quality and quantitative comparison.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Dai, F., Chen, X., Ma, Y., Jin, G., Zhao, Q.: Wide range depth estimation from binocular light field camera. In: BMVC, p. 107 (2018)
Dai, T., Cai, J., Zhang, Y., Xia, S., Zhang, L.: Second-order attention network for single image super-resolution. In: CVPR, pp. 11065–11074 (2019)
Gao, S., Cheng, M., Zhao, K., Zhang, X., Yang, M., Torr, P.H.S.: Res2Net: a new multi-scale backbone architecture. IEEE Trans. Pattern Anal. Mach. Intell. 43(2), 652–662 (2021)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: CVPR, pp. 3354–3361 (2012)
Hui, Z., Gao, X., Yang, Y., Wang, X.: Lightweight image super-resolution with information multi-distillation network. In: ACM MM, pp. 2024–2032 (2019)
Jeon, D.S., Baek, S., Choi, I., Kim, M.H.: Enhancing the spatial resolution of stereo images using a parallax prior. In: CVPR, pp. 1721–1730 (2018)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: CVPR, pp. 105–114 (2017)
Mei, Y., Fan, Y., Zhou, Y., Huang, L., Huang, T.S., Shi, H.: Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining. In: CVPR, pp. 5689–5698 (2020)
Menze, M., Geiger, A.: Object scene flow for autonomous vehicles. In: CVPR, pp. 3061–3070 (2015)
Scharstein, D., et al.: High-resolution stereo datasets with subpixel-accurate ground truth. In: Jiang, X., Hornegger, J., Koch, R. (eds.) GCPR 2014. LNCS, vol. 8753, pp. 31–42. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11752-2_3
Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: CVPR, pp. 1874–1883 (2016)
Song, W., Choi, S., Jeong, S., Sohn, K.: Stereoscopic image super-resolution with stereo consistent feature. In: AAAI, pp. 12031–12038 (2020)
Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)
Wang, L., et al.: Learning parallax attention for stereo image super-resolution. In: CVPR, pp. 12250–12259 (2019)
Xie, W., Zhang, J., Lu, Z., Cao, M., Zhao, Y.: Non-local nested residual attention network for stereo image super-resolution. In: ICASSP, pp. 2643–2647 (2020)
Yan, B., Ma, C., Bare, B., Tan, W., Hoi, S.C.H.: Disparity-aware domain adaptation in stereo image restoration. In: CVPR, pp. 13176–13184 (2020)
Ying, X., Wang, Y., Wang, L., Sheng, W., An, W., Guo, Y.: A stereo attention module for stereo image super-resolution. IEEE Signal Process. Lett. 27, 496–500 (2020)
Zhang, T., Gu, Y., Huang, X., Tu, E., Yang, J.: Stereo endoscopic image super-resolution using disparity-constrained parallel attention. In: ICLR Workshop (2020)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 294–310. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_18
Acknowledgments
This work is supported by Sichuan Science and Technology Program (No. 2021YFS0007).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, C., Zhang, D., Jiang, C., Xie, N., Shao, J. (2021). Learning Multi-dimensional Parallax Prior for Stereo Image Super-Resolution. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Communications in Computer and Information Science, vol 1517. Springer, Cham. https://doi.org/10.1007/978-3-030-92310-5_83
Download citation
DOI: https://doi.org/10.1007/978-3-030-92310-5_83
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92309-9
Online ISBN: 978-3-030-92310-5
eBook Packages: Computer ScienceComputer Science (R0)