Abstract
Light field (LF) is a emerging technology, which can be used in many fields. Furthermore, LF cameras can capture spatial and angular information of 3D real-world scenes. This information is beneficial for image super-resolution (SR). However, most existing LF approaches have the limitation of utilizing the global-view information, which contains the correlation information among all LF. Moreover, to exploit the complementary information from different views of an LF image, we propose a novel SR method that adapts each view to a global domain with the guidance of global-view information. Our method, called LF-GAGNet, uses a dual-branch network to align features across views with deformable convolutions and fuse them with an attention mechanism. The upper branch extracts global-view information and generates adaptive guidance factors for each view through a global-view adaptation-guided module (GAGM). The lower branch uses these factors as offsets for deformable convolutions to achieve feature alignment in the global domain. Furthermore, we design an angular attention fusion module (AAFM) to enhance the angular features of each view according to their importance. We evaluate our method on various real-world scenarios and show that it surpasses other state-of-the-art methods in terms of SR quality and performance. We also demonstrate that our method can handle complex realistic LF scenarios effectively.
Supported by Macao Polytechnic University.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhu, H., Wang, Q., Yu, J.: Occlusion-model guided antiocclusion depth estimation in light field. IEEE J. Sel. Topics Signal Process. 11(7), 965–978 (2017)
Zhang, M., et al.: Memory-oriented decoder for light field salient object detection. In: NeurIPS 2019, pp. 896–906 (2019)
Zhang, S., Lin, Y., Sheng, H.: Residual networks for light field image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11046–11055 (2019)
Jin, J., Hou, J., Chen, J., Kwong, S.: Light field spatial super-resolution via deep combinatorial geometry embedding and structural consistency regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2260–2269 (2020)
Y. Wang, L. Wang, J. Yang, W. An, J. Yu, Y. Guo, Spatial-angular interaction for light field image super-resolution, in: European Conference on Computer Vision, Springer, 2020, pp. 290–308
Wang, Y., et al.: Disentangling light fields for super-resolution and disparity estimation. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2022)
Zhang, W., Ke, W., Yang, D., et al.: Light field super-resolution using complementary-view feature attention [J]. Comput. Vis. Media 9(4), 843–858 (2023)
Cong, R., Sheng, H., Yang, D., Cui, Z., Chen, R.: Exploiting spatial and angular correlations with deep efficient transformers for light field image super-resolution. IEEE Trans. Multimedia (2023)
Wang, S., Zhou, T., Lu, Y., Di, H.: Detail-preserving transformer for light field image super-resolution, arXiv preprint arXiv:2201.00346 (2022)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)
Lim, B., Son, S.,Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)
Yeung, H.W.F., Hou, J., Chen, X., Chen, J., Chen, Z., Chung, Y.Y.: Light field spatial super-resolution using deep efficient spatial-angular separable convolution. IEEE Trans. Image Process. 28(5), 2319–2330 (2018)
Wang, Y., et al.: Light field image super-resolution using deformable convolution. IEEE Trans. Image Process. 30, 1057–1071 (2020)
Zhang, S., Chang, S., Lin, Y.: End-to-end light field spatial super-resolution network using multiple epipolar geometry. IEEE Trans. Image Process. 30, 5956–5968 (2021)
Liu, G., Yue, H., Wu, J., Yang, J.: Intra-inter view interaction network for light field image super-resolution. IEEE Trans. Multimedia 25, 256–266 (2021)
Wang, X., Chan, K.C., Yu, K., Dong, C., Loy, C.C.: EDVR: video restoration with enhanced deformable convolutional networks. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2019, pp. 1954–1963 (2019). https://doi.org/10.1109/CVPRW.2019.00247
Rerabek, M., Ebrahimi, T.: New light field image dataset. In: 8th International Conference on Quality of Multimedia Experience (QoMEX), no. CONF (2016)
Le Pendu, M., Jiang, X., Guillemot, C.: Light field inpainting propagation via low rank matrix completion. IEEE Trans. Image Process. 27(4), 1981–1993 (2018)
Vaish, V., Adams, A.: The (new) Stanford light field archive, Computer Graphics Laboratory, Stanford University 6 (7) (2008)
Raj, A.S., Lowney, M., Shah, R., Wetzstein, G.: Stanford Lytro light field archive (2016)
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)
Liang, Z., Wang, Y., Wang, L., Yang, J., Zhou, S.: Light field image super-resolution with transformers. IEEE Signal Process. Lett. 29, 563–567 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, W., Ke, W., Sheng, H. (2024). Light Field Image Super-Resolution via Global-View Information Adaption and Angular Attention Fusion. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1965. Springer, Singapore. https://doi.org/10.1007/978-981-99-8145-8_21
Download citation
DOI: https://doi.org/10.1007/978-981-99-8145-8_21
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8144-1
Online ISBN: 978-981-99-8145-8
eBook Packages: Computer ScienceComputer Science (R0)