Skip to main content
Springer Nature Link
Log in

Toward Understanding the Impact of Input Data for Multi-Image Super-Resolution

  • Conference paper
  • First Online:
Intelligent Information and Database Systems (ACIIDS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13758))

Included in the following conference series:

  • 871 Accesses

Abstract

Super-resolution reconstruction is a common term for a variety of techniques aimed at enhancing spatial resolution either from a single image or from multiple images presenting the same scene. While single-image super-resolution has been intensively explored with many advancements proposed attributed to the use of deep learning, multi-image reconstruction remains a much less explored field. The first solutions based on convolutional neural networks were proposed recently for super-resolving multiple Proba-V satellite images, but they have not been validated for enhancing natural images so far. Also, their sensitiveness to the characteristics of the input data, including their mutual similarity and image acquisition conditions, has not been explored in depth. In this paper, we address this research gap to better understand how to select and prepare the input data for reconstruction. We expect that the reported conclusions will help in elaborating more efficient super-resolution frameworks that could be deployed in practical applications.

This research was supported by the National Science Centre, Poland, under Research Grant 2019/35/B/ST6/03006 (MK) and co-financed by the Silesian University of Technology grant for maintaining and developing research potential (JK).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The RAMS implementation is available at https://github.com/EscVM/RAMS.

References

  1. Alvarez-Ramos, V., Ponomaryov, V., Reyes-Reyes, R.: Image super-resolution via two coupled dictionaries and sparse representation. Multimed. Tools Appl. 77(11), 13487–13511 (2018). https://doi.org/10.1007/s11042-017-4968-3

    Article  Google Scholar 

  2. Balestriero, R., Glotin, H., Baraniuk, R.G.: Interpretable Super-Resolution via a Learned Time-Series Representation (2020). arxiv.org/abs/2006.07713

  3. Benecki, P., Kawulok, M., Kostrzewa, D., Skonieczny, L.: Evaluating super-resolution reconstruction of satellite images. Acta Astronaut. 153, 15–25 (2018)

    Article  Google Scholar 

  4. Bhat, G., Danelljan, M., Timofte, R.: NTIRE 2021 challenge on burst super-resolution: methods and results. In: Proceedings IEEE/CVF CVPR Workshops, pp. 613–626 (2021)

    Google Scholar 

  5. Bhat, G., Danelljan, M., Van Gool, L., Timofte, R.: Deep burst super-resolution. In: Proceedings IEEE/CVF CVPR, pp. 9209–9218 (2021)

    Google Scholar 

  6. Bordone Molini, A., Valsesia, D., Fracastoro, G., Magli, E.: DeepSUM: deep neural network for super-resolution of unregistered multitemporal images. IEEE TGRS 58(5), 3644–3656 (2020)

    Google Scholar 

  7. Chen, H., et al.: Real-world single image super-resolution: a brief review. Inf. Fusion 79, 124–145 (2021)

    Article  Google Scholar 

  8. Deudon, M., et al.: HighRes-net: Recursive fusion for multi-frame super-resolution of satellite imagery (2020). arxiv.org/abs/2002.06460

  9. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13

    Chapter  Google Scholar 

  10. Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE TIP 13(10), 1327–1344 (2004)

    Google Scholar 

  11. Huang, Y., Li, J., Gao, X., Hu, Y., Lu, W.: Interpretable detail-fidelity attention network for single image super-resolution. IEEE TIP 30, 2325–2339 (2021)

    Google Scholar 

  12. Hui, Z., Wang, X., Gao, X.: Fast and accurate single image super-resolution via information distillation network. In: Proceedings IEEE/CVF CVPR, pp. 723–731 (2018)

    Google Scholar 

  13. Jo, Y., Kim, S.J.: Practical single-image super-resolution using look-up table. In: Proceedings IEEE/CVF CVPR, pp. 691–700 (2021)

    Google Scholar 

  14. Kappeler, A., Yoo, S., Dai, Q., Katsaggelos, A.K.: Video super-resolution with convolutional neural networks. IEEE TCI 2(2), 109–122 (2016)

    MathSciNet  Google Scholar 

  15. Kawulok, M., Benecki, P., Kostrzewa, D., Skonieczny, L.: Evolving imaging model for super-resolution reconstruction. In: Proceedings GECOO, pp. 284–285 (2018)

    Google Scholar 

  16. Kawulok, M., Benecki, P., Nalepa, J., Kostrzewa, D., Skonieczny, Ł: Towards Robust Evaluation of Super-Resolution Satellite Image Reconstruction. In: Nguyen, N.T., Hoang, D.H., Hong, T.-P., Pham, H., Trawiński, B. (eds.) ACIIDS 2018. LNCS (LNAI), vol. 10751, pp. 476–486. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75417-8_45

    Chapter  Google Scholar 

  17. Kawulok, M., Benecki, P., Piechaczek, S., Hrynczenko, K., Kostrzewa, D., Nalepa, J.: Deep learning for multiple-image super-resolution. IEEE GRSL 17(6), 1062–1066 (2020)

    Google Scholar 

  18. Kawulok, M., Tarasiewicz, T., Nalepa, J., Tyrna, D., Kostrzewa, D.: Deep learning for multiple-image super-resolution of sentinel-2 data. In: Proceedings IEEE IGARSS, pp. 3885–3888 (2021)

    Google Scholar 

  19. Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings IEEE CVPR, pp. 1646–1654 (2016)

    Google Scholar 

  20. Köhler, T., Bätz, M., Naderi, F., Kaup, A., Maier, A., Riess, C.: Toward bridging the simulated-to-real gap: benchmarking super-resolution on real data. IEEE Trans. Pattern Anal. Mach. Intell. 42(11), 2944–2959 (2020)

    Google Scholar 

  21. Kostrzewa, D., Skonieczny, Ł, Benecki, P., Kawulok, M.: B4MultiSR: a benchmark for multiple-image super-resolution reconstruction. In: Kozielski, S., Mrozek, D., Kasprowski, P., Małysiak-Mrozek, B., Kostrzewa, D. (eds.) BDAS 2018. CCIS, vol. 928, pp. 361–375. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99987-6_28

    Chapter  Google Scholar 

  22. Lai, W., Huang, J., Ahuja, N., Yang, M.: Fast and accurate image super-resolution with deep Laplacian pyramid networks. IEEE TPAMI 41(11), 2599–2613 (2019)

    Article  Google Scholar 

  23. Ledig, C., Theis, L., Huszár, F., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings IEEE CVPR, pp. 105–114 (2017)

    Google Scholar 

  24. Liu, D., Wang, Z., Wen, B., et al.: Robust single image super-resolution via deep networks with sparse prior. IEEE TIP 25(7), 3194–3207 (2016)

    MathSciNet  MATH  Google Scholar 

  25. Märtens, M., Izzo, D., Krzic, A., Cox, D.: Super-resolution of PROBA-V images using convolutional neural networks. Astrodynamics 3(4), 387–402 (2019)

    Article  Google Scholar 

  26. Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. Mach. Vision Appl. 25(6), 1423–1468 (2014). https://doi.org/10.1007/s00138-014-0623-4

    Article  Google Scholar 

  27. Nguyen, N.L., Anger, J., Davy, A., Arias, P., Facciolo, G.: Self-supervised multi-image super-resolution for push-frame satellite images. In: Proceedings IEEE/CVF CVPR, pp. 1121–1131 (2021)

    Google Scholar 

  28. Rifat Arefin, M., et al.: Multi-image super-resolution for remote sensing using deep recurrent networks. In: Proceedings IEEE CVPR Workshops, pp. 206–207 (2020)

    Google Scholar 

  29. Salvetti, F., Mazzia, V., Khaliq, A., Chiaberge, M.: Multi-image super resolution of remotely sensed images using residual attention deep neural networks. Remote Sens. 12(14), 2207 (2020)

    Article  Google Scholar 

  30. Tao, Y., Muller, J.P.: Super-resolution restoration of spaceborne ultra-high-resolution images using the UCL OpTiGAN system. Remote Sens. 13(12), 2269 (2021)

    Article  Google Scholar 

  31. Tarasiewicz, T., Nalepa, J., Kawulok, M.: A graph neural network for multiple-image super-resolution. In: Proceedings IEEE ICIP, pp. 1824–1828 (2021)

    Google Scholar 

  32. Valsesia, D., Magli, E.: Permutation invariance and uncertainty in multitemporal image super-resolution. IEEE TGRS 60, 1–12 (2021)

    Google Scholar 

  33. Wang, X., et al.: ESRGAN: enhanced super-resolution generative adversarial networks. In: Proceedings ECCV workshops (2018)

    Google Scholar 

  34. Wang, Z., Chen, J., Hoi, S.C.H.: Deep learning for image super-resolution: a survey. IEEE TPAMI 43(10), 3365–3387 (2021)

    Article  Google Scholar 

  35. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. In: IEEE TIP, pp. 600–612 (2004)

    Google Scholar 

  36. Wei, P., et al.: Component divide-and-conquer for real-world image super-resolution. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12353, pp. 101–117. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58598-3_7

    Chapter  Google Scholar 

  37. Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J., Liao, Q.: Deep learning for single image super-resolution: a brief review. IEEE Trans. Multimed. 21(12), 3106–3121 (2019)

    Article  Google Scholar 

  38. Yang, W., Feng, J., Xie, G., Liu, J., Guo, Z., Yan, S.: Video super-resolution based on spatial-temporal recurrent residual networks. Comput. Vis. Image Underst. 168, 79–92 (2018)

    Article  Google Scholar 

  39. Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., Zhang, L.: Image super-resolution: the techniques, applications, and future. Signal Process. 128, 389–408 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, Gliwice, Poland

    Jakub Adler, Jolanta Kawulok & Michal Kawulok

Authors
  1. Jakub Adler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jolanta Kawulok
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michal Kawulok
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michal Kawulok .

Editor information

Editors and Affiliations

  1. Wrocław University of Science and Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

  2. Vietnam National University, Ho Chi Minh City, Ho Chi Minh City, Vietnam

    Tien Khoa Tran

  3. Al-Farabi Kazakh National University, Almaty, Kazakhstan

    Ualsher Tukayev

  4. National University of Kaohsiung, Kaohsiung, Taiwan

    Tzung-Pei Hong

  5. Wrocław University of Science and Technology, Wrocław, Poland

    Bogdan Trawiński

  6. University of Newcastle, Newcastle, NSW, Australia

    Edward Szczerbicki

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Adler, J., Kawulok, J., Kawulok, M. (2022). Toward Understanding the Impact of Input Data for Multi-Image Super-Resolution. In: Nguyen, N.T., Tran, T.K., Tukayev, U., Hong, TP., Trawiński, B., Szczerbicki, E. (eds) Intelligent Information and Database Systems. ACIIDS 2022. Lecture Notes in Computer Science(), vol 13758. Springer, Cham. https://doi.org/10.1007/978-3-031-21967-2_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21967-2_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21966-5

  • Online ISBN: 978-3-031-21967-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics