Skip to main content
SpringerLink
Log in

B4MultiSR: A Benchmark for Multiple-Image Super-Resolution Reconstruction

  • Conference paper
  • First Online:
Beyond Databases, Architectures and Structures. Facing the Challenges of Data Proliferation and Growing Variety (BDAS 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 928))

Abstract

Super-resolution reconstruction (SRR) methods consist in processing single or multiple images to increase their spatial resolution. Deployment of such techniques is particularly important, when high resolution image acquisition is associated with high cost or risk, like for medical or satellite imaging. Unfortunately, the existing SRR techniques are not sufficiently robust to be deployed in real-world scenarios, and no real-life benchmark to validate multiple-image SRR has been published so far. As gathering a set of images presenting the same scene at different spatial resolution is not a trivial task, the SRR methods are evaluated based on different assumptions, employing various metrics and datasets, often without using any ground-truth data. In this paper, we introduce a new multi-layer benchmark dataset for systematic evaluation of multiple-image SRR techniques with particular reference to satellite imaging. We hope that the new benchmark will help the researchers to improve the state of the art in SRR, making it suitable for real-world applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aymaz, S., Köse, C.: A novel image decomposition-based hybrid technique with super-resolution method for multi-focus image fusion. Inf. Fusion 45, 113–127 (2019)

    Article  Google Scholar 

  2. Berkeley Segmentation Dataset. https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/. Accessed 27 Nov 2017

  3. Capel, D., Zisserman, A.: Super-resolution enhancement of text image sequences. In: Proceedings of 15th International Conference on Pattern Recognition, vol. 1, pp. 600–605. IEEE (2000)

    Google Scholar 

  4. Chang, K., Ding, P.L.K., Li, B.: Single image super-resolution using collaborative representation and non-local self-similarity. Sig. Process. 149, 49–61 (2018)

    Article  Google Scholar 

  5. Chavez-Roman, H., Ponomaryov, V.: Super resolution image generation using wavelet domain interpolation with edge extraction via a sparse representation. IEEE Geosci. Remote Sens. Lett. 11(10), 1777–1781 (2014)

    Article  Google Scholar 

  6. Common Objects in Context (COCO). http://cocodataset.org. Accessed 14 June 2018

  7. Del Gallego, N.P., Ilao, J.: Multiple-image super-resolution on mobile devices: an image warping approach. EURASIP J. Image Video Process. 2017(1), 8 (2017)

    Article  Google Scholar 

  8. Demirel, H., Anbarjafari, G.: Image resolution enhancement by using discrete and stationary wavelet decomposition. IEEE Trans. Image Process. 20(5), 1458–1460 (2011)

    Article  MathSciNet  Google Scholar 

  9. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 248–255. IEEE (2009)

    Google Scholar 

  10. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  11. Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004)

    Article  Google Scholar 

  12. Fattal, R.: Image upsampling via imposed edge statistics. In: ACM Transactions on Graphics (TOG), vol. 26, p. 95. ACM (2007)

    Google Scholar 

  13. Hagita, K., Higuchi, T., Jinnai, H.: Super-resolution for asymmetric resolution of FIB-SEM 3D imaging using ai with deep learning. Sci. Rep. 8(1), 5877 (2018)

    Article  Google Scholar 

  14. Heinrich, L., Bogovic, J.A., Saalfeld, S.: Deep learning for isotropic super-resolution from non-isotropic 3D electron microscopy. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 135–143. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_16

    Chapter  Google Scholar 

  15. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206 (2015)

    Google Scholar 

  16. ImageNet. http://www.image-net.org/. Accessed 14 June 2018

  17. Irani, M., Peleg, S.: Improving resolution by image registration. CVGIP Graph. Models Image Process. 53(3), 231–239 (1991)

    Article  Google Scholar 

  18. Jiang, J., Hu, R., Wang, Z., Han, Z.: Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Trans. Image Process. 23(10), 4220–4231 (2014)

    Article  MathSciNet  Google Scholar 

  19. Kawulok, M., Benecki, P., Kostrzewa, D., Skonieczny, L.: Towards evolutionary super-resolution. In: Sim, K., Kaufmann, P. (eds.) EvoApplications 2018. LNCS, vol. 10784, pp. 480–496. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77538-8_33

    Chapter  Google Scholar 

  20. Köhler, T., Haase, S., Bauer, S., Wasza, J., Kilgus, T., Maier-Hein, L., Feußner, H., Hornegger, J.: ToF meets RGB: novel multi-sensor super-resolution for hybrid 3-D endoscopy. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8149, pp. 139–146. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40811-3_18

    Chapter  Google Scholar 

  21. Li, L., Zhang, Y., Tian, Q.: Multi-face location on embedded dsp image processing system. In: Image and Signal Processing, 2008. CISP’08. Congress on. vol. 4, pp. 124–128. IEEE (2008)

    Google Scholar 

  22. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  23. Liu, D., Wang, Z., Wen, B., Yang, J., Han, W., Huang, T.S.: Robust single image super-resolution via deep networks with sparse prior. IEEE Trans. Image Process. 25(7), 3194–3207 (2016)

    Article  MathSciNet  Google Scholar 

  24. Lukinavičius, G., Umezawa, K., Olivier, N., Honigmann, A., Yang, G., Plass, T., Mueller, V., Reymond, L., Corrêa Jr., I.R., Luo, Z.G.: A near-infrared fluorophore for live-cell super-resolution microscopy of cellular proteins. Nat. Chem. 5(2), 132–139 (2013)

    Article  Google Scholar 

  25. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of 8th International Conference on Computer Vision, vol. 2, pp. 416–423, July 2001

    Google Scholar 

  26. Multi-Sensor Super-Resolution Datasets. https://www5.cs.fau.de/research/data/multi-sensor-super-resolution-datasets/. Accessed 02 Jan 2018

  27. Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. Mach. Vis. Appl. 25(6), 1423–1468 (2014)

    Article  Google Scholar 

  28. Pickup, L.C., Roberts, S.J., Zisserman, A.: Optimizing and learning for super-resolution. In: Proceedings of the British Machine Vision Conference (2006)

    Google Scholar 

  29. Qian, S.E., Chen, G.: Enhancing spatial resolution of hyperspectral imagery using sensor’s intrinsic keystone distortion. IEEE Trans. Geosci. Remote Sens. 50(12), 5033–5048 (2012)

    Article  Google Scholar 

  30. Qu, C., Luo, D., Monari, E., Schuchert, T., Beyerer, J.: Capturing ground truth super-resolution data. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 2812–2816 (2016)

    Google Scholar 

  31. Schultz, R.R., Stevenson, R.L.: Extraction of high-resolution frames from video sequences. IEEE Trans. Image Process. 5(6), 996–1011 (1996)

    Article  Google Scholar 

  32. Sun, L., Hays, J.: Super-resolution from internet-scale scene matching. In: IEEE International Conference on Computational Photography (ICCP), pp. 1–12 (2012)

    Google Scholar 

  33. Super-Resolution dataset, Audiovisual Communications Laboratory, Ecole Polytechnique Federale de Lausanne, Switzerland. http://lcav.epfl.ch/software/superresolution. Accessed 02 Jan 2018

  34. Vandewalle, P., Süsstrunk, S., Vetterli, M.: A frequency domain approach to registration of aliased images with application to super-resolution. EURASIP J. Adv. Sig. Process. 2006(1), 071459 (2006)

    Article  Google Scholar 

  35. Visual Geometry Group, Department of Engineering Science, University of Oxford, UK. http://www.robots.ox.ac.uk/~vgg/research/SR/index.html. Accessed 02 Jan 2018

  36. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  37. Wu, S., Ren, J., Chen, Z., Jin, W., Liu, X., Li, H., Pan, H., Guo, W.: Influence of reconstruction scale, spatial resolution and pixel spatial relationships on the sub-pixel mapping accuracy of a double-calculated spatial attraction model. Remote Sens. Environ. 210, 345–361 (2018)

    Article  Google Scholar 

  38. Yang, F., Chen, Y., Wang, R., Zhang, Q.: Super-resolution microwave imaging: time-domain tomography using highly accurate evolutionary optimization method. In: 2015 9th European Conference on Antennas and Propagation (EuCAP), pp. 1–4. IEEE (2015)

    Google Scholar 

  39. Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., Zhang, L.: Image super-resolution: the techniques, applications, and future. Sig. Process. 128, 389–408 (2016)

    Article  Google Scholar 

  40. Zhong, Y., Zhang, L.: Remote sensing image subpixel mapping based on adaptive differential evolution. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(5), 1306–1329 (2012)

    Article  Google Scholar 

  41. Zhu, H., Song, W., Tan, H., Wang, J., Jia, D.: Super resolution reconstruction based on adaptive detail enhancement for ZY-3 satellite images. In: Proceedings of ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, pp. 213–217 (2016)

    Google Scholar 

Download references

Acknowledgements

The reported work is a part of the SISPARE project run by Future Processing and funded by European Space Agency. In addition, the authors were partially supported by Statutory Research funds of Institute of Informatics, Silesian University of Technology, Gliwice, Poland (grants no. BKM-509/RAu2/2017 (DK) and BK-230/RAu2/2017 (MK)).

Author information

Authors and Affiliations

  1. Future Processing, Bojkowska 37A, 44-100, Gliwice, Poland

    Daniel Kostrzewa, Łukasz Skonieczny, Paweł Benecki & Michał Kawulok

  2. Institute of Informatics, Silesian University of Technology, Akademicka 16, 44-100, Gliwice, Poland

    Daniel Kostrzewa, Paweł Benecki & Michał Kawulok

Authors
  1. Daniel Kostrzewa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Łukasz Skonieczny
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paweł Benecki
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michał Kawulok
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Kostrzewa .

Editor information

Editors and Affiliations

  1. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Stanisław Kozielski

  2. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Dariusz Mrozek

  3. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Paweł Kasprowski

  4. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Bożena Małysiak-Mrozek

  5. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Daniel Kostrzewa

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kostrzewa, D., Skonieczny, Ł., Benecki, P., Kawulok, M. (2018). B4MultiSR: A Benchmark for Multiple-Image Super-Resolution Reconstruction. In: Kozielski, S., Mrozek, D., Kasprowski, P., Małysiak-Mrozek, B., Kostrzewa, D. (eds) Beyond Databases, Architectures and Structures. Facing the Challenges of Data Proliferation and Growing Variety. BDAS 2018. Communications in Computer and Information Science, vol 928. Springer, Cham. https://doi.org/10.1007/978-3-319-99987-6_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-99987-6_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-99986-9

  • Online ISBN: 978-3-319-99987-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation