Skip to main content

Advertisement

SpringerLink
Log in

Single image super-resolution with self-organization neural networks and image laplace gradient operator

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

At present, artificial neural networks have received wide applications in the field of image processing and image resolution because of their fast algorithm implementation and their high accuracy. Learning-based super-resolution methods used stochastic computation in their algorithms, leading to a manual and experimental adjustment of the regularization parameter to the solving imaging system model problem. In this paper, we present a new hybrid algorithm for low-resolution image enhancement, whose parameters are automatically adjusted by the training data and, in contrast to other super-resolution methods, do not require regular adjustment parameters. The method is a hybrid method that includes self-organizing maps as a preprocessor, the k-nearest neighbor algorithm as a classifier, and the Laplace operational edge detection operator as an edge extractor. We built a single external dictionary using a combination of low-resolution and high-resolution feature patches and then train our proposed network. Subsequently, we reconstruct the high-resolution image by Converting the low-resolution input image to feature patch vectors. Then for each vector, find the matching neuron in the network and retrieve all the vectors that belong to it. Then we train the k-nearest neighbor algorithm with these vectors plus the input vector and find the best vector most similar to the input vector and reconstruct our high super-resolution image. The proposed image super-resolution method presents in practical experiments better results with better resolution and quality than many traditional and state-of-the-art methods, both visually compared with each other using human and computational benchmarks to compare the quality of the image super-resolution algorithms. The proposed image enhancement method is best for reconstructing high-resolution images that need high-frequency details and sharp edges with a smooth slope of image objects in their structures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Ahn N, Kang B, Sohn K-A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network

  2. Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Amer Stat 46(3):175–185

    MathSciNet  Google Scholar 

  3. Andreas R, Dieter M (1999) Automatic labeling of self-organizing maps: Making a treasure-map reveal its secrets. In: Ning Z, Lizhu Z (eds) Methodologies for Knowledge Discovery and Data Mining. Springer, Berlin, pp 228–237

  4. Burger HC, Schuler CJ, Harmeling S (2012) Image denoising: Can plain neural networks compete with bm3d?. In: 2012 IEEE conference on computer vision and pattern recognition. https://doi.org/10.1109/CVPR.2012.6247952, pp 2392–2399

  5. Ding S, Zheng J, Liu Z, Zheng Y, Chen Y, Xu X, Lu J, Xie J (2021) High-resolution dermoscopy image synthesis with conditional generative adversarial networks. Biomed Signal Process Control 64:102224. https://doi.org/10.1016/j.bspc.2020.102224

    Article  Google Scholar 

  6. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution

  7. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision. Springer, pp 391–407

  8. Farsiu S, Robinson MD, Elad M, Milanfar P (2004) Fast and robust multiframe super resolution. IEEE Trans Image Process 13(10):1327–1344. https://doi.org/10.1109/TIP.2004.834669

    Article  Google Scholar 

  9. Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65. https://doi.org/10.1109/38.988747

    Article  Google Scholar 

  10. Hardie RC, Barnard KJ, Armstrong EE (1997) Joint map registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans Image Process 6(12):1621–1633. https://doi.org/10.1109/83.650116

    Article  Google Scholar 

  11. Haykin S, Network N (2004) A comprehensive foundation. Neural Netw 2(2004):41

    Google Scholar 

  12. Hou H, Andrews H (1978) Cubic splines for image interpolation and digital filtering. IEEE Trans Acoust Speech Signal Process 26(6):508–517. https://doi.org/10.1109/TASSP.1978.1163154

    Article  Google Scholar 

  13. Huang J, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2015.7299156, pp 5197–5206

  14. Hui Z, Gao X, Wang X (2020) Lightweight image super-resolution with feature enhancement residual network. Neurocomputing 404:50–60. https://doi.org/10.1016/j.neucom.2020.05.008

    Article  Google Scholar 

  15. Ignatov A, Timofte R, et al (2019) Pirm challenge on perceptual image enhancement on smartphones: report. In: European Conference on Computer Vision (ECCV) Workshops

  16. Jalali M, Behnam H, Davoodi F, Shojaeifard M (2020) Temporal super-resolution of 2d/3d echocardiography using cubic b-spline interpolation. Biomed Signal Process Control 58:101868. https://doi.org/10.1016/j.bspc.2020.101868

    Article  Google Scholar 

  17. Jiang K, Wang Z, Yi P, Jiang J (2020) Hierarchical dense recursive network for image super-resolution. Pattern Recogn 107:107475. https://doi.org/10.1016/j.patcog.2020.107475

    Article  Google Scholar 

  18. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2016.182, pp 1646–1654

  19. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2016.181, pp 1637–1645

  20. Kim KI, Kwon Y (2010) Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell 32(6):1127–1133. https://doi.org/10.1109/TPAMI.2010.25

    Article  Google Scholar 

  21. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  22. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2017.618, pp 5835–5843

  23. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network, pp 105–114

  24. Li X, Orchard MT (2001) New edge-directed interpolation. IEEE Trans Image Process 10(10):1521–1527. https://doi.org/10.1109/83.951537

    Article  Google Scholar 

  25. Lim B, Son S, Kim H, Nah S, Lee K (2017) Enhanced deep residual networks for single image super-resolution, pp 1132–1140

  26. Liu H, Lin Y, Ibragimov B, Zhang C (2020) Low dose 4d-ct super-resolution reconstruction via inter-plane motion estimation based on optical flow. Biomed Signal Process Control 62:102085. https://doi.org/10.1016/j.bspc.2020.102085

    Article  Google Scholar 

  27. Liu J, Liu Y, Wu H, Wang J, Li X, Zhang C (2021) Single image super-resolution using feature adaptive learning and global structure sparsity. Signal Process 188:108184. https://doi.org/10.1016/j.sigpro.2021.108184, https://www.sciencedirect.com/science/article/pii/S016516842100222X

    Article  Google Scholar 

  28. Macwan R, Patel N, Prajapati P, Chavda J (2014) A survey on various techniques of super resolution imaging. Int J Comput Appl 90(1)

  29. Mao X-J, Shen C, Yang Y-B (2016) Image denoising using very deep fully convolutional encoder-decoder networks with symmetric skip connections

  30. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE international conference on computer vision. ICCV 2001. https://doi.org/10.1109/ICCV.2001.937655, vol 2, pp 416–423

  31. Matsui Y, Ito K, Aramaki Y, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76. https://doi.org/10.1007/s11042-016-4020-z

  32. Nasrollahi K, Moeslund TB (2014) Super-resolution: a comprehensive survey. Mach Vis Appl 25(6):1423–1468

    Article  Google Scholar 

  33. Padraig C, Sarah D (2007) k-nearest neighbour classifiers. Mult Classif Syst

  34. Peleg T, Elad M (2014) A statistical prediction model based on sparse representations for single image super-resolution. IEEE Trans Image Process 23(6):2569–2582. https://doi.org/10.1109/TIP.2014.2305844

    Article  MathSciNet  Google Scholar 

  35. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein MS, Berg A, Fei-Fei L (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252

    Article  MathSciNet  Google Scholar 

  36. Schuler CJ, Burger HC, Harmeling S, Schölkopf B (2013) A machine learning approach for non-blind image deconvolution. In: 2013 IEEE conference on computer vision and pattern recognition. https://doi.org/10.1109/CVPR.2013.142, pp 1067–1074

  37. Schulter S, Leistner C, Bischof H (2015) Fast and accurate image upscaling with super-resolution forests. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2015.7299003, pp 3791–3799

  38. Shi W, Caballero J, Huszár F, Totz J, Aitken A, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network

  39. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2017.298, pp 2790–2798

  40. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: A persistent memory network for image restoration. In: 2017 IEEE international conference on computer vision (ICCV). https://doi.org/10.1109/ICCV.2017.486, pp 4549–4557

  41. Teuvo K (2014) Matlab implementations and applications of the self-organizing map, 1st edn. Unigrafia Oy, Helsinki

    Google Scholar 

  42. Tian C, Xu Y, Zuo W, Zhang B, Fei L, Lin C-W (2020) Coarse-to-fine cnn for image super-resolution. IEEE Trans Multimed:1–1. https://doi.org/10.1109/TMM.2020.2999182

  43. Tian C, Zhuge R, Wu Z, Xu Y, Zuo W, Chen C, Lin C-W (2020) Lightweight image super-resolution with enhanced cnn. Knowl-Based Syst 205:106235. https://doi.org/10.1016/j.knosys.2020.106235

    Article  Google Scholar 

  44. Timofte R, De V, Gool LV (2013) Anchored neighborhood regression for fast example-based super-resolution. In: 2013 IEEE international conference on computer vision. https://doi.org/10.1109/ICCV.2013.241, pp 1920–1927

  45. Timofte R, De Smet V, Van Gool L (2015) A+: Adjusted anchored neighborhood regression for fast super-resolution. 111–126. https://doi.org/10.1007/978-3-319-16817-3_8

  46. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: 2017 IEEE International Conference on Computer Vision (ICCV). https://doi.org/10.1109/ICCV.2017.514, pp 4809–4817

  47. Vesanto J, Himberg J, Alhoniemi E, Parhankangas J (2000) Self-organizing map in matlab: The som toolbox. Proceedings of the Proceedings of the Matlab DSP Conference, vol 99

  48. Wang X (2007) Laplacian operator-based edge detectors. IEEE Trans Pattern Anal Mach Intell 29(5):886–890. https://doi.org/10.1109/TPAMI.2007.1027

    Article  Google Scholar 

  49. Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: 2015 IEEE international conference on computer vision (ICCV). https://doi.org/10.1109/ICCV.2015.50, pp 370–378

  50. Wu J, Anisetti M, Wu W, Damiani E, Jeon G (2016) Bayer demosaicking with polynomial interpolation. IEEE Trans Image Process 25(11):5369–5382. https://doi.org/10.1109/TIP.2016.2604489

    Article  MathSciNet  Google Scholar 

  51. Yang F, Xu W, Tian Y (2017) Image super resolution using deep convolutional network based on topology aggregation structure. In: AIP Conference Proceedings, vol 1864. AIP Publishing LLC, p 020185

  52. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873. https://doi.org/10.1109/TIP.2010.2050625

    Article  MathSciNet  Google Scholar 

  53. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. 711–730. https://doi.org/10.1007/978-3-642-27413-8_47

  54. Zheng H, Qu X, Bai Z, Liu Y, Dong J, Peng X, Chen Z (2017) Multi-contrast brain magnetic resonance image super-resolution using the local weight similarity. BMC Med Imaging 17. https://doi.org/10.1186/s12880-016-0176-2

  55. Zheng H, Qu X, Bai Z, Liu Y, Dong J, Peng X, Chen Z (2017) Multi-contrast brain magnetic resonance image super-resolution using the local weight similarity. BMC Med Imaging 17. https://doi.org/10.1186/s12880-016-0176-2

Download references

Acknowledgements

This study was supported by Mahshar Branch, Islamic Azad University. The authors are very grateful for the constructive advice and guidance of reviewers.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Mahshahr Branch, Islamic Azad University, Mahshahr, Iran

    Khodabakhsh Ahmadian

  2. Department of Electrical and Computer Engineering, Tafresh Branch, National University, Tafresh, Iran

    Hamid-reza Reza-Alikhani

Authors
  1. Khodabakhsh Ahmadian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hamid-reza Reza-Alikhani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Khodabakhsh Ahmadian.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmadian, K., Reza-Alikhani, Hr. Single image super-resolution with self-organization neural networks and image laplace gradient operator. Multimed Tools Appl 81, 10607–10630 (2022). https://doi.org/10.1007/s11042-022-11970-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-11970-9

Keywords

Search

Navigation