Skip to main content
SpringerLink
Log in

Real-time deep satellite image quality assessment

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

A method for deep satellite image quality assessment based on no-reference satellite images is proposed. We design suitable deep convolutional neural networks, which are named satellite image quality assessment of deep convolutional neural networks (SIQA-DCNN) and SIQA-DCNN++. These sophisticated methods can remove various distorted satellite images in real-time remote sensing. The novelty of this method lies in the objective assessment and restoration of the deep model which understands various distorted satellite images in high- and low-resolution problems. The activation function has a lower computational time and ensures the deactivation of noise by making the mean activators close to zero. Our methods are also effective for transfer learning, which can be used to adequately investigate satellite image classification in deep satellite image quality assessment. Using Spearman’s rank order correlation coefficient (SROCC) and linear correlation coefficient (LCC) evaluations, we demonstrated that our methods show better performance than other algorithms, with more than 0.90 of SROCC and LCC values compared to the full-reference and no-reference satellite image in MODIS/Terra and USGS datasets. Regarding computational complexity, we obtained better performance in operational function times. As compared to other methods, SIQA-DCNN and SIQA-DCNN++ also reduced computational time by more than 40 and 56%, respectively, when applied to the USGS dataset, and by more than 46 and 60% respectively, when applied to the MODIS/Terra dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Sheikh, H.R., Bovik, A.C.: Handbook of Image and Video Processing: Information Theoretic Approaches to Image Quality Assessment, pp. 975–989. Elsevier, Amsterdam (2005). https://doi.org/10.1016/B978-012119792-6/50120-0

    Book  Google Scholar 

  2. Reeve, H.C., Lim, J.S.: Reduction of blocking effects in image coding. SPIE Opt. Eng. 23(1), 34–37 (1984). https://doi.org/10.1117/12.7973248

    Article  Google Scholar 

  3. List, P., Joch, A., Lainema, J., Bjøntegaard, G., Karczewicz, M.: Adaptive deblocking filter. IEEE Trans. Circuits Syst. Video Technol. 13(7), 614–619 (2003). https://doi.org/10.1109/TCSVT.2003.815175

    Article  Google Scholar 

  4. Oktem, H., Katkovnik, V., Egiazarin, K., Astola, J.: Local adaptive transform based image denoising with varying window size. In: Proceedings of 2001 IEEE International Conference on Image Processing, vol. 1, pp. 273–276 (2001). https://doi.org/10.1109/ICIP.2001.959006

  5. Wang, C., Zhou, J., Liu, S.: Adaptive non-local means filter for image deblocking. Elsevier Signal Process. Image Commun. 28(5), 522–530 (2013). https://doi.org/10.1016/j.image.2013.01.006

    Article  Google Scholar 

  6. Xiong, Z., Orchard, M.T., Zhang, Y.-Q.: A deblocking algorithm for JPEG compressed images using overcomplete wavelet representations. IEEE Trans. Circuits Syst. Video Technol. 7(2), 433–437 (1997). https://doi.org/10.1109/76.564123

    Article  Google Scholar 

  7. Liew, A.W.-C., Yan, H.: Blocking artifacts suppression in block-coded images using overcomplete wavelet representation. IEEE Trans. Circuits Syst. Video Technol. 14(4), 450–461 (2004). https://doi.org/10.1109/TCSVT.2004.825555

    Article  Google Scholar 

  8. Foi, A., Katkovnik, V., Egiazarian, K.: Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images. IEEE Trans. Image Process. 16(5), 1395–1411 (2007). https://doi.org/10.1109/TIP.2007.891788

    Article  MathSciNet  Google Scholar 

  9. Jancsary, J., Nowozin, S., Rother, C.: Loss-specific training of non-parametric image restoration models: a new state of the art. In: Proceedings of Springer European Conference on Computer Vision (ECCV 2012), pp. 112-125 (2012). https://doi.org/10.1007/978-3-642-33786-4-9

  10. Li, Y., Guo, F., Tan, R. T., Brown, M. S.: A contrast enhancement framework with JPEG artifacts suppression. In: Proceedings of Springer European Conference on Computer Vision (ECCV 2014), pp. 174–188 (2014). https://doi.org/10.1007/978-3-319-10605-2-12

  11. Yang, Y., Galatsanos, N.P., Katsaggelos, A.K.: Projection-based spatially adaptive reconstruction of block-transform compressed images. IEEE Trans. Image Process. 4(7), 896–908 (1995). https://doi.org/10.1109/83.392332

    Article  Google Scholar 

  12. Zou, J.J., Yan, H.: A deblocking method for BDCT compressed images based on adaptive projections. IEEE Trans. Circuits Syst. Video Technol. 15(3), 430–435 (2005). https://doi.org/10.1109/TCSVT.2004.842610

    Article  Google Scholar 

  13. Liew, A.W.-C., Yan, H., Law, N.-F.: POCS-based blocking artifacts suppression using a smoothness constraint set with explicit region modeling. IEEE Trans. Circuits Syst. Video Technol. 15(6), 795–800 (2005). https://doi.org/10.1109/TCSVT.2005.848303

    Article  Google Scholar 

  14. Jung, C., Jiao, L., Qi, H., Sun, T.: Image deblocking via sparse representation. Elsevier Signal Process. Image Commun. 27(6), 663–677 (2012). https://doi.org/10.1016/j.image.2012.03.002

    Article  Google Scholar 

  15. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  16. Sheikh, H.R., Bovik, A.C., de Veciana, G.: An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans. Image Process. 14(12), 2117–2128 (2005). https://doi.org/10.1109/TIP.2005.859389

    Article  Google Scholar 

  17. Wang, Z., Li, Q.: Information content weighting for perceptual image quality assessment. IEEE Trans. Image Process. 20(5), 1185–1198 (2011). https://doi.org/10.1109/TIP.2010.2092435

    Article  MathSciNet  MATH  Google Scholar 

  18. Xue, W., Zhang, L., Mou, X., Bovik, A.C.: Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans. Image Process. 23(2), 684–695 (2014). https://doi.org/10.1109/TIP.2013.2293423

    Article  MathSciNet  MATH  Google Scholar 

  19. Moorthy, A.K., Bovik, A.C.: Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans. Image Process. 20(12), 3350–3364 (2011). https://doi.org/10.1109/TIP.2011.2147325

    Article  MathSciNet  MATH  Google Scholar 

  20. Silverstein, D.A., Farrell, J.E.: The relationship between image fidelity and image quality. Proc. IEEE Int. Conf. Image Process. 1, 881–884 (1996). https://doi.org/10.1109/ICIP.1996.559640

    Article  Google Scholar 

  21. Wang, Z., Bovik, A. C., Evans, B. L.: Blind measurement of blocking artifacts in images. In: Proceedings of 2000 IEEE International Conference on Image Processing, vol. 3, pp. 981-984 (2000). https://doi.org/10.1109/ICIP.2000.899622

  22. Saad, M.A., Bovik, A.-C., Charrier, C.: Blind image quality assessment: a natural scene statistics approach in the dct domain. IEEE Trans. Image Process. 21(8), 3339–3352 (2012). https://doi.org/10.1109/TIP.2012.2191563

    Article  MathSciNet  MATH  Google Scholar 

  23. Jain, A.K.: Fundamental of Digital Image Processing. Prentice-Hall, Englewood Cliffs (1989)

    MATH  Google Scholar 

  24. Brownlee, J.: An Introduction to Feature Selection (2014).http://machinelearningmastery.com/an-introduction-to-feature-selection/. Accessed 1 Jun 2016

  25. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012). https://doi.org/10.1109/TIP.2012.2214050

    Article  MathSciNet  MATH  Google Scholar 

  26. Kumar, P. Ye, J., Kang, L., Doermann, D.: Unsupervised feature learning framework for no-reference image quality assessment. In: Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1098-1105 (2012). https://doi.org/10.1109/CVPR.2012.6247789

  27. Kumar, P. Ye, J., Kang, L., Doermann, D.: Real-time no-reference image quality assessment based on filter learning. In: Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 987–994 (2013). https://doi.org/10.1109/CVPR.2013.132

  28. Kumar, P. Ye, J., Kang, L., Doermann, D.: Convolutional neural networks for no-reference image quality assessment. In: Proceedings of 2014 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). (2014). https://doi.org/10.1109/CVPR.2014.224

  29. Kang, L., Ye, P., Li, Y., Doermann, D.: Simultaneous estimation of image quality and distortion via multi-task convolutional neural networks. In: Proceedings of 2015 IEEE International Conference on Image Processing (ICIP), pp. 2791–2795 (2015). https://doi.org/10.1109/ICIP.2015.7351311

  30. Krizhevsky, A., Sutskever, I., Hinton, G. E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of 25th International Conference on Neural Information Processing Systems (NIPS’12), vol. 1, pp. 1097–1105 (2012). https://doi.org/10.1145/3065386

    Article  Google Scholar 

  31. Deng, L., Yu, D.: Deep learning: methods and applications. Found. Trends Signal Process. 7(3–4), 197–387 (2014). https://doi.org/10.1561/2000000039

    Article  MathSciNet  MATH  Google Scholar 

  32. Dong, C., Loy, C. C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Proceedings of European Conference on Computer Vision (ECCV 2014), vol. 8692, pp. 184–199 (2014). https://doi.org/10.1007/978-3-319-10593-2-13

  33. Nair, V., Hinton, G. E. Rectified linear units improve restricted Boltzmann machines. In: Proceedings of 27th International Conference on Machine Learning (ICML’10), pp. 807–814. https://dl.acm.org/citation.cfm?id=3104425 (2010). Accessed 6 Jun 2016

  34. Heusel, M., Clevert, D-A., Klambauer, G., Mayr, A., Schwarzbauer, K., Unterthiner, T., Hochreiter, S.: ELU-networks: fast and accurate CNN learning on imagenet. In: Proceedings of 2015 IEEE International Conference on International Conference on Computer Vision (ICCV), vol. 1. http://image-net.org/challenges/posters/JKU-EN-RGB-Schwarz-poster.pdf (2015). Accessed 10 Jun 2016

  35. Clevert, D-A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). The 4th International Conference on Learning Representations (ICLR 2016), pp. 1–14. https://arxiv.org/abs/1511.07289 (2016). Accessed 6 Jun 2016

  36. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016). https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  37. Risnandar, Aritsugi, M.: Detection of blocking artifact on satellite image and its new evaluator. In: Proceedings of 2016 IEEE 13th International Conference on Signal Processing (ICSP), pp. 788–793 (2016). https://doi.org/10.1109/ICSP.2016.7877939

  38. Risnandar, Aritsugi, M.: Deblocking artifact of satellite image based on adaptive soft-threshold anisotropic filter in wavelet. IEICE Trans. Inf. Syst. 101(6), 1605–1620 (2018). https://doi.org/10.1587/transinf.2018EDP7013

    Article  Google Scholar 

  39. Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of International Conference on Machine Learning, vol. 28, pp. 1–6. https://ai.stanford.edu/~amaas/papers/relu-hybrid-icml2013-final.pdf (2013). Accessed 6 Jun 2016

  40. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1026–1034 (2015). https://doi.org/10.1109/ICCV.2015.123

  41. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Mag. Commun. ACM 60(6), 84–90 (2017). https://doi.org/10.1145/3065386

    Article  Google Scholar 

  42. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Proceedings of Springer European Conference on Computer Vision (ECCV 2014), pp. 818–833 (2014). https://doi.org/10.1007/978-3-319-10590-1-53

  43. Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1717–1724 (2014). https://doi.org/10.1109/CVPR.2014.222

  44. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 580–587 (2014). https://doi.org/10.1109/CVPR.2014.81

  45. Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: OverFeat: integrated recognition, localization and detection using convolutional networks. Computer Vision and Pattern Recognition (CVPR 2014), pp. 1–16. https://arxiv.org/abs/1312.6229 (2014). Accessed 6 Sept 2017

  46. Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Proceedings of 27th International Conference on Neural Information Processing Systems (NIPS’14), vol. 2, pp. 3320–3328. https://arxiv.org/pdf/1411.1792.pdf (2014). Accessed 6 Sept 2017

  47. Karpathy, A.: Convolutional neural networks for visual recognition. Course note, Department of Science, Stanford University. http://cs231n.stanford.edu (2018). Accessed 13 Feb 2018

  48. Earthdata-NASA: Earthdata-NASA Datasets. https://earthdata.nasa.gov/ (2016). Accessed 5 May 2016

  49. USGS: USGS Datasets. https://earthdata.nasa.gov/ (2016). Accessed 7 May 2016

  50. Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011). https://doi.org/10.1109/TIP.2011.2109730

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Informatics, Indonesian Institute of Sciences, Bandung, West Java, Indonesia

    Risnandar

  2. Faculty of Advanced Science and Technology, Kumamoto University, Kumamoto-shi, Kumamoto-ken, Japan

    Masayoshi Aritsugi

Authors
  1. Risnandar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Masayoshi Aritsugi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Risnandar .

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Risnandar, Aritsugi, M. Real-time deep satellite image quality assessment. J Real-Time Image Proc 15, 477–494 (2018). https://doi.org/10.1007/s11554-018-0798-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-018-0798-4

Keywords

Search

Navigation