Abstract
A method for deep satellite image quality assessment based on no-reference satellite images is proposed. We design suitable deep convolutional neural networks, which are named satellite image quality assessment of deep convolutional neural networks (SIQA-DCNN) and SIQA-DCNN++. These sophisticated methods can remove various distorted satellite images in real-time remote sensing. The novelty of this method lies in the objective assessment and restoration of the deep model which understands various distorted satellite images in high- and low-resolution problems. The activation function has a lower computational time and ensures the deactivation of noise by making the mean activators close to zero. Our methods are also effective for transfer learning, which can be used to adequately investigate satellite image classification in deep satellite image quality assessment. Using Spearman’s rank order correlation coefficient (SROCC) and linear correlation coefficient (LCC) evaluations, we demonstrated that our methods show better performance than other algorithms, with more than 0.90 of SROCC and LCC values compared to the full-reference and no-reference satellite image in MODIS/Terra and USGS datasets. Regarding computational complexity, we obtained better performance in operational function times. As compared to other methods, SIQA-DCNN and SIQA-DCNN++ also reduced computational time by more than 40 and 56%, respectively, when applied to the USGS dataset, and by more than 46 and 60% respectively, when applied to the MODIS/Terra dataset.









Similar content being viewed by others
References
Sheikh, H.R., Bovik, A.C.: Handbook of Image and Video Processing: Information Theoretic Approaches to Image Quality Assessment, pp. 975–989. Elsevier, Amsterdam (2005). https://doi.org/10.1016/B978-012119792-6/50120-0
Reeve, H.C., Lim, J.S.: Reduction of blocking effects in image coding. SPIE Opt. Eng. 23(1), 34–37 (1984). https://doi.org/10.1117/12.7973248
List, P., Joch, A., Lainema, J., Bjøntegaard, G., Karczewicz, M.: Adaptive deblocking filter. IEEE Trans. Circuits Syst. Video Technol. 13(7), 614–619 (2003). https://doi.org/10.1109/TCSVT.2003.815175
Oktem, H., Katkovnik, V., Egiazarin, K., Astola, J.: Local adaptive transform based image denoising with varying window size. In: Proceedings of 2001 IEEE International Conference on Image Processing, vol. 1, pp. 273–276 (2001). https://doi.org/10.1109/ICIP.2001.959006
Wang, C., Zhou, J., Liu, S.: Adaptive non-local means filter for image deblocking. Elsevier Signal Process. Image Commun. 28(5), 522–530 (2013). https://doi.org/10.1016/j.image.2013.01.006
Xiong, Z., Orchard, M.T., Zhang, Y.-Q.: A deblocking algorithm for JPEG compressed images using overcomplete wavelet representations. IEEE Trans. Circuits Syst. Video Technol. 7(2), 433–437 (1997). https://doi.org/10.1109/76.564123
Liew, A.W.-C., Yan, H.: Blocking artifacts suppression in block-coded images using overcomplete wavelet representation. IEEE Trans. Circuits Syst. Video Technol. 14(4), 450–461 (2004). https://doi.org/10.1109/TCSVT.2004.825555
Foi, A., Katkovnik, V., Egiazarian, K.: Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images. IEEE Trans. Image Process. 16(5), 1395–1411 (2007). https://doi.org/10.1109/TIP.2007.891788
Jancsary, J., Nowozin, S., Rother, C.: Loss-specific training of non-parametric image restoration models: a new state of the art. In: Proceedings of Springer European Conference on Computer Vision (ECCV 2012), pp. 112-125 (2012). https://doi.org/10.1007/978-3-642-33786-4-9
Li, Y., Guo, F., Tan, R. T., Brown, M. S.: A contrast enhancement framework with JPEG artifacts suppression. In: Proceedings of Springer European Conference on Computer Vision (ECCV 2014), pp. 174–188 (2014). https://doi.org/10.1007/978-3-319-10605-2-12
Yang, Y., Galatsanos, N.P., Katsaggelos, A.K.: Projection-based spatially adaptive reconstruction of block-transform compressed images. IEEE Trans. Image Process. 4(7), 896–908 (1995). https://doi.org/10.1109/83.392332
Zou, J.J., Yan, H.: A deblocking method for BDCT compressed images based on adaptive projections. IEEE Trans. Circuits Syst. Video Technol. 15(3), 430–435 (2005). https://doi.org/10.1109/TCSVT.2004.842610
Liew, A.W.-C., Yan, H., Law, N.-F.: POCS-based blocking artifacts suppression using a smoothness constraint set with explicit region modeling. IEEE Trans. Circuits Syst. Video Technol. 15(6), 795–800 (2005). https://doi.org/10.1109/TCSVT.2005.848303
Jung, C., Jiao, L., Qi, H., Sun, T.: Image deblocking via sparse representation. Elsevier Signal Process. Image Commun. 27(6), 663–677 (2012). https://doi.org/10.1016/j.image.2012.03.002
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861
Sheikh, H.R., Bovik, A.C., de Veciana, G.: An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans. Image Process. 14(12), 2117–2128 (2005). https://doi.org/10.1109/TIP.2005.859389
Wang, Z., Li, Q.: Information content weighting for perceptual image quality assessment. IEEE Trans. Image Process. 20(5), 1185–1198 (2011). https://doi.org/10.1109/TIP.2010.2092435
Xue, W., Zhang, L., Mou, X., Bovik, A.C.: Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans. Image Process. 23(2), 684–695 (2014). https://doi.org/10.1109/TIP.2013.2293423
Moorthy, A.K., Bovik, A.C.: Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans. Image Process. 20(12), 3350–3364 (2011). https://doi.org/10.1109/TIP.2011.2147325
Silverstein, D.A., Farrell, J.E.: The relationship between image fidelity and image quality. Proc. IEEE Int. Conf. Image Process. 1, 881–884 (1996). https://doi.org/10.1109/ICIP.1996.559640
Wang, Z., Bovik, A. C., Evans, B. L.: Blind measurement of blocking artifacts in images. In: Proceedings of 2000 IEEE International Conference on Image Processing, vol. 3, pp. 981-984 (2000). https://doi.org/10.1109/ICIP.2000.899622
Saad, M.A., Bovik, A.-C., Charrier, C.: Blind image quality assessment: a natural scene statistics approach in the dct domain. IEEE Trans. Image Process. 21(8), 3339–3352 (2012). https://doi.org/10.1109/TIP.2012.2191563
Jain, A.K.: Fundamental of Digital Image Processing. Prentice-Hall, Englewood Cliffs (1989)
Brownlee, J.: An Introduction to Feature Selection (2014).http://machinelearningmastery.com/an-introduction-to-feature-selection/. Accessed 1 Jun 2016
Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012). https://doi.org/10.1109/TIP.2012.2214050
Kumar, P. Ye, J., Kang, L., Doermann, D.: Unsupervised feature learning framework for no-reference image quality assessment. In: Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1098-1105 (2012). https://doi.org/10.1109/CVPR.2012.6247789
Kumar, P. Ye, J., Kang, L., Doermann, D.: Real-time no-reference image quality assessment based on filter learning. In: Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 987–994 (2013). https://doi.org/10.1109/CVPR.2013.132
Kumar, P. Ye, J., Kang, L., Doermann, D.: Convolutional neural networks for no-reference image quality assessment. In: Proceedings of 2014 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). (2014). https://doi.org/10.1109/CVPR.2014.224
Kang, L., Ye, P., Li, Y., Doermann, D.: Simultaneous estimation of image quality and distortion via multi-task convolutional neural networks. In: Proceedings of 2015 IEEE International Conference on Image Processing (ICIP), pp. 2791–2795 (2015). https://doi.org/10.1109/ICIP.2015.7351311
Krizhevsky, A., Sutskever, I., Hinton, G. E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of 25th International Conference on Neural Information Processing Systems (NIPS’12), vol. 1, pp. 1097–1105 (2012). https://doi.org/10.1145/3065386
Deng, L., Yu, D.: Deep learning: methods and applications. Found. Trends Signal Process. 7(3–4), 197–387 (2014). https://doi.org/10.1561/2000000039
Dong, C., Loy, C. C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Proceedings of European Conference on Computer Vision (ECCV 2014), vol. 8692, pp. 184–199 (2014). https://doi.org/10.1007/978-3-319-10593-2-13
Nair, V., Hinton, G. E. Rectified linear units improve restricted Boltzmann machines. In: Proceedings of 27th International Conference on Machine Learning (ICML’10), pp. 807–814. https://dl.acm.org/citation.cfm?id=3104425 (2010). Accessed 6 Jun 2016
Heusel, M., Clevert, D-A., Klambauer, G., Mayr, A., Schwarzbauer, K., Unterthiner, T., Hochreiter, S.: ELU-networks: fast and accurate CNN learning on imagenet. In: Proceedings of 2015 IEEE International Conference on International Conference on Computer Vision (ICCV), vol. 1. http://image-net.org/challenges/posters/JKU-EN-RGB-Schwarz-poster.pdf (2015). Accessed 10 Jun 2016
Clevert, D-A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). The 4th International Conference on Learning Representations (ICLR 2016), pp. 1–14. https://arxiv.org/abs/1511.07289 (2016). Accessed 6 Jun 2016
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016). https://doi.org/10.1109/TPAMI.2015.2439281
Risnandar, Aritsugi, M.: Detection of blocking artifact on satellite image and its new evaluator. In: Proceedings of 2016 IEEE 13th International Conference on Signal Processing (ICSP), pp. 788–793 (2016). https://doi.org/10.1109/ICSP.2016.7877939
Risnandar, Aritsugi, M.: Deblocking artifact of satellite image based on adaptive soft-threshold anisotropic filter in wavelet. IEICE Trans. Inf. Syst. 101(6), 1605–1620 (2018). https://doi.org/10.1587/transinf.2018EDP7013
Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of International Conference on Machine Learning, vol. 28, pp. 1–6. https://ai.stanford.edu/~amaas/papers/relu-hybrid-icml2013-final.pdf (2013). Accessed 6 Jun 2016
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1026–1034 (2015). https://doi.org/10.1109/ICCV.2015.123
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Mag. Commun. ACM 60(6), 84–90 (2017). https://doi.org/10.1145/3065386
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Proceedings of Springer European Conference on Computer Vision (ECCV 2014), pp. 818–833 (2014). https://doi.org/10.1007/978-3-319-10590-1-53
Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1717–1724 (2014). https://doi.org/10.1109/CVPR.2014.222
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 580–587 (2014). https://doi.org/10.1109/CVPR.2014.81
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: OverFeat: integrated recognition, localization and detection using convolutional networks. Computer Vision and Pattern Recognition (CVPR 2014), pp. 1–16. https://arxiv.org/abs/1312.6229 (2014). Accessed 6 Sept 2017
Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Proceedings of 27th International Conference on Neural Information Processing Systems (NIPS’14), vol. 2, pp. 3320–3328. https://arxiv.org/pdf/1411.1792.pdf (2014). Accessed 6 Sept 2017
Karpathy, A.: Convolutional neural networks for visual recognition. Course note, Department of Science, Stanford University. http://cs231n.stanford.edu (2018). Accessed 13 Feb 2018
Earthdata-NASA: Earthdata-NASA Datasets. https://earthdata.nasa.gov/ (2016). Accessed 5 May 2016
USGS: USGS Datasets. https://earthdata.nasa.gov/ (2016). Accessed 7 May 2016
Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011). https://doi.org/10.1109/TIP.2011.2109730
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Risnandar, Aritsugi, M. Real-time deep satellite image quality assessment. J Real-Time Image Proc 15, 477–494 (2018). https://doi.org/10.1007/s11554-018-0798-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-018-0798-4