Abstract
The past decades have witnessed growing development of image quality assessment (IQA) for natural images (NIs). However, since screen content images (SCIs) exhibit different visual characteristics from the NIs, few of NIs-oriented IQA methods can be directly applied on SCIs. In this paper, we present a quality prediction approach specially designed for SCIs, which is based on multi-task deep learning. First, we split a SCI into \(32\times 32\) patches and design a novel convolutional neural network (CNN) to predict the quality score of each SCI patch. Then, we propose an effective adaptive weighting algorithm for patch-level quality score aggregation. The proposed CNN is built on an end-to-end multi-task learning framework, which integrates the histogram of oriented gradient (HOG) features prediction task to the SCI quality prediction task for learning a better mapping between input SCI patch and its quality score. The proposed adaptive weighting algorithm for patch-level quality score aggregation further improves the representation ability of each SCI patch. Experimental results on two-largest SCI-oriented databases demonstrate that our proposed method is superior to the state-of-the-art no-reference IQA methods and most of the full-reference IQA methods.
Supported by the Natural Science Foundation of China under grant no. 61672375.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bai, Y., Yu, M.: Jiang: learning content-specific codebooks for blind quality assessment of screen content images. Sig. Process. 161(20), 248–258 (2019)
Chen, J., Shen, L., Zheng, L., Jiang, X.: Naturalization module in neural networks for screen content image quality assessment. IEEE Sig. Process. Lett. 25(11), 1685–1689 (2018)
Cheng, Z., Takeuchi, M.: A fast no-reference screen content image quality prediction using convolutional neural networks. In: Proceedings of the IEEE International Conference on Multimedia & Expo Workshops, pp. 1–6 (2018)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 886–893 (2005)
Fang, Y., Yan, J., Li, L., Wu, J., Lin, W.: No reference quality assessment for screen content images with both local and global feature representation. IEEE Trans. Image Process. 27(4), 1600–1610 (2017)
Fang, Y., Yan, J., Liu, J., Wang, S., Li, Q., Guo, Z.: Objective quality assessment of screen content images by uncertainty weighting. IEEE Trans. Image Process. 26(4), 2016–2027 (2017)
Fu, Y., Zeng, H., Ma, L., Ni, Z., Zhu, J., Ma, K.K.: Screen content image quality assessment using multi-scale difference of gaussian. IEEE Trans. Circ. Syst. Video Technol. 28(9), 2428–2432 (2018)
Gu, K., Qiao, J., Min, X., Yue, G., Lin, W., Thalmann, D.: Evaluating quality of screen content images via structural variation analysis. IEEE Trans. Vis. Comput. Graph. 24(10), 2689–2701 (2017)
Gu, K., et al.: Saliency-guided quality assessment of screen content images. IEEE Trans. Multimedia 18(6), 1098–1110 (2016)
Gu, K., Wang, S., Zhai, G., Ma, S., Lin, W.: Screen image quality assessment incorporating structural degradation measurement. In: Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), pp. 125–128 (2015)
Gu, K., Zhai, G., Lin, W., Yang, X., Zhang, W.: Learning a blind quality evaluation engine of screen content images. Neurocomputing 196, 140–149 (2016)
Gu, K., Zhou, J., Qiao, J.F., Zhai, G., Lin, W., Bovik, A.C.: No-reference quality assessment of screen content pictures. IEEE Trans. Image Process. 26(8), 4005–4018 (2017)
Jiang, X., Shen, L., Feng, G., Yu, L., An, P.: Deep optimization model for screen content image quality assessment using neural networks. arXiv preprint arXiv:1903.00705 (2019)
Kim, J., Lee, S.: Deep learning of human visual sensitivity in image quality assessment framework. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1676–1684 (2017)
Lu, N., Li, G.: Blind quality assessment for screen content images by orientation selectivity mechanism. Sig. Process. 145, 225–232 (2018)
Ni, Z., Zeng, H., Ma, L., Hou, J., Chen, J., Ma, K.K.: A gabor feature-based quality assessment model for the screen content images. IEEE Trans. Image Process. 27(9), 4516–4528 (2018)
Yang, J., Liu, J., Jiang, B., Lu, W.: No reference quality evaluation for screen content images considering texture feature based on sparse representation. Sig. Process. 153(89), 336–347 (2018)
Yue, G., Hou, C., Yan, W., Choi, L.K., Zhou, T., Hou, Y.: Blind quality assessment for screen content images via convolutional neural network. Digit. Sig. Process. 91, 21–30 (2019)
Zuo, L., Wang, H., Fu, J.: Screen content image quality assessment via convolutional neural network. In: Proceedings of the IEEE International Conference on Image Processing, pp. 2082–2086 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, R., Huang, Z., Liu, S. (2021). Multi-task Deep Learning for No-Reference Screen Content Image Quality Assessment. In: Lokoč, J., et al. MultiMedia Modeling. MMM 2021. Lecture Notes in Computer Science(), vol 12572. Springer, Cham. https://doi.org/10.1007/978-3-030-67832-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-67832-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67831-9
Online ISBN: 978-3-030-67832-6
eBook Packages: Computer ScienceComputer Science (R0)