Abstract
Image aesthetics assessment (IAA) aims at predicting the perceived aesthetic quality of images. Intuitively, the technical quality of an image has significant impact on its aesthetic quality, e.g., an image with noticeable distortions is not likely to have very high aesthetic quality. However, this characteristic has rarely been considered when designing modern IAA models. Motivated by this, this paper presents a new Technical Quality-assisted multi-task deep network for image Aesthetic Quality assessment, dubbed TQ4AQ. Specifically, we first extract theme-aware general aesthetic features based on the attention mechanism. Meantime, hand-crafted technical quality features are extracted from aesthetic images. Then the general aesthetic features are utilized to predict the technical quality features and the aesthetic quality simultaneously, based on which technical quality features are integrated. By this means, the aesthetic features are empowered the capability of understanding technical distortions, and more comprehensive aesthetic feature representations are obtained for IAA. Extensive experiments demonstrate the advantage of the proposed TQ4AQ model over the state-of-the-arts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, Q., et al.: Adaptive fractional dilated convolution network for image aesthetics assessment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14114–14123 (2020)
Gu, K., Zhai, G., Yang, X., Zhang, W.: Using free energy principle for blind image quality assessment. IEEE Trans. Multimedia 17(1), 50–63 (2015)
Kang, C., Valenzise, G., Dufaux, F.: EVA: an explainable visual aesthetics dataset. In: Joint Workshop on Aesthetic and Technical Quality Assessment of Multimedia and Media Analytics for Societal Trends, pp. 5–13 (2020)
Ke, J., Wang, Q., Wang, Y., Milanfar, P., Yang, F.: MUSIQ: multi-scale image quality transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5148–5157 (2021)
Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 419–426 (2006)
Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 662–679. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_40
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)
Li, L., Duan, J., Yang, Y., Xu, L., Li, Y., Guo, Y.: Psychology inspired model for hierarchical image aesthetic attribute prediction. In: Proceedings of the IEEE Conference on Multimedia and Expo, pp. 1–6 (2022)
Li, L., Zhu, H., Zhao, S., Ding, G., Lin, W.: Personality-assisted multi-task learning for generic and personalized image aesthetics assessment. IEEE Trans. Image Process. 29, 3898–3910 (2020)
Liu, X., Van De Weijer, J., Bagdanov, A.D.: Rankiqa: learning from rankings for no-reference image quality assessment. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1040–1049 (2017)
Lu, X., Lin, Z., Shen, X., Mech, R., Wang, J.Z.: Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 990–998 (2015)
Ma, S., Liu, J., Wen Chen, C.: A-lamp: adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4535–4544 (2017)
Mai, L., Jin, H., Liu, F.: Composition-preserving deep photo aesthetics assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 497–506 (2016)
Malu, G., Bapi, R.S., Indurkhya, B.: Learning photography aesthetics with deep CNNs. arXiv preprint arXiv:1707.03981 (2017)
Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)
Murray, N., Gordo, A.: A deep architecture for unified aesthetic prediction. arXiv preprint arXiv:1708.04890 (2017)
Murray, N., Marchesotti, L., Perronnin, F.: AVA: a large-scale database for aesthetic visual analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2408–2415 (2012)
Pan, B., Wang, S., Jiang, Q.: Image aesthetic assessment assisted by attributes through adversarial learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 679–686 (2019)
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text-conditional image generation with clip latents (2022)
She, D., Lai, Y.K., Yi, G., Xu, K.: Hierarchical layout-aware graph convolutional network for unified aesthetics assessment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8475–8484 (2021)
Sheng, K., Dong, W., Ma, C., Mei, X., Huang, F., Hu, B.G.: Attention-based multi-patch aggregation for image aesthetic assessment. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 879–886 (2018)
Sun, S., Yu, T., Xu, J., Zhou, W., Chen, Z.: GraphiQA: learning distortion graph representations for blind image quality assessment. IEEE Trans. Multimedia 1 (2022). https://doi.org/10.1109/TMM.2022.3152942
Sun, W.T., Chao, T.H., Kuo, Y.H., Hsu, W.H.: Photo filter recommendation by category-aware aesthetic learning. IEEE Trans. Multimedia 19(8), 1870–1880 (2017)
Talebi, H., Milanfar, P.: NIMA: neural image assessment. IEEE Trans. Image Process. 27(8), 3998–4011 (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017)
Wang, W., Shen, J., Ling, H.: A deep network solution for attention and aesthetics aware photo cropping. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1531–1544 (2018)
Xue, W., Mou, X., Zhang, L., Bovik, A.C., Feng, X.: Blind image quality assessment using joint statistics of gradient magnitude and laplacian features. IEEE Trans. Image Process. 23(11), 4850–4862 (2014)
Zeng, H., Cao, Z., Zhang, L., Bovik, A.C.: A unified probabilistic formulation of image aesthetic assessment. IEEE Trans. Image Process. 29, 1548–1561 (2019)
Zhang, W., Li, D., Ma, C., Zhai, G., Yang, X., Ma, K.: Continual learning for blind image quality assessment. IEEE Trans. Pattern Anal. Mach. Intell. 45(3), 2864–2878 (2023)
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2017)
Zhu, H., Li, L., Wu, J., Dong, W., Shi, G.: MetaiQA: deep meta-learning for no-reference image quality assessment. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14143–14152 (2020)
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grants 62171340, 62301378 and 61991451, and the OPPO Research Fund.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sheng, X. et al. (2024). Technical Quality-Assisted Image Aesthetics Quality Assessment. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14435. Springer, Singapore. https://doi.org/10.1007/978-981-99-8552-4_5
Download citation
DOI: https://doi.org/10.1007/978-981-99-8552-4_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8551-7
Online ISBN: 978-981-99-8552-4
eBook Packages: Computer ScienceComputer Science (R0)