Skip to main content
SpringerLink
Log in

Image Aesthetic Assessment Based on Perception Consistency

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11858))

Included in the following conference series:

Abstract

Automatically assessing the aesthetic quality of images that is consistent with humans is a challenging task. Previous works based on Convolution Neural Network (CNN) lacks of perception consistency in two aspects. First, they mainly extract features from the entire image without distinguishing between the foreground and background. Second, they classify images with highly-compressed semantic feature. In this paper, we proposed a visual perception network (VP-Net) to support perception consistency learning. It was designed as a double-subnet network which can learn from subject region feature and multi-level features. In addition, a subject region search algorithm was proposed to find out a composed of multiple subject regions. Experimental results on a large scale aesthetic dataset (AVA) have demonstrated the superiority of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chen, Y., Hu, Y., Zhang, L., Li, P., Zhang, C.: Engineering deep representations for modeling aesthetic perception. IEEE Trans. Cybern. 48(11), 3092–3104 (2018)

    Article  Google Scholar 

  2. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006). https://doi.org/10.1007/11744078_23

    Chapter  Google Scholar 

  3. Datta, R., Li, J., Wang, J.Z.: Algorithmic inferencing of aesthetics and emotion in natural images: an exposition. In: 2008 15th IEEE International Conference on Image Processing, ICIP 2008, pp. 105–108. IEEE (2008)

    Google Scholar 

  4. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  5. Kairanbay, M., See, J., Wong, L.K., Hii, Y.L.: Filling the gaps: reducing the complexity of networks for multi-attribute image aesthetic prediction. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3051–3055. IEEE (2017)

    Google Scholar 

  6. Kao, Y., He, R., Huang, K.: Deep aesthetic quality assessment with semantic information. IEEE Trans. Image Process. 26(3), 1482–1495 (2017)

    Article  MathSciNet  Google Scholar 

  7. Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 662–679. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_40

    Chapter  Google Scholar 

  8. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  9. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)

    Article  Google Scholar 

  10. Lin, T.Y., Dollár, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944 (2017)

    Google Scholar 

  11. Liu, L., Chen, R., Wolf, L., Cohen-Or, D.: Optimizing photo composition. In: Computer Graphics Forum, vol. 29, pp. 469–478. Wiley (2010)

    Google Scholar 

  12. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  13. Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: RAPID: rating pictorial aesthetics using deep learning. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 457–466. ACM (2014)

    Google Scholar 

  14. Lu, X., Lin, Z., Shen, X., Mech, R., Wang, J.Z.: Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 990–998 (2015)

    Google Scholar 

  15. Luo, W., Wang, X., Tang, X.: Content-based photo quality assessment. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2206–2213. IEEE (2011)

    Google Scholar 

  16. Luo, Y., Tang, X.: Photo and video quality evaluation: focusing on the subject. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 386–399. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88690-7_29

    Chapter  Google Scholar 

  17. Mai, L., Jin, H., Liu, F.: Composition-preserving deep photo aesthetics assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 497–506 (2016)

    Google Scholar 

  18. Meng, X., Gao, F., Shi, S., Zhu, S., Zhu, J.: MLANs: image aesthetic assessment via multi-layer aggregation networks. In: 2018 Eighth International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 1–6. IEEE (2018)

    Google Scholar 

  19. Murray, N., Marchesotti, L., Perronnin, F.: AVA: a large-scale database for aesthetic visual analysis. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2408–2415. IEEE (2012)

    Google Scholar 

  20. Pan, J., Sayrol, E., Giro-i Nieto, X., McGuinness, K., O’Connor, N.E.: Shallow and deep convolutional networks for saliency prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 598–606 (2016)

    Google Scholar 

  21. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

    Google Scholar 

  22. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  23. Wang, Z., Liu, D., Chang, S., Dolcos, F., Beck, D., Huang, T.: Image aesthetics assessment using Deep Chatterjee’s machine. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 941–948. IEEE (2017)

    Google Scholar 

  24. Xie, S., Tu, Z.: Holistically-nested edge detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1395–1403 (2015)

    Google Scholar 

  25. Yeh, M.C., Cheng, Y.C.: Relative features for photo quality assessment. In: 2012 19th IEEE International Conference on Image Processing (ICIP), pp. 2861–2864. IEEE (2012)

    Google Scholar 

  26. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

    Chapter  Google Scholar 

  27. Zhang, C., Zhu, C., Xu, X., Liu, Y., Xiao, J., Tillo, T.: Visual aesthetic understanding: sample-specific aesthetic classification and deep activation map visualization. Sig. Process. Image Commun. 67, 12–21 (2018)

    Article  Google Scholar 

  28. Zhang, J., Du, J., Dai, L.: Multi-scale attention with dense encoder for handwritten mathematical expression recognition. arXiv preprint arXiv:1801.03530 (2018)

  29. Zhou, X., et al.: EAST: an efficient and accurate scene text detector. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2642–2651 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. South China University of Technology, Guangzhou, China

    Weining Wang, Rui Deng, Lemin Li & Xiangmin Xu

Authors
  1. Weining Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lemin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiangmin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangmin Xu .

Editor information

Editors and Affiliations

  1. School of EECS, Peking University, Beijing, China

    Zhouchen Lin

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Liang Wang

  3. Nanjing University of Science and Technology, Nanjing, China

    Jian Yang

  4. Xidian University, Xi'an, China

    Guangming Shi

  5. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Tieniu Tan

  6. Institute of Artificial Intelligence, Xi'an Jiaotong University, Xi'an, China

    Nanning Zheng

  7. Chinese Academy of Sciences, Beijing, China

    Xilin Chen

  8. Northwestern Polytechnical University, Xi'an, China

    Yanning Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, W., Deng, R., Li, L., Xu, X. (2019). Image Aesthetic Assessment Based on Perception Consistency. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2019. Lecture Notes in Computer Science(), vol 11858. Springer, Cham. https://doi.org/10.1007/978-3-030-31723-2_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-31723-2_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-31722-5

  • Online ISBN: 978-3-030-31723-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation