Skip to main content
SpringerLink
Log in

A novel approach using deep convolutional neural network to classify the photographs based on leading-line by fine-tuning the pre-trained VGG16 neural network

  • 1203: Applications of Advanced Artificial Intelligence in Multimedia and Information Security
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Leading lines are used to draw the viewer’s attention towards the main subject present in a photograph. These lines may be straight or curved, and generally, start from the left or right bottom corner of the frame. It leads the viewer’s eyes to the region of interest, which could be any object or the vanishing point. It has been observed that the photograph containing leading lines achieves a higher aesthetic score in several photographic competitions and different photo-sharing communities. Automatic detection of the leading line may solve numerous real-time applications like on-site aesthetic value evaluation, assisting amateur photographer to get a better composition, and so on. However, implementation of the same is not easy due to the subjective nature of the problem. From the literature survey, we can notice that no such work exists that considers the presence of leading line in a photograph. This paper introduces a novel approach using deep convolutional neural network (DCNN) framework to detect the presence of leading line in a photograph as well as classify the photographs based on the leading line. The proposed model automatically extracts features by fine-tuning the pre-trained VGG16 CNN (a transfer learning method) to accomplish the classification task. The model has been examined by executing on the ground truth dataset, and the satisfying result has been observed. Another contribution of the proposed work is creating a new dataset named as Leading-line Containing Image (LCI) which will be beneficial for future work in this area.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

Data Availability

Online at https://www.somadebnath.com/post/leading-line-containing-image-dataset-lci

Code Availability

With the authors.

References

  1. Bhattacharya S, Sukthankar R, Shah M (2010) A framework for photo-quality assessment and enhancement based on visual aesthetics. In: Proceedings of the 18th ACM international conference on multimedia. ACM, pp 271–280

  2. Bishop A, Ciesielski V, Trist K (2014) Feature construction using genetic programming for classification of images by aesthetic value. In: International conference on evolutionary and biologically inspired music and art. Springer, pp 62–73

  3. Chong N, Wong L-K, See J (2019) Ganmera: reproducing aesthetically pleasing photographs using deep adversarial networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops

  4. Datta R, Wang JZ (2010) Acquine: aesthetic quality inference engine - real-time automatic rating of photo aesthetics. In: Proceedings of the 11th ACM SIGMM international conference on multimedia information retrieval, MIR. ACM, vol 4, pp 421–424

  5. Datta R, Joshi D, Li J, Wang JZ (2006) Studying aesthetics in photographic images using a computational approach. In: Computer vision – ECCV. Springer, pp 288–301

  6. Guo G, Wang H, Shen C, Yan Y, Liao H-YM (2018) Automatic image cropping for visual aesthetic enhancement using deep neural networks and cascaded regression. IEEE Trans Multimed 20(8):2073–2085

    Article  Google Scholar 

  7. Islam MB, Kuan WL, Onn WC (2017) A survey of aesthetics-driven image recomposition. Multimed Tools Appl Springer 76:9517–9542

    Article  Google Scholar 

  8. Jin X, Wu L, Li X, Zhang X, Chi J, Peng S, Ge S, Zhao G, Li S (2018) ILGNEt: inception modules with connected local and global features for efficient image aesthetic quality classification using domain adaptation. IET Comput Vis 13(2):206–212

    Article  Google Scholar 

  9. Kao Y, He R, Huang K (2017) Deep aesthetic quality assessment with semantic information. IEEE Trans Image Process 26:1482–1495

    Article  MathSciNet  Google Scholar 

  10. Kao Y, Huang K, Maybank S (2016) Hierarchical aesthetic quality assessment using deep convolutional neural networks. Signal Process Image Commun 47:500–510

    Article  Google Scholar 

  11. Lee H-J, Hong K-S, Kang H, Lee S (2017) Photo aesthetics analysis via dcnn feature encoding. IEEE Trans Multimed 19(8):1921–1932

    Article  Google Scholar 

  12. Li C, Chen T (2009) Aesthetic visual quality assessment of paintings. IEEE J Select Topics Signal Process 3:236–253

    Article  Google Scholar 

  13. Lo K-Y, Liu K-H, Chen C-S (2012) Assessment of photo aesthetics with efficiency. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012). IEEE, pp 2186–2189

  14. Lu P, Peng XJ, Yu J, Peng X (2019) Gated cnn for visual quality assessment based on color perception. Signal Process Image Commun 72:105–112

    Article  Google Scholar 

  15. Lu X, Lin Z, Jin H, Yang J, Wang JZ (2014) Rapid: rating pictorial aesthetics using deep learning. In: Proceedings of the 22nd ACM international conference on Multimedia, pp 457–466

  16. Mai L, Jin H, Liu F (2016) Composition-preserving deep photo aesthetics assessment. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 497–506

  17. Marchesotti L, Murray N, Perronnin F (2015) Discovering beautiful attributes for aesthetic image analysis. Int J Comput Vis 113(3):246–266

    Article  Google Scholar 

  18. Ni B, Xu M, Cheng B, Wang M, Yan S, Tian Q (2013) Learning to photograph: A compositional perspective. IEEE Trans Multimed 15:1138–1151

    Article  Google Scholar 

  19. Nishiyama M, Okabe T, Sato I, Sato Y (2011) Aesthetic quality classification of photographs based on color harmony. In: 24th IEEE conference on computer vision and pattern recognition, CVPR. IEEE, vol 8, pp 33–40

  20. Pramila A, Keskinarkaus A, Takala V, T. Seppanen. (2017) Extracting watermarks from printouts captured with wide angles using computational photography. Multimed Tools Appl Springer 76:16063–16084

    Article  Google Scholar 

  21. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  22. Su HH, Chen TW, Kao CC, Hsu WH, Chien SY (2012) Preference-aware view recommendation system for scenic photos based on bag-of-aesthetics-preserving features. IEEE Trans Multimed 14:833–843

    Article  Google Scholar 

  23. Suler J (2013) Photographic psychology: image and psyche http:http://truecenterpublishing.com/photopsy/psylines.htm/

  24. Tang X, Luo W, X.Wang. (2013) Content-based photo quality assessment. IEEE Trans Multimed 15:1930–1943

    Article  Google Scholar 

  25. Tian X, Dong Z, Yang K, Mei T (2015) Query-dependent aesthetic model with deep learning for photo quality assessment. IEEE Trans Multimed 17(11):2035–2048

    Article  Google Scholar 

  26. Wang W, Zhao W, Cai C, Huang J, Xu X, Li L (2015) An efficient image aesthetic analysis system using hadoop. Signal Process Image Commun 39:499–508

    Article  Google Scholar 

  27. Wang W, Zhao M, Li W, Huang J, Cai C, Xu X (2016) A multi-scene deep learning model for image aesthetic evaluation. Signal Process Image Commun 47:511–518

    Article  Google Scholar 

  28. Yang J, Huang Z, Sim K, Lu W, Liu K, Liu H (2019) No-reference image quality assessment focusing on human facial region. Signal Process Image Commun 78:51–61

    Article  Google Scholar 

  29. Yeh M, Chuang C (2014) Assessing the aesthetic quality of photographs through group comparison. In: 2014 IEEE international conference on consumer electronics-taiwan (ICCE-TW). IEEE, 26-28 May, pp 91–92

  30. Zhang C, Ce Z, Xu X, Liu Y, Xiao J, Tillo T (2018) Visual aesthetic understanding: sample-specific aesthetic classification and deep activation map visualization. Signal Process Image Commun 67:12–21

    Article  Google Scholar 

  31. Zhang F, Wang M, Hu S (2013) Aesthetic image enhancement by dependence-aware object recomposition. IEEE Trans Multimed 15:1480–1490

    Article  Google Scholar 

  32. Zhang J, Miao Y, Zhang J, Yu J (2020) Inkthetics: a comprehensive computational model for aesthetic evaluation of chinese ink paintings. IEEE Access

  33. Zhang T, Xie J, Xu Z, Choi C (2020) The effects of depth of field on subjective evaluation of aesthetic appeal and image quality of photographs. IEEE Access 8:13467–13475

    Article  Google Scholar 

  34. Zhang X, Gao X, Lu W, Yu Y, He L (2019) Fusion global and local deep representations with neural attention for aesthetic quality assessment. Signal Process Image Commun 78:42–50

    Article  Google Scholar 

  35. Zhu X, Moh TS (2016) Assessing photograph aesthetic quality with color based descriptor. In: 2016 International conference on high performance computing simulation (HPCS). IEEE, vol 8, pp 222–229

Download references

Author information

Authors and Affiliations

  1. Department of CSE, NIT, Durgapur, India

    Soma Debnath & Suvamoy Changder

  2. Dr. B.C. Roy Engineering College, Durgapur, India

    Ratnakirti Roy

Authors
  1. Soma Debnath
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ratnakirti Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Suvamoy Changder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Soma Debnath.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Debnath, S., Roy, R. & Changder, S. A novel approach using deep convolutional neural network to classify the photographs based on leading-line by fine-tuning the pre-trained VGG16 neural network. Multimed Tools Appl 83, 3189–3214 (2024). https://doi.org/10.1007/s11042-022-13338-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13338-5

Keywords

Search

Navigation