Skip to main content
SpringerLink
Log in

An image classification approach for painting using improved convolutional neural algorithm

  • Neural Networks
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The widespread availability of digitized fine art collections in museums and galleries has generated a demand for efficient software tools. These tools enable rapid retrieval and semantic classification of art images. Traditional image classification methods often rooted in shallow structure learning algorithms offer the capacity to extract various image features. However, there is need for deep understanding of fundamental painting knowledge during this process due to the reason that certain features may be lost. In this paper, we present an innovating approach for classifying digital paintings based on artist attribution. Our approach centers on the creations of multi-scale pyramid representation derived from a given painting image which facilitate the incorporation of both global and local information within a single image. The training of Convolutional Neural Network (CNN) algorithm is performed for the assignment of class labels on each level of the pyramid. To establish connections among localized image patches, we use Markov Random Fields through the optimization of the Gibbs energy function. We validate the proposed approach which referred to as the multi-scale CNN framework using fusion-based Markov random field using two challenging painting image datasets: WikiArt and Web Gallery of Art (WGA). The proposed model archives better results using WikiArt dataset for painting image classification, i.e., precision (74.12%), recall (70.23%), F1-score (72.65%), and accuracy (80.00%). Using Web Gallery of Art (WGA) dataset for painting image classification, the model achieves the following results: precision (75.88%), recall (77.25%), F1-score (76.44%), and accuracy (78.00%). Our proposed method outperforms state-of-the-art methods excelling not only in image classification but also in terms of computational efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

Data availability

Inquiries about data availability should be addressed to the corresponding author.

References

  • Alfaro-Contreras M, Ríos-Vila A, Valero-Mas JJ, Calvo-Zaragoza J (2023) Few-shot symbol classification via self-supervised learning and nearest neighbor. Pattern Recogn Lett 167:1–8

    Article  Google Scholar 

  • Ali M, Yin B, Bilal H et al (2023) Advanced efficient strategy for detection of dark objects based on spiking network with multi-box detection. Multimed Tools Appl. https://doi.org/10.1007/s11042-023-16852-2

    Article  Google Scholar 

  • Anwar MA, Tahir SF, Fahad LG, Kifayat K (2023) Image forgery detection by transforming local descriptors into deep-derived features. Appl Soft Comput 147:110730

    Article  Google Scholar 

  • Aslam XD, Jun H, Qianmu L, Rizwan U, Zhen N, Yaozong L (2020) Reliable control design for composite-driven scheme based on delay networked T-S fuzzy system. Int J Robust Nonlinear Control 30(4):1622–1642

    Article  MathSciNet  Google Scholar 

  • Bai R, Guo X (2021) Automatic orientation detection of abstract painting. Knowl Based Syst 227:107240

    Article  Google Scholar 

  • Bale AS, Kumar SS, Kiran Mohan MS, Vinay N (2022) A study of improved methods on image inpainting. Trends Adv Image Process Appl. https://doi.org/10.1007/978-3-030-75945-2_15

    Article  Google Scholar 

  • Belhi A, Ahmed HO, Alfaqheri T, Bouras A, Sadka AH, Foufou S (2021) Study and evaluation of pre-trained CNN networks for cultural heritage image classification. Data Anal Cult Herit Curr Trends Concepts 47–69

  • Bengamra S, Mzoughi O, Bigand A, Zagrouba E (2023) A comprehensive survey on object detection in visual art: taxonomy and challenge. Multimed Tools Appl. https://doi.org/10.1007/s11042-023-15968-9

    Article  Google Scholar 

  • Bhatti UA, Huang M, Neira-Molina H, Marjan S, Baryalai M, Tang H, Wu G, Bazai SU (2023) MFFCG-multi feature fusion for hyperspectral image classification using graph attention network. Expert Syst Appl 229:120496

    Article  Google Scholar 

  • Chen Z (2019) Observer-based dissipative output feedback control for network T-S fuzzy systems under time delays with mismatch premise. Nonlinear Dyn 95:2923–2941

    Article  Google Scholar 

  • Coulibaly S, Kamsu-Foguem B, Kamissoko D, Traore D (2022) Deep convolution neural network sharing for the multi-label images classification. Mach Learn Appl 10:100422

    Google Scholar 

  • Dou H, Liu Y, Chen S et al (2023) A hybrid CEEMD-GMM scheme for enhancing the detection of traffic flow on highways. Soft Comput 27:16373–16388. https://doi.org/10.1007/s00500-023-09164-y

    Article  Google Scholar 

  • Geng J, Zhang X, Yan Y, Sun M, Zhang H, Assaad M, Ren J, Li X (2023) MCCFNet: multi-channel color fusion network for cognitive classification of traditional chinese paintings. Cogn Comput 15:2050–2061

    Article  Google Scholar 

  • Guo L (2023) Art teaching interaction based on multimodal information fusion under the background of deep learning. Soft Comput. https://doi.org/10.1007/s00500-023-08669-w

    Article  Google Scholar 

  • Kumari N, Anwar S, Bhattacharjee V (2022) Automated visual stimuli evoked multi-channel EEG signal classification using EEGCapsNet. Pattern Recogn Lett 153:29–35

    Article  Google Scholar 

  • Li J, Li B, Jiang Y, Cai W (2022) MSAt-GAN: a generative adversarial network based on multi-scale and deep attention mechanism for infrared and visible light image fusion. Complex Intell Syst 8(6):4753–4781

    Article  Google Scholar 

  • Liu S, Yang J, Agaian SS, Yuan C (2021) Novel features for art movement classification of portrait paintings. Image vis Comput 108:104121

    Article  Google Scholar 

  • Lu Y, Guo C, Dai X, Wang FY (2022) Data-efficient image captioning of fine art paintings via virtual-real semantic alignment training. Neurocomputing 490:163–180

    Article  Google Scholar 

  • Maharana K, Mondal S, Nemade B (2022) A review: data pre-processing and data augmentation techniques. Glob Transit Proc 3(1):91–99

    Article  Google Scholar 

  • Manohar K, Logashanmugam E (2022) Hybrid deep learning with optimal feature selection for speech emotion recognition using improved meta-heuristic algorithm. Knowl Based Syst 246:108659

    Article  Google Scholar 

  • Muhammad A, Yin B, Kumar A, Sheikh AM et al (2020) Reduction of multiplications in convolutional neural networks. In: 2020 39th Chinese control conference (CCC). IEEE, pp 7406–7411. https://doi.org/10.23919/CCC50068.2020.9188843

  • Muhammad IQ, Abdul M, Summera S (2023) Adaptive event-triggered robust H∞ control for Takagi-Sugeno fuzzy networked Markov jump systems with time-varying delay. Asian J Control 25(1):213–228

    Article  MathSciNet  Google Scholar 

  • Narag MJG, Soriano M (2021) Discovering artistic influences of painters from expressionism, impressionism, and surrealism art movements using convolutional neural network. J Cult Herit 51:182–193

    Article  Google Scholar 

  • Nayak S, Sharma YK (2023) A modified Bayesian boosting algorithm with weight-guided optimal feature selection for sentiment analysis. Decis Anal J 8:100289

    Article  Google Scholar 

  • Nevo D, David EO, Netanyahu NS (2022) DeepArtist: a dual-stream network for painter classification of highly-varying image resolutions. In: International conference on artificial neural networks. Springer Nature, Cham, pp 582–593

  • Peng X, Peng S, Hu Q, Peng J, Wang J, Liu X, Fan J (2022) Contour-enhanced CycleGAN framework for style transfer from scenery photos to Chinese landscape paintings. Neural Comput Appl 34(20):18075–18096

    Article  Google Scholar 

  • Pham P, Nguyen LT, Pedrycz W, Vo B (2023) Deep learning, graph-based text representation and classification: a survey, perspectives and challenges. Artif Intell Rev 56(6):4893–4927

    Article  Google Scholar 

  • Shamrooz M, Li Q, Hou J (2021) Fault detection for asynchronous T-S fuzzy networked Markov jump systems with new event-triggered scheme. IET Control Theory Appl 15(11):1461–1473

    Article  MathSciNet  Google Scholar 

  • Tripuraribhatla R (2022) Exponential sailfish optimizer-based generative adversarial network for image annotation on natural scene images. Gene Expr Patterns 46:119279

    Article  Google Scholar 

  • Trombini M, Solarna D, Moser G, Dellepiane S (2023) A goal-driven unsupervised image segmentation method combining graph-based processing and Markov random fields. Pattern Recognit 134:109082

    Article  Google Scholar 

  • Ullah R, Dai X, Sheng A (2020) Event-triggered scheme for fault detection and isolation of non-linear system with time-varying delay. IET Control Theory Appl 14(16):2429–2438

    Article  MathSciNet  Google Scholar 

  • Wang L, Zhai Q, Yin B et al (2019) Second-order convolutional network for crowd counting. In: Proceedings of SPIE 11198, fourth international workshop on pattern recognition, 111980T. https://doi.org/10.1117/12.2540362

  • Wang J, Li J, Liu W, Du S, Gao S (2023) Dunhuang mural line drawing based on multi-scale feature fusion and sharp edge learning. Neural Process Lett. https://doi.org/10.1007/s11063-023-11323-z

    Article  Google Scholar 

  • Wei N (2021) Research on the algorithm of painting image style feature extraction based on intelligent vision. Futur Gener Comput Syst 123:196–200

    Article  Google Scholar 

  • Wu Q, Li X, Wang K et al (2023) Regional feature fusion for on-road detection of objects using camera and 3D-LiDAR in high-speed autonomous vehicles. Soft Comput 27:18195–18213. https://doi.org/10.1007/s00500-023-09278-3

    Article  Google Scholar 

  • Xu F, Wong MS, Zhu R, Heo J, Shi G (2023a) Semantic segmentation of urban building surface materials using multi-scale contextual attention network. ISPRS J Photogramm Remote Sens 202:158–168

    Article  Google Scholar 

  • Xu H, Sun Z, Cao Y et al (2023b) A data-driven approach for intrusion and anomaly detection using automated machine learning for the Internet of Things. Soft Comput. https://doi.org/10.1007/s00500-023-09037-4

    Article  Google Scholar 

  • Yan J, Wang W, Yu C (2022) Affective word embedding in affective explanation generation for fine art paintings. Pattern Recogn Lett 161:24–29

    Article  Google Scholar 

  • Yao W, Guo Y, Wu Y, Guo J (2017) Experimental validation of fuzzy PID control of flexible joint system in presence of uncertainties. In: 2017 36th Chinese control conference (CCC). IEEE, pp 4192–4197. https://doi.org/10.23919/ChiCC.2017.8028015

  • Yin B, Khan J, Wang L, Zhang J, Kumar A (2019) Real-time lane detection and tracking for advanced driver assistance systems. In: 2019 Chinese control conference (CCC). IEEE, pp 6772–6777. https://doi.org/10.23919/ChiCC.2019.8866334

  • Zhang H, Jiang L (2022) Fine tuning attribute weighted naive Bayes. Neurocomputing 488:402–411

    Article  Google Scholar 

  • Zhang H, Luo G, Yue Y, He K, Xu D (2023) Affective image recognition with multi-attribute knowledge in deep neural networks. Multimed Tools Appl. https://doi.org/10.1007/s11042-023-16081-7

    Article  Google Scholar 

  • Zhou W, Deng Y, Peng B, Xiang S, Kaneko SI (2023) Co-occurrence spatial-temporal model for adaptive background initialization in high-dynamic complex scenes. Signal Process Image Commun 119:117056

    Article  Google Scholar 

Download references

Funding

No funding was provided for the completion of this study.

Author information

Authors and Affiliations

  1. Department of Formative Arts, Dankook University, Yongin, 16890, Gyeonggi-do, Korea

    Qing Yu & Ce Shi

Authors
  1. Qing Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ce Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ce Shi.

Ethics declarations

Conflict of interest

No conflict of interest has been declared by the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, Q., Shi, C. An image classification approach for painting using improved convolutional neural algorithm. Soft Comput 28, 847–873 (2024). https://doi.org/10.1007/s00500-023-09420-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-023-09420-1

Keywords

Search

Navigation