Skip to main content
SpringerLink
Log in

Image feature-based affective retrieval employing improved parameter and structure identification of adaptive neuro-fuzzy inference system

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Affective computing has various challenges especially for features extraction. Semantic information and vocal messages contain much emotional information, while extracting affective from features of images, and affective computing for image dataset are regarded as a promised research direction. This paper developed an improved adaptive neuro-fuzzy inference system (ANFIS) for images’ features extraction. Affective value of valence, arousal, and dominance were the proposed system outputs, where the color, morphology, and texture were the inputs. The least-square and k-mean clustering methods were employed as learning algorithms of the system. This improved model for structure and parameter identification of ANFIS were trained and validated. The training errors of the system for the affective values were tested and compared. Data sources grouping and the ANFIS generating processes were included. In the network training process, the number of input variables and fuzzy subset membership function types has been relative to network model under different affective inputs. Finally, well-established training model was used for testing using International Affective Picture System. The resulting network predicted those affective values, which compared to the expected outputs. The results demonstrated the effect of larger deviation of the individual data. In addition, the relationships between training errors, fuzzy sample set, training data set, function type, and the number of membership functions were illustrated. The proposed model showed the effectiveness for image affective extraction modeling with maximum training errors of 14 %.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Liu N, Dellandréa E, Chen L et al (2013) Multimodal recognition of visual concepts using histograms of textual concepts and selective weighted late fusion scheme. Comput Vis Image Underst 117(5):493–512

    Article  Google Scholar 

  2. Nanni L, Brahnam S, Lumini A (2012) Random interest regions for object recognition based on texture descriptors and bag of features. Expert Syst Appl 39(1):973–977

    Article  Google Scholar 

  3. Lo EHS, Pickering MR, Frater MR, Arnold JF (2011) Image segmentation from scale and rotation invariant texture features from the double dyadic dual-tree complex wavelet transform. Image Vis Comput 29(1):15–28

    Article  Google Scholar 

  4. Janssen JH, Ijsselsteijn WA, Westerink JHDM, Tacken P, de Vries G-J (2013) The tell-tale heart: perceived emotional intensity of heartbeats. Int J Synth Emot 4(1):65–91

    Article  Google Scholar 

  5. Ying Yu, Yong C (2009) Based on curvature direction characteristic of image emotional semantic classification. Comput Syst 18(2):121–124

    Google Scholar 

  6. Hayashi T, Hagiwara M (1997) An image retrieval system to estimate impression words form images using a neural network. In: Proceedings of the IEEE international conference on systems, man and cybernetics-computational cybernetics and simulation, New York, pp 150–155

  7. Yoshida K, Kato T, Yanaru T (1998) Image retrieval system using impression words system. In: Proceedings of the IEEE international conference on systems, man and cybernetics, San Diego, pp 2780–2784

  8. Lin HC, Chiu CY, Yang SN (2002) Texture analysis and description in linguistic terms. In: Proceedings of the Asian conference on computer vision, Melbourne, Australia, pp 23–25

  9. Dai Y (2004) Intention-based image retrieval with or without a query image. In: Proceedings of the international conference on multimedia modeling, Australia, pp 26–32

  10. Abegaz T, Dillon E, Gilbert JE (2015) Exploring affective reaction during user interaction with colors and shapes. Proced Manuf 3:5253–5260

    Article  Google Scholar 

  11. Zhang P, Gong M, Su L, Liu J, Li Z (2016) Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images. ISPRS J Photogram Rem Sens 116:24–41

    Article  Google Scholar 

  12. Yun S, Bin H, Lixin X et al (2015) EEG data for knowledge modeling and emotion recognition. Chin Sci Bull 11(10):1002–1009

    Google Scholar 

  13. Lahane P, Sangaiah AK (2015) An approach to EEG based emotion recognition and classification using kernel density estimation. Proced Comput Sci 48:574–581

    Article  Google Scholar 

  14. Castillo O, Melin P, Ramírez E, Soria J (2012) Hybrid intelligent system for cardiac arrhythmia classification with Fuzzy K-Nearest Neighbors and neural networks combined with a fuzzy system. Expert Syst Appl 39(3):2947–2955

    Article  Google Scholar 

  15. Zhang YQ, Kandal A (1998) Compensatory neuro-fuzzy systems with fast learn algorithms. IEEE Trans Neural Netw 9(1):83–105

    Article  Google Scholar 

  16. Adli J, Karim A (2000) Fuzzy-wavelet RBFNN model for freeway incident detection. J Transp Eng 126(6):464–471

    Article  Google Scholar 

  17. Zhijian Z, Zongyuan M (1999) Fuzzy neural network structure and parameters optimization based on genetic algorithm. Hunan Univ Technol (Natural Science) 27(1):26–32

    Google Scholar 

  18. Qin Z, Jin C, Li D (2015) ANFIS system applications in the medical field. Chin J Clin Thorac Cardiovasc Hous 22(3):252–256

    Google Scholar 

  19. Latifoğlu F (2013) A novel approach to speckle noise filtering based on Artificial Bee Colony algorithm: an ultrasound image application. Comput Methods Programs Biomed 111(3):561–569

    Article  Google Scholar 

  20. Abdulshahed AM, Longstaff AP, Fletcher S, Myers A (2015) Thermal error modelling of machine tools based on ANFIS with fuzzy c-means clustering using a thermal imaging camera. Appl Math Model 39:1837–1852

    Article  Google Scholar 

  21. Jiang H, Kwong C, Siu K, Liu Y (2015) Rough set and PSO-based ANFIS approaches to modeling customer satisfaction for affective product design. Adv Eng Inform 29:727–738

    Article  Google Scholar 

  22. Turkmen I (2011) Efficient impulse noise detection method with ANFIS for accurate image restoration. Int J Electron Commun 65:132–139

    Article  Google Scholar 

  23. Abbaspour S, Fallah A, Lindn M, Gholamhosseini H (2016) A novel approach for removing ECG interferences from surface EMG signals using a combined ANFIS and wavelet. J Electromyogr Kinesiol 26:52–59

    Article  Google Scholar 

  24. Gupta R, ur Rehman Laghari K, Falk TH (2016) Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization. Neurocomputing 174(Part B):875–884

    Article  Google Scholar 

  25. Sharma R, Patterh M (2015) A new pose invariant face recognition system using PCA and ANFIS. Opt Int J Light Electron Opt 126:3483–3487

    Article  Google Scholar 

  26. Bhandari KA, Ramchandra RM (2016) An innovative remote sensing image retrieval techniques based on haar wavelet-LTRP and ANFIS. Proced Comput Sci 79:391–401

    Article  Google Scholar 

  27. Dai W, Han D, Dai Y, Xu D (2015) Emotion recognition and affective computing on vocal social media. Inf Manag 52:777–788

    Article  Google Scholar 

  28. Kaminskas M, Ricci F (2012) Contextual music information retrieval and recommendation: state of the art and challenges. Comput Sci Rev 6:89–119

    Article  Google Scholar 

  29. Bozhkov L, Georgieva P, Santos I, Pereira A, Silva C (2015) EEG-based subject independent affective computing models. Proced Comput Sci 53:375–382

    Article  Google Scholar 

  30. Ward R, Marsden P (2004) Affective computing: problems, reactions and intentions. Interact Comput 16:707–713

    Article  Google Scholar 

  31. Vallverd J, Talanov M, Distefano S, Mazzara M, Tchitchigin A, Nurgaliev I (2016) A cognitive architecture for the implementation of emotions in computing systems. Biol Inspired Cogn Archit 15:34–40

    Google Scholar 

  32. Hussain A, Cambria E, Schuller B, Howard N (2014) Affective neural networks and cognitive learning systems for big data analysis. Neural Netw 58:1–3

    Article  Google Scholar 

  33. Rao Y, Li Q, Wenyin L, Wu Q, Quan X (2014) Affective topic model for social emotion detection. Neural Netw 58:29–37

    Article  Google Scholar 

  34. Jiang Y, Liu C, Ma J (2016) BYY harmony learning of t-mixtures with the application to image segmentation based on contourlet texture features. Neurocomputing 188:262–274

    Article  Google Scholar 

  35. Sarafis I, Diou C, Delopoulos A (2016) Online training of concept detectors for image retrieval using streaming clickthrough data. Eng Appl Artif Intell 51:150–162

    Article  Google Scholar 

  36. Haralick RM (1979) Statistical and structural approaches to texture. Proc IEEE 67(5):786–804

    Article  Google Scholar 

  37. Vlachostergiou A, Caridakis G, Kollias S (2014) Investigating context awareness of affective computing systems: a critical approach. Proced Comput Sci 39:91–98

    Article  Google Scholar 

  38. ElAdel A, Zaied M, Amar CB (2016) Fast beta wavelet network-based feature extraction for image copy detection. Neurocomputing 173(Part 2):306–316

    Article  Google Scholar 

  39. Ponti M, Nazar TS, Thum GS (2016) Image quantization as a dimensionality reduction procedure in color and texture feature extraction. Neurocomputing 173(Part 2):385–396

    Article  Google Scholar 

  40. Shahraiyni HT, Sodoudi S, Kerschbaumer A, Cubascha U (2015) A new structure identification scheme for ANFIS and its application for the simulation of virtual air pollution monitoring stations in urban areas. Eng Appl Artif Intell 41:175–182

    Article  Google Scholar 

  41. Chiu SL (1994) Fuzzy model identification based on cluster estimation. J Intell Fuzzy Syst 2(3):267–278

    Google Scholar 

  42. Mebrabian A (1995) Framework for a comprehensive description and measurement of emotional states. Genet Soc Gen Psychol Monogr 121(3):3392–3611

    Google Scholar 

  43. Mebrabian A (1996) Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament. Curr Psychol Dev Learn Personal Soc 14(4):2612–2921

    Google Scholar 

  44. Mebrabian A, Wihardja C, Ljunggren E (1997) Emotional correlates of preferences for situation activity combinations in everyday life. Genetic Soc Gen Psychol Monogr 123(4):4612–4771

    Google Scholar 

  45. He T, Cao L, Balas VE, McCauley P, Shi F (2016) Curvature manipulation of the spectrum of Valence-Arousal-related fMRI dataset using Gaussian-shaped Fast Fourier Transform and its application to fuzzy KANSEI adjectives modeling. Neurocomputing 174(Part B):1049–1059

    Article  Google Scholar 

  46. Broekens J (2010) Modeling the experience of emotion. Int J Synth Emot 1(1):1–17

    Article  Google Scholar 

  47. Gunes H (2010) Automatic, dimensional and continuous emotion recognition. Int J Synth Emot 1:1

    Google Scholar 

  48. Cowie R, McKeown G, Douglas-Cowie E (2012) Tracing emotion: an overview. Int J Synth Emot 3(1):1–17

    Article  Google Scholar 

  49. Mortillaro M, Meuleman B, Scherer KR (2012) Advocating a componential appraisal model to guide emotion recognition. Int J Synth Emot 3(1):18–32

    Article  Google Scholar 

  50. Moral R, Navarro J, Lahoz-Beltra R, Marijuán PC (2014) Cognitive and emotional contents of laughter: framing a new neurocomputational approach. Int J Synth Emot 5(2):31–54

    Article  Google Scholar 

  51. Mikels JA, Fredrickson BL, Larkin GR, Lindberg CM, Maglio SJ, Reuter-Lorenz PA (2005) Emotional category data on images from the International Affective Picture System. Behav Res Methods 37(4):626–630

    Article  Google Scholar 

  52. Machajdik J, Hanbury A (2010) Affective image classification using features inspired by psychology and art theory. In: Proceedings of the 18th ACM international conference on multimedia. ACM, pp. 83–92

  53. Bach J (2012) A framework for emergent emotions, based on motivation and cognitive modulators. Int J Synth Emot 3(1):43–63

    Article  Google Scholar 

  54. Grunberg DK, Batula AM, Schmidt EM, Kim YE (2012) Synthetic emotions for humanoids: perceptual effects of size and number of robot platforms. Int J Synth Emot 3(2):68–83

    Article  Google Scholar 

  55. Setiawan NA (2014) Fuzzy decision support system for coronary artery disease diagnosis based on Rough set theory. Int J Rough Sets Data Anal 1(1):65–80

    Article  MathSciNet  Google Scholar 

  56. Kim H, Lee I (2015) Combining image databases for affective image classification. In: ACHI 2015: the eighth international conference on advances in computer-human interactions, pp 211–212

Download references

Acknowledgments

This work was sponsored by Zhejiang Provincial Natural Science Fund under Grant No. (LQ15A010009, Y17F030054) and National Natural Science Foundation of China under Grant No. (51205059).

Author information

Authors and Affiliations

  1. Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin, 300072, People’s Republic of China

    Dan Wang & Zairan Li

  2. Wenzhou Vocational and Technical College, Wenzhou, 325035, People’s Republic of China

    Dan Wang & Zairan Li

  3. College of Information and Engineering, Wenzhou Medical University, Wenzhou, 325035, People’s Republic of China

    Ting He, Luying Cao, Yezhi Lin & Fuqian Shi

  4. Department of IT, Techno India College of Technology, Kolkata, West Bengal, 740000, India

    Nilanjan Dey

  5. Department of Electronics and Electrical Communications Engineering, Faculty of Engineering, Tanta University, Tanta, Egypt

    Amira S. Ashour

  6. Department of Automation and Applied Informatics, Aurel Vlaicu University of Arad, 310130, Arad, Romania

    Valentina E. Balas

  7. Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL, 32825, USA

    Pamela McCauley

  8. College of Design and Innovation, Tongji University, Shanghai, 210000, People’s Republic of China

    Jiang Xu

Authors
  1. Dan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ting He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zairan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Luying Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nilanjan Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Amira S. Ashour
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Valentina E. Balas
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Pamela McCauley
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yezhi Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Jiang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Fuqian Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fuqian Shi.

Ethics declarations

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., He, T., Li, Z. et al. Image feature-based affective retrieval employing improved parameter and structure identification of adaptive neuro-fuzzy inference system. Neural Comput & Applic 29, 1087–1102 (2018). https://doi.org/10.1007/s00521-016-2512-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-016-2512-4

Keywords

Search

Navigation