Skip to main content

Advertisement

Springer Nature Link
Log in

Image fusion and enhancement based on energy of the pixel using Deep Convolutional Neural Network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a novel and generic framework for the recognition of emotions using human body expression like head, hand and leg movements. Whole body movements are among the main visual stimulus categories that are naturally associated with faces and the neuro scientific investigation of how body expressions are processed has entered the research agenda this last decade. The database was composed of 254 whole body expressions from 46 actors expressing four emotions (anger, fear, happiness, and sadness). In all pictures the face of the actor was blurred and participants were asked to categorize the emotions expressed in the stimuli in a four alternative-forced-choice task. Using Deep Convolutional Neural Network (DCNN), the input images are trained and modeled. Then the model can be tested by test images for recognizing human emotion from non-verbal communication.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Babu RG, Obaidat MS, Amudha V, Manoharan R, Sitharthan R (2020) Comparative analysis of distributive linear and non-linear optimised spectrum sensing clustering techniques in cognitive radio network systems. IET Networks. https://doi.org/10.1049/iet-net.2020.0122

    Article  Google Scholar 

  2. Battimelli G, Battimelli G, Ciccotti G, Greco P, Scalone (2020) Computer meets theoretical physics. Springer International Publishing

  3. Nele Dael, Marcello Mortillaro, Klaus R. Scherer (2012) The body action and posture coding system (BAP): development and reliability. Springer Science Business Media.

  4. Dalal N, Triggs B (2013) Histograms of oriented gradients for human detection. IEEE Comput Soc Conf Comput Vision Pattern Recognition 1:886–893

    Google Scholar 

  5. N Dalal, X He (2005) Histograms of oriented gradients for human detection. International conference on computer vision and pattern recognition, IEEE Computer Society Press, 1, June 20–25, 225–232.

  6. Dinh PH (2021) A novel approach based on grasshopper optimization algorithm for medical image fusion. Expert Syst Appl 171:114576

    Article  Google Scholar 

  7. Gopal VN, Al-Turjman F, Kumar R, Anand L, Rajesh M (2021) Feature selection and classification in breast cancer prediction using IoT and machine learning. Measurement 178:109442

    Article  Google Scholar 

  8. Goyal S, Singh V, Rani A, Yadav N (2020) FPRSGF denoised non-subsampled shearlet transform-based image fusion using sparse representation. SIViP 14(4):719–726

    Article  Google Scholar 

  9. Haris Zacharatos, Christos Gatzoulis, Yiorgos Chrysanthou (2014) Automatic emotion recognition based on body movement analysis: a survey. Computer Graphics and Applications, IEEE.

  10. Indhumathi R, Nagarajan S, Indira KP (2021) Hybrid pixel-based method for multimodal medical image fusion based on integration of Pulse-coupled neural network (PCNN) and Genetic algorithm (GA). In Advances in Machine Learning and Computational Intelligence, Springer, Singapore, pp. 853–867

  11. Jose J, Gautam N, Tiwari M, Tiwari T, Suresh A, Sundararaj V, Rejeesh MR (2021) An image quality enhancement scheme employing adolescent identity search algorithm in the NSST domain for multimodal medical image fusion. Biomed Signal Process Control 66:102480

    Article  Google Scholar 

  12. Jyoti Joshi, Roland Goecke, Gordon Parker, Michael Breakspear (2013) Can body expressions contribute to automatic depression analysis. International Conference and Workshops on Automatic Face and Gesture Recognition, IEEE.

  13. Kahlessenane F, Khaldi A, Kafi R, Euschi S (2021) A robust blind medical image watermarking approach for telemedicine applications. Clust Comput 24(3): 1–14

  14. Laptev I (2005) On space-time interest points. Int J Comput Vision. 64(4): 107–123

  15. Melissa Gross M, Elizabeth A. Crane, Barbara L. Fredrickson (2010) Methodology for assessing bodily expression of emotion. Springer Science + Business Media.

  16. Michelle Karg, Ali-Akbar Samadani, Rob Gorbet, Kolja K€uhnlenz, Jesse Hoey, Dana Kuli (2013) Body movements for affective expression: a survey of automatic recognition and generation. IEEE Trans Affect Comput. 4(4): 341-359

  17. Mohamed Bêcha Kaâniche, François Brémond (2010) Gesture recognition by learning local motion signatures. IEEE Conference on Computer Vision and Pattern Recognition.

  18. Moshika A, Thirumaran M, Natarajan B, Andal K, Sambasivam G, Manoharan R (2021) Vulnerability assessment in heterogeneous web environment using probabilistic arithmetic automata. IEEE Access 9:74659–74673. https://doi.org/10.1109/ACCESS.2021.3081567

  19. Nataraj SK, Al-Turjman F, Adom AH, Sitharthan R, Rajesh M, Kumar R (2020) Intelligent robotic chair with thought control and communication aid using higher order spectra band features. IEEE Sens J. https://doi.org/10.1109/JSEN.2020.3020971

    Article  Google Scholar 

  20. Natarajan B, Obaidat MS, Sadoun B, Manoharan R, Ramachandran S, Velusamy N (2020) New clustering-based semantic service selection and user preferential model. IEEE Syst J. https://doi.org/10.1109/JSYST.2020.3025407

    Article  Google Scholar 

  21. Nesrine Fourat, Catherine Pelachaud (2015) Multi-level classification of emotional body expression. IEEE. 5(4).

  22. Parvathy VS, Pothiraj S, Sampson J (2020) Optimal deep neural network model based multimodality fused medical image classification. Phys Commun 41:101119

    Article  Google Scholar 

  23. Purushothaman R, Rajagopalan SP, Dhandapani G (2020) Hybridizing gray wolf optimization (GWO) with Grasshopper optimization algorithm (GOA) for text feature selection and clustering. Appl Soft Comput 96:106651. https://doi.org/10.1016/j.asoc.2020.106651

    Article  Google Scholar 

  24. Pyry K. Matikainen, Martial Hebert, Rahul Sukthankar (2009) Trajectons: action recognition through the motion analysis of tracked features. Workshop on video-oriented object and event classification, ICCV.

  25. Rajesh M (2020) Streamlining radio network organizing enlargement towards microcellular frameworks. Wireless Pers Commun 113(4):2463–2475

    Article  Google Scholar 

  26. Sitharthan R, Rajesh M, Madurakavi K, Raglend J, Kumar R (2020) Assessing nitrogen dioxide (NO2) impact on health pre-and post-COVID-19 pandemic using IoT in India. Int J Pervasive Comput Commun.

  27. Sitharthan, R., Sujatha Krishnamoorthy, Padmanaban Sanjeevikumar, Jens Bo Holm-Nielsen, R. Raja Singh, M. Rajesh (2021) Torque ripple minimization of PMSM using an adaptive Elman neural network-controlled feedback linearization-based direct torque control strategy. Int Trans Electric Energy Syst 31 (1): e12685 https://doi.org/10.1002/2050-7038.12685

  28. Sitharthan R, Yuvaraj S, Padmanabhan S, Holm-Nielsen JB, Sujith M, Rajesh M, Prabaharan N, Vengatesan K (2021) Piezoelectric energy harvester converting wind aerodynamic energy into electrical energy for microelectronic application. IET Renew Power Gener. https://doi.org/10.1049/rpg2.12119

    Article  Google Scholar 

  29. Xu L, Si Y, Jiang S, Sun Y, Ebrahimian H (2020) Medical image fusion using a modified shark smell optimization algorithm and hybrid wavelet-homomorphic filter. Biomed Signal Process Control 59:101885. https://doi.org/10.1016/j.bspc.2020.101885

    Article  Google Scholar 

  30. Yadav SP, Yadav S (2020) Image fusion using hybrid methods in multimodality medical images. Med Biol Eng Comput 58(4):669–687

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science Engineering, Sanjivani College of Engineering, Kopargaon, India

    Rajesh M

  2. Department of Electrical Engineering, School of Electrical Engineering, Vellore Institute of Technology, Vellore, India

    Sitharthan R

Authors
  1. Rajesh M
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sitharthan R
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rajesh M.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rajesh M, Sitharthan R Image fusion and enhancement based on energy of the pixel using Deep Convolutional Neural Network. Multimed Tools Appl 81, 873–885 (2022). https://doi.org/10.1007/s11042-021-11501-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11501-y

Keywords