Skip to main content

Advertisement

Springer Nature Link
Log in

Full-convolution Siamese network algorithm under deep learning used in tracking of facial video image in newborns

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

This study was carried out with the aim of exploring the full-convolution Siamese network (SiamFC) in the application of neonatal facial video image tracking, achieving accurate recognition of neonatal pain and helping doctors evaluate neonatal emotions in an automatic manner. The current technology shows low accuracy on facial image recognition of newborns, so the SiamFC algorithm under the deep learning was optimized in this study. Besides, a newborn facial video image tracking model (FVIT model) was constructed based on the SiamFC algorithm in combination with the attention mechanism with face tracking algorithm, and the facial features of newborns were tracked and recognized. In addition, a newborn face database was constructed based on the adult face database to evaluate performance of the FVIT model. It was found that the accuracy of the improved algorithm is 0.889, higher by 0.036 in contrast to other models; the area under the curve (AUC) of success rate reaches 0.748, higher by 0.075 compared with other algorithms. What’s more, the improved algorithm shows good performance in tracking the facial occlusion, facial expression changes, and scale conversion of newborns. Therefore, the improved algorithm shows higher accuracy and success rate and has good effect in capturing and tracking the facial images of newborns, thereby providing an experimental basis for facial recognition and pain assessment of newborns in the later stage.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Dang LM, Hassan SI, Im S et al (2019) Face image manipulation detection based on a convolutional neural network. Expert Syst Appl 129:156–168. https://doi.org/10.1016/j.eswa.2019.04.005

    Article  Google Scholar 

  2. Deffo LL, Fute ET, Tonye E (2018) CNNSFR: a convolutional neural network system for face detection and recognition. Int J Adv Computer Sci Appl 9(12):240–244. https://doi.org/10.14569/IJACSA.2018.091235

    Article  Google Scholar 

  3. Brumancia E, Samuel SJ, Gladence LM et al (2019) Hybrid data fusion model for restricted information using Dempster-Shafer and adaptive neuro-fuzzy inference (DSANFI) system. Soft Comput 23(8):2637–2644. https://doi.org/10.1007/s00500-018-03734-1

    Article  Google Scholar 

  4. Kusiak A (2020) Convolutional and generative adversarial neural networks in manufacturing. Int J Prod Res 58(5):1594–1604. https://doi.org/10.1080/00207543.2019.1662133

    Article  Google Scholar 

  5. Chen J, Lv Y, Xu R et al (2019) Automatic social signal analysis: Facial expression recognition using difference convolution neural network. J Parallel Distrib Comput 131:97–102. https://doi.org/10.1016/j.jpdc.2019.04.017

    Article  Google Scholar 

  6. Islas MA, Rubio JJ, Muñiz S et al (2021) A fuzzy logic model for hourly electrical power demand modeling. Electronics 10(4):448. https://doi.org/10.3390/electronics10040448

    Article  Google Scholar 

  7. de Jesús RJ, Lughofer E, Pieper J et al (2021) Adapting H-infinity controller for the desired reference tracking of the sphere position in the maglev process. Inf Sci 569:669–686. https://doi.org/10.1016/j.ins.2021.05.018

    Article  MathSciNet  Google Scholar 

  8. Chiang HS, Chen MY, Huang YJ (2019) Wavelet-based EEG processing for epilepsy detection using fuzzy entropy and associative petri net. IEEE Access 7:103255–103262. https://doi.org/10.1109/ACCESS.2019.2929266

    Article  Google Scholar 

  9. de Rubio JJ (2020) Stability analysis of the modified Levenberg-Marquardt algorithm for the artificial neural network training. IEEE Trans Neural Netw Learn Syst 32(8):3510–3524. https://doi.org/10.1109/TNNLS.2020.3015200

    Article  MathSciNet  Google Scholar 

  10. Meda-Campaña JA (2018) On the estimation and control of nonlinear systems with parametric uncertainties and noisy outputs. IEEE Access 6:31968–31973. https://doi.org/10.1109/ACCESS.2018.2846483

    Article  Google Scholar 

  11. Soriano LA, Zamora E, Vazquez-Nicolas JM et al (2020) PD control compensation based on a cascade neural network applied to a robot manipulator. Front Neurorobot 14:577749. https://doi.org/10.3389/fnbot.2020.577749

    Article  Google Scholar 

  12. Al-Janabi S, Alkaim AF, Adel Z (2020) An Innovative synthesis of deep learning techniques (DCapsNet & DCOM) for generation electrical renewable energy from wind energy. Soft Comput 24(14):10943–10962. https://doi.org/10.1007/s00500-020-04905-9

    Article  Google Scholar 

  13. Wang C, Han D, Liu Q et al (2018) A deep learning approach for credit scoring of peer-to-peer lending using attention mechanism LSTM. IEEE Access 7:2161–2168. https://doi.org/10.1109/ACCESS.2018.2887138

    Article  Google Scholar 

  14. Al-Janabi S, Salman AH (2021) Sensitive integration of multilevel optimization model in human activity recognition for smartphone and smartwatch applications. Big Data Mining Anal 4(2):124–138. https://doi.org/10.1007/978-3-030-23672-4_23

    Article  Google Scholar 

  15. Al-Janabi S, Alkaim A, Al-Janabi E et al (2021) Intelligent forecaster of concentrations (PM2. 5, PM10, NO2, CO, O3, SO2) caused air pollution (IFCsAP). Neural Comput Appl. https://doi.org/10.1007/s00521-021-06067-7

    Article  Google Scholar 

  16. Al-Janabi S, Mohammad M, Al-Sultan A (2020) A new method for prediction of air pollution based on intelligent computation. Soft Comput 24(1):661–680. https://doi.org/10.1007/s00500-019-04495-1

    Article  Google Scholar 

  17. Al-Janabi S, Al-Shourbaji I (2016) A hybrid image steganography method based on genetic algorithm. In: 2016 7th international conference on sciences of electronics, technologies of information and telecommunications (SETIT). IEEE, pp. 398–404. https://doi.org/10.1109/SETIT.2016.7939903

  18. Omer Y, Sapir R, Hatuka Y et al (2019) What is a face? Critical Features Face Detect Percep 48(5):437–446. https://doi.org/10.1177/0301006619838734

    Article  Google Scholar 

  19. Al-Janabi S, Alkaim AF (2020) A nifty collaborative analysis to predicting a novel tool (DRFLLS) for missing values estimation[J]. Soft Comput 24(1):555–569. https://doi.org/10.1007/s00500-019-03972-x

    Article  Google Scholar 

  20. Al-Janabi S, Al-Shourbaji I (2016) A smart and effective method for digital video compression. In: 2016 7th international conference on sciences of electronics, technologies of information and telecommunications (SETIT). IEEE, pp. 532–538. https://doi.org/10.1109/SETIT.2016.7939927

  21. Chrysos GG, Antonakos E, Snape P et al (2018) A comprehensive performance evaluation of deformable face tracking “in-the-wild.” Int J Comput Vision 126(2–4):198–232. https://doi.org/10.1007/s11263-017-0999-5

    Article  MathSciNet  Google Scholar 

  22. Sonkusare S, Ahmedt-Aristizabal D, Aburn MJ et al (2019) Detecting changes in facial temperature induced by a sudden auditory stimulus based on deep learning-assisted face tracking. Sci Rep 9(1):1–11. https://doi.org/10.1038/s41598-019-41172-7

    Article  Google Scholar 

  23. Low CC, Ong LY, Koo VC et al (2020) Multi-audience tracking with RGB-D camera on digital signage. Heliyon 6(9):e05107. https://doi.org/10.1016/j.heliyon.2020.e05107

    Article  Google Scholar 

  24. Yang A, Yang X, Wu W et al (2019) Research on feature extraction of tumor image based on convolutional neural network. IEEE Access 7:24204–24213. https://doi.org/10.1109/ACCESS.2019.2897131

    Article  Google Scholar 

  25. Rajan AP, Mathew AR (2019) Evaluation and applying feature extraction techniques for face detection and recognition. Indonesian J Elect Eng Inform (IJEEI) 7(4):742–749. https://doi.org/10.52549/ijeei.v7i4.935

    Article  Google Scholar 

  26. Tao X, Zhang D, Ma W et al (2018) Automatic metallic surface defect detection and recognition with convolutional neural networks. Appl Sci 8(9):1575. https://doi.org/10.3390/app8091575

    Article  Google Scholar 

  27. Jangid M, Srivastava S (2018) Handwritten devanagari character recognition using layer-wise training of deep convolutional neural networks and adaptive gradient methods. J Imaging 4(2):41. https://doi.org/10.3390/jimaging4020041

    Article  Google Scholar 

  28. Yuan F, Zhang L, Wan B et al (2019) Convolutional neural networks based on multi-scale additive merging layers for visual smoke recognition. Mach Vis Appl 30(2):345–358. https://doi.org/10.1007/s00138-018-0990-3

    Article  Google Scholar 

  29. Ashwin TS, Guddeti RMR (2020) Automatic detection of students’ affective states in classroom environment using hybrid convolutional neural networks. Educ Inf Technol 25(2):1387–1415. https://doi.org/10.1007/s10639-019-10004-6

    Article  Google Scholar 

  30. Saeedimoghaddam M, Stepinski TF (2020) Automatic extraction of road intersection points from USGS historical map series using deep convolutional neural networks. Int J Geogr Inf Sci 34(5):947–968. https://doi.org/10.1080/13658816.2019.1696968

    Article  Google Scholar 

  31. Jumani SZ, Ali F, Guriro S et al (2019) Facial expression recognition with histogram of oriented gradients using CNN. Indian J Sci Technol 12(24):1–8. https://doi.org/10.17485/ijst/2019/v12i24/145093

    Article  Google Scholar 

  32. Achour B, Belkadi M, Filali I et al (2020) Image analysis for individual identification and feeding behaviour monitoring of dairy cows based on Convolutional Neural Networks (CNN). Biosys Eng 198:31–49. https://doi.org/10.1016/j.biosystemseng.2020.07.019

    Article  Google Scholar 

  33. Rauber J, Zimmermann R, Bethge M et al (2020) Foolbox Native: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX. J Open Source Softw 5(53):2607. https://doi.org/10.21105/joss.02607

    Article  Google Scholar 

  34. Bendjillali RI, Beladgham M, Merit K et al (2019) Improved facial expression recognition based on DWT feature for deep CNN. Electronics 8(3):324. https://doi.org/10.3390/electronics8030324

    Article  Google Scholar 

  35. Zhu R, Gong X, Hu S et al (2019) Power quality disturbances classification via fully-convolutional Siamese network and k-nearest neighbor. Energies 12(24):4732. https://doi.org/10.3390/en12244732

    Article  Google Scholar 

  36. Yang L, Jiang P, Wang F et al (2018) Robust real-time visual object tracking via multi-scale full-convolution Siamese networks. Multimed Tools Appl 77(17):22131–22143. https://doi.org/10.1007/s11042-018-5664-7

    Article  Google Scholar 

  37. Li D, Yu Y, Chen X (2019) Object tracking framework with Siamese network and re-detection mechanism. EURASIP J Wirel Commun Netw 2019(1):261. https://doi.org/10.1186/s13638-019-1579-x

    Article  Google Scholar 

  38. Nguyen TL, Han DY (2020) Detection of road surface changes from multi-temporal unmanned aerial vehicle images using a convolutional Siamese network. Sustainability 12(6):2482. https://doi.org/10.3390/su12062482

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the following projects: 1. Research on Publicity Channels of Traditional Chinese Medicine Culture in Primary and Middle Schools in Ethnic Minority Areas, a Project of Collaborative Development and Research Center for Sichuan Traditional Chinese Medicine Culture, Project No. ZYYWH1813. 2. Research on Home Protection Methods of Multi-Dimensional Linkage for Tibetan and Yi Infants in Major Public Health Emergencies—Exemplified by COVID-19 Pandemic, a Project of Sichuan 0-3 Years Old Infants’ Early Development and Education Research Center, Project No. SCLS20-13. 3. Research on AIDS Prevention Publicity Channels for Medical Students of Yi Ethnic Group to Serve the Hometown, a project of Sichuan Sex Sociology and Sex Education Research Center, Project No. SXJYB1927.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Shanxi Polytechnic College, Taiyuan, 030006, China

    Yun Wang

  2. Institute of Microelectronics, Chinese Academy of Sciences, Beijing, 100029, China

    Lu Huang

  3. Department of Oral Biology, Division of Orthodontics, Harvard School of Dental Medicine, Harvard University, Boston, 02115, USA

    Austin Lin Yee

Authors
  1. Yun Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Lu Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Austin Lin Yee
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Austin Lin Yee.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Huang, L. & Yee, A.L. Full-convolution Siamese network algorithm under deep learning used in tracking of facial video image in newborns. J Supercomput 78, 14343–14361 (2022). https://doi.org/10.1007/s11227-022-04439-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04439-x

Keywords