Skip to main content
SpringerLink
Log in

An efficient cybersecurity framework for facial video forensics detection based on multimodal deep learning

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In cloud services and Internet-of-Things (IoT) applications, the cybersecurity in video transmission technologies has drawn much attention in recent researches due to the rapid growth of cyber-risks on both individuals and institutions. Unfortunately, the spoofing attack, a kind of cyber-risk, has increased the number of cyber-criminals in data transfer applications without being detected, especially in smart cities. Several applications based on online video communications, such as online testing and video conferences, are involved in smart cities. The video displays various variations of a person, which makes face recognition an important concept in security implementation. The face spoofing attacks are mainly based on the person's face replication by replaying a video or by printed photos. Therefore, video forgery detection and related spoof attack detection have become a new topic in cybersecurity research. From this perspective, this paper presents a deep learning approach for video face forensic detection with a cyber facial spoofing attack using two methodologies. The first methodology is based on a convolutional neural network (CNN) to extract features from the input video frames. The model has five convolutional layers followed by five pooling layers. The second methodology is based on convolutional long short-term memory (ConvLSTM). This model comprises two pooling layers, two convolutional layers, and a convolutional LSTM layer. Each methodology includes a fully-connected layer to interconnect between the feature map resulting from the feature extraction process and the classification layer. A SoftMax layer performs the classification task in each method. This paper aims to achieve an optimum modality for face forensic detection to overcome spoofing attacks. Simulation results reveal that the ConvLSTM with CNN methodology achieves better classification results as the extracted features are more comprehensive than those of other conventional approaches. Also, it achieves an accuracy of 99%, and the works presented in the literature achieve an accuracy levels up to 95%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35

Similar content being viewed by others

References

  1. Scarfò, A. (2018). The cyber security challenges in the IoT era. In: Security and resilience in intelligent data-centric systems and communication networks. Academic Press, 53–76

  2. Xiao B, Wei Y, Bi X, Li W, Ma J (2020) Image splicing forgery detection combining coarse to refined convolutional neural network and adaptive clustering. Inf Sci 511:172–191

    Article  MathSciNet  Google Scholar 

  3. Fadl, S., Megahed, A., Han, Q., & Qiong, L. (2020). Frame duplication and shuffling forgery detection technique in surveillance videos based on temporal average and gray level co-occurrence matrix. Multimed Tools Appl 1–25

  4. Li L, Correia PL, Hadid A (2018) Face recognition under spoofing attacks: countermeasures and research directions. IET Biometrics 7(1):3–14

    Article  Google Scholar 

  5. Kaiiali M, Ozkaya A, Altun H, Haddad H, Alier M (2016) Designing a secure exam management system (SEMS) for M-learning environments. IEEE Trans Learn Technol 9(3):258–271

    Article  Google Scholar 

  6. Arashloo SR, Kittler J, Christmas W (2017) An anomaly detection approach to face spoofing detection: a new formulation and evaluation protocol. IEEE Access 5:13868–13882

    Article  Google Scholar 

  7. Mohan K, Chandrasekhar P, Jilani SAK (2017) A combined histogram of oriented gradient-local phase quantization (HOG-LPQ) with fuzzy-support vector machine (Fuz-SVM) classifier for object face liveness detection. In: 2017 International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) (pp. 531–537). IEEE

  8. Faseela T, Jayasree M (2016) Spoof face recognition in video using KSVM. Proc Technol 24:1285–1291

    Article  Google Scholar 

  9. Li L, Feng X, Boulkenafet Z, Xia Z, Li M, Hadid A (2016) An original face anti-spoofing approach using partial convolutional neural network. In: 2016 Sixth international conference on image processing theory, tools and applications (IPTA) (pp. 1–6). IEEE

  10. Fourati E, Elloumi W, Chetouani A (2020) Anti-spoofing in face recognition-based biometric authentication using image quality assessment. Multimed Tools Appl 79(1–2):865–889

    Article  Google Scholar 

  11. Katika BR, Karthik K (2020) Face anti-spoofing by identity masking using random walk patterns and outlier detection. Pattern Anal Appl. https://doi.org/10.1007/s10044-020-00875-8

    Article  Google Scholar 

  12. Elloumi W, Chetouani A, Charrada TB, Fourati E (2020) Anti-spoofing in face recognition: deep learning and image quality assessment-based approaches. In: Jiang R, Li C-T, Crookes D, Meng W, Rosenberger C (eds) Deep biometrics. Springer, Cham, pp 51–69

    Chapter  Google Scholar 

  13. Anjos A, & Marcel S (2011) Counter-measures to photo attacks in face recognition: a public database and a baseline. In: 2011 international joint conference on Biometrics (IJCB) (pp. 1–7). IEEE

  14. Galbally J, Marcel S, Fierrez J (2013) Image quality assessment for fake biometric detection: application to iris, fingerprint, and face recognition. IEEE Trans Image Process 23(2):710–724

    Article  MathSciNet  Google Scholar 

  15. Erdogmus N, Marcel S (2014) Spoofing face recognition with 3D masks. IEEE Trans Inf Forensics Secur 9(7):1084–1097

    Article  Google Scholar 

  16. Patel K, Han H, Jain AK (2016) Secure face unlock: spoof detection on smartphones. IEEE Trans Inf Forensics Secur 11(10):2268–2283

    Article  Google Scholar 

  17. Liu S, Yang B, Yuen PC, Zhao G (2016) A 3D mask face anti-spoofing database with real world variations. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 100–106).

  18. Feng L, Po LM, Li Y, Xu X, Yuan F, Cheung TCH, Cheung KW (2016) Integration of image quality and motion cues for face anti-spoofing: a neural network approach. J Vis Commun Image Represent 38:451–460

    Article  Google Scholar 

  19. Lucena O, Junior A, Moia V, Souza R, Valle E, Lotufo R (2017) Transfer learning using convolutional neural networks for face anti-spoofing. In: International conference image analysis and recognition. (pp. 27–34). Springer, Cham

  20. George A, Mostaani Z, Geissenbuhler D, Nikisins O, Anjos A, Marcel S (2019) Biometric face presentation attack detection with multi-channel convolutional neural network. IEEE Trans Inf Forensics Secur 15:42–55

    Article  Google Scholar 

  21. Tolosana R, Gomez-Barrero M, Busch C, Ortega-Garcia J (2019) Biometric presentation attack detection: beyond the visible spectrum. IEEE Trans Inf Forensics Secur 15:1261–1275

    Article  Google Scholar 

  22. Scherhag U, Rathgeb C, Merkle J, Breithaupt R, Busch C (2019) Face recognition systems under morphing attacks: a survey. IEEE Access 7:23012–23026

    Article  Google Scholar 

  23. Liu A, Wan J, Escalera S, Jair Escalante H, Tan Z, Yuan Q, Li SZ (2019). Multi-modal face anti-spoofing attack detection challenge at cvpr2019. In Proceedings of the IEEE Conference on computer vision and pattern recognition workshops (pp. 0–0).

  24. Tolosana R, Vera-Rodriguez R, Fierrez J, Morales A, Ortega-Garcia J (2020) Deepfakes and beyond: a survey of face manipulation and fake detection. arXiv preprint http://arxiv.org/abs/2001.00179

  25. Liu YC, Bianchin G, Pasqualetti F (2020) Secure trajectory planning against undetectable spoofing attacks. Automatica 112:108655

    Article  MathSciNet  Google Scholar 

  26. Mahmoud AA, El-Shafai W, Taha TE, El-Rabaie ESM, Zahran O, El-Fishawy AS, Abd El-Samie FE (2020) A statistical framework for breast tumor classification from ultrasonic images. Multimed Tools Appl 80:5977

    Article  Google Scholar 

  27. Whittington JC, Bogacz R (2019) Theories of error back-propagation in the brain. Trends Cognit Sci. https://doi.org/10.1016/j.tics.2018.12.005

    Article  Google Scholar 

  28. El-Gindy SAE, Hamad A, El-Shafai W, Khalaf AA, El-Dolil SM, Taha TE, Abd El-Samie FE (2021) Efficient communication and EEG signal classification in wavelet domain for epilepsy patients. J Ambient Intell Humaniz Comput 22:1–16

    Google Scholar 

  29. Labati RD, Muñoz E, Piuri V, Sassi R, Scotti F (2019) Deep-ECG: convolutional neural networks for ECG biometric recognition. Pattern Recogn Lett 126:78–85

    Article  Google Scholar 

  30. Shan W, Yi Y, Huang R, Xie Y (2019) Robust contrast enhancement forensics based on convolutional neural networks. Signal Process Image Commun 71:138–146

    Article  Google Scholar 

  31. El-Hag NA, Sedik A, El-Shafai W, El-Hoseny HM, Khalaf AA, El-Fishawy AS, El-Banby GM (2020) Classification of retinal images based on convolutional neural network. Microsc Res Tech 84:394

    Article  Google Scholar 

  32. Xu Q, Zhang M, Gu Z, Pan G (2019) Overfitting remedy by sparsifying regularization on fully-connected layers of CNNs. Neurocomputing 328:69–74

    Article  Google Scholar 

  33. Bappy JH, Simons C, Nataraj L, Manjunath BS, Roy-Chowdhury AK (2019) Hybrid lstm and encoder–decoder architecture for detection of image forgeries. IEEE Trans Image Process 28(7):3286–3300

    Article  MathSciNet  Google Scholar 

  34. Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Adv Neural Inf Process Syst pp 3104–3112

  35. Bhunia AK, Konwer A, Bhunia AK, Bhowmick A, Roy PP, Pal U (2019) Script identification in natural scene image and video frames using an attention based convolutional-LSTM network. Pattern Recogn 85:172–184

    Article  Google Scholar 

  36. Bock S, Weiß M (2019) A proof of local convergence for the Adam optimizer. In: 2019 International Joint Conference On Neural Networks (IJCNN) (pp. 1–8). IEEE.

  37. Smith DF, Wiliem A, Lovell BC (2015) Face recognition on consumer devices: reflections on replay attacks. IEEE Trans Inf Forensics Secur 10(4):736–745

    Article  Google Scholar 

  38. Chui KT, Tsang KF, Chi HR, Ling BWK, Wu CK (2016) An accurate ECG-based transportation safety drowsiness detection scheme. IEEE Trans Ind Inf 12(4):1438–1452

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Deanship of Scientific Research, Taif University Researchers Supporting Project number (TURSP-2020/08), Taif University, Taif, Saudi Arabia.

Author information

Authors and Affiliations

  1. Department of the Robots and Intelligent Machines, Faculty of AI, Kafrelsheikh University, Kafr El-Shaikh, Egypt

    Ahmed Sedik

  2. Department of Information Technology, College of Computers and Information Technology, Taif University, P. O. Box 11099, Taif, 21944, Saudi Arabia

    Osama S. Faragallah

  3. Department of Electrical Engineering, Faculty of Engineering, Menoufia University, Shebin El-Kom, 32511, Egypt

    Hala S. El-sayed

  4. Department of Industrial Electronics and Control Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf, 32952, Egypt

    Ghada M. El-Banby

  5. Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf, 32952, Egypt

    Fathi E. Abd El-Samie & Walid El-Shafai

  6. Department of Electronics and Electrical Communications, Faculty of Engineering, Minia University, El Minya, Egypt

    Ashraf A. M. Khalaf

  7. Security Engineering Lab, Computer Science Department, Prince Sultan University, Riyadh, 11586, Saudi Arabia

    Walid El-Shafai

  8. Department of Information Technology, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, Riyadh, 21974, Saudi Arabia

    Fathi E. Abd El-Samie

Authors
  1. Ahmed Sedik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Osama S. Faragallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hala S. El-sayed
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ghada M. El-Banby
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fathi E. Abd El-Samie
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ashraf A. M. Khalaf
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Walid El-Shafai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Osama S. Faragallah.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sedik, A., Faragallah, O.S., El-sayed, H.S. et al. An efficient cybersecurity framework for facial video forensics detection based on multimodal deep learning. Neural Comput & Applic 34, 1251–1268 (2022). https://doi.org/10.1007/s00521-021-06416-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06416-6

Keywords

Search

Navigation