Skip to main content

Potential Threat of Face Swapping to eKYC with Face Registration and Augmented Solution with Deepfake Detection

  • Conference paper
  • First Online:
Future Data and Security Engineering (FDSE 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13076))

Included in the following conference series:

  • 1193 Accesses

Abstract

It is necessary to develop an efficient and secure mechanism to verify customers digitally for various online transactions. Integrating biometric solutions into the online user registration and verification processes is a promising trend for electronic Know Your Customer (eKYC) systems. However, Deepfake or face manipulation techniques may become a threat for eKYC with face authentication. In this paper, we introduce this potential attack of Deepfake on eKYC by swapping and manipulating faces between source and target faces. We then propose to augment the security for current eKYC systems with Deepfake detection. We conduct the experiments on the 10K video clips in the private test of Deepfake Detection Challenge 2020, and our method, following the Capsule-forensics approach, achieves the Logloss score of 0.5189, among the top 6% best results among the 2114 teams worldwide. This result demonstrates that our deepfake detection algorithm can be a promising method to provide extra protection for eKYC solutions with face registration and authentication.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Faceswap (2017). https://github.com/MarekKowalski/FaceSwap

  2. Deepfake (2018). https://github.com/deepfakes/faceswap

  3. Terrifying high-tech porn: Creepy deepfake videos are on the rise (2018). https://www.foxnews.com/tech/terrifying-high-tech-porn-creepy-deepfake-videos-are-on-the-rise

  4. Agarwal, S., Farid, H., Gu, Y., He, M., Nagano, K., Li, H.: Protecting world leaders against deep fakes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (June 2019)

    Google Scholar 

  5. Bappy, J.H., Simons, C., Nataraj, L., Manjunath, B.S., Roy-Chowdhury, A.K.: Hybrid lstm and encoder-decoder architecture for detection of image forgeries. IEEE Trans. Image Process. 28(7), 3286–3300 (2019)

    Article  MathSciNet  Google Scholar 

  6. Bhattacharjee, S., Mohammadi, A., Anjos, A., Marcel, S.: Recent advances in face presentation attack detection. In: Marcel, S., Nixon, M.S., Fierrez, J., Evans, N. (eds.) Handbook of Biometric Anti-Spoofing. ACVPR, pp. 207–228. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-92627-8_10

    Chapter  Google Scholar 

  7. Bonettini, N., Cannas, E.D., Mandelli, S., Bondi, L., Bestagini, P., Tubaro, S.: Video face manipulation detection through ensemble of CNNs. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 5012–5019 (2021)

    Google Scholar 

  8. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1800–1807. IEEE, Honolulu, HI (July 2017)

    Google Scholar 

  9. Costa-Pazo, A., Vazquez-Fernandez, E., Alba-Castro, J.L., González-Jiménez, D.: Challenges of face presentation attack detection in real scenarios. In: Marcel, S., Nixon, M.S., Fierrez, J., Evans, N. (eds.) Handbook of Biometric Anti-Spoofing. ACVPR, pp. 247–266. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-92627-8_12

    Chapter  Google Scholar 

  10. Davletshin, A.: (2020). https://github.com/ntech-lab/deepfakedetection-challenge

  11. Deng, J., Guo, J., Ververas, E., Kotsia, I., Zafeiriou, S.: RetinaFace: single-shot multi-level face localisation in the wild. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5202–5211. IEEE, Seattle, WA, USA (June 2020)

    Google Scholar 

  12. Dolhansky, B., et al.: The deepfake detection challenge dataset. CoRR abs/2006.07397 (2020)

    Google Scholar 

  13. Hernandez-Ortega, J., Fierrez, J., Morales, A., Galbally, J.: Introduction to face presentation attack detection. In: Marcel, S., Nixon, M.S., Fierrez, J., Evans, N. (eds.) Handbook of Biometric Anti-Spoofing. ACVPR, pp. 187–206. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-92627-8_9

    Chapter  Google Scholar 

  14. Jain, V., Learned-Miller, E.: FDDB: a benchmark for face detection in unconstrained settings. Technical report UM-CS-2010-009, University of Massachusetts, Amherst (2010)

    Google Scholar 

  15. Khalid, H., Woo, S.S.: Oc-FakeDect: classifying deepfakes using one-class variational autoencoder. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (June 2020)

    Google Scholar 

  16. Kim, H., et al.: Deep video portraits. ACM Trans. Graph. (TOG) 37(4), 163 (2018)

    Google Scholar 

  17. Li, J., et al.: DSFD: dual shot face detector. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5055–5064. IEEE, Long Beach, CA, USA (June 2019)

    Google Scholar 

  18. Li, L., et al.: Face x-ray for more general face forgery detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5000–5009 (2020)

    Google Scholar 

  19. Li, Y., Chang, M.C., Lyu, S.: In ictu oculi: exposing ai created fake videos by detecting eye blinking. In: 2018 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–7 (2018)

    Google Scholar 

  20. Li, Y., Lyu, S.: Exposing deepfake videos by detecting face warping artifacts. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2019)

    Google Scholar 

  21. Marcel, S., Nixon, M.S., Fiérrez, J., Evans, N.W.D. (eds.): Handbook of Biometric Anti-Spoofing - Presentation Attack Detection, Second Edition. Advances in Computer Vision and Pattern Recognition, Springer, Heidelberg (2019)

    Google Scholar 

  22. Mirsky, Y., Lee, W.: The creation and detection of deepfakes: a survey. ACM Comput. Surv. 54(1) (2021)

    Google Scholar 

  23. Mondal, P.C., Deb, R., Huda, M.N.: Transaction authorization from know your customer (kyc) information in online banking. In: 2016 9th International Conference on Electrical and Computer Engineering (ICECE), pp. 523–526 (2016)

    Google Scholar 

  24. Nguyen, H.H., Yamagishi, J., Echizen, I.: Capsule-forensics: using capsule networks to detect forged images and videos. In: ICASSP 2019–2019 IEEE International Conference on Acoustics. Speech and Signal Processing (ICASSP), pp. 2307–2311. IEEE, Brighton, United Kingdom (May 2019)

    Google Scholar 

  25. Nirkin, Y., Keller, Y., Hassner, T.: Fsgan: subject agnostic face swapping and reenactment. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (October 2019)

    Google Scholar 

  26. Perov, I., et al.: Deepfacelab: a simple, flexible and extensible face swapping framework (2020)

    Google Scholar 

  27. Raghavendra, R., Raja, K.B., Venkatesh, S., Busch, C.: Transferable deep-cnn features for detecting digital and print-scanned morphed face images. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1822–1830 (2017)

    Google Scholar 

  28. Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Niessner, M.: FaceForensics++: learning to detect manipulated facial images. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1–11. IEEE, Seoul, Korea (South) (October 2019)

    Google Scholar 

  29. Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., Natarajan, P.: Recurrent convolutional strategies for face manipulation detection in videos. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, 16–20 June 2019, pp. 80–87. Computer Vision Foundation/IEEE (2019)

    Google Scholar 

  30. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 3859–3869. NIPS 2017, Curran Associates Inc., Red Hook, NY, USA (2017)

    Google Scholar 

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1409.1556

  32. Tang, X., Du, D.K., He, Z., Liu, J.: Pyramidbox: a context-assisted single shot face detector. In: Proceedings of the European Conference on Computer Vision (ECCV) (September 2018)

    Google Scholar 

  33. Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., Niessner, M.: Face2face: real-time face capture and reenactment of rgb videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2016)

    Google Scholar 

  34. Thies, J., Zollhöfer, M., Nießner, M.: Deferred neural rendering: image synthesis using neural textures. ACM Trans. Graph. 38(4), 1–12 (2019)

    Article  Google Scholar 

  35. Wang, J.S.: Exploring biometric identification in fintech applications based on the modified tam. Financ. Innov. 7(1), 1–24 (2021)

    Article  Google Scholar 

  36. Yang, S., Luo, P., Loy, C.C., Tang, X.: WIDER FACE: a face detection benchmark. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5525–5533. IEEE, Las Vegas, NV, USA (June 2016)

    Google Scholar 

  37. Yang, X., Li, Y., Lyu, S.: Exposing deep fakes using inconsistent head poses. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8261–8265 (2019)

    Google Scholar 

  38. Zakharov, E., Shysheya, A., Burkov, E., Lempitsky, V.: Few-shot adversarial learning of realistic neural talking head models. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9458–9467. IEEE (October 2019)

    Google Scholar 

  39. Zhou, P., Han, X., Morariu, V.I., Davis, L.S .: Two-stream neural networks for tampered face detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (July 2017)

    Google Scholar 

Download references

Acknowledgment

Trong-Le Do and Mai-Khiem Tran were funded by Vingroup Joint Stock Company and supported by the Domestic Master/ PhD Scholarship Programme of Vingroup Innovation Foundation (VINIF), Vingroup Big Data Institute (VINBIGDATA), code VINIF.2019.ThS.22 and VINIF.2020.ThS.JVN.06, respectively.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Minh-Triet Tran .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Do, TL., Tran, MK., Nguyen, H.H., Tran, MT. (2021). Potential Threat of Face Swapping to eKYC with Face Registration and Augmented Solution with Deepfake Detection. In: Dang, T.K., Küng, J., Chung, T.M., Takizawa, M. (eds) Future Data and Security Engineering. FDSE 2021. Lecture Notes in Computer Science(), vol 13076. Springer, Cham. https://doi.org/10.1007/978-3-030-91387-8_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-91387-8_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-91386-1

  • Online ISBN: 978-3-030-91387-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics