Sergey Zotov
ABSTRACT
We analyzed the developed methods of computer vision in areas associated with recognition and detection of DeepFakes using various models and architectures of neural networks: mainly GAN and CNN. We also discussed the main types and models of these networks that are most effective in detecting and recognizing objects from different data sets, which were provided in the studied articles.
- Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. Mesonet: a compact facial video forgery detection network. In IEEE International Workshop on Information Forensics and Security (WIFS), 2018.Google Scholar
Cross Ref
- Xin Yang, Yuezun Li, and Siwei Lyu. Exposing deep fakes using inconsistent head poses. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019.Google ScholarCross Ref
- Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A Efros. Generative visual manipulation on the natural image manifold. In European Conference on Computer Vision, pages 597–613. Springer, 2016.Google Scholar
- Raymond A Yeh, Chen Chen, Teck Yian Lim, Alexander G Schwing, Mark Hasegawa-Johnson, and Minh N Do. Semantic image inpainting with deep generative models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5485–5493, 2017.Google ScholarCross Ref
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.Google Scholar
- A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2012, pp. 1097–1105/Google ScholarDigital Library
- Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In CVPR, 2017.Google ScholarCross Ref
- K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.Google Scholar
- Pavel Korshunov and Sébastien Marcel. Deepfakes: a new threat to face recognition? assessment and detection. arXiv preprint arXiv:1812.08685, 2018.Google Scholar
- Y. Li and S.Li. Exposing DeepFake videos by detecting face warping artifacts. arXiv preprint arXiv: 1811.00656, 2019Google Scholar
- Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.Google Scholar
- Aayush Bansal, Shugao Ma, Deva Ramanan, and Yaser Sheikh. Recycle-gan: Unsupervised video retargeting. In ECCV, 2018.Google Scholar
- Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In CVPR, 2018.Google ScholarCross Ref
- Ming-Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. In NIPS, 2017.Google ScholarDigital Library
- A.Karnewar, O.Wang, R.S. Iyengar. MSG-GAN: Multi-scale gradient GAN for stable image synthesis. arXiv preprint arXiv: 1903.06048, 2019Google Scholar
- D. O'Sullivan. When seeing is no longer believing. CNN Business, 2017.Google Scholar
- DeepTrace. The state of DeepFakes: Landscape, Threats and Impact. September 2019Google Scholar
- G. Saon, H.-K. J. Kuo, S. Rennie, and M. Picheny. The ibm 2015 english conversational telephone speech recognition system. arXiv preprint arXiv:1505.05899, 2015Google Scholar
- M. Alzantot, B. Balaji, and M. Srivastava. Did you hear that? adversarial examples against automatic speech recognition. arXiv preprint arXiv:1801.00554, 2018Google Scholar
- A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, R. Prenger, S. Satheesh, S. Sengupta, A. Coates, and A. Y. Ng. Deepspeech: Scaling up end-to-end speech recognition. arXiv:1412.5567,2014.Google Scholar
- D. Wang and J. Chen. Supervised speech separation based on deeplearning: An overview. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 26(10):1702–1726, 2018.Google Scholar
- Thanh Thi Nguyen, Cuong M. Nguyen, Dung Tien Nguyen, Duc Thanh Nguyen, Saeid Nahavandi, Deep learning for Deepfakes creation and Detection. Preprint. September 2019. URL: https://arxiv.org/abs/1909.11573Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.Google ScholarCross Ref
- Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096, 2018.Google Scholar
- M. Pividori, G.L. Grinblat, L.C. Uzal. Exploiting GAN Internal capacity for high-quality reconstruction of natural images. arXiv preprint arXiv: 1911.05630, 2019Google Scholar
- Y. LeCun and C. Cortes, “MNIST handwritten digit database,” http://yann.lecun.com/exdb/mnist/, 2010.Google Scholar
- A. Krizhevsky, V. Nair, and G. Hinton, “Cifar-10 (canadian institute for advanced research),” http://www.cs.toronto.edu/ kriz/cifar.html, 2010.Google Scholar
- K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” CoRR, vol. abs/1502.01852, 2015.Google Scholar
- J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” arXiv preprint arXiv:1804.02767, 2018.Google Scholar
- S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in Advances in neural information processing systems, 2015, pp. 91–99.Google Scholar
- G. Tzelepis, A.Azif, S. Baci, S. Cavdar, E.E. Aksoy. Deep neural network compression for image classification and object detection. . arXiv preprint arXiv: 1910.02747, 2019Google Scholar
- Y.Li, X. Yang, P.Sun, H. Qi, S. Lyu. Celeb-DF(v2): a new dataset for DeepFake Forensics. arXiv preprint arXiv: 1909.12962, 2019.Google Scholar
- F. Marra, D. Gragnaniello, D. Cozzolino, , “Detection of gan generated fake images over social networks,” in 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), 384–389, IEEE (2018).Google ScholarCross Ref
- Xin Yang, Yuezun Li, and Siwei Lyu. Exposing deep fakes using inconsistent head poses. In ICASSP, 2019.Google ScholarCross Ref
- Peng Zhou, Xintong Han, Vlad I Morariu, and Larry S Davis. Two-stream neural networks for tampered face detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, pages 1831–1839, 2017.Google ScholarCross Ref
- R. Valle, U. CNMAT, W. Cai, , “Tequilagan: How to easily identify gan samples,” arXiv preprint arXiv:1807.04919Google Scholar
- S. McCloskey and M. Albright, “Detecting gan-generated imagery using color cues,” arXiv preprint arXiv:1812.08247Google Scholar
- A.E. Dirik, S.Bayram, H.T.Sencar, , “New features to identify computer generated images,” in Image Processing, 2007.ICIP2007. IEEE International Conference on, 4, IV–433, IEEEGoogle Scholar
- B. Mader, M. S. Banks, and H. Farid, “Identifying computer generated portraits: The importance of training and incentives,” Perception 46(9), 1062–1076(2017)Google ScholarCross Ref
- J.-Y. Zhu, T. Park, P. Isola, , “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in IEEE International Conference on Computer Vision, (2017).Google ScholarCross Ref
- L. Nataraj, T.M. Mohammed, S. Chandrasekaran, A. Flenner, J.H. Bappy, A.K. Roy-Chowdhury, B.S. Manjunath. Detecting gan generated fake images using co-occurrence matrices. arXiv preprint arXiv: 1903.06836, 2019Google Scholar
- Z. Liu, P. Luo, X. Wang, , “Deep learning face attributes in the wild,” in Proceedings of the IEEE International Conference on Computer Vision, 3730–3738 (2015).Google ScholarDigital Library
- Y. Choi, M. Choi, M. Kim, , “Stargan: Unified generative adversarial networks for multi-domain image-to-image translation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8789–8797 (2018).Google ScholarCross Ref
- X. Zhang, S. Karaman, S.F Chang. Detecting and simulating artifacts in GAN Fake images (Extended version). arXiv preprint arXiv: 1907.06515,2019.Google Scholar
- Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.Google Scholar
- S. Klus, P. Gelb. Tensor-based algorithms for image classification. arXiv preprint arXiv: 1910.02150, 2019Google Scholar
- S. McCloskey, M. Albright. Detecting GAN-generated Imagery using color cues. arXiv preprint arXiv: 1812.08247, 2018Google Scholar
- T. Murakami, R. Fujita, T. Ohki, Y. Kaga, M. Fujio, and K. Takahashi ‘‘Cancelable permutation-based indexing for secure and efficient biometric identification,’’ in IEEE Access, vol. 7, pp. 45563-45582, 2019.Google ScholarCross Ref
- T. Yanagawa, S. Aoki, and T. Oyama, ‘‘Diversity of human finger vein patterns and its application to personal identification,’’ Bull. Inform. Cybern.,vol. 41, no. 1, pp. 1–9, 2007.Google Scholar
- T. Piplani, N. Merill and J. Chuang, "Faking it, Making it: Fooling and Improving Brain-Based Authentication with Generative Adversarial Networks," 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), 2018, pp. 1-7.Google Scholar
- T.Adviser [Электронный ресурс] URL: http://www.tadviser.ru/index.php/%D0%A1%D1%82%D0%B0%D1%82%D1%8C%D1%8F:DeepFakeGoogle Scholar
- H. Hukkelas, R.Mestar and F. Lindseth. DeepPrivacy: A generative adversarial network for face anonymization. arXiv preprint arXiv: 1909.04538, 2019Google Scholar
- Roman, D., Natalia, P. Artificial intelligence legal policy: Limits of use of some kinds of AI // ACM International Conference Proceeding Series, Part F147956, 2019. P. 343-346.Google Scholar
Recommendations
Deepfake Detection Using CNN Trained on Eye Region
Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial IntelligenceAbstractIn this work, we will develop a simple convolutional neural network to detect deepfakes in videos on a frame-by-frame level, focusing on the region around the eyes. Since deepfakes are increasingly being created using forms of CNN, it should be ...
An integrated spatiotemporal-based methodology for deepfake detection
AbstractRapid advances in deep learning models have made it easier for public and crackers to generate hyper-realistic deepfake videos in which faces are swapped. Such deepfake videos may constitute a significant threat to the world if they are misused to ...
Deepfake Video Detection by Using Convolutional Gated Recurrent Unit
ICMLC '21: Proceedings of the 2021 13th International Conference on Machine Learning and ComputingRapid development in deep learning is making it easier to create fake videos known as “deepfake” videos in which human faces are swapped. Since deepfake videos are difficult to recognize by human eyes, it becomes important to automatically detect these ...
Comments