skip to main content
10.1145/3512527.3531350acmconferencesArticle/Chapter ViewAbstractPublication PagesicmrConference Proceedingsconference-collections
short-paper

Real-Time Deepfake System for Live Streaming

Published: 27 June 2022 Publication History

Abstract

This paper proposes a real-time deepfake framework to assist users use deep forgery to conduct live streaming, further to protect privacy and increase interesting by selecting different reference faces to create a non-existent fake face. Nowadays, because of the demand for live broadcast functions such as selling goods, playing games, and auctions, the opportunities for anchor exposure are increasing, which leads live streamers pay more attention to their privacy protection. Meanwhile, the traditional technology of deepfake is more likely to infring on the portrait rights of others, so our framework supports users to select different face features for facial tampering to avoid infringement. In our framework, through feature extractor, heatmap transformer, heatmap regression and face blending, face reenactment could be confirmed effectively. Users can enrich the personal face feature database by uploading different photos, and then select the desired picture for tampering on this basis, and finally real-time tampering live broadcast is achieved. Moreover, our framework is a closed loop self-adaptation system as it allows users to update the database themselves to extend face feature data and improve conversion efficiency.

Supplementary Material

MP4 File (ICMR22-271.mp4)
Presentation video

References

[1]
Renwang Chen, Xuanhong Chen, Bingbing Ni, and Yanhao Ge. 2020. Simswap: An efficient framework for high fidelity face swapping. In Proceedings of the 28th ACM International Conference on Multimedia. 2003--2011.
[2]
Davis E King. 2009. Dlib-ml: A machine learning toolkit. The Journal of Machine Learning Research, Vol. 10 (2009), 1755--1758.
[3]
Yuval Nirkin, Yosi Keller, and Tal Hassner. 2019. FSGAN: Subject Agnostic Face Swapping and Reenactment. CoRR, Vol. abs/1908.05932 (2019). http://arxiv.org/abs/1908.05932
[4]
Yuval Nirkin, Iacopo Masi, Anh Tran Tuan, Tal Hassner, and Gerard Medioni. 2018. On face segmentation, face swapping, and face perception. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, 98--105.
[5]
Andreas Rö ssler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019. FaceForensics+: Learning to Detect Manipulated Facial Images. CoRR, Vol. abs/1901.08971 (2019). http://arxiv.org/abs/1901.08971
[6]
Jizhe Zhou and Chi-Man Pun. 2021. Personal Privacy Protection via Irrelevant Faces Tracking and Pixelation in Video Live Streaming. IEEE Transactions on Information Forensics and Security, Vol. 16 (2021), 1088--1103. https://doi.org/10.1109/TIFS.2020.3029913

Cited By

View all
  • (2024)Gen-AI for User Safety: A Survey2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825656(5315-5324)Online publication date: 15-Dec-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMR '22: Proceedings of the 2022 International Conference on Multimedia Retrieval
June 2022
714 pages
ISBN:9781450392389
DOI:10.1145/3512527
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. face blending
  2. live streaming
  3. one image driven
  4. real-time deepfake

Qualifiers

  • Short-paper

Conference

ICMR '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)83
  • Downloads (Last 6 weeks)7
Reflects downloads up to 08 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Gen-AI for User Safety: A Survey2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825656(5315-5324)Online publication date: 15-Dec-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media