Kurtis Haut
ABSTRACT
This paper explores the complexity of what constitutes the demographic features of race and how race is perceived. "Race" is composed of a variety of factors including skin tone, facial features, and accent. Isolating these interrelated race features is a difficult problem and failure to do so properly can easily invite confounding factors. Here we propose a novel method to isolate features of race by using AI-based technology and measure the impact these modifications have on an outcome variable of interest; i.e., perceived credibility. We used videos from a deception dataset for which the ground-truth is known and create three conditions: 1) a Black vs White CycleGAN image condition; 2) an original vs deepfake video condition; 3) an original vs deepfake still frame condition. We crowd-sourced 1736 responses to measure how credibility was influenced by changing the perceived race. We found that it is possible to alter perceived race through modifying demographically visual features alone. However, we did not find any statistically significant differences for credibility across our experiments based on these changes. Our findings help quantify intuitions from prior research that the relationship between racial perception and credibility is more complex than visual features alone. Our presented deepfake framework could be incorporated to precisely measure the impact of a wider range of demographic features (such as gender or age) due to the fine-grained isolation and control that was previously impossible in a lab setting.
Supplemental Material
Available for Download
- [n.d.]. National Library of Medicine How to control confounding effects by statistical analysis. How to control confounding effects by statistical analysis. Accessed: 2022-04-14.Google Scholar
- [n.d.]. Political Analytics An Empirical Justification for the Use of Racially Distinctive Names to Signal Race in Experiments. www.cambridge.org/core/journals/political-analysis/article/an-empirical- justification-for-the-use-of-racially-distinctive-names-to-signal-race-in- experiments/DBC39F875F2DC0F65E7140FC721CE1EB. Accessed: 2022-04-14.Google Scholar
- [n.d.]. Science Direct Confounding: What it is and how to deal with it. https://www.sciencedirect.com/science/article/pii/S0085253815529748. Accessed: 2022-04-14.Google Scholar
- [n.d.]. United States Sentencing Commission demographic sentencing. https://www.ussc.gov/research/research-reports/demographic-differences-sentencing. Accessed: 2022-04--14.Google Scholar
- [n.d.]. Wiley Online Library Overcoming confounding of race with socio- economic status and segregation to explore race disparities in smoking. https: //onlinelibrary.wiley.com/doi/full/10.1111/j.1360-0443.2007.01956.x. Accessed: 2022-04-14.Google Scholar
- Marianne Bertrand and Sendhil Mullainathan. 2004. Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. American economic review 94, 4 (2004), 991--1013.Google Scholar
- Johnny Botha and Heloise Pieterse. 2020. Fake news and deepfakes: A dangerous threat for 21st century information security. In ICCWS 2020 15th International Conference on Cyber Warfare and Security. Academic Conferences and publishing limited. 57.Google Scholar
- Michael Buhrmester, Tracy Kwang, and Samuel D Gosling. 2016. Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality data? (2016).Google Scholar
- Daniel M Butler and Jonathan Homola. 2017. An empirical justification for the use of racially distinctive names to signal race in experiments. Political Analysis 25, 1 (2017), 122--130.Google ScholarCross Ref
- Simone Fabbrizzi, Symeon Papadopoulos, Eirini Ntoutsi, and Ioannis Kompat- siaris. 2021. A survey on bias in visual datasets. arXiv preprint arXiv:2107.07919 (2021).Google Scholar
- Chloë FitzGerald and Samia Hurst. 2017. Implicit bias in healthcare professionals: a systematic review. BMC medical ethics 18, 1 (2017), 1--18.Google Scholar
- Victoria Groom, Jeremy N Bailenson, and Clifford Nass. 2009. The influence of racial embodiment on racial bias in immersive virtual environments. Social Influence 4, 3 (2009), 231--248.Google ScholarCross Ref
- Gül Günaydin, Vivian Zayas, Emre Selcuk, and Cindy Hazan. 2012. I like you but I don't know why: Objective facial resemblance to significant others influences snap judgments. Journal of Experimental Social Psychology 48, 1 (2012), 350--353.Google ScholarCross Ref
- John J Horton, David G Rand, and Richard J Zeckhauser. 2011. The online labo- ratory: Conducting experiments in a real labor market. Experimental economics 14, 3 (2011), 399--425.Google Scholar
- KJ Jager, C Zoccali, A Macleod, and FW Dekker. 2008. Confounding: what it is and how to deal with it. Kidney international 73, 3 (2008), 256--260.Google ScholarCross Ref
- Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator ar- chitecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4401--4410.Google ScholarCross Ref
- Elizabeth A Klonoff and Hope Landrine. 2000. Is skin color a marker for racial discrimination? Explaining the skin color--hypertension relationship. Journal of behavioral medicine 23, 4 (2000), 329--338.Google ScholarCross Ref
- Margaret Bull Kovera. 2019. Racial disparities in the criminal justice system: Prevalence, causes, and a search for solutions. Journal of Social Issues 75, 4 (2019), 1139--1164.Google ScholarCross Ref
- Thomas A LaVeist, Roland J Thorpe Jr, GiShawn A Mance, and John Jackson. 2007. Overcoming confounding of race with socio-economic status and segregation to explore race disparities in smoking. Addiction 102 (2007), 65--70.Google ScholarCross Ref
- Zeus Leonardo. 2004. The color of supremacy: Beyond the discourse of ?white privilege'. Educational philosophy and theory 36, 2 (2004), 137--152.Google Scholar
- Ivy W Maina, Tanisha D Belton, Sara Ginzberg, Ajit Singh, and Tiffani J Johnson. 2018. A decade of studying implicit racial/ethnic bias in healthcare providers using the implicit association test. Social Science & Medicine 199 (2018), 219--229.Google ScholarCross Ref
- Jason P Nance. 2015. Over-disciplining students, racial bias, and the school-to- prison pipeline. U. Rich. L. Rev. 50 (2015), 1063.Google Scholar
- Eirini Ntoutsi, Pavlos Fafalios, Ujwal Gadiraju, Vasileios Iosifidis, Wolfgang Nejdl, Maria-Esther Vidal, Salvatore Ruggieri, Franco Turini, Symeon Papadopoulos, Emmanouil Krasanakis, et al. 2020. Bias in data-driven artificial intelligence systems-An introductory survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 10, 3 (2020), e1356.Google ScholarCross Ref
- Konstantin A Pantserev. 2020. The malicious use of AI-based deepfake technology as the new threat to psychological security and political stability. In Cyber defence in the age of AI, smart societies and augmented humanity. Springer, 37--55.Google Scholar
- Gabriele Paolacci, Jesse Chandler, and Panagiotis G Ipeirotis. 2010. Running experiments on amazon mechanical turk. Judgment and Decision making 5, 5 (2010), 411--419.Google Scholar
- Tabitha C Peck, Sofia Seinfeld, Salvatore M Aglioti, and Mel Slater. 2013. Putting yourself in the skin of a black avatar reduces implicit racial bias. Consciousness and cognition 22, 3 (2013), 779--787.Google Scholar
- Ivan Perov, Daiheng Gao, Nikolay Chervoniy, Kunlin Liu, Sugasa Marangonda, Chris Umé, Mr. Dpfks, Carl Shift Facenheim, Luis RP, Jian Jiang, Sheng Zhang, Pingyu Wu, Bo Zhou, and Weiming Zhang. 2020. DeepFaceLab: A simple, flexible and extensible face swapping framework. ArXiV (2020).Google Scholar
- Joan Petersilia. 1985. Racial disparities in the criminal justice system: A summary. Crime & Delinquency 31, 1 (1985), 15--34.Google ScholarCross Ref
- Mohamad Amin Pourhoseingholi, Ahmad Reza Baghestani, and Mohsen Vahedi. 2012. How to control confounding effects by statistical analysis. Gastroenterology and hepatology from bed to bench 5, 2 (2012), 79.Google Scholar
- KR Prajwal, Rudrabha Mukhopadhyay, Vinay P Namboodiri, and CV Jawahar. 2020. A lip sync expert is all you need for speech to lip generation in the wild. In Proceedings of the 28th ACM International Conference on Multimedia. 484--492.Google ScholarDigital Library
- Prasanna Sattigeri, Samuel C Hoffman, Vijil Chenthamarakshan, and Kush R Varshney. 2018. Fairness gan. arXiv preprint arXiv:1805.09910 (2018).Google Scholar
- Maya Sen and Omar Wasow. 2016. Race as a bundle of sticks: Designs that estimate effects of seemingly immutable characteristics. Annual Review of Political Science 19 (2016), 499--522.Google ScholarCross Ref
- Taylan Sen, Md Kamrul Hasan, Zach Teicher, and Mohammed Ehsan Hoque. 2018. Automated dyadic data recorder (ADDR) framework and analysis of facial cues in deceptive communication. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 1--22.Google ScholarDigital Library
- Sandra Susan Smith. 2010. Race and trust. Annual Review of Sociology 36 (2010), 453--475.Google ScholarCross Ref
- Cheryl Staats. 2014. Implicit racial bias and school discipline disparities. Exploring the connection (2014).Google Scholar
- Tyler J VanderWeele and Whitney R Robinson. 2014. On causal interpretation of race in regressions adjusting for confounding and mediating variables. Epidemiology (Cambridge, Mass.) 25, 4 (2014), 473.Google ScholarCross Ref
- Konstantinos Vougioukas, Stavros Petridis, and Maja Pantic. 2020. Realistic speech-driven facial animation with gans. International Journal of Computer Vision 128, 5 (2020), 1398--1413.Google ScholarDigital Library
- Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Pro- ceedings of the IEEE international conference on computer vision. 2223--2232.Google Scholar
Index Terms
- Demographic Feature Isolation for Bias Research using Deepfakes
Recommendations
Questioning Racial and Gender Bias in AI-based Recommendations: Do Espoused National Cultural Values Matter?
AbstractOne realm of AI, recommender systems have attracted significant research attention due to concerns about its devastating effects to society’s most vulnerable and marginalised communities. Both media press and academic literature provide compelling ...
Using Intersectional Representation & Embodied Identification in Standard Video Game Play to Reduce Societal Biases
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsWhile virtual character embodiment has been studied as a mitigator of singular societal biases in fully immersive VR and empathy games, there have been no major studies on representation featuring standard game play or intersectional identities. In our ...
Surfacing Racial Stereotypes through Identity Portrayal
FAccT '22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and TransparencyContent warning: this paper discusses and contains content that may be offensive or upsetting.
People express racial stereotypes through conversations with others, increasingly in a digital format; as a result, the ability to computationally identify ...
Comments