skip to main content
research-article

Gendering algorithms in social media

Published: 29 May 2021 Publication History

Abstract

Social media platforms employ inferential analytics methods to guess user preferences and may include sensitive attributes such as race, gender, sexual orientation, and political opinions. These methods are often opaque, but they can have significant effects such as predicting behaviors for marketing purposes, influencing behavior for profit, serving attention economics, and reinforcing existing biases such as gender stereotyping. Although two international human rights treaties include express obligations relating to harmful and wrongful stereotyping, these stereotypes persist both online and offline, and platforms often appear to fail to understand that gender is not merely a binary of being a 'man' or a 'woman,' but is socially constructed. Our study investigates the impact of algorithmic bias on inadvertent privacy violations and the reinforcement of social prejudices of gender and sexuality through a multidisciplinary perspective including legal, computer science, and queer media viewpoints. We conducted an online survey to understand whether and how Twitter inferred the gender of users. Beyond Twitter's binary understanding of gender and the inevitability of the gender inference as part of Twitter's personalization trade-off, the results show that Twitter misgendered users in nearly 20% of the cases (N=109). Although not apparently correlated, only 8% of the straight male respondents were misgendered, compared to 25% of gay men and 16% of straight women. Our contribution shows how the lack of attention to gender in gender classifiers exacerbates existing biases and affects marginalized communities. With our paper, we hope to promote the online account for privacy, diversity, and inclusion and advocate for the freedom of identity that everyone should have online and offline.

References

[1]
Convention on the elimination of all forms of discrimination against women. https://www.ohchr.org/en/professionalinterest/pages/cedaw. aspx#: :text=On%2018%20December%201979%2C%20th e,twentieth%20country%20had%20ratified%20it, 1979.
[2]
Convention on the rights of persons with disabilities. https://www.un.org/development/desa/disabilities/conventio n-on-the-rights-of-persons-with-disabilities/article-8- awareness-raising.html, 2008.
[3]
Article 29 Data Protection Working Party. Opinion 06/2014 on the notion of legitimate interests of the data controller under Article 7 of Directive 95/46/EC, https://ec.europa.eu/ justice/article-29/documentation/opinionrecommendation/ files/2014/wp217_en.pdf, 2014.
[4]
Ashley, F. No, pronouns won't send you to jail: The misunderstood scope of Bill C-16. Medium, https://medium.com/@florence.ashley/no-pronouns-wontsend- you-to-jail-43c268cffd55, 2017.
[5]
Bray, F. Gender and technology. Annual Review of Anthropology, 36(1): 37--53, 2007.
[6]
Brown, G. Thinking beyond homonormativity: Performative explorations of diverse gay economies. Environment and Planning A: Economy and Space, 41(6): 1496--1510, 2009.
[7]
Buolamwini, J. and Gebru, T. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the First Conference on Fairness, Accountability and Transparency, 77--91. PMLR, 2018.
[8]
Butler, J. Gender trouble, feminist theory, and psychoanalytic discourse. Routledge, 1990.
[9]
Calders, T. and Custers, B. What Is data mining and how does it work? In: Discrimination and privacy in the information society: Data mining and profiling in large databases (eds. Custers et al.), pages 27--42. Springer, 2013.
[10]
Caliskan, A., Bryson, J. J. and Narayanan, A. Semantics derived automatically from language corpora contain humanlike biases. Science, 356(6334): 183--186, 2017.
[11]
Campa, S., Davis, M. and Gonzalez, D. Deep & machine learning approaches to analyzing gender representations in journalism.https://web.stanford.edu/class/archive/cs/cs224n/ cs224n.1194/reports/custom/15787612.pdf, 2019.
[12]
Corney, M., Vel, O. d., Anderson, A. and Mohay, G. Genderpreferential text mining of e-mail discourse. In Proceedings of the 18th Annual Computer Security Applications Conference, 2002, pages 282--289, 2002.
[13]
Custers, B. Data dilemmas in the information society: Introduction and overview. In B. Custers, T. Calders, B. Schermer and T. Zarsky, editors, Discrimination and privacy in the information society, pages 3--26, Springer, 2013.
[14]
Custers, B. Profiling as inferred data: Amplifier effects and positive feedback loops. Amsterdam University Press, http://www.jstor.org/stable/j.ctvhrd092.23, 2018.
[15]
Davenport, T. D. and Beck, J. C. The attention economy: Understanding the new currency of business. Harvard Business School Press, 2002.
[16]
Fergus, J. Twitter is guessing users' genders to sell ads and often getting it wrong. Input, https://www.inputmag.com/ tech/twitter-guesses-your-gender-to-serve-you-ads-relevanttweets- wrong-misgendered, 2020.
[17]
Fink, C., Kopecky, J. and Morawski, M. Inferring gender from the content of tweets: A region specific example. In Proceedings of the Sixth International AAAI Conference on Weblogs and Social Medi, pages 459--452, AAAI, 2012.
[18]
Font, J. E. and Costa-jussà, M. R. Equalizing gender bias in neural machine translation with word embeddings techniques. In Proceedings of the 1st Workshop on Gender Bias in Natural Language Processing, pages 147--154. Association for Computational Linguistics, 2019. SIGKDD Explorations Volume 23, Issue 1 29
[19]
Garibo i Orts, O. A big data approach to gender classification in Twitter. In Proceedings of the Ninth International Conference of the CLEF Association, 2018.
[20]
GLAAD GLAAD media reference guide -- transgender. https://www.glaad.org/reference/transgender, n.d.
[21]
Gomes, A., Antonialli, D. and Olivia, T. D. Drag queens and Artificial Intelligence: Should computers decide what is ?toxic' on the internet?, https://www.internetlab.org.br/en/ freedom-of-expression/drag-queens-and-artificialintelligence- should-computers-decide-what-is-toxic-on-theinternet/, 2019.
[22]
Grant, A., Grey, S. and van Hell, J. G. Male fashionistas and female football fans: Gender stereotypes affect neurophysiological correlates of semantic processing during speech comprehension. Journal of Neurolinguistics, 53: 100876, 2020.
[23]
Hamidi, F., Scheuerman, M. K. and Branham, S. M. Gender recognition or gender reductionism? The social implications of embedded gender recognition systems. In Proceedings of the Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Paper 8, ACM, 2018.
[24]
Hao, K. Facebook's ad-serving algorithm discriminates by gender and race. MIT Technology Review, https://www.technologyreview.com/2019/04/05/1175/faceb ook-algorithm-discriminates-ai-bias/, 2019.
[25]
Harari, Y. N. The world after coronavirus. https://www.ft.com/content/19d90308--6858--11ea-a3c9- 1fe6fedcca75, 2020.
[26]
Hentschel, T., Heilman, M. E. and Peus, C. V. The multiple dimensions of gender stereotypes: A current look at men's and women's characterizations of others and themselves. Frontiers in Psychology, 10(11), 2019.
[27]
Information Commissioner's Office Legitimate interests. https://ico.org.uk/for-organisations/guide-to-dataprotection/ guide-to-the-general-data-protection-regulationgdpr/ lawful-basis-for-processing/legitimate-interests/, n.d.
[28]
Ito, J. Supposedly ?fair' algorithms can perpetuate discrimination. Wired, https://www.wired.com/story/ideasjoi- ito-insurance-algorithms/, 2019.
[29]
Jenkins, H., Ito, M. and boyd, d. Participatory culture in a networked era: A conversation on youth, learning, commerce, and politics. Polity Press, Cambridge, UK, 2016.
[30]
Jernigan, C. and Mistree, B.F.T. Gaydar: Facebook friendships expose sexual orientation. First Monday, 14(10), 2009.
[31]
Kamiran, F., Calders, T. and Pechenizkiy, M. Techniques for discrimination-free predictive models. In B. Custers, T. Calders, B. Schermer and T. Zarsky, editors, Discrimination and privacy in the information society, pages 223--239, Berlin, Springer, Berlin, Heidelberg. 2013.
[32]
Keating, A. and McLoughlin, D. Understanding the emergence of markets: A social constructionist perspective on gay economy. Consumption Markets & Culture, 8(2): 131--152, 2005.
[33]
Keyes, O. The misgendering machines: Trans/HCI Implications of automatic gender recognition. Proc. ACM Hum.-Comput. Interact., 2(CSCW): Article 88, 2018.
[34]
Khan, S. A., Ahmad, M., Nazir, M. and Riaz, N. A comparative analysis of gender classification techniques. International Journal of Bio-Science and Bio-Technology, 5(4): 223--244, 2013.
[35]
Kosinski, M., Stillwell, D. and Graepel, T. Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences, 110(15): 5802--5805, 2013.
[36]
Kumar, D., Gupta, R., Sharma, A. and Saroj, S. K. Gender classification using skin patterns. In Proceedings of the 2nd International Conference on Advanced Computing and Software Engineering (ICACSE), 2019.
[37]
Lambrecht, A. and Tucker, C. Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Management Science, 65(7): 2966--2981, 2019.
[38]
Li, B., Lian, X.-C. and Lu, B.-L. Gender classification by combining clothing, hair and facial component classifiers. Neurocomputing, 76(1): 18--27, 2012.
[39]
Lin, F., Wu, Y., Zhuang, Y., Long, X. and Xu, W. Human gender classification: A review. International Journal of Biometrics, 8(3--4): 275--300, 2016.
[40]
Lopes Filho, J. A. B., Pasti, R. and de Castro, L. N. Gender classification of Twitter data based on textual meta-attributes extraction. In Á. Rocha, A. M. Correia, H. Adeli, L. P. Reis and M. Mendonça Teixeira, editors, New advances in information systems and technologies, pages 1025--1034, Cham, Springer International Publishing, 2016.
[41]
Mathivanan, P. and Poornima, K. Biometric authentication for gender classification techniques: A review. Journal of The Institution of Engineers (India): Series B, 99(1): 79--85, 2018.
[42]
Matz, S. C., Menges, J. I., Stillwell, D. J. and Schwartz, H. A. Predicting individual-level income from Facebook profiles. PLOS ONE, 14(3): e0214369, 2019.
[43]
McDuff, D., Song, Y., Kapoor, A. and Ma, S. Characterizing bias in classifiers using generative models. In Proceedings of the 33rd Conference on Neural Information Processing Systems, 2019.
[44]
McLemore, K. A. Experiences with misgendering: Identity misclassification of transgender spectrum individuals. Self and Identity, 14(1): 51--74, 2015.
[45]
Nieuwenhuis, M. and Wilkens, J. Twitter text and image gender classification with a logistic regression n-gram model. In Proceedings of the Ninth International Conference of the CLEF Association, pages, 2018.
[46]
Noble, S. U. Algorithms of oppression: How search engines reinforce racism. NYU Press, New York, 2018.
[47]
Nosek, B. A., Banaji, M. R. and Greenwald, A. G. Harvesting implicit group attitudes and beliefs from a demonstration web site. Group Dynamics: Theory, Research, and Practice, 6(1): 101--115, 2002.
[48]
Nosek, B. A., Banaji, M. R. and Greenwald, A. G. Math = male, me = female, therefore math ? me. Journal of Personality and Social Psychology, 83(1): 44--59, 2002.
[49]
O'Neil, C. Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway Books, New York, 2016.
[50]
Park, S. and Woo, J. Gender classification using sentiment analysis and deep learning in a health web forum. Applied Sciences, 9, 2019.
[51]
Rai, P. and Khanna, P. Gender classification techniques: A review. In D.C. Wyld et al. (editors), Advances in computer science, engineering & applications, pages 51--59, Berlin, Springer, 2012.
[52]
Robinson, L., Cotten, S. R., Ono, H., Quan-Haase, A., Mesch, G., Chen, W., Schulz, J., Hale, T. M. and Stern, M. J. Digital inequalities and why they matter. Information, Communication & Society, 18(5): 569--582, 2015. 30 Volume 23, Issue 1 SIGKDD Explorations
[53]
Roosendaal, A. Facebook tracks and traces everyone: Like this! Tilburg Law School Legal Studies Research Paper Series, 2010.
[54]
Schiebinger, L. Scientific research must take gender into account. Nature, 507(7490): 9, 2014.
[55]
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W. and Wang, W. Y. Mitigating gender bias in natural language processing: Literature review. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1630--1640, 2019.
[56]
Thorson, K., Cotter, K., Medeiros, M. and Pak, C. Algorithmic inference, political interest, and exposure to news and politics on Facebook. Information, Communication & Society: 1--18, 2019.
[57]
Torralba, A. and Efros, A. A. Unbiased look at dataset bias. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) 2011, pages 1521--1528, 2011.
[58]
Um, N.-H. Seeking the holy grail through gay and lesbian consumers: An exploratory content analysis of ads with gay/lesbian-specific content. Journal of Marketing Communications, 18(2): 133--149, 2012.
[59]
UN Office of the High Commissioner Human Rights Gender stereotyping. https://www.ohchr.org/EN/Issues/Women/WRGS/Pages/Ge nderStereotypes.aspx, n.d.
[60]
University of Washington Human Resources Office Terminology. https://hr.uw.edu/ops/transgender-resources/ terminology/, n.d.
[61]
Ur, B., Leon, P. G., Cranor, L. F., Shay, R. and Wang, Y. Smart, useful, scary, creepy: perceptions of online behavioral advertising. In Proceedings of the Eighth Symposium on Usable Privacy and Security, Article 4. ACM, 2012.
[62]
Wachter, S. Affinity profiling and discrimination by association in online behavioural advertising. Berkeley Technology Law Journal, 35(2), 2020.
[63]
Wachter, S. and Mittelstadt, B. A right to reasonable inferences: Re-thinking data protection law in the age of big data and AI. Columbia Business Law Review, 2019(2): 494-- 620, 2019.
[64]
Willson, M. Algorithms (and the) everyday. Information, Communication & Society, 20(1): 137--150, 2017.
[65]
Zhao, J., Wang, T., Yatskar, M., Ordonez, V. and Chang, K. W. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979--2989, 2017.
[66]
Zhou, P., Zhao, J., Huang, K.-H. and Shi, W. Examining gender bias in languages with grammatical gender. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP), pages 5279--5287, 2019.
[67]
?liobait?, I. and Custers, B. Using sensitive personal data may be necessary for avoiding discrimination in data-driven decision models. Artificial Intelligence and Law, 24(2): 183- 201, 2016.
[68]
Zuboff, S. Big other: Surveillance capitalism and the prospects of an information civilization. Journal of Information Technology, 30(1): 75--89, 2015.
[69]
Fosch-Villaronga, E., Poulsen, A., Søraa, R. A., & Custers, B. H. M. (2021). A little bird told me your gender: Gender inferences in social media. Information Processing & Management, 58(3), 102541.

Cited By

View all
  • (2024)Exploring the Intersection Between Data-Driven Marketing and EconomicsData-Driven Marketing for Strategic Success10.4018/979-8-3693-3455-3.ch003(73-100)Online publication date: 12-Jul-2024
  • (2024)Setting the Social Media Stage, a Narrative Review: The Role of Theory and Research in Understanding Adolescent Online Sexual Health Information-SeekingSexes10.3390/sexes50400375:4(544-578)Online publication date: 12-Oct-2024
  • (2024)La prédiction de l’homosexualité à l’ère de l’intelligence artificielle : une analyse de trois controverses technoscientifiquesCanadian Journal of Communication10.3138/cjc-2023-003649:3(456-485)Online publication date: 1-Aug-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGKDD Explorations Newsletter
ACM SIGKDD Explorations Newsletter  Volume 23, Issue 1
June 2021
99 pages
ISSN:1931-0145
EISSN:1931-0153
DOI:10.1145/3468507
Issue’s Table of Contents
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 May 2021
Published in SIGKDD Volume 23, Issue 1

Check for updates

Author Tags

  1. algorithmic bias
  2. discrimination
  3. gender
  4. gender classifier
  5. gender stereotyping
  6. inference
  7. lgbtqai+
  8. privacy
  9. social media
  10. twitter

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)510
  • Downloads (Last 6 weeks)37
Reflects downloads up to 15 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Exploring the Intersection Between Data-Driven Marketing and EconomicsData-Driven Marketing for Strategic Success10.4018/979-8-3693-3455-3.ch003(73-100)Online publication date: 12-Jul-2024
  • (2024)Setting the Social Media Stage, a Narrative Review: The Role of Theory and Research in Understanding Adolescent Online Sexual Health Information-SeekingSexes10.3390/sexes50400375:4(544-578)Online publication date: 12-Oct-2024
  • (2024)La prédiction de l’homosexualité à l’ère de l’intelligence artificielle : une analyse de trois controverses technoscientifiquesCanadian Journal of Communication10.3138/cjc-2023-003649:3(456-485)Online publication date: 1-Aug-2024
  • (2024)Afropolitan Influence: Gender, Comedy, and Social Media in Global AfricaSocial Media + Society10.1177/2056305124130833010:4Online publication date: 19-Dec-2024
  • (2024)Recommendation Systems: Issues, Challenges and Regulations2024 IEEE Symposium on Computers and Communications (ISCC)10.1109/ISCC61673.2024.10733684(1-6)Online publication date: 26-Jun-2024
  • (2024)Bias of responses to white and black dance companies and their dancers on InstagramResearch in Dance Education10.1080/14647893.2024.2411208(1-13)Online publication date: 3-Oct-2024
  • (2024)Algorithmic gender bias: investigating perceptions of discrimination in automated decision-makingBehaviour & Information Technology10.1080/0144929X.2024.230648443:16(4208-4221)Online publication date: 31-Jan-2024
  • (2024)Social media influencers and adolescents’ health: A scoping review of the research fieldSocial Science & Medicine10.1016/j.socscimed.2023.116387340(116387)Online publication date: Jan-2024
  • (2024)Fathers’ use of social media for social comparison is associated with their food parenting practicesAppetite10.1016/j.appet.2024.107201194(107201)Online publication date: Mar-2024
  • (2024)Legal and Ethical Aspects of AI in RadiologyAI Implementation in Radiology10.1007/978-3-031-68942-0_6(87-106)Online publication date: 27-Nov-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media