research-article

Mitigating Cognitive Biases in Machine Learning Algorithms for Decision Making

Author:

Christopher G. HarrisAuthors Info & Claims

WWW '20: Companion Proceedings of the Web Conference 2020

Pages 775 - 781

https://doi.org/10.1145/3366424.3383562

Published: 20 April 2020 Publication History

Abstract

Cognitive biases are an ingrained part of the human decision-making process. Nearly all machine learning algorithms that mimic human decision-making use human judgments as training data, which propagates these biases. In this paper, we conduct an empirical study in which 150 applicants are rated for suitability for three separate job openings. We develop an algorithm that learns from human judgments and consequently develops biases based on these human-generated inputs. Next, we explore and apply techniques to mitigate these algorithmic biases, using a combination of pre-processing, in-processing, and post-processing algorithms. The results from our study show that biases can be mitigated using these approaches but involve a tradeoff between complexity and effectiveness.

References

[1]

Tversky, A., & Kahneman, D. (1986). Judgment under uncertainty: Heuristics and biases. Judgment and decision making: An interdisciplinary reader, 38-55.

[2]

Kahneman, D., Slovic, S. P., Slovic, P., & Tversky, A. (Eds.). (1982). Judgment under uncertainty: Heuristics and biases. Cambridge university press.

[3]

Wilson, T. D., Houston, C. E., Etling, K. M., & Brekke, N. (1996). A new look at anchoring effects: basic anchoring and its antecedents. Journal of Experimental Psychology: General, 125(4), 387.

[4]

Friedman, B., & Nissenbaum, H. (1996). Bias in computer systems. ACM Transactions on Information Systems (TOIS), 14(3), 330-347.

Digital Library

[5]

Pedreshi, D., Ruggieri, S., & Turini, F. (2008). Discrimination-aware data mining. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 560-568). ACM.

Digital Library

[6]

Bantilan, N. Themis-ml: A fairness-aware machine learning interface for end-to-end discrimination discovery and mitigation. Journal of Technology in Human Services, 36(1):15–30, 2018. URL https://github.com/cosmicBboy/themis-ml.

[7]

Narayanan, A. (2018, February). Translation tutorial: 21 fairness definitions and their politics. In Proc. Conf. Fairness Accountability Transp., New York, USA.

[8]

Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807.

[9]

Berk, R., Heidari, H., Jabbari, S., Joseph, M., Kearns, M., Morgenstern, J., ... & Roth, A. (2017). A convex framework for fair regression. arXiv preprint arXiv:1706.02409.

[10]

Fish, B., Kun, J., & Lelkes, Á. D. (2016, June). A confidence-based approach for balancing fairness and accuracy. In Proceedings of the 2016 SIAM International Conference on Data Mining (pp. 144-152). Society for Industrial and Applied Mathematics.

[11]

Hajian, S., Bonchi, F., & Castillo, C. (2016, August). Algorithmic bias: From discrimination discovery to fairness-aware data mining. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 2125-2126). ACM.

Digital Library

[12]

Bellamy, R. K., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., ... & Nagar, S. (2018). AI fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv preprint arXiv:1810.01943.

[13]

Zemel, R., Wu, Y., Swersky, K., Pitassi, T., & Dwork, C. (2013, February). Learning fair representations. In International Conference on Machine Learning (pp. 325-333).

[14]

Calmon, F., Wei, D., Vinzamuri, B., Ramamurthy, K. N., & Varshney, K. R. (2017). Optimized pre-processing for discrimination prevention. In Advances in Neural Information Processing Systems (pp. 3992-4001).

[15]

Kamiran, F., and Calders, T. Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1):1–33, 2012. org/10.1007/s10115-011-0463-8.

[16]

Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015, August). Certifying and removing disparate impact. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 259-268). ACM.

Digital Library

[17]

The U.S. EEOC. Uniform guidelines on employee selection procedures, March 2, 1979

[18]

Zhang, B. H., Lemoine, B., & Mitchell, M. (2018, December). Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (pp. 335-340). ACM.

Digital Library

[19]

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems (pp. 2672-2680).

[20]

Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2012, September). Fairness-aware classifier with prejudice remover regularizer. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (pp. 35-50). Springer, Berlin, Heidelberg.

[21]

Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. In Advances in neural information processing systems (pp. 3315-3323).

[22]

Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., & Weinberger, K. Q. (2017). On fairness and calibration. In Advances in Neural Information Processing Systems (pp. 5680-5689).

[23]

Kamiran, F., Karim, A., & Zhang, X. (2012, December). Decision theory for discrimination-aware classification. In 2012 IEEE 12th International Conference on Data Mining (pp. 924-929). IEEE.

Digital Library

[24]

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159-174.

[25]

Ma, D. S., Correll, J., & Wittenbrink, B. (2015). The Chicago face database: A free stimulus set of faces and norming data. Behavior research methods, 47(4), 1122-1135.

[26]

Pennebaker, J. W., Booth, R. J., & Francis, M. E. (2015). Linguistic inquiry and word count: LIWC [Computer software]. Austin, TX: liwc. net, 135.

[27]

Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16, 321-357.

[28]

Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: a module in human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17(11), 4302-4311.

[29]

Sergent, J., Ohta, S., & MACDONALD, B. (1992). Functional neuroanatomy of face and object processing: a positron emission tomography study. Brain, 115(1), 15-36.

[30]

Roscigno, V. J., Mong, S., Byron, R., & Tester, G. (2007). Age discrimination, social closure, and employment. Social Forces, 86(1), 313-334.

[31]

Rosen, B., & Jerdee, T. H. (1976). The influence of age stereotypes on managerial decisions. Journal of applied psychology, 61(4), 428.

[32]

Van't Wout, M., & Sanfey, A. G. (2008). Friend or foe: The effect of implicit trustworthiness judgments in social decision-making. Cognition, 108(3), 796-803.

[33]

Card, D., Zhang, M., & Smith, N. A. (2019, January). Deep Weighted Averaging Classifiers. In Proceedings of the Conference on Fairness, Accountability, and Transparency (pp. 369-378). ACM.

Digital Library

[34]

Harris, C.G. (2020), Methods to Evaluate Temporal Cognitive Biases in Machine Learning Prediction Models. In Companion Proceedings of the Web Conference 2020 (WWW '20 Companion), Taipei, Taiwan. ACM, New York, NY, USA. https://doi.org/10.1145/3366424.3383418

Digital Library

Cited By

Kekez ILauwaert LBegičević Ređep N(2025)Is Artificial Intelligence (AI) Research Biased and Conceptually Vague? A Systematic Review of Research on Bias and Discrimination in the context of using AI in Human Resource ManagementTechnology in Society10.1016/j.techsoc.2025.102818(102818)Online publication date: Jan-2025
https://doi.org/10.1016/j.techsoc.2025.102818
Yang MArai HYamashita NBaba Y(2024)Fair Machine Guidance to Enhance Fair Decision Making in Biased PeopleProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642627(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642627
Harris C(2024)Combining Human-in-the-Loop Systems and AI Fairness Toolkits to Reduce Age Bias in AI Job Hiring Algorithms2024 IEEE International Conference on Big Data and Smart Computing (BigComp)10.1109/BigComp60711.2024.00019(60-66)Online publication date: 18-Feb-2024
https://doi.org/10.1109/BigComp60711.2024.00019
Show More Cited By

Index Terms

Mitigating Cognitive Biases in Machine Learning Algorithms for Decision Making

Index terms have been assigned to the content through auto-classification.

Recommendations

How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
AIES '22: Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society

The field of eXplainable Artificial Intelligence (XAI) aims to bring transparency to complex AI systems. Although it is usually considered an essentially technical field, effort has been made recently to better understand users' human explanation ...
Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
CSCW1

Several strands of research have aimed to bridge the gap between artificial intelligence (AI) and human decision-makers in AI-assisted decision-making, where humans are the consumers of AI model predictions and the ultimate decision-makers in high-...
Designing Information for Remediating Cognitive Biases in Decision-Making
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Software is playing an increasingly important role in supporting human decision-making. Previous HCI research on decision support systems (DSS) has improved the information visualization aspect of DSS information design, but has somewhat overlooked the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '20: Companion Proceedings of the Web Conference 2020

April 2020

854 pages

ISBN:9781450370240

DOI:10.1145/3366424

Editors:
Amal El Fallah Seghrouchni
Sorbonne University, France
,
Gita Sukthankar
University of Central Florida, United States
,
Tie-Yan Liu
Microsoft Research Asia, China
,
Maarten van Steen
University of Twente, Netherlands

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '20

Sponsor:

SIGWEB

WWW '20: The Web Conference 2020

April 20 - 24, 2020

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
774
Total Downloads

Downloads (Last 12 months)134
Downloads (Last 6 weeks)15

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kekez ILauwaert LBegičević Ređep N(2025)Is Artificial Intelligence (AI) Research Biased and Conceptually Vague? A Systematic Review of Research on Bias and Discrimination in the context of using AI in Human Resource ManagementTechnology in Society10.1016/j.techsoc.2025.102818(102818)Online publication date: Jan-2025
https://doi.org/10.1016/j.techsoc.2025.102818
Yang MArai HYamashita NBaba Y(2024)Fair Machine Guidance to Enhance Fair Decision Making in Biased PeopleProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642627(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642627
Harris C(2024)Combining Human-in-the-Loop Systems and AI Fairness Toolkits to Reduce Age Bias in AI Job Hiring Algorithms2024 IEEE International Conference on Big Data and Smart Computing (BigComp)10.1109/BigComp60711.2024.00019(60-66)Online publication date: 18-Feb-2024
https://doi.org/10.1109/BigComp60711.2024.00019
González-Sendino RSerrano EBajo J(2024)Mitigating bias in artificial intelligenceFuture Generation Computer Systems10.1016/j.future.2024.02.023155:C(384-401)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.future.2024.02.023
Rabonato RBerton L(2024)A systematic review of fairness in machine learningAI and Ethics10.1007/s43681-024-00577-5Online publication date: 19-Sep-2024
https://doi.org/10.1007/s43681-024-00577-5
Harris C(2022)Age Bias: A Tremendous Challenge for Algorithms in the Job Candidate Screening Process2022 IEEE International Symposium on Technology and Society (ISTAS)10.1109/ISTAS55053.2022.10227135(1-5)Online publication date: 10-Nov-2022
https://doi.org/10.1109/ISTAS55053.2022.10227135
Schönau A(2022)Agency in augmented reality: exploring the ethics of Facebook’s AI-powered predictive recommendation systemAI and Ethics10.1007/s43681-022-00158-43:2(407-417)Online publication date: 25-Apr-2022
https://doi.org/10.1007/s43681-022-00158-4

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten