Skip to main content
SpringerLink
Log in

Does a Compromise on Fairness Exist in Using AI Models?

  • Conference paper
  • First Online:
AI 2022: Advances in Artificial Intelligence (AI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13728))

Included in the following conference series:

  • 1524 Accesses

Abstract

Artificial Intelligence (AI) has been increasingly used to assist decision making in different domains. Multiple parties are usually affected by decisions in decision making, e.g. decision-maker and people affected by decisions. While various parties of users may have different responses to decisions regarding ethical concerns such as fairness, it is important to understand whether a compromise on fairness exists in using AI models. This paper takes AI-assisted talent shortlisting as a case study and investigates perception of fairness, trust, and satisfaction with decisions of both recruiters and applicants in AI-informed decision making. The compromises on fairness between decision-maker and people affected by decisions are identified which are then explained by social and psychological theories. The findings can be used to help find compromising points between decision-maker and people affected by decisions so that both parties can reach for a balanced state in decision making.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aggarwal, P., Larrick, R.P.: When consumers care about being treated fairly: the interaction of relationship norms and fairness norms. J. Consumer Psychol. 22(1, SI), 114–127 (2012)

    Google Scholar 

  2. Alam, L., Mueller, S.: Examining the effect of explanation on satisfaction and trust in AI diagnostic systems. BMC Med. Inform. Decision Making 21(1), 178 (2021). https://doi.org/10.1186/s12911-021-01542-6

    Article  Google Scholar 

  3. Bellamy, R.K.E., et al.: AI fairness 360: an extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv:1810.01943 [cs] (2018)

  4. Berk, R., Heidari, H., Jabbari, S., Kearns, M., Roth, A.: Fairness in criminal justice risk assessments: the state of the art. Sociological Methods & Researchm p. 0049124118782533 (2018)

    Google Scholar 

  5. Blau, P.M.: Exchange and Power in Social Life. Wiley, New York, NY (1964)

    Google Scholar 

  6. van den Bos, K.: Uncertainty management: the influence of uncertainty salience on reactions to perceived procedural fairness. J. Person. Soc. Psychol. 80(6), 931–941 (2001)

    Article  Google Scholar 

  7. Carson, P.P., Carson, K.D., Knouse, S.B., Roe, C.W.: Balance theory applied to service quality: a focus on the organization, provider, and consumer triad. J. Bus. Psychol. 12(2), 99–120 (1997). https://doi.org/10.1023/A:1025061816323,

  8. Chen, F., Zhou, J.: Humanity Driven AI: Productivity, Well-being, Sustainability and Partnership. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-72188-6

  9. Colquitt, J.A., Rodell, J.B.: Measuring justice and fairness. In: Cropanzano, R.S., Ambrose, M.L. (eds.) The Oxford Handbook of Justice in the Workplace, pp. 187–202. Oxford University Press (2015)

    Google Scholar 

  10. Corbett-Davies, S., Goel, S.: The measure and mismeasure of fairness: a critical review of fair machine learning. arXiv preprint arXiv:1808.00023 (2018)

  11. Feldman, M., Friedler, S.A., Moeller, J., Scheidegger, C., Venkatasubramanian, S.: Certifying and removing disparate impact. In: Proceedings of KDD2015, pp. 259–268 (2015)

    Google Scholar 

  12. Glymour, B., Herington, J.: Measuring the biases that matter: the ethical and casual foundations for measures of fairness in algorithms. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 269–278 (2019)

    Google Scholar 

  13. Heider, F.: The Psychology of Interpersonal Relations. Wiley (1958)

    Google Scholar 

  14. Helberger, N., Araujo, T., de Vreese, C.H.: Who is the fairest of them all? public attitudes and expectations regarding automated decision-making. Comput. Law Secur. Rev. 39, 105456 (2020)

    Article  Google Scholar 

  15. Jawahar, I.M.: The influence of perceptions of fairness on performance appraisal reactions. J. Labor Res. 28(4), 735–754 (2007)

    Article  Google Scholar 

  16. Lee, M.K., Jain, A., Cha, H.J., Ojha, S., Kusbit, D.: Procedural justice in algorithmic fairness: leveraging transparency and outcome control for fair algorithmic mediation. Proc. ACM Hum. Comput. Interact. 3, 1–26 (2019)

    Google Scholar 

  17. Lin, C.F., Fu, C.S., Chen, Y.T.: Exploring customer perceptions toward different service volumes: an integration of means-end chain and balance theories. Food Qual. Preferen. 73, 86–96 (2019)

    Article  Google Scholar 

  18. Lind, E.: Fairness heuristic theory: justice judgments as pivotal cognitions in organizational relations. In: Advances in Organizational Justice, pp. 56–88. Stanford University Press (2001)

    Google Scholar 

  19. Liu, S., Vicente, L.N.: Accuracy and fairness trade-offs in machine learning: a stochastic multi-objective approach. arXiv preprint arXiv:2008.01132 (2020)

  20. Martinez, N., Bertran, M., Sapiro, G.: Fairness with minimal harm: a pareto-optimal approach for healthcare. arXiv preprint arXiv:1911.06935 (2019)

  21. Merritt, S.M., Heimbaugh, H., LaChapell, J., Lee, D.: I trust it, but i don’t know why: effects of implicit attitudes toward automation on trust in an automated system. Hum. Fact. 55(3), 520–534 (2013)

    Article  Google Scholar 

  22. Nabi, R., Shpitser, I.: Fair inference on outcomes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 2018, p. 1931. NIH Public Access (2018)

    Google Scholar 

  23. Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., Weinberger, K.Q.: On fairness and calibration. Adv. Neural Inform. Process. Syst. 30 (2017)

    Google Scholar 

  24. Roy, S.K., Devlin, J.F., Sekhon, H.: The impact of fairness on trustworthiness and trust in banking. J. Market. Manage. 31(9–10), 996–1017 (2015)

    Article  Google Scholar 

  25. Sholihin, M.: How does procedural fairness affect performance evaluation system satisfaction? (evidence from a UK police force). Gadjah Mada Int. J. Bus. 15, 231–247 (2013). https://doi.org/10.22146/gamaijb.5445

  26. Starke, C., Baleis, J., Keller, B., Marcinkowski, F.: Fairness perceptions of algorithmic decision-making: a systematic review of the empirical literature (2021)

    Google Scholar 

  27. Wang, Y., Wang, X., Beutel, A., Prost, F., Chen, J., Chi, E.H.: Understanding and improving fairness-accuracy trade-offs in multi-task learning. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1748–1757 (2021)

    Google Scholar 

  28. Yang, K., Stoyanovich, J.: Measuring fairness in ranked outputs. SSDBM 2017 (2017). https://doi.org/10.1145/3085504.3085526

  29. Yu, K., Berkovsky, S., Taib, R., Zhou, J., Chen, F.: Do i trust my machine teammate? an investigation from perception to decision. In: Proceedings of the 24th International Conference on Intelligent User Interfaces, pp. 460–468. IUI 2019, ACM (2019)

    Google Scholar 

  30. Zhang, Y., Liao, Q.V., Bellamy, R.K.E.: Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 295–305. FAT* 2020 (2020)

    Google Scholar 

  31. Zhao, H., Gordon, G.: Inherent tradeoffs in learning fair representations. Adv. Neural Inform. Process. Syst. 32 (2019)

    Google Scholar 

  32. Zhao, J., Wang, T., Yatskar, M., Ordonez, V., Chang, K.W.: Men also like shopping: reducing gender bias amplification using corpus-level constraints. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2979–2989. Copenhagen, Denmark (2017)

    Google Scholar 

  33. Zhou, J., Bridon, C., Chen, F., Khawaji, A., Wang, Y.: Be informed and be involved: effects of uncertainty and correlation on user’s confidence in decision making. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, pp. 923–928 (2015)

    Google Scholar 

  34. Zhou, J., Chen, F. (eds.): Human and Machine Learning: Visible, Explainable, Trustworthy and Transparent. Springer, Cham (2018)

    Google Scholar 

  35. Zhou, J., Hu, H., Li, Z., Yu, K., Chen, F.: Physiological indicators for user trust in machine learning with influence enhanced fact-checking. In: Machine Learning and Knowledge Extraction, pp. 94–113 (2019)

    Google Scholar 

  36. Zhou, J., et al.: Measurable decision making with GSR and pupillary analysis for intelligent user interface. ACM Trans. Comput. Hum. Interact. 21(6), 1–23 (2015)

    Google Scholar 

  37. Zhou, J., Verma, S., Mittal, M., Chen, F.: Understanding relations between perception of fairness and trust in algorithmic decision making. In: Proceedings of the International Conference on Behavioral and Social Computing (BESC 2021), pp. 1–5 (2021)

    Google Scholar 

  38. Zliobaite, I.: On the relation between accuracy and fairness in binary classification. arXiv preprint arXiv:1505.05723 (2015)

Download references

Author information

Authors and Affiliations

  1. Data Science Institute, University of Technology Sydney, Sydney, Australia

    Jianlong Zhou, Zhidong Li & Fang Chen

  2. Research Office, University of Technology Sydney, Sydney, Australia

    Chun Xiao

Authors
  1. Jianlong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhidong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chun Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianlong Zhou .

Editor information

Editors and Affiliations

  1. University of New South Wales, Sydney, NSW, Australia

    Haris Aziz

  2. University of Western Australia, Perth, WA, Australia

    Débora Corrêa

  3. University of Western Australia, Perth, WA, Australia

    Tim French

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhou, J., Li, Z., Xiao, C., Chen, F. (2022). Does a Compromise on Fairness Exist in Using AI Models?. In: Aziz, H., Corrêa, D., French, T. (eds) AI 2022: Advances in Artificial Intelligence. AI 2022. Lecture Notes in Computer Science(), vol 13728. Springer, Cham. https://doi.org/10.1007/978-3-031-22695-3_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-22695-3_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-22694-6

  • Online ISBN: 978-3-031-22695-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation