research-article

Designing Fairly Fair Classifiers Via Economic Fairness Notions

Authors:

Safwan Hossain,

Andjela Mladenovic,

Nisarg ShahAuthors Info & Claims

WWW '20: Proceedings of The Web Conference 2020

Pages 1559 - 1569

https://doi.org/10.1145/3366423.3380228

Published: 20 April 2020 Publication History

Abstract

The past decade has witnessed a rapid growth of research on fairness in machine learning. In contrast, fairness has been formally studied for almost a century in microeconomics in the context of resource allocation, during which many general-purpose notions of fairness have been proposed. This paper explore the applicability of two such notions — envy-freeness and equitability — in machine learning. We propose novel relaxations of these fairness notions which apply to groups rather than individuals, and are compelling in a broad range of settings. Our approach provides a unifying framework by incorporating several recently proposed fairness definitions as special cases. We provide generalization bounds for our approach, and theoretically and experimentally evaluate the tradeoff between loss minimization and our fairness guarantees.

References

[1]

A. Agarwal, A. Beygelzimer, M. Dudik, J. Langford, and H. Wallach. 2018. A Reductions Approach to Fair Classification. In Proceedings of the 35th International Conference on Machine Learning (ICML). 60–69.

[2]

M. Aleksandrov and T. Walsh. 2018. Group envy freeness and group pareto efficiency in fair division with indivisible items. In In Proceedings of the 41st German Conference on Artificial Intelligence (KI). 57–72.

[3]

M. Altman, A. Wood, and E. Vayena. 2018. A harm-reduction framework for algorithmic fairness. IEEE Security & Privacy 16, 3 (2018), 34–45.

[4]

H. Aziz. 2019. Justifications of Welfare Guarantees under Normalized Utilities. arXiv preprint arXiv:1905.10774.

[5]

M. F. Balcan, F. Constantin, S. Iwata, and L. Wang. 2012. Learning valuation functions. In Proceedings of the 25th Conference on Computational Learning Theory (COLT). 4.1–4.24.

[6]

M. F. Balcan, T. Dick, R. Noothigattu, and A. D. Procaccia. 2019. Envy-Free Classification. In Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS). 1238–1248.

[7]

S. Barocas and A. D. Selbst. 2016. Big data’s disparate impact. Calif. L. Rev. 104(2016), 671.

[8]

J. Bartholdi, C. A. Tovey, and M. A. Trick. 1992. How hard Is It to Control an Election. Mathematical and Computer Modelling 16 (1992), 27–40.

Digital Library

[9]

O. Ben-Porat, F. Sandomirskiy, and M. Tennenholtz. 2019. Protecting the Protected Group: Circumventing Harmful Fairness. arXiv:1905.10546.

[10]

A. Blum, J. Jackson, T. Sandholm, and M. Zinkevich. 2004. Preference elicitation and query learning. Journal of Machine Learning Research 5, Jun (2004), 649–667.

[11]

C. Boutilier, K. Regan, and P. Viappiani. 2010. Simultaneous elicitation of preference features and utility. In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI). 1160–1167.

[12]

E. Budish. 2011. The combinatorial assignment problem: Approximate competitive equilibrium from equal incomes. Journal of Political Economy 119, 6 (2011), 1061–1103.

[13]

T. Calders, F. Kamiran, and M. Pechenizkiy. 2009. Building classifiers with independency constraints. In Proceedings of the IEEE International Conference on Data Mining Workshops. 13–18.

[14]

I. Caragiannis, D. Kurokawa, H. Moulin, A. D. Procaccia, N. Shah, and J. Wang. 2019. The unreasonable fairness of maximum Nash welfare. ACM Transactions on Economics and Computation (TEAC) 7, 3 (2019), 12.

Digital Library

[15]

U. Chajewska, D. Koller, and D. Ormoneit. 2001. Learning an agent’s utility function by observing behavior. In Proceedings of the 18th International Conference on Machine Learning (ICML). 35–42.

[16]

X. Chen, B. Fain, L. Lyu, and K. Munagala. 2019. Proportionally Fair Clustering. In Proceedings of the 36th International Conference on Machine Learning (ICML). 1032–1041.

[17]

A. Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2 (2017), 153–163.

[18]

V. Conitzer, R. Freeman, N. Shah, and J. W. Vaughan. 2019. Group fairness for the allocation of indivisible goods. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI). 1853–1860.

[19]

S. Diamond and S. Boyd. 2016. CVXPY: A Python-Embedded Modeling Language for Convex Optimization. Journal of Machine Learning Research 17, 83 (2016), 1–5.

Digital Library

[20]

C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (ITCS). 214–226.

[21]

B. Fain, A. Goel, and K. Munagala. 2016. The core of the participatory budgeting problem. In Proceedings of the 12th Conference on Web and Internet Economics (WINE). 384–399.

[22]

D. Foley. 1967. Resource allocation and the public sector. Yale Economics Essays 7(1967), 45–98.

[23]

D. Gale and L. S. Shapley. 1962. College Admissions and the Stability of Marriage. Americal Mathematical Monthly 69, 1 (1962), 9–15.

[24]

P. Gölz, A. Kahng, and A. D. Procaccia. 2019. Paradoxes in Fair Machine Learning. In Proceedings of the 32nd Conference on Neural Information Processing Systems (NeurIPS). 8340–8350.

[25]

M. Hardt, E. Price, and N. Srebro. 2016. Equality of opportunity in supervised learning. In Proceedings of the 30th Conference on Neural Information Processing Systems (NeurIPS). 3315–3323.

[26]

Ú. Hébert-Johnson, M. Kim, O. Reingold, and G. Rothblum. 2018. Multicalibration: Calibration for the (computationally-identifiable) masses. In Proceedings of the 35th International Conference on Machine Learning (ICML). 1944–1953.

[27]

H. Heidari, M. Loi, K. P. Gummadi, and A. Krause. 2019. A Moral Framework for Understanding Fair ML through Economic Models of Equality of Opportunity. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*). 181–190.

[28]

K. Holstein, J. W. Vaughan, H. Daumé III, M. Dudik, and H. Wallach. 2019. Improving fairness in machine learning systems: What do industry practitioners need?. In Proceedings of the Conference on Human Factors in Computing Systems (CHI). 600.

[29]

M. Kearns, S. Neel, A. Roth, and Z. S. Wu. 2018. Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness. In Proceedings of the 35th International Conference on Machine Learning (ICML). 2569–2577.

[30]

J. M. Kleinberg, S. Mullainathan, and M. Raghavan. 2017. Inherent Trade-Offs in the Fair Determination of Risk Scores. In Proceedings of the 8th Innovations in Theoretical Computer Science Conference (ITCS). 43:1–43:23.

[31]

M. J. Kusner, J. Loftus, C. Russell, and R. Silva. 2017. Counterfactual fairness. In Proceedings of the 31st Conference on Neural Information Processing Systems (NeurIPS). 4066–4076.

[32]

N. Lambert and Y. Shoham. 2009. Eliciting truthful answers to multiple-choice questions. In Proceedings of the 10th ACM Conference on Economics and Computation (EC). 109–118.

[33]

R. J. Lipton, E. Markakis, E. Mossel, and A. Saberi. 2004. On approximately fair allocations of indivisible goods. In Proceedings of the 6th ACM Conference on Economics and Computation (EC). 125–131.

[34]

L. T. Liu, S. Dean, E. Rolf, M. Simchowitz, and M. Hardt. 2018. Delayed impact of fair machine learning. arXiv:1803.04383.

[35]

A. K. Menon and R. C. Williamson. 2018. The cost of fairness in binary classification. In Proceedings of the Conference on Fairness, Accountability and Transparency (FAT). 107–118.

[36]

H. Moulin. 2004. Fair Division and Collective Welfare. MIT Press.

[37]

S. Mullainathan. 2018. Algorithmic Fairness and the Social Welfare Function. In Proceedings of the 19th ACM Conference on Economics and Computation (EC). 1–1.

Digital Library

[38]

A. Narayanan. 2018. Translation tutorial: 21 fairness definitions and their politics. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT).

[39]

G. Ramnarayan. 2018. Equalizing Financial Impact in Supervised Learning. arXiv:1806.09211.

[40]

J. Rawls. 1971. A Theory of Justice. Harvard University Press.

[41]

S. Shalev-Shwartz and S. Ben-David. 2014. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press.

[42]

L. Shapley and H. Scarf. 1974. On cores and indivisibility. Journal of Mathematical Economics 1, 1 (1974), 23–37.

[43]

H. Steinhaus. 1948. The problem of fair division. Econometrica 16(1948), 101–104.

[44]

W. Stromquist. 1980. How to cut a cake fairly. Amer. Math. Monthly 87, 8 (1980), 640–644.

[45]

F. E. Su. 1999. Rental harmony: Sperner’s lemma in fair division. Amer. Math. Monthly 106, 10 (1999), 930–942.

[46]

B. Ustun, Y. Liu, and D. C. Parkes. 2019. Fairness without harm: Decoupled classifiers with preference guarantees. In Proceedings of the 36th International Conference on Machine Learning (ICML). 6373–6382.

[47]

H. Varian. 1974. Equity, envy and efficiency. Journal of Economic Theory 9 (1974), 63–91.

[48]

B. Woodworth, S. Gunasekar, M. I. Ohannessian, and N. Srebro. 2017. Learning Non-Discriminatory Predictors. In Proceedings of the 30th Conference on Computational Learning Theory (COLT). 1920–1953.

[49]

M. B. Zafar, I. Valera, M. Rodriguez, K. Gummadi, and A. Weller. 2017. From parity to preference-based notions of fairness in classification. In Proceedings of the 31st Conference on Neural Information Processing Systems (NeurIPS). 229–239.

Cited By

Wei WMa XWang JOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Fair adaptive experimentsProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666962(19157-19169)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666962
Shah NElkind E(2023)Pushing the limits of fairness in algorithmic decision-makingProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/806(7051-7056)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/806
Mashiat TGitiaux XRangwala HFowler PDas S(2022)Trade-offs between Group Fairness Metrics in Societal Resource AllocationProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency10.1145/3531146.3533171(1095-1105)Online publication date: 21-Jun-2022
https://dl.acm.org/doi/10.1145/3531146.3533171
Show More Cited By

Index Terms

Designing Fairly Fair Classifiers Via Economic Fairness Notions
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees
2. Theory of computation

Index terms have been assigned to the content through auto-classification.

Recommendations

Airtime Fairness for IEEE 802.11 Multirate Networks

Under a multi rate network scenario, the IEEE 802.11 DCF MAC fails to provide air-time fairness for all competing stations since the protocol is designed for ensuring max-min throughput fairness and the maximum achievable throughput by any station gets ...
Fairness in multi-hop wireless backhaul networks: a dynamic estimation approach
QShine '08: Proceedings of the 5th International ICST Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness

In this work, we consider the problem of fairness for Transit Access Points (TAP) in multi-hop wireless backhaul networks. Existing approaches are not practical due to the requirement for modifications to the MAC layer or queueing operations of TAPs, or ...
Fair Allocation through Competitive Equilibrium from Generic Incomes
FAT* '19: Proceedings of the Conference on Fairness, Accountability, and Transparency

Two food banks catering to populations of different sizes with different needs must divide among themselves a donation of food items. What constitutes a "fair" allocation of the items among them?

Competitive equilibrium from equal incomes (CEEI) is a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '20: Proceedings of The Web Conference 2020

April 2020

3143 pages

ISBN:9781450370233

DOI:10.1145/3366423

Editors:
Yennun Huang
Acadmica sinica, Taiwan
,
Irwin King
The Chinese University of Hong Kong, Hong Kong
,
Tie-Yan Liu
Microsoft Research Asia, China
,
Maarten van Steen
University of Twente, Netherlands

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '20

Sponsor:

SIGWEB

WWW '20: The Web Conference 2020

April 20 - 24, 2020

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
394
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)4

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wei WMa XWang JOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Fair adaptive experimentsProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666962(19157-19169)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666962
Shah NElkind E(2023)Pushing the limits of fairness in algorithmic decision-makingProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/806(7051-7056)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/806
Mashiat TGitiaux XRangwala HFowler PDas S(2022)Trade-offs between Group Fairness Metrics in Societal Resource AllocationProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency10.1145/3531146.3533171(1095-1105)Online publication date: 21-Jun-2022
https://dl.acm.org/doi/10.1145/3531146.3533171
Chien IDeliu NTurner RWeller AVillar SKilbertus N(2022)Multi-disciplinary fairness considerations in machine learning for clinical trialsProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency10.1145/3531146.3533154(906-924)Online publication date: 21-Jun-2022
https://dl.acm.org/doi/10.1145/3531146.3533154
Simonetto ANotarnicola IConitzer VTasioulas JScheutz MCalo RMara MZimmermann A(2022)Achievement and Fragility of Long-term EquitabilityProceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3514094.3534132(675-685)Online publication date: 26-Jul-2022
https://dl.acm.org/doi/10.1145/3514094.3534132
He SShin K(2022)Socially-Equitable Interactive Graph Information Fusion-based Prediction for Urban Dockless E-Scooter SharingProceedings of the ACM Web Conference 202210.1145/3485447.3512145(3269-3279)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3485447.3512145
Rouzot JFerry JHuguet M(2022)Learning Optimal Fair Scoring Systems for Multi-Class Classification2022 IEEE 34th International Conference on Tools with Artificial Intelligence (ICTAI)10.1109/ICTAI56018.2022.00036(197-204)Online publication date: Oct-2022
https://doi.org/10.1109/ICTAI56018.2022.00036
Do VCorbett-Davies SAtif JUsunier NRanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Two-sided fairness in rankings via Lorenz dominanceProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3540919(8596-8608)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3540919
Biswas AMukherjee SFourcade MKuipers BLazar SMulligan D(2021)Ensuring Fairness under Prior Probability ShiftsProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3461702.3462596(414-424)Online publication date: 21-Jul-2021
https://dl.acm.org/doi/10.1145/3461702.3462596
Shah KGupta PDeshpande ABhattacharyya CFourcade MKuipers BLazar SMulligan D(2021)Rawlsian Fair Adaptation of Deep Learning ClassifiersProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3461702.3462592(936-945)Online publication date: 21-Jul-2021
https://dl.acm.org/doi/10.1145/3461702.3462592
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten