Skip to main content

LimeOut: An Ensemble Approach to Improve Process Fairness

  • Conference paper
  • First Online:
ECML PKDD 2020 Workshops (ECML PKDD 2020)

Abstract

Artificial Intelligence and Machine Learning are becoming increasingly present in several aspects of human life, especially, those dealing with decision making. Many of these algorithmic decisions are taken without human supervision and through decision making processes that are not transparent. This raises concerns regarding the potential bias of these processes towards certain groups of society, which may entail unfair results and, possibly, violations of human rights. Dealing with such biased models is one of the major concerns to maintain the public trust.

In this paper, we address the question of process or procedural fairness. More precisely, we consider the problem of making classifiers fairer by reducing their dependence on sensitive features while increasing (or, at least, maintaining) their accuracy. To achieve both, we draw inspiration from “dropout” techniques in neural based approaches, and propose a framework that relies on “feature drop-out” to tackle process fairness. We make use of “LIME Explanations” to assess a classifier’s fairness and to determine the sensitive features to remove. This produces a pool of classifiers (through feature dropout) whose ensemble is shown empirically to be less dependent on sensitive features, and with improved or no impact on accuracy.

This research was partially supported by TAILOR, a project funded by EU Horizon 2020 research and innovation programme under GA No. 952215, and the Inria Project Lab “Hybrid Approaches for Interpretable AI” (HyAIAI).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.zdnet.com/article/gdpr-an-executive-guide-to-what-you-need-to-know/.

  2. 2.

    General Data Protection Regulation (GDPR): https://gdpr-info.eu/.

  3. 3.

    Terms unfairness and bias are used interchangeably.

  4. 4.

    The name comes from drop-out techniques [5, 6] in neural networks. The github repository of LimeOut can be found here:

    https://github.com/vaishnavi026/LimeOut.

  5. 5.

    https://www.kaggle.com/bittlingmayer/amazonreviews.

  6. 6.

    LableEncoder Class is given in the sklearn preprocessing library

    https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html.

  7. 7.

    https://en.wikipedia.org/wiki/COMPAS_(software).

  8. 8.

    https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.

  9. 9.

    Here we focus on binary classifiers that output the probability for each class label.

  10. 10.

    In [18] the authors argue that the submodular pick is a better method than random pick. We still experimented random pick on the datasets of Sect. 4, but the relative importance of features remained similar.

  11. 11.

    In this study we focused on the top 10 features. However this parameter can be set by the user and changed according to his use case.

  12. 12.

    Adult Dataset: http://archive.ics.uci.edu/ml/datasets/Adult.

  13. 13.

    https://imbalanced-learn.readthedocs.io/en/stable/generated/imblearn.over_sampling.SMOTE.html.

  14. 14.

    https://machinelearningmastery.com/threshold-moving-for-imbalanced-classification/.

  15. 15.

    We performed the t-test.

  16. 16.

    https://archive.ics.uci.edu/ml/datasets/statlog+(german+credit+data).

  17. 17.

    It depicts if a person gave aphone number. Due to privacy reasons, the number may not be given. Thus it should not be considered important.

  18. 18.

    Interestingly, there is an accuracy increase when that variable is dropped. However, the current implementation of LimeOut does not take action in these cases.

References

  1. Binns, R.: On the apparent conflict between individual and group fairness. In: Conference on Fairness, Accountability, and Transparency (FAT20), pp. 514–524 (2020)

    Google Scholar 

  2. Chouldechova, A.: Fair prediction with disparate impact: a study of bias in recidivism prediction instruments. Big Data 5(2), 153–163 (2017)

    Article  Google Scholar 

  3. Dressel, J., Farid, H.: The accuracy, fairness, and limits of predicting recidivism. Sci. Adv. 4(eaao5580), 1 (2018)

    Google Scholar 

  4. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.S.: Fairness through awareness. In: Goldwasser, S. (ed.) Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 8–10, 2012, pp. 214–226. ACM (2012)

    Google Scholar 

  5. Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: Balcan, M., Weinberger, K.Q. (eds.) International Conference on Machine Learning, ICML16. JMLR Workshop and Conference Proceedings, vol. 48, pp. 1050–1059 (2016)

    Google Scholar 

  6. Gal, Y., Ghahramani, Z.: A theoretically grounded application of dropout in recurrent neural networks. In: Lee, D.D., Sugiyama, M., von Luxburg, U., Guyon, I., Garnett, R. (eds.) Neural Information Processing Systems (NIPS16), pp. 1019–1027 (2016)

    Google Scholar 

  7. Garreau, D., von Luxburg, U.: Explaining the explainer: a first theoretical analysis of LIME. CoRR abs/2001.03447 (2020)

    Google Scholar 

  8. Grgic-Hlaca, N., Redmiles, E.M., Gummadi, K.P., Weller, A.: Human perceptions of fairness in algorithmic decision making: a case study of criminal risk prediction. In: World Wide Web (WWW18), pp. 903–912 (2018)

    Google Scholar 

  9. Grgic-Hlaca, N., Zafar, M.B., Gummadi, K.P., Weller, A.: The case for process fairness in learning: feature selection for fair decision making. In: NIPS Symposium on Machine Learning and the Law, vol. 1, p. 2 (2016)

    Google Scholar 

  10. Grgić-Hlača, N., Zafar, M.B., Gummadi, K.P., Weller, A.: Beyond distributive fairness in algorithmic decision making: feature selection for procedurally fair learning. In: Proceedings of the Conference on Artificial Intelligence (AAAI18), pp. 51–60 (2018)

    Google Scholar 

  11. Guegan, D., Addo, P.M., Hassani, B.: Credit risk analysis using machine and deep learning models. Risks 6(2), 38 (2018)

    Article  Google Scholar 

  12. Iskandar, B.: Terrorism detection based on sentiment analysis using machine learning. J. Eng. Appl. Sci. 12(3), 691–698 (2017)

    Google Scholar 

  13. Kearns, M., Neel, S., Roth, A., Wu, Z.S.: Preventing fairness gerrymandering: auditing and learning for subgroup fairness. In: International Conference on Machine Learning (ICML18), pp. 2564–2572 (2018)

    Google Scholar 

  14. Laugel, T., Renard, X., Lesot, M.J., Marsala, C., Detyniecki, M.: Defining locality for surrogates in post-hoc interpretablity. arXiv preprint arXiv:1806.07498 (2018)

  15. van der Linden, I., Haned, H., Kanoulas, E.: Global aggregations of local explanations for black box models. arXiv abs/1907.03039 (2019)

    Google Scholar 

  16. Lundberg, S.M., Lee, S.: A unified approach to interpreting model predictions. In: Conference on Neural Information Processing Systems (NIPS17), pp. 4765–4774 (2017)

    Google Scholar 

  17. Maes, S., Tuyls, K., Vanschoenwinkel, B., Manderick, B.: Credit card fraud detection using Bayesian and neural networks. In: NAISO Congress on Neuro Fuzzy Technologies, pp. 261–270 (2002)

    Google Scholar 

  18. Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should i trust you?”: explaining the predictions of any classifier. In: International Conference on Knowledge Discovery and Data Mining (SIGKDD16), pp. 1135–1144 (2016)

    Google Scholar 

  19. Ribeiro, M.T., Singh, S., Guestrin, C.: Anchors: high-precision model-agnostic explanations. In: AAAI Conference on Artificial Intelligence, AAAI18, pp. 1527–1535 (2018)

    Google Scholar 

  20. Roh, Y., Heo, G., Whang, S.E.: A survey on data collection for machine learning: a big data - AI integration perspective. arXiv abs/1811.03402 (2018)

    Google Scholar 

  21. Speicher, T., et al.: A unified approach to quantifying algorithmic unfairness: measuring individual & group unfairness via inequality indices. In: International Conference on Knowledge Discovery & Data Mining (SIGKDD18), pp. 2239–2248 (2018)

    Google Scholar 

  22. Zafar, M.B., Valera, I., Gomez Rodriguez, M., Gummadi, K.P.: Fairness beyond disparate treatment & disparate impact: learning classification without disparate mistreatment. In: World Wide Web (WWW17), pp. 1171–1180 (2017)

    Google Scholar 

  23. Zafar, M.B., Valera, I., Rodriguez, M.G., Gummadi, K.P.: Fairness constraints: mechanisms for fair classification. In: Artificial Intelligence and Statistics (AISTATS17), pp. 962–970 (2017)

    Google Scholar 

  24. Zemel, R., Wu, Y., Swersky, K., Pitassi, T., Dwork, C.: Learning fair representations. In: International Conference on Machine Learning (ICML13), pp. 325–333 (2013)

    Google Scholar 

  25. Zhang, Z., Neill, D.B.: Identifying significant predictive bias in classifiers. CoRR abs/1611.08292 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Miguel Couceiro .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bhargava, V., Couceiro, M., Napoli, A. (2020). LimeOut: An Ensemble Approach to Improve Process Fairness. In: Koprinska, I., et al. ECML PKDD 2020 Workshops. ECML PKDD 2020. Communications in Computer and Information Science, vol 1323. Springer, Cham. https://doi.org/10.1007/978-3-030-65965-3_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-65965-3_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-65964-6

  • Online ISBN: 978-3-030-65965-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics