Skip to main content
SpringerLink
Log in
Book cover

Transactions on Computational Intelligence XIII pp 176–200Cite as

Validating Generic Metrics of Fairness in Game-Based Resource Allocation Scenarios with Crowdsourced Annotations

  • Chapter

Part of the book series: Lecture Notes in Computer Science ((TCCI,volume 8342))

Abstract

Being able to effectively measure the notion of fairness is of vital importance as it can provide insight into the formation and evolution of complex patterns and phenomena, such as social preferences, collaboration, group structures and social conflicts. This paper presents a comparative study for quantitatively modelling the notion of fairness in one-to-many resource allocation scenarios — i.e. one provider agent has to allocate resources to multiple receiver agents. For this purpose, we investigate the efficacy of six metrics and cross-validate them on crowdsourced human ranks of fairness annotated through a computer game implementation of the one-to-many resource allocation scenario. Four of the fairness metrics examined are well-established metrics of data dispersion, namely standard deviation, normalised entropy, the Gini coefficient and the fairness index. The fifth metric, proposed by the authors, is an ad-hoc context-based measure which is based on key aspects of distribution strategies. The sixth metric, finally, is machine learned via ranking support vector machines (SVMs) on the crowdsourced human perceptions of fairness. Results suggest that all ad-hoc designed metrics correlate well with the human notion of fairness, and the context-based metrics we propose appear to have a predictability advantage over the other ad-hoc metrics. On the other hand, the normalised entropy and fairness index metrics appear to be the most expressive and generic for measuring fairness for the scenario adopted in this study and beyond. The SVM model can automatically model fairness more accurately than any ad-hoc metric examined (with an accuracy of 81.86%) but it is limited by its expressivity and generalisability.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Akerlof, G.A., Kranton, R.E.: Economics and Identity. The Quarterly Journal of Economics 115(3), 715–753 (2000)

    Article  Google Scholar 

  2. Axelrod, R., Hamilton, W.D.: The Evolution of Cooperation (1981)

    Google Scholar 

  3. Bolton, G.E., Katok, E., Zwick, R.: Dictator Game Giving: Rules of Fairness Versus Acts of Kindness. International Journal of Game Theory 27, 269–299 (1998)

    Article  MATH  Google Scholar 

  4. Charness, G., Rabin, M.: Understanding Social Preferences with Simple Tests. The Quarterly Journal of Economics 117(3), 817–869 (2002)

    Article  MATH  Google Scholar 

  5. Charness, G., Rigotti, L., Rustichini, A.: Individual Behavior and Group Membership (2006), SSRN 894685

    Google Scholar 

  6. Chen, Y., Li, S.X.: Group Identity and Social Preferences. The American Economic Review, 431–457 (2009)

    Google Scholar 

  7. Cheong, Y.G., Khaled, R., Grappiolo, C., Campos, J., Martinho, C., Ingram, G.P.D., Paiva, A., Yannakakis, G.N.: A Computational Approach Towards Conflict Resolution for Serious Games. In: Proceedings of the Sixth International Conference on the Foundations of Digital Games. ACM (2010)

    Google Scholar 

  8. Chu, W., Ghahramani, Z.: Preference Learning with Gaussian Processes. In: Proceedings of the 22nd International Conference on Machine Learning. ACM (2005)

    Google Scholar 

  9. Dawes, R.M., Messick, D.M.: Social Dilemmas. International Journal of Psychology 2(35), 111–116 (2000)

    Article  Google Scholar 

  10. De Jong, S., Tuyls, K., Verbeeck, K.: Artificial Agents Learning Human Fairness. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 863–870 (2008)

    Google Scholar 

  11. Dianati, M., Shen, X., Naik, S.: A New Fairness Index for Radio Resource Allocation in Wireless Networks. In: Wireless Communications and Networking Conference, vol. 2, pp. 712–717 (2005)

    Google Scholar 

  12. Ducheneaut, N., Yee, N., Nickell, E., Moore, R.J.: Alone Together? Exploring the Social Dynamics of Massively Multiplayer Online Games. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 407–416. ACM (2006)

    Google Scholar 

  13. Eagle, N., Pentland, A.S., Lazer, D.: Inferring Friendship Network Structure by Using Mobile Phone Data. Proceedings of the National Academy of Sciences 106(36), 15274–15278 (2009)

    Article  Google Scholar 

  14. Epstein, J.M., Axtell, R.L.: Growing Artificial Societies: Social Science from the Bottom Up (Complex Adaptive Systems). The MIT Press (1996)

    Google Scholar 

  15. Fehr, E., Fischbacher, U.: Why Social Preferences Matter — The Impact of Non-Selfish Motives on Competition, Cooperation and Incentives. Economic Journal 112, 1–33 (2002)

    Article  Google Scholar 

  16. Fehr, E., Schmidt, K.M.: A Theory of Fairness, Competition, and Cooperation. The Quarterly Journal of Economics 114(3), 817–868 (1999)

    Article  MATH  Google Scholar 

  17. Forsythe, R.: Fairness in Simple Bargaining Experiments. Games and Economic Behavior 6(3), 347–369 (1994)

    Article  MATH  Google Scholar 

  18. Gal, Y., Grosz, B.J., Kraus, S., Pfeffer, A., Shieber, S.: Colored Trails: A Formalism for Investigating Decision-making in Strategic Environments. In: Proceedings of the 2005 IJCAI Workshop on Reasoning, Representation, and Learning in Computer Games, pp. 25–30 (2005)

    Google Scholar 

  19. Gini, C.: Measurement of Inequality of Incomes. The Economic Journal 31(121), 124–126 (1921)

    Article  Google Scholar 

  20. Girvan, M., Newman, M.E.: Community structure in social and biological networks. Proceedings of the National Academy of Sciences 99(12), 7821–7826 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  21. Grappiolo, C., Cheong, Y.G., Khaled, R., Yannakakis, G.N.: Modelling Global Pattern Formation for Collaborative Learning Environments. In: Proceedings of the IEEE International Conference on Advanced Learning Technologies (2012)

    Google Scholar 

  22. Grappiolo, C., Cheong, Y.G., Togelius, J., Khaled, R., Yannakakis, G.N.: Towards Player Adaptivity in a Serious Game for Conflict Resolution. In: Proceedings of the 3rd IEEE International Conference in Games and Virtual Worlds for Serious Applications. pp. 192–198 (2011)

    Google Scholar 

  23. Grappiolo, C., Togelius, J., Yannakakis, G.N.: Interaction-based Group Identity Detection via Reinforcement Learning and Artificial Evolution. In: Proceedings of the Evolutionary Computation and Multi-agent Systems and Simulation Workshop, Genetic and Evolutionary Computation Conference, pp. 1423–1430. ACM (2013)

    Google Scholar 

  24. Grappiolo, C., Yannakakis, G.N.: Towards Detecting Group Identities in Complex Artificial Societies. In: Proceedings of the Simulation of Adaptive Behaviour Conference. pp. 421–430 (2012)

    Google Scholar 

  25. Greenwood, G.W., Ashlock, D.: Evolutionary Games and the Study of Cooperation: Why Has So Little Progress Been Made? In: Proceedings of the IEEE World Congress on Computational Intelligence (2012)

    Google Scholar 

  26. Hammond, R.A., Axelrod, R.: The Evolution of Ethnocentrism. Journal of Conflict Resolution 50(6), 926–936 (2006)

    Article  Google Scholar 

  27. Herbrich, R., Graepel, T., Obermayer, K.: Support Vector Learning for Ordinal Regression. In: Proceedings of the International Conference on Artificial Neural Networks, vol. 1, p. 97 (1999)

    Google Scholar 

  28. Huberman, B.A., Glance, N.S.: Evolutionary Games and Computer Simulations. Proceedings National Academy of Science 90(16), 7716–7718 (1993)

    Article  MATH  Google Scholar 

  29. Jain, R., Chiu, D.M., Hawe, W.R.: A Quantitative Measure of Fairness and Discrimination for Resource Allocation in Shared Computer System. Eastern Research Laboratory, Digital Equipment Corporation (1984)

    Google Scholar 

  30. Joachims, T.: Learning to Classify Text Using Support Vector Machines — Methods, Theory, and Algorithms. Kluwer/Springer (2002)

    Google Scholar 

  31. Joachims, T.: Optimizing Search Engines Using Clickthrough Data. In: Proceedings of the 8th SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 133–142. ACM (2002)

    Google Scholar 

  32. Joe-Wong, C., Sen, S., Lan, T., Chiang, M.: Multi-resource Allocation: Fairness-efficiency Tradeoffs in a Unifying Framework. In: Proceedings of the IEEE International Conference on Computer Communications, pp. 1206–1214. IEEE (2012)

    Google Scholar 

  33. Kagel, J.H., Kim, C., Moser, D.: Fairness in Ultimatum Games with Asymmetric Information and Asymmetric Payoffs. Games and Economic Behavior 13(1), 100–110 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  34. Kim, J.H.: The Role of Identity in Intra-and Inter-Group Bargaining in the Ultimatum Game. Undergraduate Economic Review 4(1), 6 (2008)

    Google Scholar 

  35. Kranton, R., Pease, M., Sanders, S., Huettel, S.: Identity, Group Conflict, and Social Preferences. Working Paper (2012)

    Google Scholar 

  36. Lancichinetti, A., Fortunato, S.: Limits of Modularity Maximization in Community Detection. Physical Review E 84(6), 066122 (2011)

    Article  Google Scholar 

  37. Lansing, S.J.: Complex Adaptive Systems. Annual Review of Anthropology 32, 183–204 (2003)

    Article  Google Scholar 

  38. Mahlmann, T., Togelius, J., Yannakakis, G.N.: Modelling and Evaluation of Complex Scenarios with the Strategy Game Description Language. In: Proceedings of the IEEE Conference for Computational Intelligence and Games, Seoul, KR (2011)

    Google Scholar 

  39. Martínez, H.P., Yannakakis, G.N.: Mining multimodal sequential patterns: a case study on affect detection. In: Proceedings of International Conference on Multimodal Interfaces (ICMI), pp. 3–10. ACM (2011)

    Google Scholar 

  40. Martínez, H.P., Yannakakis, G.N.: Genetic Search Feature Selection for Affective Modeling: a Case Study on Reported Preferences. In: Proceedings of the 3rd International Workshop on Affective Interaction in Natural Environments, pp. 15–20. ACM (2010)

    Google Scholar 

  41. Martinez, R., Kay, J., Wallace, J.R., Yacef, K.: Modelling Symmetry of Activity as an Indicator of Collocated Group Collaboration. In: Konstan, J.A., Conejo, R., Marzo, J.L., Oliver, N. (eds.) UMAP 2011. LNCS, vol. 6787, pp. 207–218. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  42. Marzo, F., Grosz, B.J., Pfeffer, A.: Social preferences in Relational Contexts. In: Fourth Conference in Collective Intentionality (2005)

    Google Scholar 

  43. Montuno, K., Zhacfi, Y.: Fairness of Resource Allocation in Cellular Networks: A Survey. Resource Allocation in Next Generation Wireless Networks, 249–266 (2006)

    Google Scholar 

  44. Nowak, M.A.: Five Rules for the Evolution of Cooperation. Science 314(5805), 1560–1563 (2006)

    Article  Google Scholar 

  45. Palla, G., Barabási, A.L., Vicsek, T.: Quantifying Social Group Evolution. Nature 446(7136), 664–667 (2007)

    Article  Google Scholar 

  46. Pandremmenou, K., Kondi, L.P., Parsopoulos, K.E.: Fairness Issues in Resource Allocation Schemes for Wireless Visual Sensor Networks. In: IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, pp. 866601–866601 (2013)

    Google Scholar 

  47. Prada, R., Paiva, A.: Teaming Up Humans with Autonomous synthetic Characters. Artificial Intelligence 173(1), 80–103 (2009)

    Article  Google Scholar 

  48. Rabin, M.: Incorporating Fairness into Game Theory and Economics. The American Economic Review, 1281–1302 (1993)

    Google Scholar 

  49. Rocha, J.B., Mascarenhas, S., Prada, R.: Game Mechanics for Cooperative Games, pp. 73–80. Universidade do Minho (2008)

    Google Scholar 

  50. El-Nasr, M.S., Aghabeigi, B., Milam, D., Erfani, M., Lameman, B., Maygoli, H., Mah, S.: Understanding and Evaluating Cooperative Games. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 253–262. ACM (2010)

    Google Scholar 

  51. Shaker, N., Yannakakis, G., Togelius, J.: Crowd-Sourcing the Aesthetics of Platform Games. IEEE Transactions on Computational Intelligence and AI in Games (2012)

    Google Scholar 

  52. Shi, H., Venkatesha Prasad, R., Rao, V.S., Niemegeers, I.G.M.M.: A Fairness Model for Resource Allocation in Wireless Networks. In: Becvar, Z., Bestak, R., Kencl, L. (eds.) NETWORKING 2012 Workshops. LNCS, vol. 7291, pp. 1–9. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  53. Sonntagbauer, P., Aizstrauts, A., Ginters, E., Aizstrauta, D.: Policy Simulation and E-Governance. In: IADIS International Conference e-Society (2012)

    Google Scholar 

  54. Szell, M., Thurner, S.: Measuring Social Dynamics in a Massive Multiplayer Online Game. Social Networks 32(4), 313–329 (2010)

    Article  Google Scholar 

  55. Tan, G., Guttag, J.V.: Time-based Fairness Improves Performance in Multi-Rate WLANs. In: USENIX Annual Technical Conference, General Track, pp. 269–282 (2004)

    Google Scholar 

  56. Xianyu, B.: Social Preference, Incomplete Information, and the Evolution of Ultimatum Game in the Small World Networks: An Agent-Based Approach. Journal of Artificial Societies and Social Simulation 13(2), 7 (2010)

    Google Scholar 

  57. Yannakakis, G.N., Hallam, J.: Towards Optimizing Entertainment in Computer Games. Applied Artificial Intelligence 21(10), 933–971 (2007)

    Article  Google Scholar 

  58. Yannakakis, G.N., Hallam, J.: Ranking vs. Preference: A Comparative Study of Self-reporting. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011, Part I. LNCS, vol. 6974, pp. 437–446. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  59. Yannakakis, G.N., Martnez, H.P., Jhala, A.: Towards Affective Camera Control in Games. User Modeling and User-Adapted Interaction 20, 313–340 (2010)

    Article  Google Scholar 

  60. Yannakakis, G.N., Togelius, J.: Experience-Driven Procedural Content Generation. IEEE Transactions on Affective Computing 2, 147–161 (2011)

    Article  Google Scholar 

  61. Yannakakis, G.N., Togelius, J., Khaled, R., Jhala, A., Karpouzis, K., Paiva, A., Vasalou, A.: Siren: Towards Adaptive Serious Games for Teaching Conflict Resolution. In: Proceedings European Conference on Games-Based Learning (ECGBL), pp. 412–417. Copenhagen (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Computer Games Research, IT University of Copenhagen, Denmark

    Corrado Grappiolo

  2. Institute of Digital Games, University of Malta, Malta

    Héctor P. Martínez & Georgios N. Yannakakis

Authors
  1. Corrado Grappiolo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Héctor P. Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Georgios N. Yannakakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Informatics, Wrocław University of Technology, Wrocław, Poland

    Ngoc-Thanh Nguyen

  2. Laboratory of Theoretical and Applied Computer Science, University of Lorraine, 57045, Metz Cedex, France

    Hoai An Le-Thi

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Grappiolo, C., Martínez, H.P., Yannakakis, G.N. (2014). Validating Generic Metrics of Fairness in Game-Based Resource Allocation Scenarios with Crowdsourced Annotations. In: Nguyen, NT., Le-Thi, H.A. (eds) Transactions on Computational Intelligence XIII. Lecture Notes in Computer Science, vol 8342. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-54455-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-54455-2_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-54454-5

  • Online ISBN: 978-3-642-54455-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation