Christine Bauer
Lennard Chung
ABSTRACT
FairRecKit is a web-based analysis software that supports researchers in performing, analyzing, and understanding recommendation computations. The idea behind FairRecKit is to facilitate the in-depth analysis of recommendation outcomes considering fairness aspects. With (nested) filters on user or item attributes, metrics can easily be compared across user and item subgroups. Further, (nested) filters can be used on the dataset level; this way, recommendation outcomes can be compared across several sub-datasets to analyze for differences considering fairness aspects. The software currently features five datasets, 11 metrics, and 21 recommendation algorithms to be used in computational experimentation. It is open source and developed in a modular manner to facilitate extension. The analysis software consists of two components: A software package (FairRecKitLib) for running recommendation algorithms on the available datasets and a web-based user interface (FairRecKitApp) to start experiments, retrieve results of previous experiments, and analyze details. The application also comes with extensive documentation and options for result customization, which makes for a flexible tool that supports in-depth analysis.
- Vito Walter Anelli, Alejandro Bellogín, Antonio Ferrara, Daniele Malitesta, Felice Antonio Merra, Claudio Pomo, Francesco Maria Donini, and Tommaso Di Noia. 2021. Elliot: A Comprehensive and Rigorous Framework for Reproducible Recommender Systems Evaluation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada) (SIGIR ’21). ACM, New York, NY, USA, 2405–2414. https://doi.org/10.1145/3404835.3463245Google Scholar
Digital Library
- Joeran Beel, Corinna Breitinger, Stefan Langer, Andreas Lommatzsch, and Bela Gipp. 2016. Towards reproducibility in recommender-systems research. User Modeling and User-Adapted Interaction 26, 1 (2016), 69–101. https://doi.org/10.1007/s11257-016-9174-xGoogle ScholarDigital Library
- Alejandro Bellogín and Alan Said. 2021. Improving accountability in recommender systems research through reproducibility. User Modeling and User-Adapted Interaction 31, 5 (2021), 941–977. https://doi.org/10.1007/s11257-021-09302-xGoogle ScholarDigital Library
- Òscar Celma. 2010. Music Recommendation. Springer Berlin Heidelberg, Berlin, Heidelberg, Germany, Chapter 3, 43–85. https://doi.org/10.1007/978-3-642-13287-2_3Google ScholarCross Ref
- Karlijn Dinnissen and Christine Bauer. 2022. Fairness in music recommender systems: a stakeholder-centered mini review. Frontiers in Big Data 5, Article 913608 (2022), 9 pages. https://doi.org/10.3389/fdata.2022.913608Google ScholarCross Ref
- Michael D. Ekstrand. 2020. LensKit for Python: Next-Generation Software for Recommender Systems Experiments. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (Virtual Event, Ireland) (CIKM ’20). ACM, New York, NY, USA, 2999–3006. https://doi.org/10.1145/3340531.3412778Google ScholarDigital Library
- Michael D Ekstrand, Anubrata Das, Robin Burke, and Fernando Diaz. 2022. Fairness in Information Access Systems. Foundations and Trends® in Information Retrieval 16, 1–2 (2022), 1–177. https://doi.org/10.1561/1500000079Google ScholarDigital Library
- Michael D. Ekstrand and Daniel Kluver. 2021. Exploring author gender in book rating and recommendation. User Modeling and User-Adapted Interaction 31, 3 (2021), 377–420. https://doi.org/10.1007/s11257-020-09284-2Google ScholarDigital Library
- Michael D. Ekstrand, Michael Ludwig, Joseph A. Konstan, and John T. Riedl. 2011. Rethinking the Recommender Research Ecosystem: reproducibility, openness, and LensKit. In Proceedings of the 5th ACM Conference on Recommender Systems (Chicago, IL, USA) (RecSys ’11). ACM, New York, NY, USA, 133–140. https://doi.org/10.1145/2043932.2043958Google ScholarDigital Library
- Maurizio Ferrari Dacrema, Simone Boglio, Paolo Cremonesi, and Dietmar Jannach. 2021. A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research. ACM Transactions on Information Systems 39, 2, Article 20 (jan 2021), 49 pages. https://doi.org/10.1145/3434185Google ScholarDigital Library
- Andrés Ferraro, Xavier Serra, and Christine Bauer. 2021. Break the Loop: Gender Imbalance in Music Recommenders. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval (Virtual Event) (CHIIR ’21). ACM, New York, NY, USA, 249–254. https://doi.org/10.1145/3406522.3446033Google ScholarDigital Library
- Andrés Ferraro, Xavier Serra, and Christine Bauer. 2021. What is fair? exploring the artists’ perspective on the fairness of music streaming platforms. In Human-Computer Interaction – INTERACT 2021. Lecture Notes in Computer Science, Vol. 12933. Springer International Publishing, Cham, Germany, 562–584. https://doi.org/10.1007/978-3-030-85616-8_33Google ScholarDigital Library
- Ben Frederickson. 2017. Implicit package. https://implicit.readthedocs.io/en/latest/index.html.Google Scholar
- F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems 5, 4, Article 19 (Dec. 2015), 19 pages. https://doi.org/10.1145/2827872Google ScholarDigital Library
- Nicolas Hug. 2015. Surprise package. https://surprise.readthedocs.io/en/stable/index.html.Google Scholar
- Benedek Rozemberczki, Sebastian Nilsson, and Piotr Grabowski. 2022. rexmex package. https://rexmex.readthedocs.io/en/latest/index.html.Google Scholar
- Markus Schedl. 2016. The LFM-1b Dataset for Music Retrieval and Recommendation. In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval (New York, NY, USA) (ICMR ’16). ACM, New York, NY, USA, 103–110. https://doi.org/10.1145/2911996.2912004Google ScholarDigital Library
- Markus Schedl, Stefan Brandl, Oleg Lesota, Emilia Parada-Cabaleiro, David Penz, and Navid Rekabsaz. 2022. LFM-2b: A Dataset of Enriched Music Listening Events for Recommender Systems Research and Fairness Analysis. In ACM SIGIR Conference on Human Information Interaction and Retrieval (Regensburg, Germany) (CHIIR ’22). ACM, New York, NY, USA, 337–341. https://doi.org/10.1145/3498366.3505791Google ScholarDigital Library
- Nasim Sonboli, Robin Burke, Zijun Liu, and Masoud Mansoury. 2020. Fairness-Aware Recommendation with Librec-Auto. In Fourteenth ACM Conference on Recommender Systems(RecSys ’20). ACM, New York, NY, USA, 594–596. https://doi.org/10.1145/3383313.3411525Google ScholarDigital Library
- Eva Zangerle and Christine Bauer. 2022. Evaluating recommender systems: survey and framework. Comput. Surveys 55, 8, Article 170 (2022), 38 pages. https://doi.org/10.1145/3556536Google ScholarDigital Library
Index Terms
- FairRecKit: A Web-based Analysis Software for Recommender Evaluations
Recommendations
A Scalable, Accurate Hybrid Recommender System
WKDD '10: Proceedings of the 2010 Third International Conference on Knowledge Discovery and Data MiningRecommender systems apply machine learning techniques for filtering unseen information and can predict whether a user would like a given resource. There are three main types of recommender systems: collaborative filtering, content-based filtering, and ...
On Unexpectedness in Recommender Systems: Or How to Better Expect the Unexpected
Special Sections on Diversity and Discovery in Recommender Systems, Online Advertising and Regular PapersAlthough the broad social and business success of recommender systems has been achieved across several domains, there is still a long way to go in terms of user satisfaction. One of the key dimensions for significant improvement is the concept of ...
KMulE: a framework for user-based comparison of recommender algorithms
IUI '12: Proceedings of the 2012 ACM international conference on Intelligent User InterfacesCollaborative Filtering Recommender Systems come in a wide variety of variants. In this paper we present a system for visualizing and comparing recommendations provided by different collaborative recommendation algorithms. The system utilizes a set of ...
Comments