Joeran Beel
Haley Dixon
ABSTRACT
The impact of Graphical User Interfaces (GUI) for recommender systems is a little explored area. Therefore, we conduct an empirical study in which we create, deploy, and evaluate seven different GUI variations. We use these variations to display 68.260 related-blog-post recommendations to 10.595 unique visitors of our blog. The study shows that the GUIs have a strong effect on the recommender systems’ performance, measured in click-through rate (CTR). The best performing GUI achieved a 66% higher CTR than the worst performing GUI (statist. significant with p<0.05). In other words, with a few days of work to develop different GUIs, a recommender-system operator could increase CTR notably – maybe even more than by tuning the recommendation algorithm. In analogy to the ‘unreasonable effectiveness of data’ discussion by Google and others, we conclude that the effectiveness of graphical user interfaces for recommender systems is equally ‘unreasonable’. Hence, the recommender system community should spend more time on researching GUIs for recommender systems. In addition, we conduct a survey and find that the ACM Recommender Systems Conference has a strong focus on algorithms – 81% of all short and full papers published in 2019 and 2020 relate to algorithm development, and none to GUIs for recommender systems. We also surveyed the recommender systems of 50 blogs. While most displayed a thumbnail (86%) and had a mouseover interaction (62%) other design elements were rare. Only few highlighted top recommendations (8%), displayed rankings or relevance scores (6%), or offered a ‘view more’ option (4%).
Supplemental Material
- Afridi, A.H. 2019. Transparency for Beyond-Accuracy Experiences: A Novel User Interface for Recommender Systems. Procedia Computer Science. 151, (2019), 335–344.Google Scholar
- Banko, M. and Brill, E. 2001. Scaling to Very Very Large Corpora for Natural Language Disambiguation. Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics (Toulouse, France, 2001), 26–33.Google Scholar
- Baudisch, P. and Terveen, L. 1999. Interacting with recommender systems. CHI’99 Extended Abstracts on Human Factors in Computing Systems (1999), 164–164.Google Scholar
- Beel, J. 2019. Darwin & Goliath: Recommendations-As-a-Service with Automated Algorithm-Selection and White-Labels. 13th ACM Conference on Recommender Systems (RecSys) (2019).Google ScholarDigital Library
- Beel, J. 2013. Persistence in Recommender Systems: Giving the Same Recommendations to the Same Users Multiple Times. Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL 2013) (Valletta, Malta, Sep. 2013), 390–394.Google ScholarCross Ref
- Beel, J. 2013. Sponsored vs. Organic (Research Paper) Recommendations and the Impact of Labeling. Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL 2013) (Valletta, Malta, Sep. 2013), 395–399.Google Scholar
- Beel, J. 2014. Utilizing Mind-Maps for Information Retrieval and User Modelling. Proceedings of the 22nd Conference on User Modelling, Adaption, and Personalization (UMAP) (2014), 301–313.Google ScholarCross Ref
- Beel, J. and Dinesh, S. 2017. Real-World Recommender Systems for Academia: The Gain and Pain in Developing, Operating, and Researching them. Proceedings of the Fifth Workshop on Bibliometric-enhanced Information Retrieval (BIR) co-located with the 39th European Conference on Information Retrieval (ECIR 2017) (2017), 6–17.Google Scholar
- Beierle, F. 2019. Choice overload and recommendation effectiveness in related-article recommendations. International Journal of Digital Libraries (IJDL). (2019), 1–16.Google ScholarCross Ref
- Beierle, F. 2017. Exploring Choice Overload in Related-Article Recommendations in Digital Libraries. 5th International Workshop on Bibliometric-enhanced Information Retrieval (BIR) at the 39th European Conference on Information Retrieval (ECIR) (2017), 51–61.Google Scholar
- Bendada, W. 2020. Carousel Personalization in Music Streaming Apps with Contextual Bandits. Fourteenth ACM Conference on Recommender Systems (Virtual Event, Brazil, 2020), 420–425.Google Scholar
- Bonhard, P. and Sasse, M.A. 2006. I thought it was terrible and everyone else loved it" — A New Perspective for Effective Recommender System Design. People and Computers XIX—The Bigger Picture. Springer. 251–265.Google Scholar
- Cardoso, B. 2019. IntersectionExplorer, a multi-perspective approach for exploring recommendations. International Journal of Human-Computer Studies. 121, (2019), 73–92.Google Scholar
- Chen, L. 2013. Workshop on human decision making in recommender systems: decisions@ RecSys’ 13. Proceedings of the 7th ACM conference on Recommender systems (2013), 479–480.Google ScholarDigital Library
- Chen, L. and Tsoi, H.K. 2011. Users’ decision behavior in recommender interfaces: Impact of layout design. RecSys’ 11 Workshop on Human Decision Making in Recommender Systems (2011).Google Scholar
- Collins, A. 2018. Position Bias in Recommender Systems for Digital Libraries. Proceedings of the iConference (2018), 335–344.Google ScholarCross Ref
- Cosley, D. 2003. Is seeing believing? How recommender system interfaces affect users’ opinions. Proceedings of the SIGCHI conference on Human factors in computing systems (2003), 585–592.Google Scholar
- Devendorf, L. 2012. TopicLens: An interactive recommender system based on topical and social connections. First International Workshop on Recommendation Technologies for Lifestyle Change (LIFESTYLE 2012) (2012), 41.Google Scholar
- Dong, Q. 2017. Interactive design on recommender system. 2017 IEEE 17th International Conference on Communication Technology (ICCT) (2017), 1884–1890.Google ScholarCross Ref
- Feyer, S. 2017. Integration of the Scientific Recommender System Mr. DLib into the Reference Manager JabRef. Proceedings of the 39th European Conference on Information Retrieval (ECIR) (2017), 770–774.Google ScholarCross Ref
- Galitz, W.O. 2007. The essential guide to user interface design: an introduction to GUI design principles and techniques. John Wiley & Sons.Google ScholarDigital Library
- Garett, R. 2016. A literature review: website design and user engagement. Online journal of communication and media technologies. 6, 3 (2016), 1.Google Scholar
- Gipp, B. 2009. Scienstein: A Research Paper Recommender System. Proceedings of the International Conference on Emerging Trends in Computing (ICETiC’09) (Virudhunagar (India), 2009), 309–315.Google Scholar
- Guntuku, S.C. 2016. Personalizing User Interfaces for improving quality of experience in VoD recommender systems. 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX) (2016), 1–6.Google ScholarCross Ref
- Halevy, A. 2009. The unreasonable effectiveness of data. IEEE Intelligent Systems. 24, 2 (2009), 8–12.Google ScholarDigital Library
- Helander, M.G. 2014. Handbook of human-computer interaction. Elsevier.Google Scholar
- Herlocker, J.L. 2004. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems (TOIS). 22, 1 (2004), 5–53.Google ScholarDigital Library
- Herlocker, J.L. 2000. Explaining collaborative filtering recommendations. Proceedings of the 2000 ACM conference on Computer supported cooperative work (2000), 241–250.Google ScholarDigital Library
- Hofmann, K. 2014. Effects of Position Bias on Click-Based Recommender Evaluation. Advances in Information Retrieval. Springer. 624–630.Google Scholar
- Jannach, D. 2021. Exploring Multi-List User Interfaces for Similar-Item Recommendations. Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization (2021).Google ScholarDigital Library
- Jannach, D. 2017. Interacting with Recommender Systems. Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion (2017), 25–27.Google ScholarDigital Library
- Jugovac, M. and Jannach, D. 2017. Interacting with Recommenders—Overview and Research Directions. ACM Trans. Interact. Intell. Syst. 7, 3 (2017).Google ScholarDigital Library
- Knijnenburg, B.P. 2012. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction. 22, 4-5 (2012), 441–504.Google ScholarDigital Library
- Konstan, J.A. and Riedl, J. 2012. Recommender systems: from algorithms to user experience. User Modeling and User-Adapted Interaction. (2012), 1–23.Google Scholar
- McInerney, J. 2018. Explore, Exploit, and Explain: Personalizing Explainable Recommendations with Bandits. Proceedings of the 12th ACM Conference on Recommender Systems (Vancouver, British Columbia, Canada, 2018), 31–39.Google ScholarDigital Library
- McNee, S.M. 2003. Interfaces for eliciting new user preferences in recommender systems. International Conference on User Modeling (2003), 178–187.Google ScholarDigital Library
- Murphy-Hill, E. and Murphy, G.C. 2014. Recommendation delivery. Recommendation systems in software engineering. Springer. 223–242.Google Scholar
- Neumann, A.W. 2008. RecoDiver: Browsing behavior-based recommendations on dynamic graphs. AI Communications. 21, 2-3 (2008), 177–183.Google ScholarCross Ref
- Ozok, A.A. 2010. Design Guidelines for Effective Recommender System Interfaces Based on a Usability Criteria Conceptual Model: Results from a College Student Population. Behav. Inf. Technol. 29, 1 (2010), 57–83.Google ScholarDigital Library
- Pu, P. 2011. A user-centric evaluation framework for recommender systems. Proceedings of the fifth ACM conference on Recommender systems (2011), 157–164.Google ScholarDigital Library
- Pu, P. and Chen, L. 2007. Trust-inspiring explanation interfaces for recommender systems. Knowledge-Based Systems. 20, 6 (2007), 542–556.Google ScholarDigital Library
- Schafer, J.B. 2002. Meta-Recommendation Systems: User-Controlled Integration of Diverse Recommendations. Proceedings of the Eleventh International Conference on Information and Knowledge Management (McLean, Virginia, USA, 2002), 43–51.Google ScholarDigital Library
- Schendel, Z.A. 2020. A Human Perspective on Algorithmic Similarity. Fourteenth ACM Conference on Recommender Systems (Virtual Event, Brazil, 2020), 562.Google ScholarDigital Library
- Schnabel, T. 2018. Improving Recommender Systems Beyond the Algorithm. arXiv 1802.07578. (2018).Google Scholar
- Swearingen, K. and Sinha, R. 2001. Beyond algorithms: An HCI perspective on recommender systems. ACM SIGIR 2001 workshop on recommender systems (2001), 1–11.Google Scholar
- Di Tommaso, G. and Stilo, G. 2017. Twixonomy Visualization Interface: How to Wander Around User Preferences. EnCHIReS@ EICS (2017), 32–41.Google Scholar
- Verbert, K. 2013. Visualizing recommendations to support exploration, transparency and controllability. Proceedings of the 2013 international conference on Intelligent user interfaces (2013), 351–362.Google ScholarDigital Library
Recommendations
Acquiring User Information Needs for Recommender Systems
WI-IAT '13: Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 03Most recommender systems attempt to use collaborative filtering, content-based filtering or hybrid approach to recommend items to new users. Collaborative filtering recommends items to new users based on their similar neighbours, and content-based ...
User Personality and User Satisfaction with Recommender Systems
In this study, we show that individual users' preferences for the level of diversity, popularity, and serendipity in recommendation lists cannot be inferred from their ratings alone. We demonstrate that we can extract strong signals about individual ...
Personalizing graphical user interfaces on flexible widget layout
EICS '09: Proceedings of the 1st ACM SIGCHI symposium on Engineering interactive computing systemsThe authors propose a method for personalizing the flexible widget layout (FWL) by adjusting the desirability of widgets with a pairwise comparison method, and show its implementation and that it actually works. Personalization of graphical user ...
Comments