skip to main content
research-article

MARLUI: Multi-Agent Reinforcement Learning for Adaptive Point-and-Click UIs

Published: 17 June 2024 Publication History

Abstract

As the number of selectable items increases, point-and-click interfaces rapidly become complex, leading to a decrease in usability. Adaptive user interfaces can reduce this complexity by automatically adjusting an interface to only display the most relevant items. A core challenge for developing adaptive interfaces is to infer user intent and chose adaptations accordingly. Current methods rely on tediously hand-crafted rules or carefully collected user data. Furthermore, heuristics need to be recrafted and data regathered for every new task and interface. To address this issue, we formulate interface adaptation as a multi-agent reinforcement learning problem. Our approach learns adaptation policies without relying on heuristics or real user data, facilitating the development of adaptive interfaces across various tasks with minimal adjustments needed. In our formulation, a user agent mimics a real user and learns to interact with an interface via point-and-click actions. Simultaneously, an interface agent learns interface adaptations, to maximize the user agent's efficiency, by observing the user agent's behavior. For our evaluation, we substituted the simulated user agent with actual users. Our study involved twelve participants and concentrated on automatic toolbar item assignment. The results show that the policies we developed in simulation effectively apply to real users. These users were able to complete tasks with fewer actions and in similar times compared to methods trained with real data. Additionally, we demonstrated our method's efficiency and generalizability across four different interfaces and tasks.

References

[1]
John R Anderson, Michael Matessa, and Christian Lebiere. 1997. ACT-R: A theory of higher level cognition and its relation to visual attention. Human-Computer Interaction 12, 4 (1997), 439--462.
[2]
Karl Johan Åström. 1965. Optimal control of Markov processes with incomplete state information. Journal of mathematical analysis and applications 10, 1 (1965), 174--205.
[3]
Bowen Baker, Ingmar Kanitscheider, Todor Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. 2019. Emergent tool use from multi-agent autocurricula. arXiv preprint arXiv:1909.07528 (2019).
[4]
Pauline M Berry, Melinda Gervasio, Bart Peintner, and Neil Yorke-Smith. 2011. PTIME: Personalized assistance for calendaring. ACM Transactions on Intelligent Systems and Technology (TIST) 2, 4 (2011), 1--22.
[5]
Wauter Bosma and Elisabeth André. 2004. Exploiting Emotions to Disambiguate Dialogue Acts. In Proceedings of the 9th International Conference on Intelligent User Interfaces (Funchal, Madeira, Portugal) (IUI '04). Association for Computing Machinery, New York, NY, USA, 85--92. https://doi.org/10.1145/964442.964459
[6]
Matthew Michael Botvinick. 2012. Hierarchical reinforcement learning and decision making. Current opinion in neurobiology 22, 6 (2012), 956--962.
[7]
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. arXiv:1606.01540 [cs.LG]
[8]
Dermot Browne, Peter Totterdell, and Mike Norman. 2016. Adaptive user interfaces. Elsevier.
[9]
S Card, T Moran, and A Newell. 1983. T he Psychology of Human Computer Interaction.
[10]
Stuart K. Card, Thomas P. Moran, and Allen Newell. 1980. The Keystroke-Level Model for User Performance Time with Interactive Systems. Commun. ACM 23, 7 (jul 1980), 396--410. https://doi.org/10.1145/358886.358895
[11]
Stuart. K. Card, Thomas. P. Moran, and Allen Newell. 1986. The model human processor- An engineering model of human performance. Handbook of perception and human performance. 2, 45-1 (1986).
[12]
Noshaba Cheema, Laura A Frey-Law, Kourosh Naderi, Jaakko Lehtinen, Philipp Slusallek, and Perttu Hämäläinen. 2020. Predicting mid-air interaction movements and fatigue using deep reinforcement learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--13.
[13]
Minmin Chen, Alex Beutel, Paul Covington, Sagar Jain, Francois Belletti, and Ed H Chi. 2019. Top-K Off-Policy Correction for a REINFORCE Recommender System. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining (WSDM '19). ACM, 456--464. https://doi.org/10.1145/3289600.3290999
[14]
Xiuli Chen, Gilles Bailly, Duncan P Brumby, Antti Oulasvirta, and Andrew Howes. 2015. The emergence of interactive behavior: A model of rational menu search. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 4217--4226.
[15]
Yi Fei Cheng, Christoph Gebhardt, and Christian Holz. 2023. InteractionAdapt: Interaction-driven Workspace Adaptation for Situated Virtual Reality Environments. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1--14.
[16]
Sammy Christen, Lan Feng, Wei Yang, Yu-Wei Chao, Otmar Hilliges, and Jie Song. 2023. SynH2R: Synthesizing Hand-Object Motions for Learning Human-to-Robot Handovers. arXiv preprint arXiv:2311.05599 (2023).
[17]
Sammy Christen, Lukas Jendele, Emre Aksan, and Otmar Hilliges. 2021. Learning Functionally Decomposed Hierarchies for Continuous Control Tasks With Path Planning. IEEE Robotics and Automation Letters 6, 2 (2021), 3623--3630. https://doi.org/10.1109/LRA.2021.3060403
[18]
Sammy Christen, Stefan Stevšić, and Otmar Hilliges. 2019. Guided deep reinforcement learning of control policies for dexterous human-robot interaction. In 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2161--2167.
[19]
Sammy Christen, Wei Yang, Claudia Pérez-D'Arpino, Otmar Hilliges, Dieter Fox, and Yu-Wei Chao. 2023. Learning human-to-robot handovers from point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9654--9664.
[20]
Andy Cockburn, Carl Gutwin, and Saul Greenberg. 2007. A Predictive Model of Menu Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '07). Association for Computing Machinery, New York, NY, USA, 627--636. https://doi.org/10.1145/1240624.1240723
[21]
Peng Dai, Christopher Lin, Mausam Mausam, and Daniel Weld. 2013. POMDP-based control of workflows for crowdsourcing. Artificial Intelligence 202 (09 2013), 52--85. https://doi.org/10.1016/j.artint.2013.06.002
[22]
Quentin Debard, Jilles Steeve Dibangoye, Stéphane Canu, and Christian Wolf. 2020. Learning 3D Navigation Protocols on Touch Interfaces with Cooperative Multi-agent Reinforcement Learning. In Machine Learning and Knowledge Discovery in Databases, Ulf Brefeld, Elisa Fromont, Andreas Hotho, Arno Knobbe, Marloes Maathuis, and Céline Robardet (Eds.). Springer International Publishing, Cham, 35--52.
[23]
Stephanie Denison, Elizabeth Bonawitz, Alison Gopnik, and Thomas L Griffiths. 2013. Rational variability in children's causal inferences: The sampling hypothesis. Cognition 126, 2 (2013), 285--300.
[24]
Andrew T Duchowski, Krzysztof Krejtz, Izabela Krejtz, Cezary Biele, Anna Niedzielska, Peter Kiefer, Martin Raubal, and Ioannis Giannopoulos. 2018. The index of pupillary activity: Measuring cognitive load vis-à-vis task difficulty with pupil oscillation. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1--13.
[25]
Andrew Faulring, Brad Myers, Ken Mohnkern, Bradley Schmerl, Aaron Steinfeld, John Zimmerman, Asim Smailagic, Jeffery Hansen, and Daniel Siewiorek. 2010. Agent-Assisted Task Management That Reduces Email Overload. In Proceedings of the 15th International Conference on Intelligent User Interfaces (Hong Kong, China) (IUI '10). Association for Computing Machinery, New York, NY, USA, 61--70. https://doi.org/10.1145/1719970.1719980
[26]
Stefano Ferretti, Silvia Mirri, Catia Prandi, and Paola Salomoni. 2014. Exploiting reinforcement learning to profile users and personalize web pages. In 2014 IEEE 38th International Computer Software and Applications Conference Workshops. IEEE, 252--257.
[27]
Florian Fischer, Miroslav Bachinski, Markus Klar, Arthur Fleig, and Jörg Müller. 2021. Reinforcement learning control of a biomechanical model of the upper extremity. Scientific Reports 11, 1 (2021), 1--15.
[28]
Paul M Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of experimental psychology 47, 6 (1954), 381.
[29]
Jakob Foerster, Ioannis Alexandros Assael, Nando De Freitas, and Shimon Whiteson. 2016. Learning to communicate with deep multi-agent reinforcement learning. Advances in neural information processing systems 29 (2016).
[30]
Michael J Frank and David Badre. 2012. Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. Cerebral cortex 22, 3 (2012), 509--526.
[31]
Milica Gasić and Steve Young. 2014. Gaussian processes for POMDP-based dialogue manager optimization. IEEE Transactions on Audio, Speech and Language Processing 22, 1 (2014), 28--40. https://doi.org/10.1109/TASL.2013.2282190
[32]
Daniel Gaspar-Figueiredo, Silvia Abrahão, Emilio Insfrán, and Jean Vanderdonckt. 2023. Measuring User Experience of Adaptive User Interfaces using EEG: A Replication Study. In Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. 52--61.
[33]
Daniel Gaspar-Figueiredo, Marta Fernández-Diego, Silvia Abrahao, and Emilio Insfran. 2023. A Comparative Study on Reward Models for UI Adaptation with Reinforcement Learning. methods 13 (2023), 14.
[34]
Christoph Gebhardt, Brian Hecox, Bas van Opheusden, Daniel Wigdor, James Hillis, Otmar Hilliges, and Hrvoje Benko. 2019. Learning Cooperative Personalized Policies from Gaze Data. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST '19). ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3332165.3347933
[35]
Christoph Gebhardt and Otmar Hilliges. 2021. Optimal Control to Support High-Level User Goals in Human-Computer Interaction. In Artificial Intelligence for Human Computer Interaction: A Modern Approach. Springer, 33--72.
[36]
Christoph Gebhardt, Antti Oulasvirta, and Otmar Hilliges. 2021. Hierarchical Reinforcement Learning as a Model of Human Task Interleaving. Computational Brain and Behavior (2021). https://arxiv.org/pdf/2001.02122.pdf
[37]
Daniel Gerber, Urwashi Kapasiya, Lukas Rosenbauer, and Jörg Hähner. 2023. Automation of User Interface Testing by Reinforcement Learning-Based Monkey Agents. In International Conference on Complex Computational Ecosystems. Springer, 3--15.
[38]
Samuel J Gershman, Eric J Horvitz, and Joshua B Tenenbaum. 2015. Computational rationality: A converging paradigm for intelligence in brains, minds, and machines. Science 349, 6245 (2015), 273--278.
[39]
Samuel J Gershman, Edward Vul, and Joshua B Tenenbaum. 2012. Multistability and perceptual inference. Neural computation 24, 1 (2012), 1--24.
[40]
Dorota Glowacka. 2019. Bandit algorithms in recommender systems. In Proceedings of the 13th ACM Conference on Recommender Systems. 574--575.
[41]
Yves Guiard and Olivier Rioul. 2015. A mathematical description of the speed/accuracy trade-off of aimed movement. In Proceedings of the 2015 British HCI Conference. 91--100.
[42]
Tanay Gupta and Julien Gori. 2023. Modeling reciprocal adaptation in HCI: a Multi-Agent Reinforcement Learning Approach. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1--6.
[43]
William E Hick. 1952. On the rate of gain of information. Quarterly Journal of experimental psychology 4, 1 (1952), 11--26.
[44]
Eric Horvitz, Jack Breese, David Heckerman, David Hovel, and Koos Rommelse. 1998. The LumièRe Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence (Madison, Wisconsin) (UAI'98). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 256--265.
[45]
Ronald A Howard. 1960. Dynamic programming and markov processes. (1960).
[46]
Andrew Howes, Xiuli Chen, Aditya Acharya, and Richard L Lewis. 2018. Interaction as an emergent property of a Partially Observable Markov Decision Process. Computational interaction (2018), 287--310.
[47]
Zehong Hu, Yitao Liang, Jie Zhang, Zhao Li, and Yang Liu. 2018. Inference aided reinforcement learning for incentive mechanism design in crowdsourcing. In Advances in Neural Information Processing Systems (NIPS '18). 5508--5518. https://arxiv.org/abs/1806.00206
[48]
Zool Hilmi Ismail and Nohaidda Sariff. 2018. A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions. In Applications of Mobile Robots, Efren Gorrostieta Hurtado (Ed.). IntechOpen, Rijeka, Chapter 1. https://doi.org/10.5772/intechopen.79337
[49]
Max Jaderberg, Wojciech M. Czarnecki, Iain Dunning, Luke Marris, Guy Lever, Antonio Garcia Castañeda, Charles Beattie, Neil C. Rabinowitz, Ari S. Morcos, Avraham Ruderman, Nicolas Sonnerat, Tim Green, Louise Deason, Joel Z. Leibo, David Silver, Demis Hassabis, Koray Kavukcuoglu, and Thore Graepel. 2019. Human-level performance in 3D multiplayer games with population-based reinforcement learning. Science 364, 6443 (2019), 859--865. https://doi.org/10.1126/science.aau6249
[50]
Bonnie E John and Dario D Salvucci. 2005. Multipurpose prototypes for assessing user interfaces in pervasive computing systems. IEEE pervasive computing 4, 4 (2005), 27--34.
[51]
Jussi Jokinen, Aditya Acharya, Mohammad Uzair, Xinhui Jiang, and Antti Oulasvirta. 2021. Touchscreen Typing as Optimal Supervisory Control. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). ACM. https://userinterfaces.aalto.fi/touchscreen-typing/
[52]
Jussi PP Jokinen, Tuomo Kujala, and Antti Oulasvirta. 2021. Multitasking in driving as optimal adaptation under uncertainty. Human factors 63, 8 (2021), 1324--1341.
[53]
Ioannis Kangas, Maud Schwoerer, and Lucas Bernardi. 2022. Scalable User Interface Optimization Using Combinatorial Bandits. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 3375--3379.
[54]
Charles C Kemp, Cressel D Anderson, Hai Nguyen, Alexander J Trevor, and Zhe Xu. 2008. A point-and-click interface for the real world: laser designation of objects for mobile manipulation. In Proceedings of the 3rd ACM/IEEE international conference on human robot interaction. 241--248.
[55]
Davis E Kieras and Davis E Meyer. 1997. An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human-Computer Interaction 12, 4 (1997), 391--438.
[56]
Janin Koch, Andrés Lucero, Lena Hegemann, and Antti Oulasvirta. 2019. May AI? Design ideation with cooperative contextual bandits. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--12.
[57]
Sucheta V Kolekar, Sriram G Sanjeevi, and DS Bormane. 2010. Learning style recognition using artificial neural network for adaptive user interface in e-learning. In 2010 IEEE International conference on computational intelligence and computing research. IEEE, 1--5.
[58]
Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2014. Crowd-powered parameter analysis for visual design exploration. Proceedings of the 27th annual ACM symposium on User interface software and technology - UIST '14 (2014), 65--74. https://doi.org/10.1145/2642918.2647386
[59]
Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2016. SelPh: Progressive Learning and Support of Manual Photo Color Enhancement. Proc. of CHI '16 (2016). https://doi.org/10.1145/2858036.2858111
[60]
Melchor Lafuente, Sonia Elizondo, Unai J Fernández, and Asier Marzo. 2023. Comparing a Mid-air Two-Hand Pinching Point-and-Click Technique with Mouse, Keyboard and TouchFree. In XXIII International Conference on Human Computer Interaction. 1--4.
[61]
Thomas Langerak, Sammy Christen, Anna Maria Feit, and Otmar Hilliges. 2021. Generalizing User Models through Hybrid Hierarchical Control. (2021).
[62]
Yezdi Lashkari, Max Metral, and Pattie Maes. 1997. Collaborative interface agents. Readings in agents (1997), 111--116.
[63]
Joel Z Leibo, Vinicius Zambaldi, Marc Lanctot, Janusz Marecki, and Thore Graepel. 2017. Multi-agent reinforcement learning in sequential social dilemmas. arXiv preprint arXiv:1702.03037 (2017).
[64]
Katri Leino, Kashyap Todi, Antti Oulasvirta, and Mikko Kurimo. 2019. Computer-Supported Form Design Using Keystroke-Level Modeling with Reinforcement Learning. In Proceedings of the 24th International Conference on Intelligent User Interfaces: Companion (Marina del Ray, California) (IUI '19). Association for Computing Machinery, New York, NY, USA, 85--86. https://doi.org/10.1145/3308557.3308704
[65]
Eric Liang, Richard Liaw, Philipp Moritz, Robert Nishihara, Roy Fox, Ken Goldberg, Joseph E. Gonzalez, Michael I. Jordan, and Ion Stoica. 2018. RLlib: Abstractions for Distributed Reinforcement Learning. arXiv:1712.09381 [cs.AI]
[66]
Elad Liebman, Maytal Saar-Tsechansky, and Peter Stone. 2015. DJ-MC: A Reinforcement-Learning Agent for Music Playlist Recommendation. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems (AAMAS '15). 591--599. https://arxiv.org/abs/1401.1880
[67]
David Lindlbauer, Anna Maria Feit, and Otmar Hilliges. 2019. Context-aware online adaptation of mixed reality interfaces. In Proceedings of the 32nd annual ACM symposium on user interface software and technology. 147--160.
[68]
Feng Liu, Ruiming Tang, Xutao Li, Weinan Zhang, Yunming Ye, Haokun Chen, Huifeng Guo, and Yuzhou Zhang. 2018. Deep reinforcement learning based recommendation with explicit user-item interactions modeling. arXiv preprint arXiv:1810.12027 (2018). https://arxiv.org/abs/1810.12027
[69]
J Derek Lomas, Jodi Forlizzi, Nikhil Poonwala, Nirmal Patel, Sharan Shodhan, Kishan Patel, Ken Koedinger, and Emma Brunskill. 2016. Interface design optimization as a multi-armed bandit problem. In Proceedings of the 2016 CHI conference on human factors in computing systems. 4142--4153.
[70]
Qian Long, Zihan Zhou, Abhinav Gupta, Fei Fang, Yi Wu, and Xiaolong Wang. 2020. Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning. In International Conference on Learning Representations.
[71]
Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30 (2017).
[72]
Wendy Mackay. 2000. Responding to cognitive overload: Co-adaptation between users and technology. Intellectica 30, 1 (2000), 177--193.
[73]
Pattie Maes. 1995. Agents that reduce work and information overload. In Readings in human-computer interaction. Elsevier, 811--821.
[74]
Eric McCreath, Judy Kay, and Elisabeth Crawford. 2006. IEMS-an approach that combines handcrafted rules with learnt instance based rules. Aust. J. Intell. Inf. Process. Syst. 9, 1 (2006), 40--53.
[75]
Abhinav Mehrotra and Robert Hendley. 2015. Designing Content-driven Intelligent Notification Mechanisms for Mobile Applications. (2015), 813--824.
[76]
Nesrine Mezhoudi and Jean Vanderdonckt. 2021. Toward a task-driven intelligent GUI adaptation by mixed-initiative. International Journal of Human-Computer Interaction 37, 5 (2021), 445--458.
[77]
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).
[78]
Roderick Murray-Smith, Antti Oulasvirta, Andrew Howes, Jörg Müller, Aleksi Ikkala, Miroslav Bachinski, Arthur Fleig, Florian Fischer, and Markus Klar. 2022. What simulation can do for HCI research. Interactions 29, 6 (2022), 48--53.
[79]
Jun Ota. 2006. Multi-agent robot systems as distributed autonomous systems. Advanced Engineering Informatics 20, 1 (2006), 59--70. https://doi.org/10.1016/j.aei.2005.06.002
[80]
Antti Oulasvirta, Niraj Ramesh Dayama, Morteza Shiripour, Maximilian John, and Andreas Karrenbauer. 2020. Combinatorial Optimization of Graphical User Interface Designs. Proc. IEEE 108, 3 (2020), 434--464. https://doi.org/10.1109/JPROC.2020.2969687
[81]
Antti Oulasvirta, Samuli De Pascale, Janin Koch, Thomas Langerak, Jussi Jokinen, Kashyap Todi, Markku Laine, Manoj Kristhombuge, Yuxi Zhu, Aliaksei Miniukovich, et al. 2018. Aalto interface metrics (AIM) a service and codebase for computational GUI evaluation. In Adjunct Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 16--19.
[82]
Antti Oulasvirta, Anna Feit, Perttu Lähteenlahti, and Andreas Karrenbauer. 2017. Computational Support for Functionality Selection in Interaction Design. 24, 5, Article 34 (oct 2017), 30 pages. https://doi.org/10.1145/3131608
[83]
Antti Oulasvirta, Jussi PP Jokinen, and Andrew Howes. 2022. Computational Rationality as a Theory of Interaction. In CHI Conference on Human Factors in Computing Systems. 1--14.
[84]
Antti Oulasvirta, Per Ola Kristensson, Xiaojun Bi, and Andrew Howes. 2018. Computational interaction. Oxford University Press.
[85]
Seonwook Park, Christoph Gebhardt, Roman Rädle, Anna Maria Feit, Hana Vrzakova, Niraj Ramesh Dayama, Hui-Shyong Yeo, Clemens N Klokmose, Aaron Quigley, Antti Oulasvirta, et al. 2018. Adam: Adapting multi-user interfaces for collaborative environments in real-time. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1--14.
[86]
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825--2830.
[87]
Veljko Pejovic and Mirco Musolesi. 2014. InterruptMe: Designing Intelligent Prompting Mechanisms for Pervasive Applications. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (2014), 897--908. https://doi.org/10.1145/2632048.2632062
[88]
Athanasios S Polydoros and Lazaros Nalpantidis. 2017. Survey of model-based reinforcement learning: Applications on robotics. Journal of Intelligent & Robotic Systems 86, 2 (2017), 153--173.
[89]
Derek Reilly, Michael Welsman-Dinelle, Colin Bate, and Kori Inkpen. 2005. Just point and click? Using handhelds to interact with paper maps. In Proceedings of the 7th international conference on Human computer interaction with mobile devices & services. 239--242.
[90]
Charles Rich, Candy Sidner, Neal Lesh, Andrew Garland, Shane Booth, and Markus Chimani. 2005. DiamondHelp: A collaborative interface framework for networked home appliances. In 25th IEEE International Conference on Distributed Computing Systems Workshops. IEEE, 514--519.
[91]
Charles Rich and Candace L Sidner. 1998. COLLAGEN: A collaboration manager for software interface agents. In Computational Models of Mixed-Initiative Interaction. Springer, 149--184.
[92]
Fabio Rizzoglio, Maura Casadio, Dalia De Santis, and Ferdinando A. Mussa-Ivaldi. 2021. Building an adaptive interface via unsupervised tracking of latent manifolds. Neural Networks 137 (2021), 174--187. https://doi.org/10.1016/j.neunet.2021.01.009
[93]
Dario D Salvucci. 2001. An integrated model of eye movements and visual encoding. Cognitive Systems Research 1, 4 (2001), 201--220.
[94]
Immo Schuetz, T Scott Murdison, Kevin J MacKenzie, and Marina Zannoli. 2019. An Explanation of Fitts' Law-like Performance in Gaze-Based Selection Tasks Using a Psychophysics Approach. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--13.
[95]
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347 [cs.LG]
[96]
Young-Woo Seo and Byoung-Tak Zhang. 2000. A reinforcement learning agent for personalized information filtering. In Proceedings of the 5th international conference on Intelligent user interfaces. 248--251.
[97]
Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P Adams, and Nando De Freitas. 2015. Taking the human out of the loop: A review of Bayesian optimization. Proc. IEEE 104, 1 (2015), 148--175.
[98]
Lloyd S Shapley. 1953. Stochastic games. Proceedings of the national academy of sciences 39, 10 (1953), 1095--1100.
[99]
Jianqiang Shen, Erin Fitzhenry, and Thomas G Dietterich. 2009. Discovering frequent work procedures from resource connections. In Proceedings of the 14th international conference on Intelligent user interfaces. 277--286.
[100]
Jianqiang Shen, Jed Irvine, Xinlong Bao, Michael Goodman, Stephen Kolibaba, Anh Tran, Fredric Carl, Brenton Kirschner, Simone Stumpf, and Thomas G Dietterich. 2009. Detecting and correcting user activity switches: algorithms and interfaces. In Proceedings of the 14th international conference on Intelligent user interfaces. 117--126.
[101]
KGGH Silva, WAPS Abeyasekare, DMHE Dasanayake, TB Nandisena, Dharshana Kasthurirathna, and Archchana Kugathasan. 2021. Dynamic user interface personalization based on deep reinforcement learning. In 2021 3rd International Conference on Advancements in Computing (ICAC). IEEE, 25--30.
[102]
Dustin A Smith and Henry Lieberman. 2010. The why UI: using goal networks to improve user interfaces. In Proceedings of the 15th international conference on Intelligent user interfaces. 377--380.
[103]
Harold Soh, Scott Sanner, Madeleine White, and Greg Jamieson. 2017. Deep sequential recommendation for personalized adaptive user interfaces. In Proceedings of the 22nd international conference on intelligent user interfaces. 589--593.
[104]
Constantine Stephanidis, Charalampos Karagiannidis, and Adamantios Koumpis. 1997. Decision making in intelligent user interfaces. In Proceedings of the 2nd international conference on Intelligent user interfaces. 195--202.
[105]
Pei-Hao Su, Pawel Budzianowski, Stefan Ultes, Milica Gasic, and Steve Young. 2017. Sample-efficient actor-critic reinforcement learning with supervised data for dialogue management. arXiv preprint arXiv:1707.00130 (2017). https://arxiv.org/abs/1707.00130
[106]
Richard S Sutton, Andrew G Barto, et al. 1998. Introduction to reinforcement learning. (1998).
[107]
Zheng Tian, Shihao Zou, Ian Davies, Tim Warr, Lisheng Wu, Haitham Bou Ammar, and Jun Wang. 2020. Learning to communicate implicitly by actions. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 7261--7268.
[108]
Kashyap Todi, Gilles Bailly, Luis Leiva, and Antti Oulasvirta. 2021. Adapting User Interfaces with Model-based Reinforcement Learning. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). ACM. https://userinterfaces.aalto.fi/adaptive/
[109]
Weixun Wang, Tianpei Yang, Yong Liu, Jianye Hao, Xiaotian Hao, Yujing Hu, Yingfeng Chen, Changjie Fan, and Yang Gao. 2020. From few to more: Large-scale dynamic multiagent curriculum learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 7293--7300.
[110]
Zhibo Yang, Lihan Huang, Yupei Chen, Zijun Wei, Seoyoung Ahn, Gregory Zelinsky, Dimitris Samaras, and Minh Hoai. 2020. Predicting goal-directed human attention using inverse reinforcement learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 193--202.
[111]
Neil Yorke-Smith, Shahin Saadati, Karen L Myers, and David N Morley. 2012. The design of a proactive personal agent for task management. International Journal on Artificial Intelligence Tools 21, 01 (2012), 1250004.
[112]
Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. 2022. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems 35 (2022), 24611--24624.
[113]
Chao Yu, Akash Velu, Eugene Vinitsky, Yu Wang, Alexandre Bayen, and Yi Wu. 2021. The surprising effectiveness of ppo in cooperative, multi-agent games. arXiv preprint arXiv:2103.01955 (2021).
[114]
Chao Yu, Minjie Zhang, Fenghui Ren, and Guozhen Tan. 2015. Emotional Multiagent Reinforcement Learning in Spatial Social Dilemmas. IEEE Transactions on Neural Networks and Learning Systems 26, 12 (2015), 3083--3096. https://doi.org/10.1109/TNNLS.2015.2403394
[115]
Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. 2021. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control (2021), 321--384.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 8, Issue EICS
EICS
June 2024
589 pages
EISSN:2573-0142
DOI:10.1145/3673909
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 June 2024
Accepted: 01 April 2024
Revised: 01 April 2024
Received: 01 February 2023
Published in PACMHCI Volume 8, Issue EICS

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Adaptive User Interfaces
  2. Intelligent User Interfaces
  3. Multi-Agent Reinforcement Learning

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 267
    Total Downloads
  • Downloads (Last 12 months)267
  • Downloads (Last 6 weeks)46
Reflects downloads up to 12 Feb 2025

Other Metrics

Citations

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media