“I Want It That Way”: Enabling Interactive Decision Support Using Large Language Models and Constraint Programming
Authors: 
Connor Lawless, 
Jakob Schoeffer, Lindy Le, 
Kael Rowan, Shilad Sen, Cristina St. Hill, Jina Suh, Bahareh SarrafzadehAuthors Info & Claims
Connor Lawless, Jakob Schoeffer, Lindy Le, Kael Rowan, Shilad Sen, Cristina St. Hill, Jina Suh, Bahareh SarrafzadehAuthors Info & ClaimsArticle No.: 22, Pages 1 - 33
Abstract
A critical factor in the success of many decision support systems is the accurate modeling of user preferences. Psychology research has demonstrated that users often develop their preferences during the elicitation process, highlighting the pivotal role of system-user interaction in developing personalized systems. This paper introduces a novel approach, combining Large Language Models (LLMs) with Constraint Programming to facilitate interactive decision support. We study this hybrid framework through the lens of meeting scheduling, a time-consuming daily activity faced by a multitude of information workers. We conduct three studies to evaluate the novel framework, including a diary study to characterize contextual scheduling preferences, a quantitative evaluation of the system’s performance, and a user study to elicit insights with a technology probe that encapsulates our framework. Our work highlights the potential for a hybrid LLM and optimization approach for iterative preference elicitation, and suggests design considerations for building systems that support human-system collaborative decision-making processes.
References
[1]
Marah I. Abdin, Suriya Gunasekar, Varun Chandrasekaran, Jerry Li, Mert Yuksekgonul, Rahee Ghosh Peshawaria, Ranjita Naik, and Besmira Nushi. 2024. KITAB: Evaluating LLMs on constraint satisfaction for information retrieval. In Proceedings of the 12th International Conference on Learning Representations, 1–23.
[2]
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 technical report. arXiv:2303.08774. Retrieved from https://arxiv.org/abs/2303.08774
[3]
Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access 6 (2018), 52138–52160.
[4]
Rakesh Agrawal, Sreenivas Gollapudi, Alan Halverson, and Samuel Ieong. 2009. Diversifying search results. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining, 5–14.
[5]
John A. Aloysius, Fred D. Davis, Darryl D. Wilson, A. Ross Taylor, and Jeffrey E. Kottemann. 2006. User acceptance of multi-criteria decision support systems: The impact of preference elicitation techniques. European Journal of Operational Research 169, 1 (2006), 273–285.
[6]
Liliana Ardissono, Alexander Felfernig, Gerhard Friedrich, Anna Goy, Dietmar Jannach, Giovanna Petrone, Ralph Schafer, and Markus Zanker. 2003. A framework for the development of personalized, distributed web-based configuration systems. AI Magazine 24, 3 (2003), 93–93.
[7]
Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58 (2020), 82–115.
[8]
Michiel Bakker, Martin Chadwick, Hannah Sheahan, Michael Tessler, Lucy Campbell-Gillingham, Jan Balaguer, Nat McAleese, Amelia Glaese, John Aslanides, Matt Botvinick, and Christopher Summerfield. 2022. Fine-tuning language models to find agreement among humans with diverse preferences. Advances in Neural Information Processing Systems 35 (2022), 38176–38189.
[9]
Pauline Berry, Melinda Gervasio, Bart Peintner, and Neil Yorke-Smith. 2007. Balancing the Needs of Personalization and Reasoning in a User-Centric Scheduling Assistant. Technical Report. Artificial Intelligence Center, SRI International.
[10]
Pauline M. Berry, Melinda Gervasio, Bart Peintner, and Neil Yorke-Smith. 2011. PTIME: Personalized assistance for calendaring. ACM Transactions on Intelligent Systems and Technology (TIST) 2, 4 (2011), 1–22.
[11]
Dimitris Bertsimas and John N. Tsitsiklis. 1997. Introduction to Linear Optimization, Vol. 6. Athena Scientific, Belmont, MA.
[12]
James R. Bettman, Mary Frances Luce, and John W. Payne. 1998. Constructive consumer choice processes. Journal of Consumer Research 25, 3 (1998), 187–217.
[13]
Kirsten Boehner, William Gaver, and Andy Boucher. 2012. 14 probes. Inventive Methods 185 (2012), 185–201.
[14]
Kirsten Boehner, Janet Vertesi, Phoebe Sengers, and Paul Dourish. 2007. How HCI interprets the probes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1077–1086.
[15]
Virginia Braun, Victoria Clarke, and Nikki Hayfield. 2023. Thematic analysis: A reflexive approach. In Qualitative Psychology: A Practical Guide to Research Methods. Carla Willig and Wendy Stainton Rogers (Eds.), SAGE Publications, New York, NY.
[16]
Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tramèr. 2022. What does it mean for a language model to preserve privacy?. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 2280–2292.
[17]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 33, 1877–1901.
[18]
Mike Brzozowski, Kendra Carattini, Scott R. Klemmer, Patrick Mihelich, Jiang Hu, and Andrew Y. Ng. 2006. Grouptime: Preference based group scheduling. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1047–1056.
[19]
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv:2303.12712. Retrieved from https://arxiv.org/abs/2303.12712
[20]
Giuseppe Carenini and David Poole. 2002. Constructed preferences and value-focused thinking: Implications for AI research on preference elicitation. In Proceedings of the AAAI Workshop on Preferences in AI and CP: Symbolic Approaches, 1–10.
[21]
Giuseppe Carenini, Jocelyin Smith, and David Poole. 2003. Towards more conversational and collaborative recommender systems. In Proceedings of the 8th International Conference on Intelligent User Interfaces, 12–18.
[22]
Urszula Chajewska, Daphne Koller, and Ronald Parr. 2000. Making rational decisions using adaptive utility elicitation. In AAAI Proceedings. AAAI, New York, NY, 363–369.
[23]
Kathy Charmaz. 2006. Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. Sage, New York, NY.
[24]
Hao Chen, Gonzalo E. Constante-Flores, and Can Li. 2023. Diagnosing infeasible optimization problems using large language models. arXiv:2308.12923. Retrieved from https://arxiv.org/abs/2308.12923
[25]
Li Chen and Pearl Pu. 2004. Survey of Preference Elicitation Methods. Technical Report. EPFL.
[26]
Li Chen and Pearl Pu. 2009. Interaction design guidelines on critiquing-based recommender systems. User Modeling and User-Adapted Interaction 19 (2009), 167–206.
[27]
Li Chen and Pearl Pu. 2012. Critiquing-based recommenders: Survey and emerging trends. User Modeling and User-Adapted Interaction 22 (2012), 125–150.
[28]
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, Wojciech Zaremba 2021. Evaluating large language models trained on code. arXiv:2107.03374. Retrieved from https://arxiv.org/abs/2107.03374
[29]
Justin Cranshaw, Emad Elwany, Todd Newman, Rafal Kocielnik, Bowen Yu, Sandeep Soni, Jaime Teevan, and Andrés Monroy-Hernández. 2017. Calendar.help: Designing a workflow-based scheduling agent with humans in the loop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2382–2393.
[30]
Jared R. Curhan, Margaret A. Neale, and Lee Ross. 2004. Dynamic valuation: Preference changes in the context of face-to-face negotiation. Journal of Experimental Social Psychology 40, 2 (2004), 142–151.
[31]
Parag Pravin Dakle, Serdar Kadi̇oğlu, Karthik Uppuluri, Regina Politi, Preethi Raghavan, SaiKrishna Rallabandi, and Ravisutha Srinivasamurthy. 2023. Ner4Opt: Named entity recognition for optimization modelling from natural language. In Proceedings of the International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research. Springer, 299–319.
[32]
Lisa Dent, Jesus Boticario, John McDermott, Tom Mitchell, and David Zabowski. 1992. A personal learning apprentice. In Proceedings of the Tenth National Conference on Artificial Intelligence, 96–103.
[33]
Victor Dibia. 2023. LIDA: A tool for automatic generation of grammar-agnostic visualizations and infographics using large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), 113–126.
[34]
Jon Doyle. 2004. Prospects for preferences. Computational Intelligence 20, 2 (2004), 111–136.
[35]
Ward Edwards and J. Robert Newman. 1982. Multiattribute evaluation. Quantitative Applications in the Social Sciences 26 (1982), 7–32.
[36]
Bill Gaver, Tony Dunne, and Elena Pacenti. 1999. Design: Cultural probes. Interactions 6, 1 (1999), 21–29.
[37]
Melinda T. Gervasio, Michael D. Moffitt, Martha E. Pollack, Joseph M. Taylor, and Tomas E. Uribe. 2005. Active preference learning for personalized calendar scheduling assistance. In Proceedings of the 10th International Conference on Intelligent User Interfaces, 90–97.
[38]
Judy Goldsmith and Ulrich Junker. 2008. Preference handling for artificial intelligence. AI Magazine 29, 4 (2008), 9.
[39]
Connor Graham and Mark Rouncefield. 2008. Probes and participation. In Proceedings of the 10th Anniversary Conference on Participatory Design 2008. ACM, New York, NY, 194–197.
[40]
Robin Gregory, Sarah Lichtenstein, and Paul Slovic. 1993. Valuing environmental resources: A constructive approach. Journal of Risk and Uncertainty 7, 2 (1993), 177–197.
[41]
John S Hammond, Ralph L Keeney, and Howard Raiffa. 2015. Smart Choices: A Practical Guide to Making Better Decisions. Harvard Business Review Press, Cambridge, MA.
[42]
Thomas Haynes, Sandip Sen, Neeraj Arora, and Rajani Nadella. 1997. An automated meeting scheduling system that utilizes user preferences. In Proceedings of the 1st International Conference on Autonomous Agents. ACM, New York, NY, 308–315.
[43]
Kenneth Holstein, Maria De-Arteaga, Lakshmi Tumati, and Yanghuidi Cheng. 2023. Toward supporting perceptual complementarity in human-AI collaboration via reflection on unobservables. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1–20.
[44]
Hilary Hutchinson, Wendy Mackay, Bo Westerlund, Benjamin B. Bederson, Allison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, Helen Evans, Heiko Hansen, Nicolas Roussel, and Björn Eiderbäck. 2003. Technology probes: Inspiring design for and with families. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 17–24.
[45]
Irving L. Janis and Leon Mann. 1977. Decision Making: A Psychological Analysis of Conflict, Choice, and Commitment. Free Press, New York, NY.
[46]
Dietmar Jannach, Ahtsham Manzoor, Wanling Cai, and Li Chen. 2021. A survey on conversational recommender systems. ACM Computing Surveys (CSUR) 54, 5 (2021), 1–36.
[47]
Eric J. Johnson, Mary Steffel, and Daniel G. Goldstein. 2005. Making better decisions: From measuring to constructing preferences. Health Psychology 24, 4S (2005), S17.
[48]
Ralph L. Keeney and Howard Raiffa. 1993. Decisions with Multiple Objectives: Preferences and Value Trade-Offs. Cambridge University Press, Cambridge, UK.
[49]
Donghyeon Kim, Jinhyuk Lee, Donghee Choi, Jaehoon Choi, and Jaewoo Kang. 2018. Learning user preferences and understanding calendar contexts for event scheduling. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. ACM, New York, NY, 337–346.
[50]
Alfred Krzywicki, Wayne Wobcke, and Anna Wong. 2010. An adaptive calendar assistant using pattern mining for user preference modelling. In Proceedings of the 15th International Conference on Intelligent User Interfaces. ACM, New York, NY, 71–80.
[51]
Markus Langer, Daniel Oster, Timo Speith, Holger Hermanns, Lena Kästner, Eva Schmidt, Andreas Sesing, and Kevin Baum. 2021. What do we want from explainable artificial intelligence (XAI)? A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research. Artificial Intelligence 296 (2021), 103473.
[52]
Seth Lazar. 2024. Legitimacy, authority, and democratic duties of explanation. Oxford Studies in Political Philosophy 10 (2024), 28.
[53]
Sarah Lebovitz, Hila Lifshitz-Assaf, and Natalia Levina. 2022. To engage or not to engage with AI for critical judgments: How professionals deal with opacity when using AI for medical diagnosis. Organization Science 33, 1 (2022), 126–148.
[54]
John D. Lee and Katrina A. See. 2004. Trust in automation: Designing for appropriate reliance. Human Factors 46, 1 (2004), 50–80.
[55]
Beibin Li, Konstantina Mellou, Bo Zhang, Jeevan Pathuri, and Ishai Menache. 2023. Large language models for supply chain optimization. arXiv:2307.03875. Retrieved from https://arxiv.org/abs/2307.03875
[56]
Belinda Z. Li, Alex Tamkin, Noah Goodman, and Jacob Andreas. 2023. Eliciting human preferences with language models. DOI:
[57]
Sarah Lichtenstein and Paul Slovic. 2006. The Construction of Preference. Cambridge University Press.
[58]
Jessy Lin, Nicholas Tomlin, Jacob Andreas, and Jason Eisner. 2024. Decision-oriented dialogue for human-AI collaboration. In Proceedings of the ICLR 2024 Workshop on Large Language Model (LLM) Agents, 1–19.
[59]
Lorraine McGinty and James Reilly. 2010. On the evolution of critiquing recommenders. In Proceedings of the Recommender Systems Handbook. Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor (Eds.), Springer, New York, NY, 419–453.
[60]
Grégoire Mialon, Roberto Dessi, Maria Lomeli, Christoforos Nalmpantis, Ramakanth Pasunuru, Roberta Raileanu, Baptiste Roziere, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, and Thomas Scialom. 2024. Augmented language models: A survey. Transactions on Machine Learning Research (2024).
[61]
Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 267 (2019), 1–38.
[62]
Judson Mills and Edgar O’Neal. 1971. Anticipated choice, attention, and halo effect. Psychonomic Science 22, 4 (1971), 231–233.
[63]
Tom M. Mitchell, Rich Caruana, Dayne Freitag, John McDermott, and David Zabowski. 1994. Experience with a learning personal assistant. Communications of the ACM 37, 7 (1994), 80–91.
[64]
Lillio Mok, Lu Sun, Shilad Sen, and Bahareh Sarrafzadeh. 2023. Challenging but connective: Large-scale characteristics of synchronous collaboration across time zones. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1–17.
[65]
Henry Montgomery and Helena Willén. 2007. Decision making and action: The search for a good structure. In Judgement and Decision Making. Peter Juslin and Henry Montgomery (Eds.), Psychology Press, 147–173.
[66]
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. 2021. WebGPT: Browser-assisted question-answering with human feedback. arXiv:2112.09332. Retrieved from https://arxiv.org/abs/2112.09332
[67]
Yuting Ning, Jiayu Liu, Longhu Qin, Tong Xiao, Shangzi Xue, Zhenya Huang, Qi Liu, Enhong Chen, and Jinze Wu. 2023. A novel approach for auto-formulation of optimization problems. arXiv:2302.04643. Retrieved from https://arxiv.org/abs/2302.04643
[68]
Jean Oh and Stephen F. Smith. 2005. Calendar assistants that learn preferences. In Proceedings of the AAAI Spring Symposium: Persistent Assistants: Living and Working With AI, 7–13.
[69]
John W. Payne, James R. Bettman, and Eric J. Johnson. 1992. Behavioral decision research: A constructive processing perspective. Annual Review of Psychology 43, 1 (1992), 87–131.
[70]
John W. Payne, James R. Bettman, and Eric J. Johnson. 1993. The Adaptive Decision Maker. Cambridge University Press, Cambridge, UK.
[71]
John W. Payne, James R. Bettman, David A. Schkade, Norbert Schwarz, and Robin Gregory. 2000. Measuring constructed preferences: Towards a building code. Elicitation of Preferences 19 (2000), 243–275.
[72]
Bart Peintner, Paolo Viappiani, and Neil Yorke-Smith. 2008. Preferences in interactive systems: Technical challenges and case studies. AI Magazine 29, 4 (2008), 13–13.
[73]
Alina Pommeranz, Joost Broekens, Pascal Wiggers, Willem-Paul Brinkman, and Catholijn M. Jonker. 2012. Designing interfaces for explicit preference elicitation: A user-centered investigation of preference representation and elicitation process. User Modeling and User-Adapted Interaction 22 (2012), 357–397.
[74]
Ganesh Prasath and Shirish Karande. 2023. Synthesis of mathematical programs from natural language specifications. arXiv:2304.03287. Retrieved from https://arxiv.org/abs/2304.03287
[75]
Pearl Pu, Boi Faltings, and Marc Torrens. 2003. User-Involved Preference Elicitation. Technical Report. EPFL.
[76]
Jing Qian, Hong Wang, Zekun Li, Shiyang Li, and Xifeng Yan. 2023. Limitations of language models in arithmetic and symbolic induction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 9285–9298.
[77]
Rindra Ramamonjison, Haley Li, Timothy Yu, Shiqi He, Vishnu Rengan, Amin Banitalebi-Dehkordi, Zirui Zhou, and Yong Zhang. 2022. Augmenting operations research with auto-formulation of optimization models from problem descriptions. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track, 29–62.
[78]
Marcel K. Richter. 1966. Revealed preference theory. Econometrica: Journal of the Econometric Society 34 (1966), 635–645.
[79]
Francesca Rossi. 1999. Constraint (logic) programming: A survey on research and applications. In Compulog Net/ERCIM Workshop on Constraints. Krzysztof R. Apt, Eric Monfroy, Antonis C. Kakas, and Francesca Rossi (Eds.), Springer, New York, NY, 40–74.
[80]
J. Edward Russo, Victoria Husted Medvec, and Margaret G. Meloy. 1996. The distortion of information during decisions. Organizational Behavior and Human Decision Processes 66, 1 (1996), 102–110.
[81]
Max Schemmer, Niklas Kuehl, Carina Benz, Andrea Bartos, and Gerhard Satzger. 2023. Appropriate reliance on AI advice: Conceptualization and the effect of explanations. In Proceedings of the 28th International Conference on Intelligent User Interfaces, 410–422.
[82]
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2024. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems 36 (2024), 1–13.
[83]
Jakob Schoeffer, Maria De-Arteaga, and Niklas Kuehl. 2024. Explanations, fairness, and appropriate reliance in human-AI decision-making. In Proceedings of the CHI Conference on Human Factors in Computing Systems, 1–18.
[84]
Jakob Schoeffer, Johannes Jakubik, Michael Voessing, Niklas Kuehl, and Gerhard Satzger. 2023. On the interdependence of reliance behavior and accuracy in AI-assisted decision-making. In Proceedings of the : Augmenting Human Intellect (HHAI ’23). IOS Press, 46–59.
[85]
Sandip Sen, Thomas Haynes, and Neeraj Arora. 1997. Satisfying user preferences while negotiating meetings. International Journal of Human-Computer Studies 47, 3 (1997), 407–427.
[86]
Dan Simon, Daniel C. Krawczyk, Airom Bleicher, and Keith J. Holyoak. 2008. The transience of constructed preferences. Journal of Behavioral Decision Making 21, 1 (2008), 1–14.
[87]
Paul Slovic. 1995. The construction of preference. American Psychologist 50 (1995), 364–371.
[88]
Lu Sun, Lillio Mok, Shilad Sen, and Bahar Sarrafzadeh. 2024. Rhythm of work: Mixed-methods characterization of information workers scheduling preferences and practices. In Proceedings of the 27th ACM Conference on Computer-Supported Cooperative Work and Social Computing, 1–34.
[89]
Atena M. Tabakhi. 2017. Preference elicitation in DCOPs for scheduling devices in smart buildings. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI, New York, NY, 64–78.
[90]
Atena M. Tabakhi. 2021. Preference Elicitation in Constraint-Based Models: Models, Algorithms, and Applications. Ph.D. Dissertation. Washington University in St. Louis.
[91]
Atena M Tabakhi, Tiep Le, Ferdinando Fioretto, and William Yeoh. 2017. Preference elicitation for DCOPs. In Principles and Practice of Constraint Programming. Springer, New York, NY, 278–296.
[92]
Atena M Tabakhi, William Yeoh, and Roie Zivan. 2022. Incomplete distributed constraint optimization problems: Model, algorithms, and heuristics. In Distributed Artificial Intelligence. Springer, New York, NY, 64–78.
[93]
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoc Le. 2022. LaMDA: Language models for dialog applications. arXiv:2201.08239. Retrieved from https://arxiv.org/abs/2201.08239
[94]
Dimos Tsouros, Hélène Verhaeghe, Serdar Kadi̇oğlu, and Tias Guns. 2023. Holy Grail 2.0: From natural language to constraint models. arXiv:2308.01589. Retrieved from https://arxiv.org/abs/2308.01589
[95]
Joe Tullio, Jeremy Goecks, Elizabeth D. Mynatt, and David H. Nguyen. 2002. Augmenting shared personal calendars. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, 11–20.
[96]
Paolo Viappiani, Boi Faltings, and Pearl Pu. 2006. Evaluating preference-based search tools: A tale of two approaches. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI ’06). AAAI Press, 205–211.
[97]
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 35, 24824–24837.
[98]
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2020. Neural text generation with unlikelihood training. In Proceedings of the International Conference on Learning Representations, 1–18.
[99]
Yuanming Xiao. 2020. Embedding Preference Elicitation Within the Search for DCOP Solutions. Ph.D. Dissertation. Washington University in St. Louis.
[100]
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 36, 1–14.
[101]
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing reasoning and acting in language models. In Proceedings of the The 11th International Conference on Learning Representations.
[102]
Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: Story writing with large language models. In Proceedings of the 27th International Conference on Intelligent User Interfaces. ACM, New York, NY, 841–852.
[103]
Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi, Ece Kamar, and Besmira Nushi. 2024. Attention satisfies: A constraint-satisfaction lens on factual errors of language models. In Proceedings of the 12th International Conference on Learning Representations, 1–25.
Index Terms
- “I Want It That Way”: Enabling Interactive Decision Support Using Large Language Models and Constraint Programming
Recommendations
Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Making
AAMAS '18: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent SystemsIn multi-objective decision planning and learning, much attention is paid to producing optimal solution sets that contain an optimal policy for every possible user preference profile. We argue that the step that follows, i.e, determining which policy to ...
Large language models present new questions for decision support
AbstractLarge language models (LLMs) have proven capable of assisting with many aspects of organizational decision making, such as helping to collect information from databases and helping to brainstorm possible courses of action ahead of making a ...
Highlights- We discuss how language models might change decision support systems.
- We argue large language models are more versatile than prior decision support tools.
- We propose that such versatility introduces new questions during decision ...
Decision support for preference elicitation in multi-attribute electronic procurement auctions through an agent-based intermediary
Multi-attribute auctions have become increasingly popular in enterprise procurement. In the auctions, the elicitation of the preference of an auctioneer concerning multiple attributes is a central task in determining the winner(s). Considering the ...
Comments
Information & Contributors
Information
Published In
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 September 2024
Online AM: 01 August 2024
Accepted: 19 June 2024
Revised: 14 May 2024
Received: 02 February 2024
Published in TIIS Volume 14, Issue 3
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 542Total Downloads
- Downloads (Last 12 months)542
- Downloads (Last 6 weeks)138
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in