skip to main content
10.1145/3544548.3581469acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

RetroLens: A Human-AI Collaborative System for Multi-step Retrosynthetic Route Planning

Published: 19 April 2023 Publication History

Abstract

Multi-step retrosynthetic route planning (MRRP) is the core task in synthetic chemistry, in which chemists recursively deconstruct a target molecule to find a set of reactants that make up the target. MRRP is challenging in that the search space is vast, and chemists are often lost in the process. Existing AI models can achieve automatic MRRP fast, but they only work on relatively simple targets, which leaves complex molecules under chemists’ expertise. To facilitate MRRP of complex molecules, we proposed a human-AI collaborative system, RetroLens, through a participatory design process. AI can contribute by two approaches: joint action and algorithm-in-the-loop. Deconstruction steps are allocated to chemists or AI based on their capabilities and AI recommends candidate revision steps to fix problems along the way. A within-subjects study (N=18) showed that chemists who used RetroLens reported faster MRRP, broader design space exploration, higher confidence in their planning, and lower cognitive load.

Supplementary Material

Supplemental Materials (3544548.3581469-supplemental-materials.zip)
MP4 File (3544548.3581469-talk-video.mp4)
Pre-recorded Video Presentation

References

[1]
Laura KG Ackerman-Biegasiewicz, Daniela M Arias-Rotondo, Kyle F Biegasiewicz, Elizabeth Elacqua, Matthew R Golder, Laure V Kayser, Jessica R Lamb, Christine M Le, Nathan A Romero, Sidney M Wilkerson-Hill, 2020. Organic Chemistry: A Retrosynthetic Approach to a Diverse Field., 1845–1850 pages.
[2]
Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.
[3]
Zahra Ashktorab, Q Vera Liao, Casey Dugan, James Johnson, Qian Pan, Wei Zhang, Sadhana Kumaravel, and Murray Campbell. 2020. Human-ai collaboration in a cooperative game setting: Measuring social perception and outcomes. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2(2020), 1–20.
[4]
Atanas G Atanasov, Sergey B Zotchev, Verena M Dirsch, and Claudiu T Supuran. 2021. Natural products in drug discovery: advances and opportunities. Nature Reviews Drug Discovery 20, 3 (2021), 200–216.
[5]
Tomasz Badowski, Karol Molga, and Bartosz A Grzybowski. 2019. Selection of cost-effective yet chemically diverse pathways from the networks of computer-generated retrosynthetic plans. Chemical Science 10, 17 (2019), 4640–4651.
[6]
Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16.
[7]
Javier L Baylon, Nicholas A Cilfone, Jeffrey R Gulcher, and Thomas W Chittenden. 2019. Enhancing retrosynthetic reaction prediction with deep learning using multiscale reaction classification. Journal of Chemical Information and Modeling 59, 2 (2019), 673–688.
[8]
William Bort, Igor I Baskin, Timur Gimadiev, Artem Mukanov, Ramil Nugmanov, Pavel Sidorov, Gilles Marcou, Dragos Horvath, Olga Klimchuk, Timur Madzhidov, 2021. Discovery of novel chemical reactions by deep generative recurrent neural network. Scientific Reports 11, 1 (2021), 1–15.
[9]
Riccardo Bovo, Nicola Binetti, Duncan P Brumby, and Simon Julier. 2020. Detecting errors in pick and place procedures: detecting errors in multi-stage and sequence-constrained manual retrieve-assembly procedures. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 536–545.
[10]
Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health 11, 4 (2019), 589–597.
[11]
Zana Buçinca, Phoebe Lin, Krzysztof Z Gajos, and Elena L Glassman. 2020. Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 454–464.
[12]
Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1(2021), 1–21.
[13]
Carrie J Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. " Hello AI": uncovering the onboarding needs of medical practitioners for human-AI collaborative decision-making. Proceedings of the ACM on Human-computer Interaction 3, CSCW(2019), 1–24.
[14]
Liwei Chan, Yi-Chi Liao, George B Mo, John J Dudley, Chun-Lien Cheng, Per Ola Kristensson, and Antti Oulasvirta. 2022. Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques. In CHI Conference on Human Factors in Computing Systems. 1–14.
[15]
Binghong Chen, Chengtao Li, Hanjun Dai, and Le Song. 2020. Retro*: learning retrosynthetic planning with neural guided A* search. In International Conference on Machine Learning. PMLR, 1608–1616.
[16]
Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. 2019. This looks like that: deep learning for interpretable image recognition. Advances in Neural Information Processing Systems 32 (2019).
[17]
Lydia B Chilton, Juho Kim, Paul André, Felicia Cordeiro, James A Landay, Daniel S Weld, Steven P Dow, Robert C Miller, and Haoqi Zhang. 2014. Frenzy: collaborative data organization for creating conference sessions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1255–1264.
[18]
Penny Chong, Ngai-Man Cheung, Yuval Elovici, and Alexander Binder. 2021. Toward Scalable and Unified Example-Based Explanation and Outlier Detection. IEEE Transactions on Image Processing 31 (2021), 525–540.
[19]
Connor W Coley, Luke Rogers, William H Green, and Klavs F Jensen. 2018. SCScore: synthetic complexity learned from a reaction corpus. Journal of Chemical Information and Modeling 58, 2 (2018), 252–261.
[20]
Elias James Corey. 1967. General methods for the construction of complex molecules. Pure and Applied chemistry 14, 1 (1967), 19–38.
[21]
Hanjun Dai, Chengtao Li, Connor Coley, Bo Dai, and Le Song. 2019. Retrosynthesis prediction with conditional graph logic network. Advances in Neural Information Processing Systems 32 (2019).
[22]
Kahneman Daniel. 2017. Thinking, fast and slow.
[23]
Ian W Davies. 2019. The digitization of organic synthesis. Nature 570, 7760 (2019), 175–181.
[24]
John J Dudley and Per Ola Kristensson. 2018. A review of user interface design for interactive machine learning. ACM Transactions on Interactive Intelligent Systems (TiiS) 8, 2(2018), 1–37.
[25]
Jerry Alan Fails and Dan R Olsen Jr. 2003. Interactive machine learning. In Proceedings of the 8th International Conference on Intelligent User Interfaces. 39–45.
[26]
William Finnigan, Sabine L Flitsch, Lorna J Hepworth, and Nicholas J Turner. 2021. Enzyme Cascade Design: Retrosynthesis Approach. In Enzyme Cascade Design and Modelling. Springer, 7–30.
[27]
Susanne Gaube, Harini Suresh, Martina Raue, Alexander Merritt, Seth J Berkowitz, Eva Lermer, Joseph F Coughlin, John V Guttag, Errol Colak, and Marzyeh Ghassemi. 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ Digital Medicine 4, 1 (2021), 1–8.
[28]
Samuel Genheden, Amol Thakkar, Veronika Chadimová, Jean-Louis Reymond, Ola Engkvist, and Esben Bjerrum. 2020. AiZynthFinder: a fast, robust and flexible open-source software for retrosynthetic planning. Journal of Cheminformatics 12, 1 (2020), 1–9.
[29]
Subhash Ghosh and Tapan Kumar Pradhan. 2008. The first total synthesis of emericellamide A. Tetrahedron Letters 49, 22 (2008), 3697–3700.
[30]
Samuel Gratzl, Alexander Lex, Nils Gehlenborg, Hanspeter Pfister, and Marc Streit. 2013. Lineup: Visual analysis of multi-attribute rankings. IEEE Transactions on Visualization and Computer Graphics 19, 12(2013), 2277–2286.
[31]
Ben Green and Yiling Chen. 2019. The principles and limits of algorithm-in-the-loop decision making. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–24.
[32]
Bartosz A Grzybowski, Sara Szymkuć, Ewa P Gajewska, Karol Molga, Piotr Dittwald, Agnieszka Wołos, and Tomasz Klucznik. 2018. Chematica: a story of computer code that started to think like a chemist. Chem 4, 3 (2018), 390–398.
[33]
Corina Haita-Falah. 2017. Sunk-cost fallacy and cognitive ability in individual decision-making. Journal of Economic Psychology 58 (2017), 44–59.
[34]
Peng Han, Peilin Zhao, Chan Lu, Junzhou Huang, Jiaxiang Wu, Shuo Shang, Bin Yao, and Xiangliang Zhang. 2022. GNN-Retro: Retrosynthetic Planning with Graph Neural Networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 4014–4021.
[35]
Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in Psychology. Vol. 52. Elsevier, 139–183.
[36]
Peter Hase, Chaofan Chen, Oscar Li, and Cynthia Rudin. 2019. Interpretable image recognition with hierarchical prototypes. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 32–40.
[37]
Shoichi Ishida, Kei Terayama, Ryosuke Kojima, Kiyosei Takasu, and Yasushi Okuno. 2022. AI-Driven Synthetic Route Design Incorporated with Retrosynthesis Knowledge. Journal of Chemical Information and Modeling 62, 6 (2022), 1357–1367.
[38]
Ellen Jiang, Edwin Toh, Alejandra Molina, Kristen Olson, Claire Kayacik, Aaron Donsbach, Carrie J Cai, and Michael Terry. 2022. Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models. In CHI Conference on Human Factors in Computing Systems. 1–19.
[39]
Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. 2018. To trust or not to trust a classifier. Advances in Neural Information Processing Systems 31 (2018).
[40]
Ekaterina Jussupow, Izak Benbasat, and Armin Heinzl. 2020. Why are we averse towards algorithms? A comprehensive literature review on algorithm aversion. (2020).
[41]
Bo Kang, Arun Kulshreshth, and Joseph J LaViola Jr. 2016. Analyticalink: An interactive learning environment for math word problem solving. In Proceedings of the 21st International Conference on Intelligent User Interfaces. 419–430.
[42]
Pavel Karpov, Guillaume Godin, and Igor V Tetko. 2019. A transformer model for retrosynthesis. In International Conference on Artificial Neural Networks. Springer, 817–830.
[43]
Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. 2020. Interpreting interpretability: understanding data scientists’ use of interpretability tools for machine learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.
[44]
Harmanpreet Kaur, Alex Williams, and WS Lasecki. 2019. Building shared mental models between humans and ai for effective collaboration. CHI’19, May 2019, Glasgow, Scotland(2019).
[45]
Junhan Kim, Jana Muhic, Lionel Peter Robert, and Sun Young Park. 2022. Designing Chatbots with Black Americans with Chronic Conditions: Overcoming Challenges against COVID-19. In CHI Conference on Human Factors in Computing Systems. 1–17.
[46]
Tomasz Klucznik, Barbara Mikulak-Klucznik, Michael P McCormack, Heather Lima, Sara Szymkuć, Manishabrata Bhowmick, Karol Molga, Yubai Zhou, Lindsey Rickershauser, Ewa P Gajewska, 2018. Efficient syntheses of diverse, medicinally relevant targets planned by computer and executed in the laboratory. Chem 4, 3 (2018), 522–532.
[47]
Levente Kocsis and Csaba Szepesvári. 2006. Bandit based monte-carlo planning. In European Conference on Machine Learning. Springer, 282–293.
[48]
Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In International Conference on Machine Learning. PMLR, 1885–1894.
[49]
Caitlin Kuhlman, Diana Doherty, Malika Nurbekova, Goutham Deva, Zarni Phyo, Paul-Henry Schoenhagen, MaryAnn VanValkenburg, Elke Rundensteiner, and Lane Harrison. 2019. Evaluating preference collection methods for interactive ranking analytics. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11.
[50]
Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q Vera Liao, Yunfeng Zhang, and Chenhao Tan. 2022. Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation. In CHI Conference on Human Factors in Computing Systems. 1–18.
[51]
Vivian Lai, Han Liu, and Chenhao Tan. 2020. " Why is’ Chicago’deceptive?" Towards Building Model-Driven Tutorials for Humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[52]
Vivian Lai and Chenhao Tan. 2019. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 29–38.
[53]
Greg Landrum 2013. RDKit: A software suite for cheminformatics, computational chemistry, and predictive modeling.
[54]
Min Hun Lee, Daniel P Siewiorek, Asim Smailagic, Alexandre Bernardino, and Sergi Bermúdez i Badia. 2021. A human-ai collaborative approach for clinical decision making on rehabilitation assessment. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
[55]
Yang Li, Sayan Sarcar, Yilin Zheng, and Xiangshi Ren. 2021. Exploring Text Revision with Backspace and Caret in Virtual Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12.
[56]
Kangjie Lin, Youjun Xu, Jianfeng Pei, and Luhua Lai. 2020. Automatic retrosynthetic route planning using template-free models. Chemical Science 11, 12 (2020), 3355–3364.
[57]
Bowen Liu, Bharath Ramsundar, Prasad Kawthekar, Jade Shi, Joseph Gomes, Quang Luu Nguyen, Stephen Ho, Jack Sloane, Paul Wender, and Vijay Pande. 2017. Retrosynthetic reaction prediction using neural sequence-to-sequence models. ACS Central Science 3, 10 (2017), 1103–1113.
[58]
Ryan Louie, Andy Coenen, Cheng Zhi Huang, Michael Terry, and Carrie J Cai. 2020. Novice-AI music co-creation via AI-steering tools for deep generative models. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[59]
Hao Lü and Yang Li. 2012. Gesture coder: a tool for programming multi-touch gestures by demonstration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2875–2884.
[60]
Shuai Ma, Mingfei Sun, and Xiaojuan Ma. 2022. Modeling Adaptive Expression of Robot Learning Engagement and Exploring its Effects on Human Teachers. ACM Transactions on Computer-Human Interaction (2022).
[61]
Shuai Ma, Zijun Wei, Feng Tian, Xiangmin Fan, Jianming Zhang, Xiaohui Shen, Zhe Lin, Jin Huang, Radomír Měch, Dimitris Samaras, 2019. SmartEye: assisting instant photo taking via integrating user preference with deep view proposal network. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–12.
[62]
Shuai Ma, Taichang Zhou, Fei Nie, and Xiaojuan Ma. 2022. Glancee: An adaptable system for instructors to grasp student learning status in synchronous online classes. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–25.
[63]
Maximilian Mackeprang, Claudia Müller-Birn, and Maximilian Timo Stauss. 2019. Discovering the sweet spot of human-computer configurations: A case study in information extraction. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–30.
[64]
Barbara Mikulak-Klucznik, Patrycja Gołębiowska, Alison A Bayly, Oskar Popik, Tomasz Klucznik, Sara Szymkuć, Ewa P Gajewska, Piotr Dittwald, Olga Staszewska-Krajewska, Wiktor Beker, 2020. Computational planning of the synthesis of complex natural products. Nature 588, 7836 (2020), 83–88.
[65]
Swati Mishra and Jeffrey M Rzeszotarski. 2021. Designing Interactive Transfer Learning Tools for ML Non-Experts. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.
[66]
Ramaravind K Mothilal, Amit Sharma, and Chenhao Tan. 2020. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 607–617.
[67]
Vishnu H Nair, Philippe Schwaller, and Teodoro Laino. 2019. Data-driven chemical reaction prediction and retrosynthesis. CHIMIA International Journal for Chemistry 73, 12 (2019), 997–1000.
[68]
Stephan Pajer, Marc Streit, Thomas Torsney-Weir, Florian Spechtenhauser, Torsten Möller, and Harald Piringer. 2016. Weightlifter: Visual weight space exploration for multi-criteria decision making. IEEE Transactions on Visualization and Computer Graphics 23, 1(2016), 611–620.
[69]
Joon Sung Park, Rick Barber, Alex Kirlik, and Karrie Karahalios. 2019. A slow algorithm improves users’ assessments of the algorithm’s accuracy. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–15.
[70]
Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. 2021. Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–52.
[71]
Koji Satoh and Kimito Funatsu. 1999. A novel approach to retrosynthetic analysis using knowledge bases derived from reaction databases. Journal of Chemical Information and Computer Sciences 39, 2 (1999), 316–325.
[72]
John S Schreck, Connor W Coley, and Kyle JM Bishop. 2019. Learning retrosynthetic planning through simulated experience. ACS Central Science 5, 6 (2019), 970–981.
[73]
Philippe Schwaller, Riccardo Petraglia, Valerio Zullo, Vishnu H Nair, Rico Andreas Haeuselmann, Riccardo Pisoni, Costas Bekas, Anna Iuliano, and Teodoro Laino. 2020. Predicting retrosynthetic pathways using transformer-based models and a hyper-graph exploration strategy. Chemical Science 11, 12 (2020), 3316–3325.
[74]
Oliver B Scott and AW Edith Chan. 2020. ScaffoldGraph: an open-source library for the generation and analysis of molecular scaffold networks and scaffold trees. Bioinformatics 36, 12 (2020), 3930–3931.
[75]
Marwin HS Segler, Mike Preuss, and Mark P Waller. 2018. Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555, 7698 (2018), 604–610.
[76]
Marwin HS Segler and Mark P Waller. 2017. Modelling chemical reasoning to predict and invent reactions. Chemistry–A European Journal 23, 25 (2017), 6118–6128.
[77]
Seung-Woo Seo, You Young Song, June Yong Yang, Seohui Bae, Hankook Lee, Jinwoo Shin, Sung Ju Hwang, and Eunho Yang. 2021. GTA: Graph truncated attention for retrosynthesis. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 531–539.
[78]
Chuhan Shi, Zhihan Jiang, Xiaojuan Ma, and Qiong Luo. 2022. A Personalized Visual Aid for Selections of Appearance Building Products with Long-term Effects. In CHI Conference on Human Factors in Computing Systems. 1–18.
[79]
Chence Shi, Minkai Xu, Hongyu Guo, Ming Zhang, and Jian Tang. 2020. A graph to graphs framework for retrosynthesis prediction. In International Conference on Machine Learning. PMLR, 8818–8827.
[80]
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484–489.
[81]
Teague Sterling and John J Irwin. 2015. ZINC 15–ligand discovery for everyone. Journal of Chemical Information and Modeling 55, 11 (2015), 2324–2337.
[82]
Sarah Jean Stevens. 2011. Progress toward the synthesis of providencin. Ph. D. Dissertation. Colorado State University.
[83]
Dong Sun, Renfei Huang, Yuanzhe Chen, Yong Wang, Jia Zeng, Mingxuan Yuan, Ting-Chuen Pong, and Huamin Qu. 2019. PlanningVis: A visual analytics approach to production planning in smart factories. IEEE Transactions on Visualization and Computer Graphics 26, 1(2019), 579–589.
[84]
Sara Szymkuć, Ewa P Gajewska, Tomasz Klucznik, Karol Molga, Piotr Dittwald, Michał Startek, Michał Bajczyk, and Bartosz A Grzybowski. 2016. Computer-assisted synthetic planning: the end of the beginning. Angewandte Chemie International Edition 55, 20 (2016), 5904–5937.
[85]
Chunqiao Tan and Xiaohong Chen. 2010. Intuitionistic fuzzy Choquet integral operator for multi-criteria decision making. Expert Systems with Applications 37, 1 (2010), 149–157.
[86]
Stefano Teso and Kristian Kersting. 2019. Explanatory interactive machine learning. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 239–245.
[87]
Igor V Tetko, Pavel Karpov, Ruud Van Deursen, and Guillaume Godin. 2020. State-of-the-art augmented NLP transformer models for direct and single-step retrosynthesis. Nature Communications 11, 1 (2020), 1–11.
[88]
Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. Interpretable predictions of tree-based ensembles via actionable feature tweaking. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 465–474.
[89]
Chun-Hua Tsai, Yue You, Xinning Gui, Yubo Kou, and John M Carroll. 2021. Exploring and promoting diagnostic transparency and explainability in online symptom checkers. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–17.
[90]
Kurt VanLehn. 2016. Regulative loops, step loops and task loops. International Journal of Artificial Intelligence in Education 26, 1(2016), 107–112.
[91]
Chandrabhan Verma, LO Olasunkanmi, Eno E Ebenso, and MA Quraishi. 2018. Substituents effect on corrosion inhibition performance of organic compounds in aggressive ionic solutions: a review. Journal of Molecular Liquids 251 (2018), 100–118.
[92]
Emily Wall, Subhajit Das, Ravish Chawla, Bharath Kalidindi, Eli T Brown, and Alex Endert. 2017. Podium: Ranking data using mixed-initiative visual analytics. IEEE Transactions on Visualization and Computer Graphics 24, 1(2017), 288–297.
[93]
Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y Lim. 2019. Designing theory-driven user-centric explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–15.
[94]
Yiting Wang, Walker M White, and Erik Andersen. 2017. Pathviewer: Visualizing pathways through student data. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 960–964.
[95]
Zhuang Wang, Wenhan Zhang, and Bo Liu. 2021. Computational Analysis of Synthetic Planning: Past and Future. Chinese Journal of Chemistry 39, 11 (2021), 3127–3143.
[96]
David Weininger. 1988. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. Journal of Chemical Information and Computer Sciences 28, 1 (1988), 31–36.
[97]
Di Weng, Ran Chen, Zikun Deng, Feiran Wu, Jingmin Chen, and Yingcai Wu. 2019. SRVis: Towards Better Spatial Integration in Ranking Visualization. IEEE Transactions on Visualization and Computer Graphics 25, 1(2019), 459–469. https://doi.org/10.1109/TVCG.2018.2865126
[98]
Di Weng, Chengbo Zheng, Zikun Deng, Mingze Ma, Jie Bao, Yu Zheng, Mingliang Xu, and Yingcai Wu. 2020. Towards better bus networks: A visual analytics approach. IEEE transactions on visualization and computer graphics 27, 2(2020), 817–827.
[99]
Di Weng, Heming Zhu, Jie Bao, Yu Zheng, and Yingcai Wu. 2018. Homefinder revisited: Finding ideal homes with reachability-centric multi-criteria decision making. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.
[100]
Chaochao Yan, Qianggang Ding, Peilin Zhao, Shuangjia Zheng, Jinyu Yang, Yang Yu, and Junzhou Huang. 2020. Retroxpert: Decompose retrosynthesis prediction like a chemist. Advances in Neural Information Processing Systems 33 (2020), 11248–11258.
[101]
Litao Yan, Elena L Glassman, and Tianyi Zhang. 2021. Visualizing Examples of Deep Neural Networks at Scale. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
[102]
Stelios H Zanakis, Anthony Solomon, Nicole Wishart, and Sandipa Dublish. 1998. Multi-attribute decision making: A simulation comparison of select methods. European Journal of Operational Research 107, 3 (1998), 507–529.
[103]
Haoqi Zhang, Edith Law, Rob Miller, Krzysztof Gajos, David Parkes, and Eric Horvitz. 2012. Human computation tasks with global constraints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 217–226.
[104]
Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295–305.
[105]
Yong Zheng. 2017. Criteria chains: a novel multi-criteria recommendation approach. In Proceedings of the 22nd International Conference on Intelligent User Interfaces. 29–33.
[106]
Oren Zuckerman, Viva Sarah Press, Ehud Barda, Benny Megidish, and Hadas Erel. 2022. Tangible Collaboration: A Human-Centered Approach for Sharing Control With an Actuated-Interface. In CHI Conference on Human Factors in Computing Systems. 1–13.

Cited By

View all
  • (2024)Rolling-Type Collaborative Training for False Comment Identification: Enhancing Accuracy through Multi-Characteristic FusionA Practitioner's Approach to Problem-Solving using AI10.2174/9789815305364124010010(130-141)Online publication date: 17-Oct-2024
  • (2024)DiscipLink: Unfolding Interdisciplinary Information Seeking Process via Human-AI Co-ExplorationProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676366(1-20)Online publication date: 13-Oct-2024
  • (2024)A Taxonomy for Human-LLM Interaction Modes: An Initial ExplorationExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650786(1-11)Online publication date: 11-May-2024
  • Show More Cited By

Index Terms

  1. RetroLens: A Human-AI Collaborative System for Multi-step Retrosynthetic Route Planning

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
    April 2023
    14911 pages
    ISBN:9781450394215
    DOI:10.1145/3544548
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 April 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Human-AI collaboration
    2. multi-criteria decision making
    3. multi-step problem solving

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • The Research Grants Council of the Hong Kong Special Administrative Region under General Research Fund

    Conference

    CHI '23
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)605
    • Downloads (Last 6 weeks)50
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Rolling-Type Collaborative Training for False Comment Identification: Enhancing Accuracy through Multi-Characteristic FusionA Practitioner's Approach to Problem-Solving using AI10.2174/9789815305364124010010(130-141)Online publication date: 17-Oct-2024
    • (2024)DiscipLink: Unfolding Interdisciplinary Information Seeking Process via Human-AI Co-ExplorationProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676366(1-20)Online publication date: 13-Oct-2024
    • (2024)A Taxonomy for Human-LLM Interaction Modes: An Initial ExplorationExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650786(1-11)Online publication date: 11-May-2024
    • (2024)Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with StudentsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642807(1-19)Online publication date: 11-May-2024
    • (2024)ARTiST: Automated Text Simplification for Task Guidance in Augmented RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642772(1-24)Online publication date: 11-May-2024
    • (2024)“Are You Really Sure?” Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision MakingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642671(1-20)Online publication date: 11-May-2024
    • (2024)SLInterpreter: An Exploratory and Iterative Human-AI Collaborative System for GNN-Based Synthetic Lethal PredictionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345632531:1(919-929)Online publication date: 24-Sep-2024
    • (2024)The Role of AI in Drug DiscoveryChemBioChem10.1002/cbic.20230081625:14Online publication date: 26-Jun-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media