Large Language Model Agents Enabled Generative Design of Fluidic Computation Interfaces
Authors: 
Qiuyu Lu, Jiawei Fang, Zhihao Yao, 
Yue Yang, Shiqing Lyu, 
Haipeng Mi, Lining YaoAuthors Info & Claims
Qiuyu Lu, Jiawei Fang, Zhihao Yao, Yue Yang, Shiqing Lyu, Haipeng Mi, Lining YaoAuthors Info & ClaimsArticle No.: 76, Pages 1 - 3
Abstract
The creation of interactive devices is a major area of interest. However, traditional design tools in this field often require a significant learning curve and may not effectively support creative ideation. This study explores the use of fluidic computation interfaces as a case study to examine the potential of enhancing design tools for physical devices with Large Language Model (LLM) agents. With LLM agents, the Generative Design Tool (GDT) can understand the capabilities and limitations of new devices, suggest diverse, insightful, and practical application scenarios, and recommend designs that are technically and contextually appropriate. Additionally, it generates the necessary design parameters for the traditional components of the design tool to visualize results and create files for fabrication.
References
[1]
Byoungkwon An, Ye Tao, Jianzhe Gu, Tingyu Cheng, Xiang ’Anthony’ Chen, Xiaoxiao Zhang, Wei Zhao, Youngwook Do, Shigeo Takahashi, Hsiang-Yun Wu, Teng Zhang, and Lining Yao. 2018. Thermorph: Democratizing 4D Printing of Self-Folding Materials and Interfaces(CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173834
[2]
Bhavya Bhavya, Jinjun Xiong, and Chengxiang Zhai. 2023. CAM: A Large Language Model-based Creative Analogy Mining Framework(WWW ’23). Association for Computing Machinery, New York, NY, USA, 12 pages. https://doi.org/10.1145/3543507.3587431
[3]
Yu Cheng, Jieshan Chen, Qing Huang, Zhenchang Xing, Xiwei Xu, and Qinghua Lu. 2023. Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chains. (dec 2023). https://doi.org/10.1145/3638247
[4]
Peitong Duan, Jeremy Warner, and Bjoern Hartmann. 2023. Towards Generating UI Design Feedback with LLMs(UIST ’23 Adjunct). Association for Computing Machinery, New York, NY, USA, 3 pages. https://doi.org/10.1145/3586182.3615810
[5]
Emily Guan, Di Wu, Qiuyu Lu, and Lining Yao. 2024. Design and Simulation Tool for Sequentially and Conditionally Programmable Waxpaper Morphing Interfaces. In Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’24). Association for Computing Machinery, New York, NY, USA, Article 399, 5 pages. https://doi.org/10.1145/3613905.3648661
[6]
Hyung-Kwon Ko, Gwanmo Park, Hyeon Jeon, Jaemin Jo, Juho Kim, and Jinwook Seo. 2023. Large-scale Text-to-Image Generation Models for Visual Artists’ Creative Works(IUI ’23). Association for Computing Machinery, New York, NY, USA, 15 pages. https://doi.org/10.1145/3581641.3584078
[7]
Qiuyu Lu, Yejun Liu, and Haipeng Mi. 2020. MotionFlow: Time-axis-based Multiple Robots Expressive Motion Programming. In Proceedings of the 3rd International Conference on Computer Science and Software Engineering (Beijing, China) (CSSE ’20). Association for Computing Machinery, New York, NY, USA, 145–149. https://doi.org/10.1145/3403746.3403919
[8]
Qiuyu Lu, Chengpeng Mao, Liyuan Wang, and Haipeng Mi. 2016. LIME: LIquid MEtal Interfaces for Non-Rigid Interaction. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). Association for Computing Machinery, New York, NY, USA, 449–452. https://doi.org/10.1145/2984511.2984562
[9]
Qiuyu Lu, Jifei Ou, João Wilbert, André Haben, Haipeng Mi, and Hiroshi Ishii. 2019. milliMorph – Fluid-Driven Thin Film Shape-Change Materials for Interaction Design. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology(UIST ’19). Association for Computing Machinery, New York, NY, USA, 663–672. https://doi.org/10.1145/3332165.3347956
[10]
Qiuyu Lu, Jifei Ou, Lining Yao, and Hiroshi Ishii. 2024. milleCrepe: Extending Capabilities of Fluid-driven Interfaces with Multilayer Structures and Diverse Actuation Media. In Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems(CHI EA ’24). Association for Computing Machinery, New York, NY, USA, Article 383, 10 pages. https://doi.org/10.1145/3613905.3650891
[11]
Qiuyu Lu, Haiqing Xu, Yijie Guo, Joey Yu Wang, and Lining Yao. 2023. Fluidic Computation Kit: Towards Electronic-free Shape-changing Interfaces. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems(CHI ’23). Association for Computing Machinery, New York, NY, USA, 1–21. https://doi.org/10.1145/3544548.3580783
[12]
Qiuyu Lu, Tianyu Yu, Semina Yi, Yuran Ding, Haipeng Mi, and Lining Yao. 2023. Sustainflatable: Harvesting, Storing and Utilizing Ambient Energy for Pneumatic Morphing Interfaces. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (, San Francisco, CA, USA, ) (UIST ’23). Association for Computing Machinery, New York, NY, USA, Article 32, 20 pages. https://doi.org/10.1145/3586183.3606721
[13]
Di Wu, Emily Guan, Yunjia Zhang, Hsuanju Lai, Qiuyu Lu, and Lining Yao. 2024. Waxpaper Actuator: Sequentially and Conditionally Programmable Wax Paper for Morphing Interfaces. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 865, 16 pages. https://doi.org/10.1145/3613904.3642373
[14]
Humphrey Yang, Kuanren Qian, Haolin Liu, Yuxuan Yu, Jianzhe Gu, Matthew McGehee, Yongjie Jessica Zhang, and Lining Yao. 2020. SimuLearn: Fast and Accurate Simulator to Support Morphing Materials Design and Workflows. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’20). Association for Computing Machinery, New York, NY, USA, 71–84. https://doi.org/10.1145/3379337.3415867
[15]
Tianyu Yu, Mengjia Niu, Haipeng Mi, and Qiuyu Lu. 2024. MilliWare: Parametric Modeling and Simulation of Millifluidic Shape-changing Interface. In Proceedings of the Eleventh International Symposium of Chinese CHI (Denpasar, Bali, Indonesia) (CHCHI ’23). Association for Computing Machinery, New York, NY, USA, 461–467. https://doi.org/10.1145/3629606.3629654
Index Terms
- Large Language Model Agents Enabled Generative Design of Fluidic Computation Interfaces
Recommendations
A practical generative design method
A generative CAD based design exploration method is proposed. It is suitable for complex multi-criteria design problems where important performance criteria are uncomputable. The method is based on building a genotype of the design within a history ...
A generative design technique for exploring shape variations
Highlights- A new generative design technique for creation of aesthetic CAD designs.
- The ...
AbstractBecause innovative and creative design is essential to a successful product, this work brings the benefits of generative design in the conceptual phase of the product development process so that designers/engineers can effectively ...
Generative urban design: integrating financial and energy goals for automated neighborhood layout
SIMAUD '18: Proceedings of the Symposium on Simulation for Architecture and Urban DesignThis paper demonstrates an application of Generative Design to an urban scale through the design of a real-world residential neighborhood development project in Alkmaar, Netherlands. Problems in urban design can benefit greatly from the Generative ...
Comments
Information & Contributors
Information
Published In
October 2024
394 pages
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 October 2024
Check for updates
Author Tags
Qualifiers
- Poster
- Research
- Refereed limited
Conference
UIST '24: The 37th Annual ACM Symposium on User Interface Software and Technology
October 13 - 16, 2024
PA, Pittsburgh, USA
Acceptance Rates
Overall Acceptance Rate 355 of 1,733 submissions, 20%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 172Total Downloads
- Downloads (Last 12 months)172
- Downloads (Last 6 weeks)37
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format