ABSTRACT
Many digital design tasks require a user to set a large number of parameters. Gallery-based interfaces provide a way to quickly evaluate examples and explore the space of potential designs, but require systems to predict which designs from a high-dimensional space are the right ones to present to the user. In this paper we present the design adjectives framework for building parameterized design tools in high dimensional design spaces. The framework allows users to create and edit design adjectives, machine-learned models of user intent, to guide exploration through high-dimensional design spaces. We provide a domain-agnostic implementation of the design adjectives framework based on Gaussian process regression, which is able to rapidly learn user intent from only a few examples. Learning and sampling of the design adjective occurs at interactive rates, making the system suitable for iterative design workflows. We demonstrate use of the design adjectives framework to create design tools for three domains: materials, fonts, and particle systems. We evaluate these tools in a user study showing that participants were able to easily explore the design space and find designs that they liked, and in professional case studies that demonstrate the framework's ability to support professional design concepting workflows.
Supplemental Material
Available for Download
README This is the supplemental material and code for "Design Adjectives: A Framework for Interactive Model-Guided Exploration of Parameterized Design Spaces." There are two folders contained here. DesignAdjectives-code contains the prototype server and interface code. Follow the instructions in that folder's readme to compile and run the program. The study-data folder contains the raw user study data collected for this paper's evaluation, and also contains two PDFs displaying designs created by users during the user study. The raw study data can be viewed with Microsoft Excel (or any spreadsheet reader that can handle the Excel format), and the created designs can be viewed with any PDF reader.
- Saleema Amershi. 2012. Designing for effective end-user interaction with machine learning. Ph.D. Dissertation. University of Washington.Google Scholar
- Saleema Amershi, Maya Cakmak, William Bradley Knox, and Todd Kulesza. 2014. Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 35, 4 (Dec. 2014), 105. http://dx.doi.org/10.1609/aimag.v35i4.2513Google ScholarDigital Library
- Autodesk Inc. 2020. Autodesk Maya. (2020). https://www.autodesk.com/products/maya/overviewGoogle Scholar
- Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, and Tony D. DeRose. 1993. Toolglass and magic lenses: the see-through interface. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques - SIGGRAPH '93. ACM Press, 73--80. http://dx.doi.org/10.1145/166117.166126Google Scholar
- Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian Interactive Optimization Approach to Procedural Animation Design. (2010), 10.Google Scholar
- Siddhartha Chaudhuri, Evangelos Kalogerakis, Stephen Giguere, and Thomas Funkhouser. 2013. Attribit: content creation with semantic attributes. In Proceedings of the 26th annual ACM symposium on User interface software and technology - UIST '13. ACM Press, St. Andrews, Scotland, United Kingdom, 193--202. http://dx.doi.org/10.1145/2501988.2502008Google ScholarDigital Library
- Minh Dang, Stefan Lienhard, Duygu Ceylan, Boris Neubert, Peter Wonka, and Mark Pauly. 2015. Interactive Design of Probability Density Functions for Shape Grammars. ACM Trans. Graph. 34, 6 (Oct. 2015), 206:1--206:13. http://dx.doi.org/10.1145/2816795.2818069Google ScholarDigital Library
- Ruta Desai, Fraser Anderson, Justin Matejka, Stelian Coros, James McCann, George Fitzmaurice, and Tovi Grossman. 2019. Geppetto: Enabling Semantic Design of Expressive Robot Behaviors. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA, 369:1--369:14. http://dx.doi.org/10.1145/3290605.3300599 event-place: Glasgow, Scotland Uk.Google ScholarDigital Library
- Electron. 2020. Electron. (2020). https://www.electronjs.org/Google Scholar
- Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. 2009. Describing objects by their attributes. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 1778--1785. http://dx.doi.org/10.1109/CVPR.2009.5206772 ISSN: 1063--6919.Google ScholarCross Ref
- Vittorio Ferrari and Andrew Zisserman. 2008. Learning Visual Attributes. In Advances in Neural Information Processing Systems 20, J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis (Eds.). Curran Associates, Inc., 433--440. http://papers.nips.cc/paper/3217-learning-visual-attributes.pdfGoogle Scholar
- Martin R. Frank and James D. Foley. 1993. Model-based user interface design by example and by interview. In Proceedings of the 6th annual ACM symposium on User interface software and technology (UIST '93). Association for Computing Machinery, Atlanta, Georgia, USA, 129--137. http://dx.doi.org/10.1145/168642.168655Google Scholar
- Jacob R. Gardner, Geoff Pleiss, David Bindel, Kilian Q. Weinberger, and Andrew Gordon Wilson. 2018. GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration. arXiv:1809.11165 [cs, stat] (Sept. 2018). http://arxiv.org/abs/1809.11165 arXiv: 1809.11165.Google Scholar
- Vincent Garreau. 2019. particles.js. (Oct. 2019). https://github.com/ebshimizu/particles.js original-date: 2019--10-01T18:30:48Z.Google Scholar
- Vinod Goel and Peter Pirolli. 1992. The structure of Design Problem Spaces. Cognitive Science 16, 3 (1992), 395--429. http://dx.doi.org/10.1207/s15516709cog1603_3Google ScholarCross Ref
- William B. Kerr and Fabio Pellacini. 2010. Toward evaluating material design interface paradigms for novice users. ACM Transactions on Graphics 29, 4 (July 2010), 1. http://dx.doi.org/10.1145/1778765.1778772Google ScholarDigital Library
- Sandeep Kochhar. 1990. A prototype system for design automation via the browsing paradigm. In Proceedings of Graphics Interface '90, Vol. Halifax. 156--166. http://dx.doi.org/10.20380/gi1990.19Google Scholar
- Adriana Kovashka, Devi Parikh, and Kristen Grauman. 2015. WhittleSearch: Interactive Image Search with Relative Attribute Feedback. International Journal of Computer Vision 115, 2 (Nov. 2015), 185--210. http://dx.doi.org/10.1007/s11263-015-0814-0 arXiv: 1505.04141.Google ScholarDigital Library
- Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2014. Crowd-powered parameter analysis for visual design exploration. In Proceedings of the 27th annual ACM symposium on User interface software and technology - UIST '14. ACM Press, Honolulu, Hawaii, USA, 65--74. http://dx.doi.org/10.1145/2642918.2647386Google ScholarDigital Library
- Pierre-Yves Laffont, Zhile Ren, Xiaofeng Tao, Chao Qian, and James Hays. 2014. Transient attributes for high-level understanding and editing of outdoor scenes. In ACM Transactions on Graphics, Vol. 33. 1--11. http://dx.doi.org/10.1145/2601097.2601101Google ScholarDigital Library
- Bryan Lawson. 2006. How Designers Think: The Design Process Demystified. Elsevier/Architectural.Google Scholar
- Brian Lee, Savil Srivastava, Ranjitha Kumar, Ronen Brafman, and Scott R. Klemmer. 2010. Designing with interactive example galleries. In Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. ACM Press, Atlanta, Georgia, USA, 2257. http://dx.doi.org/10.1145/1753326.1753667Google Scholar
- J. Marks, W. Ruml, K. Ryall, J. Seims, S. Shieber, B. Andalman, P. A. Beardsley, W. Freeman, S. Gibson, J. Hodgins, T. Kang, B. Mirtich, and H. Pfister. 1997. Design galleries: a general approach to setting parameters for computer graphics and animation. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97. ACM Press, 389--400. http://dx.doi.org/10.1145/258734.258887Google Scholar
- Justin Matejka, Michael Glueck, Erin Bradner, Ali Hashemi, Tovi Grossman, and George Fitzmaurice. 2018. Dream Lens: Exploration and Visualization of Large-Scale Generative Design Datasets. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, 369:1--369:12. http://dx.doi.org/10.1145/3173574.3173943 event-place: Montreal QC, Canada.Google ScholarDigital Library
- Wojciech Matusik, Hanspeter Pfister, Matt Brand, and Leonard McMillan. 2003. A Data-driven Reflectance Model. In ACM SIGGRAPH 2003 Papers (SIGGRAPH '03). ACM, New York, NY, USA, 759--769. http://dx.doi.org/10.1145/1201775.882343 event-place: San Diego, California.Google ScholarDigital Library
- Richard G. McDaniel and Brad A. Myers. 1998. Building applications using only demonstration. In Proceedings of the 3rd international conference on Intelligent user interfaces (IUI '98). Association for Computing Machinery, San Francisco, California, USA, 109--116. http://dx.doi.org/10.1145/268389.268409Google Scholar
- Vittorio Megaro, Bernhard Thomaszewski, Maurizio Nitti, Otmar Hilliges, Markus Gross, and Stelian Coros. 2015. Interactive Design of 3D-printable Robotic Creatures. ACM Trans. Graph. 34, 6 (Oct. 2015), 216:1--216:9. http://dx.doi.org/10.1145/2816795.2818137Google ScholarDigital Library
- Mr.doob. 2019. three.js. (Nov. 2019). https://github.com/mrdoob/three.js original-date: 2010-03--23T18:58:01Z.Google Scholar
- Peter O'Donovan, JAseem Agarwala, and Aaron Hertzmann. 2014. Exploratory font selection using crowdsourced attributes. ACM Transactions on Graphics 33, 4 (July 2014), 1--9. http://dx.doi.org/10.1145/2601097.2601110Google ScholarDigital Library
- Devi Parikh and Kristen Grauman. 2011. Relative attributes. In 2011 International Conference on Computer Vision. 503--510. http://dx.doi.org/10.1109/ICCV.2011.6126281 ISSN: 1550--5499.Google ScholarDigital Library
- Prototypo. 2019. Prototypo. (2019). https://www.prototypo.io/Google Scholar
- Carl Edward Rasmussen and Christopher K. I. Williams. 2006. Gaussian processes for machine learning. MIT Press, Cambridge, Mass. OCLC: ocm61285753.Google ScholarDigital Library
- Daniel Ritchie, Ankita Arvind Kejriwal, and Scott R. Klemmer. 2011. d.tour: style-based exploration of design example galleries. In Proceedings of the 24th annual ACM symposium on User interface software and technology - UIST '11. ACM Press, Santa Barbara, California, USA, 165. http://dx.doi.org/10.1145/2047196.2047216Google Scholar
- Adriana Schulz, Jie Xu, Bo Zhu, Changxi Zheng, Eitan Grinspun, and Wojciech Matusik. 2017. Interactive Design Space Exploration and Optimization for CAD Models. ACM Trans. Graph. 36, 4 (July 2017), 157:1--157:14. http://dx.doi.org/10.1145/3072959.3073688Google ScholarDigital Library
- Evan Shimizu, Matt Fisher, Sylvain Paris, and Kayvon Fatahalian. 2019. Finding Layers Using Hover Visualizations. Proceedings of Graphics Interface 2019 Kingston (2019), 28--31 May 2019. http://dx.doi.org/10.20380/gi2019.16Google Scholar
- Ben Shneiderman, Gerhard Fischer, Mary Czerwinski, Mitch Resnick, Brad Myers, Linda Candy, Ernest Edmonds, Mike Eisenberg, Elisa Giaccardi, Tom Hewett, Pamela Jennings, Bill Kules, Kumiyo Nakakoji, Jay Nunamaker, Randy Pausch, Ted Selker, Elisabeth Sylvan, and Michael Terry. 2005. Creativity Support Tools: Report From a U. S. National Science Foundation Sponsored Workshop. (Sept. 2005). http://www.cs.umd.edu/hcil/CST/report.htmlGoogle Scholar
- Maria Shugrina, Ariel Shamir, and Wojciech Matusik. 2015. Fab forms: customizable objects for fabrication with validity and geometry caching. ACM Transactions on Graphics 34, 4 (July 2015), 100:1--100:12. http://dx.doi.org/10.1145/2766994Google ScholarDigital Library
- Leonid Sigal, Moshe Mahler, Spencer Diaz, Kyna McIntosh, Elizabeth Carter, Timothy Richards, and Jessica Hodgins. 2015. A perceptual control space for garment simulation. ACM Transactions on Graphics 34, 4 (July 2015), 117:1--117:10. http://dx.doi.org/10.1145/2766971Google ScholarDigital Library
- Herbert A. Simon. 1973. The structure of ill structured problems. Artificial Intelligence 4, 3--4 (1973), 181--201. http://dx.doi.org/10.1016/0004--3702(73)90011--8Google ScholarCross Ref
- Substance3D. 2019. Substance Designer. (Jan. 2019). https://www..substance3d.comGoogle Scholar
- Adobe Systems. 2019. Adobe Photoshop Lightroom CC. (2019). https://www.adobe.com/products/photoshop-lightroom.htmlGoogle Scholar
- Jerry O. Talton, Daniel Gibson, Lingfeng Yang, Pat Hanrahan, and Vladlen Koltun. 2009. Exploratory modeling with collaborative design spaces. ACM Transactions on Graphics 28, 5 (Dec. 2009), 1. http://dx.doi.org/10.1145/1618452.1618513Google ScholarDigital Library
- Unity Technologies. 2019. Unity3D. (2019). https://unity3d.com/Google Scholar
- Michael Terry and Elizabeth D. Mynatt. 2002. Side views: persistent, on-demand previews for open-ended tasks. In Proceedings of the 15th annual ACM symposium on User interface software and technology - UIST '02. ACM Press, Paris, France, 71. http://dx.doi.org/10.1145/571985.571996Google Scholar
- David G. Ullman, Thomas G. Dietterich, and Larry A. Stauffer. 1988. A model of the mechanical design process based on empirical data. AI EDAM 2, 1 (Feb. 1988), 33--52. http://dx.doi.org/10.1017/S0890060400000536Google Scholar
- Nobuyuki Umetani, Danny M. Kaufman, Takeo Igarashi, and Eitan Grinspun. 2011. Sensitive Couture for Interactive Garment Modeling and Editing. In ACM SIGGRAPH 2011 Papers (SIGGRAPH '11). ACM, New York, NY, USA, 90:1--90:12. http://dx.doi.org/10.1145/1964921.1964985 event-place: Vancouver, British Columbia, Canada.Google Scholar
- Nobuyuki Umetani, Yuki Koyama, Ryan Schmidt, and Takeo Igarashi. 2014. Pteromys: Interactive Design and Optimization of Free-formed Free-flight Model Airplanes. ACM Trans. Graph. 33, 4 (July 2014), 65:1--65:10. http://dx.doi.org/10.1145/2601097.2601129Google ScholarDigital Library
- Mehmet Ersin Yumer, Siddhartha Chaudhuri, Jessica K. Hodgins, and Levent Burak Kara. 2015. Semantic shape editing using deformation handles. ACM Transactions on Graphics 34, 4 (July 2015), 86:1--86:12. http://dx.doi.org/10.1145/2766908Google ScholarDigital Library
- Károly Zsolnai-Fehér, Peter Wonka, and Michael Wimmer. 2018. Gaussian material synthesis. ACM Transactions on Graphics 37, 4 (July 2018), 1--14. http://dx.doi.org/10.1145/3197517.3201307Google ScholarDigital Library
Index Terms
- Design Adjectives: A Framework for Interactive Model-Guided Exploration of Parameterized Design Spaces
Recommendations
Design as traversal and consequences: an exploration tool for experimental designs
GI '07: Proceedings of Graphics Interface 2007We present a design space explorer for the space of experimental designs. For many design problems, design decisions are determined by the consequences of the design rather than its elemental parts. To support this need, the explorer is constructed to ...
Blended Interaction Design: A Spatial Workspace Supporting HCI and Design Practice
INTERACT '09: Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part IIThis research investigates novel methods and techniques along with tool support that result from a conceptual blend of human-computer interaction with design practice. Using blending theory with material anchors as a theoretical framework, we frame both ...
Mixplorer: Scaffolding Design Space Exploration through Genetic Recombination of Multiple Peoples’ Designs to Support Novices’ Creativity
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing SystemsThe ability to consider a wide range of solutions to a design problem is a crucial skill for designers, and is a major differentiator between experts and novices. One reason for this is that novices are unaware of the full extent of the design space in ...
Comments