skip to main content
10.1145/3379337.3415866acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article
Open Access

Design Adjectives: A Framework for Interactive Model-Guided Exploration of Parameterized Design Spaces

Published:20 October 2020Publication History

ABSTRACT

Many digital design tasks require a user to set a large number of parameters. Gallery-based interfaces provide a way to quickly evaluate examples and explore the space of potential designs, but require systems to predict which designs from a high-dimensional space are the right ones to present to the user. In this paper we present the design adjectives framework for building parameterized design tools in high dimensional design spaces. The framework allows users to create and edit design adjectives, machine-learned models of user intent, to guide exploration through high-dimensional design spaces. We provide a domain-agnostic implementation of the design adjectives framework based on Gaussian process regression, which is able to rapidly learn user intent from only a few examples. Learning and sampling of the design adjective occurs at interactive rates, making the system suitable for iterative design workflows. We demonstrate use of the design adjectives framework to create design tools for three domains: materials, fonts, and particle systems. We evaluate these tools in a user study showing that participants were able to easily explore the design space and find designs that they liked, and in professional case studies that demonstrate the framework's ability to support professional design concepting workflows.

Skip Supplemental Material Section

Supplemental Material

ufp6584pv.mp4

mp4

21.6 MB

ufp6584vf.mp4

mp4

68.9 MB

3379337.3415866.mp4

Presentation Video

mp4

21.7 MB

References

  1. Saleema Amershi. 2012. Designing for effective end-user interaction with machine learning. Ph.D. Dissertation. University of Washington.Google ScholarGoogle Scholar
  2. Saleema Amershi, Maya Cakmak, William Bradley Knox, and Todd Kulesza. 2014. Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 35, 4 (Dec. 2014), 105. http://dx.doi.org/10.1609/aimag.v35i4.2513Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Autodesk Inc. 2020. Autodesk Maya. (2020). https://www.autodesk.com/products/maya/overviewGoogle ScholarGoogle Scholar
  4. Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, and Tony D. DeRose. 1993. Toolglass and magic lenses: the see-through interface. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques - SIGGRAPH '93. ACM Press, 73--80. http://dx.doi.org/10.1145/166117.166126Google ScholarGoogle Scholar
  5. Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian Interactive Optimization Approach to Procedural Animation Design. (2010), 10.Google ScholarGoogle Scholar
  6. Siddhartha Chaudhuri, Evangelos Kalogerakis, Stephen Giguere, and Thomas Funkhouser. 2013. Attribit: content creation with semantic attributes. In Proceedings of the 26th annual ACM symposium on User interface software and technology - UIST '13. ACM Press, St. Andrews, Scotland, United Kingdom, 193--202. http://dx.doi.org/10.1145/2501988.2502008Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Minh Dang, Stefan Lienhard, Duygu Ceylan, Boris Neubert, Peter Wonka, and Mark Pauly. 2015. Interactive Design of Probability Density Functions for Shape Grammars. ACM Trans. Graph. 34, 6 (Oct. 2015), 206:1--206:13. http://dx.doi.org/10.1145/2816795.2818069Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Ruta Desai, Fraser Anderson, Justin Matejka, Stelian Coros, James McCann, George Fitzmaurice, and Tovi Grossman. 2019. Geppetto: Enabling Semantic Design of Expressive Robot Behaviors. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA, 369:1--369:14. http://dx.doi.org/10.1145/3290605.3300599 event-place: Glasgow, Scotland Uk.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Electron. 2020. Electron. (2020). https://www.electronjs.org/Google ScholarGoogle Scholar
  10. Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. 2009. Describing objects by their attributes. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 1778--1785. http://dx.doi.org/10.1109/CVPR.2009.5206772 ISSN: 1063--6919.Google ScholarGoogle ScholarCross RefCross Ref
  11. Vittorio Ferrari and Andrew Zisserman. 2008. Learning Visual Attributes. In Advances in Neural Information Processing Systems 20, J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis (Eds.). Curran Associates, Inc., 433--440. http://papers.nips.cc/paper/3217-learning-visual-attributes.pdfGoogle ScholarGoogle Scholar
  12. Martin R. Frank and James D. Foley. 1993. Model-based user interface design by example and by interview. In Proceedings of the 6th annual ACM symposium on User interface software and technology (UIST '93). Association for Computing Machinery, Atlanta, Georgia, USA, 129--137. http://dx.doi.org/10.1145/168642.168655Google ScholarGoogle Scholar
  13. Jacob R. Gardner, Geoff Pleiss, David Bindel, Kilian Q. Weinberger, and Andrew Gordon Wilson. 2018. GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration. arXiv:1809.11165 [cs, stat] (Sept. 2018). http://arxiv.org/abs/1809.11165 arXiv: 1809.11165.Google ScholarGoogle Scholar
  14. Vincent Garreau. 2019. particles.js. (Oct. 2019). https://github.com/ebshimizu/particles.js original-date: 2019--10-01T18:30:48Z.Google ScholarGoogle Scholar
  15. Vinod Goel and Peter Pirolli. 1992. The structure of Design Problem Spaces. Cognitive Science 16, 3 (1992), 395--429. http://dx.doi.org/10.1207/s15516709cog1603_3Google ScholarGoogle ScholarCross RefCross Ref
  16. William B. Kerr and Fabio Pellacini. 2010. Toward evaluating material design interface paradigms for novice users. ACM Transactions on Graphics 29, 4 (July 2010), 1. http://dx.doi.org/10.1145/1778765.1778772Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Sandeep Kochhar. 1990. A prototype system for design automation via the browsing paradigm. In Proceedings of Graphics Interface '90, Vol. Halifax. 156--166. http://dx.doi.org/10.20380/gi1990.19Google ScholarGoogle Scholar
  18. Adriana Kovashka, Devi Parikh, and Kristen Grauman. 2015. WhittleSearch: Interactive Image Search with Relative Attribute Feedback. International Journal of Computer Vision 115, 2 (Nov. 2015), 185--210. http://dx.doi.org/10.1007/s11263-015-0814-0 arXiv: 1505.04141.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2014. Crowd-powered parameter analysis for visual design exploration. In Proceedings of the 27th annual ACM symposium on User interface software and technology - UIST '14. ACM Press, Honolulu, Hawaii, USA, 65--74. http://dx.doi.org/10.1145/2642918.2647386Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Pierre-Yves Laffont, Zhile Ren, Xiaofeng Tao, Chao Qian, and James Hays. 2014. Transient attributes for high-level understanding and editing of outdoor scenes. In ACM Transactions on Graphics, Vol. 33. 1--11. http://dx.doi.org/10.1145/2601097.2601101Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Bryan Lawson. 2006. How Designers Think: The Design Process Demystified. Elsevier/Architectural.Google ScholarGoogle Scholar
  22. Brian Lee, Savil Srivastava, Ranjitha Kumar, Ronen Brafman, and Scott R. Klemmer. 2010. Designing with interactive example galleries. In Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. ACM Press, Atlanta, Georgia, USA, 2257. http://dx.doi.org/10.1145/1753326.1753667Google ScholarGoogle Scholar
  23. J. Marks, W. Ruml, K. Ryall, J. Seims, S. Shieber, B. Andalman, P. A. Beardsley, W. Freeman, S. Gibson, J. Hodgins, T. Kang, B. Mirtich, and H. Pfister. 1997. Design galleries: a general approach to setting parameters for computer graphics and animation. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97. ACM Press, 389--400. http://dx.doi.org/10.1145/258734.258887Google ScholarGoogle Scholar
  24. Justin Matejka, Michael Glueck, Erin Bradner, Ali Hashemi, Tovi Grossman, and George Fitzmaurice. 2018. Dream Lens: Exploration and Visualization of Large-Scale Generative Design Datasets. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, 369:1--369:12. http://dx.doi.org/10.1145/3173574.3173943 event-place: Montreal QC, Canada.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Wojciech Matusik, Hanspeter Pfister, Matt Brand, and Leonard McMillan. 2003. A Data-driven Reflectance Model. In ACM SIGGRAPH 2003 Papers (SIGGRAPH '03). ACM, New York, NY, USA, 759--769. http://dx.doi.org/10.1145/1201775.882343 event-place: San Diego, California.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Richard G. McDaniel and Brad A. Myers. 1998. Building applications using only demonstration. In Proceedings of the 3rd international conference on Intelligent user interfaces (IUI '98). Association for Computing Machinery, San Francisco, California, USA, 109--116. http://dx.doi.org/10.1145/268389.268409Google ScholarGoogle Scholar
  27. Vittorio Megaro, Bernhard Thomaszewski, Maurizio Nitti, Otmar Hilliges, Markus Gross, and Stelian Coros. 2015. Interactive Design of 3D-printable Robotic Creatures. ACM Trans. Graph. 34, 6 (Oct. 2015), 216:1--216:9. http://dx.doi.org/10.1145/2816795.2818137Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Mr.doob. 2019. three.js. (Nov. 2019). https://github.com/mrdoob/three.js original-date: 2010-03--23T18:58:01Z.Google ScholarGoogle Scholar
  29. Peter O'Donovan, JAseem Agarwala, and Aaron Hertzmann. 2014. Exploratory font selection using crowdsourced attributes. ACM Transactions on Graphics 33, 4 (July 2014), 1--9. http://dx.doi.org/10.1145/2601097.2601110Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Devi Parikh and Kristen Grauman. 2011. Relative attributes. In 2011 International Conference on Computer Vision. 503--510. http://dx.doi.org/10.1109/ICCV.2011.6126281 ISSN: 1550--5499.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Prototypo. 2019. Prototypo. (2019). https://www.prototypo.io/Google ScholarGoogle Scholar
  32. Carl Edward Rasmussen and Christopher K. I. Williams. 2006. Gaussian processes for machine learning. MIT Press, Cambridge, Mass. OCLC: ocm61285753.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Daniel Ritchie, Ankita Arvind Kejriwal, and Scott R. Klemmer. 2011. d.tour: style-based exploration of design example galleries. In Proceedings of the 24th annual ACM symposium on User interface software and technology - UIST '11. ACM Press, Santa Barbara, California, USA, 165. http://dx.doi.org/10.1145/2047196.2047216Google ScholarGoogle Scholar
  34. Adriana Schulz, Jie Xu, Bo Zhu, Changxi Zheng, Eitan Grinspun, and Wojciech Matusik. 2017. Interactive Design Space Exploration and Optimization for CAD Models. ACM Trans. Graph. 36, 4 (July 2017), 157:1--157:14. http://dx.doi.org/10.1145/3072959.3073688Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Evan Shimizu, Matt Fisher, Sylvain Paris, and Kayvon Fatahalian. 2019. Finding Layers Using Hover Visualizations. Proceedings of Graphics Interface 2019 Kingston (2019), 28--31 May 2019. http://dx.doi.org/10.20380/gi2019.16Google ScholarGoogle Scholar
  36. Ben Shneiderman, Gerhard Fischer, Mary Czerwinski, Mitch Resnick, Brad Myers, Linda Candy, Ernest Edmonds, Mike Eisenberg, Elisa Giaccardi, Tom Hewett, Pamela Jennings, Bill Kules, Kumiyo Nakakoji, Jay Nunamaker, Randy Pausch, Ted Selker, Elisabeth Sylvan, and Michael Terry. 2005. Creativity Support Tools: Report From a U. S. National Science Foundation Sponsored Workshop. (Sept. 2005). http://www.cs.umd.edu/hcil/CST/report.htmlGoogle ScholarGoogle Scholar
  37. Maria Shugrina, Ariel Shamir, and Wojciech Matusik. 2015. Fab forms: customizable objects for fabrication with validity and geometry caching. ACM Transactions on Graphics 34, 4 (July 2015), 100:1--100:12. http://dx.doi.org/10.1145/2766994Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Leonid Sigal, Moshe Mahler, Spencer Diaz, Kyna McIntosh, Elizabeth Carter, Timothy Richards, and Jessica Hodgins. 2015. A perceptual control space for garment simulation. ACM Transactions on Graphics 34, 4 (July 2015), 117:1--117:10. http://dx.doi.org/10.1145/2766971Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Herbert A. Simon. 1973. The structure of ill structured problems. Artificial Intelligence 4, 3--4 (1973), 181--201. http://dx.doi.org/10.1016/0004--3702(73)90011--8Google ScholarGoogle ScholarCross RefCross Ref
  40. Substance3D. 2019. Substance Designer. (Jan. 2019). https://www..substance3d.comGoogle ScholarGoogle Scholar
  41. Adobe Systems. 2019. Adobe Photoshop Lightroom CC. (2019). https://www.adobe.com/products/photoshop-lightroom.htmlGoogle ScholarGoogle Scholar
  42. Jerry O. Talton, Daniel Gibson, Lingfeng Yang, Pat Hanrahan, and Vladlen Koltun. 2009. Exploratory modeling with collaborative design spaces. ACM Transactions on Graphics 28, 5 (Dec. 2009), 1. http://dx.doi.org/10.1145/1618452.1618513Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Unity Technologies. 2019. Unity3D. (2019). https://unity3d.com/Google ScholarGoogle Scholar
  44. Michael Terry and Elizabeth D. Mynatt. 2002. Side views: persistent, on-demand previews for open-ended tasks. In Proceedings of the 15th annual ACM symposium on User interface software and technology - UIST '02. ACM Press, Paris, France, 71. http://dx.doi.org/10.1145/571985.571996Google ScholarGoogle Scholar
  45. David G. Ullman, Thomas G. Dietterich, and Larry A. Stauffer. 1988. A model of the mechanical design process based on empirical data. AI EDAM 2, 1 (Feb. 1988), 33--52. http://dx.doi.org/10.1017/S0890060400000536Google ScholarGoogle Scholar
  46. Nobuyuki Umetani, Danny M. Kaufman, Takeo Igarashi, and Eitan Grinspun. 2011. Sensitive Couture for Interactive Garment Modeling and Editing. In ACM SIGGRAPH 2011 Papers (SIGGRAPH '11). ACM, New York, NY, USA, 90:1--90:12. http://dx.doi.org/10.1145/1964921.1964985 event-place: Vancouver, British Columbia, Canada.Google ScholarGoogle Scholar
  47. Nobuyuki Umetani, Yuki Koyama, Ryan Schmidt, and Takeo Igarashi. 2014. Pteromys: Interactive Design and Optimization of Free-formed Free-flight Model Airplanes. ACM Trans. Graph. 33, 4 (July 2014), 65:1--65:10. http://dx.doi.org/10.1145/2601097.2601129Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Mehmet Ersin Yumer, Siddhartha Chaudhuri, Jessica K. Hodgins, and Levent Burak Kara. 2015. Semantic shape editing using deformation handles. ACM Transactions on Graphics 34, 4 (July 2015), 86:1--86:12. http://dx.doi.org/10.1145/2766908Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Károly Zsolnai-Fehér, Peter Wonka, and Michael Wimmer. 2018. Gaussian material synthesis. ACM Transactions on Graphics 37, 4 (July 2018), 1--14. http://dx.doi.org/10.1145/3197517.3201307Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Design Adjectives: A Framework for Interactive Model-Guided Exploration of Parameterized Design Spaces

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology
          October 2020
          1297 pages
          ISBN:9781450375146
          DOI:10.1145/3379337

          Copyright © 2020 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 20 October 2020

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article

          Acceptance Rates

          Overall Acceptance Rate842of3,967submissions,21%

          Upcoming Conference

          UIST '24

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader