skip to main content
10.1145/3313831.3376618acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

ICONATE: Automatic Compound Icon Generation and Ideation

Published: 23 April 2020 Publication History

Abstract

Compound icons are prevalent on signs, webpages, and infographics, effectively conveying complex and abstract concepts, such as "no smoking" and "health insurance", with simple graphical representations. However, designing such icons requires experience and creativity, in order to efficiently navigate the semantics, space, and style features of icons. In this paper, we aim to automate the process of generating icons given compound concepts, to facilitate rapid compound icon creation and ideation. Informed by ethnographic interviews with professional icon designers, we have developed ICONATE, a novel system that automatically generates compound icons based on textual queries and allows users to explore and customize the generated icons. At the core of ICONATE is a computational pipeline that automatically finds commonly used icons for sub-concepts and arranges them according to inferred conventions. To enable the pipeline, we collected a new dataset, Compicon1k, consisting of 1000 compound icons annotated with semantic labels (i.e., concepts). Through user studies, we have demonstrated that our tool is able to automate or accelerate the compound icon design process for both novices and professionals.

Supplemental Material

MP4 File
MP4 File
Preview video
MP4 File
Supplemental video
ZIP File
A supplementary document including more details about our IconVoc152 dataset, our annotation tool, and results from user studies with professionals. A zip file including our IconVoc152 dataset.

References

[1]
Accessed: 2019-09--17. Full Emoji List, v12.0. https://unicode.org/emoji/charts/full-emoji-list.html. (Accessed: 2019-09--17).
[2]
Accessed: 2019-09--17. Microsoft PowerPoint(v16.27). (Accessed: 2019-09--17).
[3]
Scott Bateman, Regan L Mandryk, Carl Gutwin, Aaron Genest, David McDine, and Christopher Brooks. 2010. Useful junk?: the effects of visual embellishment on comprehension and memorability of charts. In Proceedings SIGCHI. ACM, 2573--2582.
[4]
Michelle A Borkin, Zoya Bylinskii, Nam Wook Kim, Constance May Bainbridge, Chelsea S Yeh, Daniel Borkin, Hanspeter Pfister, and Aude Oliva. 2015. Beyond memorability: Visualization recognition and recall. IEEE TVCG 22, 1 (2015), 519--528.
[5]
Michelle A Borkin, Azalea A Vo, Zoya Bylinskii, Phillip Isola, Shashank Sunkavalli, Aude Oliva, and Hanspeter Pfister. 2013. What makes a visualization memorable? IEEE TVCG 19, 12 (2013), 2306--2315.
[6]
Bill Buxton. 2010. Sketching user experiences: getting the design right and the right design. Morgan kaufmann.
[7]
Zoya Bylinskii, Nam Wook Kim, Peter O'Donovan, Sami Alsheikh, Spandan Madan, Hanspeter Pfister, Fredo Durand, Bryan Russell, and Aaron Hertzmann. 2017. Learning Visual Importance for Graphic Designs and Data Visualizations. In Proceedings UIST (UIST '17). ACM, NY, NY, USA, 57--69.
[8]
SHI-KUO CHANG. 1987. Icon Semantics a Formal Approach tO Icon System Design. IJPRAI 01, 01 (1987), 103--120.
[9]
SHI-KUO CHANG, MICHAEL J. TAUBER, BING YU, and JING-SHENG YU. 1988. The Sillicon Complier an Icon-oriented System Generator. IJPRAI 02, 02 (1988), 241--273.
[10]
SHI-KUO CHANG, G. TORTORA, BING YU, and A. GUERCIO. 1987. Icon Purify Toward a Formal Theory of Icons. IJPRAI 01, 03n04 (1987), 377--392.
[11]
S. . Cheng, W. Chen, and H. Sundaram. 1998. Semantic visual templates: linking visual features to semantics. In Proceedings ICIP. 531--535 vol.3.
[12]
Lydia B Chilton, Savvas Petridis, and Maneesh Agrawala. 2019. VisiBlends: A flexible workflow for visual blends. In Proceedings SIGCHI. ACM, 172.
[13]
Marianela Ciolfi Felice, Nolwenn Maudet, Wendy E. Mackay, and Michel Beaudouin-Lafon. 2016. Beyond Snapping: Persistent, Tweakable Alignment and Distribution with StickyLines. In Proceedings UIST (UIST '16). ACM, NY, NY, USA, 133--144.
[14]
Aaron James Draplin. 2016. Draplin Design Co.: Pretty Much Everything (1 ed.). Abrams.
[15]
Michael Evamy. 2015. Logo: The Reference Guide to Symbols and Logotypes. Laurence King Publishing.
[16]
C. Ailie Fraser, Mira Dontcheva, Holger Winnemöller, Sheryl Ehrlich, and Scott Klemmer. 2016. DiscoverySpace: Suggesting Actions in Complex Software. In Proceedings DIS (DIS '16). ACM, NY, NY, USA, 1221--1232.
[17]
Elena Garces, Aseem Agarwala, Diego Gutierrez, and Aaron Hertzmann. 2014. A similarity measure for illustration style. ACM TOG 33, 4 (2014), 93.
[18]
Steve Haroz, Robert Kosara, and Steven L Franconeri. 2015. Isotype visualization: Working memory, performance, and engagement with pictographs. In Proceedings SIGCHI. ACM, 1191--1200.
[19]
Jon Hicks. 2011. The Icon Handbook. Five Simple Steps Ltd.
[20]
William K Horton. 1994. The icon book: Visual symbols for computer systems and documentation. John Wiley & Sons, Inc.
[21]
Ghita Jalal, Nolwenn Maudet, and Wendy E. Mackay. 2015. Color Portraits: From Color Picking to Interacting with Color. In Proceedings SIGCHI (CHI '15). ACM, NY, NY, USA, 4207--4216.
[22]
Sujay Khandekar, Joseph Higg, Yuanzhe Bian, Chae Won Ryu, Jerry O Talton Iii, and Ranjitha Kumar. 2019. Opico: A Study of Emoji-first Communication in a Mobile Social App. In Companion Proceedings of The 2019 World Wide Web Conference. ACM, 450--458.
[23]
Nathalie Cindy Kuicheu, Ning Wang, Gile Narcisse Fanzou Tchuissang, Francois Siewe, and De Xu. 2012. Description logic based icons semantics: An ontology for icons. In IEEE ICSP, Vol. 2. IEEE, 1260--1263.
[24]
Ranjitha Kumar, Jerry O. Talton, Salman Ahmad, and Scott R. Klemmer. 2011. Bricolage: Example-based Retargeting for Web Design. In Proceedings SIGCHI (CHI '11). ACM, NY, NY, USA, 2197--2206.
[25]
Manuel Lagunas, Elena Garces, and Diego Gutierrez. 2019. Learning icons appearance similarity. Multimedia tools and applications 78, 8 (2019), 10733--10751.
[26]
Brian Lee, Savil Srivastava, Ranjitha Kumar, Ronen Brafman, and Scott R. Klemmer. 2010. Designing with Interactive Example Galleries. In Proceedings SIGCHI (CHI '10). ACM, NY, NY, USA, 2257--2266.
[27]
Jianan Li, Tingfa Xu, Jianming Zhang, Aaron Hertzmann, and Jimei Yang. 2019a. LayoutGAN: Generating Graphic Layouts with Wireframe Discriminator. In ICLR. https://openreview.net/forum?id=HJxB5sRcFQ
[28]
Jianan Li, Jimei Yang, Aaron Hertzmann, Jianming Zhang, and Tingfa Xu. 2019b. LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators. arXiv:1901.06767 (2019).
[29]
Rungtai Lin. 1994. A Study of Visual Features for Icon Design. Design Studies 15, 2 (1994), 185 -- 197.
[30]
Yiming Liu, Aseem Agarwala, Jingwan Lu, and Szymon Rusinkiewicz. 2016. Data-driven iconification. In Proceedings of the Joint Symposium on Computational Aesthetics and Sketch Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering. Eurographics Association, 113--124.
[31]
Logo Design Love. A Guide to Creating Iconic Brand Identities. (????).
[32]
Kurt Luther, Amy Pavel, Wei Wu, Jari-lee Tolentino, Maneesh Agrawala, Björn Hartmann, and Steven P. Dow. 2014. CrowdCrit: Crowdsourcing and Aggregating Visual Design Critique. In Proceedings CSCW (CSCW Companion '14). ACM, NY, NY, USA, 21--24.
[33]
Wakako Machida and Takayuki Itoh. 2011. Lyricon: A visual music selection interface featuring multiple icons. In 2011 15th International Conference on Information Visualisation. IEEE, 145--150.
[34]
Spandan Madan, Zoya Bylinskii, Matthew Tancik, Adrià Recasens, Kimberli Zhong, Sami Alsheikh, Hanspeter Pfister, Aude Oliva, and Fredo Durand. 2018. Synthetically Trained Icon Proposals for Parsing and Summarizing Infographics. arXiv preprint arXiv:1807.10441 (2018).
[35]
Nolwenn Maudet, Ghita Jalal, Philip Tchernavskij, Michel Beaudouin-Lafon, and Wendy E. Mackay. 2017. Beyond Grids: Interactive Graphical Substrates to Structure Digital Layout. In Proceedings SIGCHI (CHI '17). ACM, NY, NY, USA, 5053--5064.
[36]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv:1301.3781 (2013).
[37]
David R. Millen. 2000. Rapid Ethnography: Time deepening strategies for HCI field research. Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (2000), 280--286.
[38]
George A Miller. 1995. WordNet: a lexical database for English. Commun. ACM 38, 11 (1995), 39--41.
[39]
Peter O'Donovan, Aseem Agarwala, and Aaron Hertzmann. 2015. DesignScape: Design with Interactive Layout Suggestions. In Proceedings of SIGCHI (CHI '15). ACM, NY, NY, USA, 1221--1224.
[40]
Peter O'Donovan, Janis Lebeks, Aseem Agarwala, and Aaron Hertzmann. 2014. Exploratory Font Selection Using Crowdsourced Attributes. ACM TOG 33, 4, Article 92 (July 2014), 9 pages.
[41]
Yuting Qiang, Yanwei Fu, Yanwen Guo, Zhi-Hua Zhou, and Leonid Sigal. 2016. Learning to generate posters of scientific papers. In Proceedings AAAI.
[42]
Daniel Ritchie, Ankita Arvind Kejriwal, and Scott R. Klemmer. 2011. D.Tour: Style-based Exploration of Design Example Galleries. In Proceedings UIST (UIST '11). ACM, NY, NY, USA, 165--174.
[43]
Gabriele Rohr. 1986. Using Visual Concepts. Springer, Boston, MA, 325--348.
[44]
Vidya Setlur, Conrad Albrecht-Buehler, Amy A. Gooch, Sam Rossoff, and Bruce Gooch. 2005. Semanticons: Visual metaphors as file icons. In Computer Graphics Forum, Vol. 24. Wiley Online Library, 647--656.
[45]
Vidya Setlur and Jock D. Mackinlay. 2014. Automatic Generation of Semantic Icon Encodings for Visualizations. In Proceedings SIGCHI. ACM, 541--550.
[46]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[47]
Maarten Someren, Yvonne Barnard, and Jacobijn Sandberg. 1994. The Think Aloud Method - A Practical Guide to Modelling CognitiveProcesses.
[48]
Amanda Swearngin, Mira Dontcheva, Wilmot Li, Joel Brandt, Morgan Dixon, and Andrew J. Ko. 2018. Rewire: Interface Design Assistance from Examples. In Proceedings SIGCHI (CHI '18). ACM, NY, NY, USA, Article 504, 12 pages.
[49]
Anna Szekely, Thomas Jacobsen, Simona D'Amico, Antonella Devescovi, Elena Andonova, Daniel Herron, Ching Ching Lu, Thomas Pechmann, Csaba Pléh, Nicole Wicha, and others. 2004. A new on-line resource for psycholinguistic studies. Journal of memory and language 51, 2 (2004), 247--250.
[50]
Alex Taikh, Ian S Hargreaves, Melvin J Yap, and Penny M Pexman. 2015. Semantic classification of pictures and words. The Quarterly Journal of Experimental Psychology 68, 8 (2015), 1502--1518.
[51]
Kashyap Todi, Daryl Weir, and Antti Oulasvirta. 2016. Sketchplore: Sketch and Explore with a Layout Optimiser. In Proceedings DIS (DIS '16). ACM, NY, NY, USA, 543--555.
[52]
Haijun Xia, Bruno Araujo, and Daniel Wigdor. 2017. Collection Objects: Enabling Fluid Formation and Manipulation of Aggregate Selections. In Proceedings SIGCHI (CHI '17). ACM, NY, NY, USA, 5592--5604.
[53]
Nanxuan Zhao, Ying Cao, and Rynson WH Lau. 2018. What characterizes personalities of graphic designs? ACM TOG 37, 4 (2018), 116.
[54]
Xinru Zheng, Xiaotian Qiao, Ying Cao, and Rynson W. H. Lau. 2019. Content-aware Generative Modeling of Graphic Design Layouts. ACM TOG 38, 4, Article 133 (July 2019), 15 pages.
[55]
C Lawrence Zitnick and Devi Parikh. 2013. Bringing semantics into focus using visual abstraction. In IEEE Proceedings CVPR.
[56]
C Lawrence Zitnick, Ramakrishna Vedantam, and Devi Parikh. 2016. Adopting abstract images for semantic scene understanding. IEEE TPAMI 38, 4 (2016), 627--638.

Cited By

View all
  • (2024)Generative AI for Secure User Interface (UI) DesignReshaping CyberSecurity With Generative AI Techniques10.4018/979-8-3693-5415-5.ch010(333-394)Online publication date: 13-Sep-2024
  • (2024)Demystifying Tacit Knowledge in Graphic Design: Characteristics, Instances, Approaches, and GuidelinesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642886(1-18)Online publication date: 11-May-2024
  • (2024)CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642794(1-25)Online publication date: 11-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
April 2020
10688 pages
ISBN:9781450367080
DOI:10.1145/3313831
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. compound icon
  2. design tools
  3. graphic design
  4. icon design
  5. ideogram
  6. pictogram

Qualifiers

  • Research-article

Conference

CHI '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)188
  • Downloads (Last 6 weeks)13
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Generative AI for Secure User Interface (UI) DesignReshaping CyberSecurity With Generative AI Techniques10.4018/979-8-3693-5415-5.ch010(333-394)Online publication date: 13-Sep-2024
  • (2024)Demystifying Tacit Knowledge in Graphic Design: Characteristics, Instances, Approaches, and GuidelinesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642886(1-18)Online publication date: 11-May-2024
  • (2024)CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642794(1-25)Online publication date: 11-May-2024
  • (2024)TypeDance: Creating Semantic Typographic Logos from Image through Personalized GenerationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642185(1-18)Online publication date: 11-May-2024
  • (2024)Intelligent Graphic Layout Generation: Current Status and Future Perspectives2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD61410.2024.10580676(2632-2637)Online publication date: 8-May-2024
  • (2024)Coherent visual design through attribute-specific feedback: a hybrid approach to intelligent design agentsDigital Creativity10.1080/14626268.2023.230135534:4(311-330)Online publication date: 7-Jan-2024
  • (2024)When and How to Use AI in the Design Process? Implications for Human-AI Design CollaborationInternational Journal of Human–Computer Interaction10.1080/10447318.2024.235345141:2(1569-1584)Online publication date: 22-May-2024
  • (2023)Understanding Design Collaboration Between Designers and Artificial Intelligence: A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36102177:CSCW2(1-35)Online publication date: 4-Oct-2023
  • (2023)The Prompt ArtistsProceedings of the 15th Conference on Creativity and Cognition10.1145/3591196.3593515(75-87)Online publication date: 19-Jun-2023
  • (2023)Cells, Generators, and Lenses: Design Framework for Object-Oriented Interaction with Large Language ModelsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606833(1-18)Online publication date: 29-Oct-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media