skip to main content
research-article

Juxtaform: interactive visual summarization for exploratory shape design

Published: 26 July 2023 Publication History

Abstract

We present juxtaform, a novel approach to the interactive summarization of large shape collections for conceptual shape design. We conduct a formative study to ascertain design goals for creative shape exploration tools. Motivated by a mathematical formulation of these design goals, juxtaform integrates the exploration, analysis, selection, and refinement of large shape collections to support an interactive divergence-convergence shape design workflow. We exploit sparse, segmented sketch-stroke visual abstractions of shape and a novel visual summarization algorithm to balance the needs of shape understanding, in-situ shape juxtaposition, and visual clutter. Our evaluation is three-fold: we show that existing shape and stroke clustering algorithms do not address our design goals compared to our proposed shape corpus summarization algorithm; we compare juxtaform against a structured image gallery interface for various shape design and analysis tasks; and we present multiple compelling 2D/3D applications using juxtaform.

Supplementary Material

ZIP File (papers_698-supplemental.zip)
supplemental material
MP4 File (papers_698_VOD.mp4)
presentation

References

[1]
Aseem Agarwala, Mira Dontcheva, Maneesh Agrawala, Steven Drucker, Alex Colburn, Brian Curless, David Salesin, and Michael Cohen. 2004. Interactive digital photomontage. In ACM SIGGRAPH 2004 Papers. 294--302.
[2]
Andrés Arias-Rosales. 2022. The perceived value of human-AI collaboration in early shape exploration: An exploratory assessment. PloS one 17, 9 (2022), e0274496.
[3]
Rahul Arora, Ishan Darolia, Vinay P Namboodiri, Karan Singh, and Adrien Bousseau. 2017. Sketchsoup: Exploratory ideation using design sketches. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 302--312.
[4]
Rahul Arora, Rubaiat Habib Kazi, Tovi Grossman, George Fitzmaurice, and Karan Singh. 2018. Symbiosissketch: Combining 2d & 3d sketching for designing detailed 3d objects in situ. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--15.
[5]
Jackie Assa, Yaron Caspi, and Daniel Cohen-Or. 2005. Action synopsis: pose selection and illustration. ACM Transactions on Graphics (TOG) 24, 3 (2005), 667--676.
[6]
Melinos Averkiou, Vladimir G Kim, Youyi Zheng, and Niloy J Mitra. 2014. Shapesynth: Parameterizing model collections for coupled shape exploration and synthesis. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 125--134.
[7]
Pierre Bénard and Aaron Hertzmann. 2019. Line Drawings from 3D Models: A Tutorial. Found. Trends. Comput. Graph. Vis. 11, 1--2 (sep 2019), 1--159.
[8]
Silvia Biasotti, Andrea Cerri, Alex Bronstein, and Michael Bronstein. 2016a. Recent trends, applications, and perspectives in 3d shape similarity assessment. In Computer graphics forum, Vol. 35. Wiley Online Library, 87--119.
[9]
Silvia Biasotti, Bianca Falcidieno, Daniela Giorgi, and Michela Spagnuolo. 2016b. 3D Objects Exploration: Guidelines for Future Research. In 3DOR@ Eurographics.
[10]
Silvia Biasotti, Elia Moscoso Thompson, and Michela Spagnuolo. 2019. Context-adaptive navigation of 3D model collections. Computers & Graphics 79 (2019), 1--13.
[11]
Umberto Castellani and Adrien Bartoli. 2020. 3d shape registration. In 3D Imaging, Analysis and Applications. Springer, 353--411.
[12]
Tony F Chan and Luminita A Vese. 2001. Active contours without edges. IEEE Transactions on image processing 10, 2 (2001), 266--277.
[13]
Andy Cockburn, Amy Karlson, and Benjamin B Bederson. 2009. A review of overview+ detail, zooming, and focus+ context interfaces. ACM Computing Surveys (CSUR) 41, 1 (2009), 1--31.
[14]
Daniel Cohen-Or and Hao Zhang. 2016. From Inspired Modeling to Creative Modeling. Vis. Comput. 32, 1 (jan 2016), 7--14.
[15]
Noa Fish, Melinos Averkiou, Oliver Van Kaick, Olga Sorkine-Hornung, Daniel Cohen-Or, and Niloy J Mitra. 2014. Meta-representation of shape families. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1--11.
[16]
Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. 2022. GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images. In Advances In Neural Information Processing Systems.
[17]
Lin Gao, Yan-Pei Cao, Yu-Kun Lai, Hao-Zhi Huang, Leif Kobbelt, and Shi-Min Hu. 2014. Active exploration of large 3d model repositories. IEEE transactions on visualization and computer graphics 21, 12 (2014), 1390--1402.
[18]
Yunhao Ge, Sami Abu-El-Haija, Gan Xin, and Laurent Itti. 2021. Zero-shot Synthesis with Group-Supervised Learning. In International Conference on Learning Representations. https://openreview.net/forum?id=8wqCDnBmnrT
[19]
Stéphane Grabli, Frédo Durand, and Francois X Sillion. 2004. Density measure for line-drawing simplification. In 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings. IEEE, 309--318.
[20]
Yulia Gryaditskaya, Felix Hähnlein, Chenxi Liu, Alla Sheffer, and Adrien Bousseau. 2020. Lifting freehand concept sketches into 3D. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--16.
[21]
Felix Hähnlein, Changjian Li, Niloy J Mitra, and Adrien Bousseau. 2022. CAD2Sketch: Generating Concept Sketches from CAD Sequences. ACM Transactions on Graphics (TOG) 41, 6 (2022), 1--18.
[22]
Almoctar Hassoumi, María Jesús Lobo, Gabriel Jarry, Vsevolod Peysakhovich, and Christophe Hurter. 2019. Interactive shape based brushing technique for trail sets. (2019).
[23]
James W Hennessey, Han Liu, Holger Winnemöller, Mira Dontcheva, and Niloy J Mitra. 2016. How2Sketch: generating easy-to-follow tutorials for sketching 3D objects. arXiv preprint arXiv:1607.07980 (2016).
[24]
Josh Holinaty, Alec Jacobson, and Fanny Chevalier. 2021. Supporting Reference Imagery for Digital Drawing. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2434--2442.
[25]
Min Hua. 2019. The roles of sketching in supporting creative design. The Design Journal 22, 6 (2019), 895--904.
[26]
Ruqi Huang, Panos Achlioptas, Leonidas Guibas, and Maks Ovsjanikov. 2019. Limit Shapes - A Tool for Understanding Shape Differences and Variability in 3D Model Collections. SGP (2019).
[27]
Shi-Sheng Huang, Ariel Shamir, Chao-Hui Shen, Hao Zhang, Alla Sheffer, Shi-Min Hu, and Daniel Cohen-Or. 2013. Qualitative organization of collections of shapes via quartet analysis. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1--10.
[28]
Moos Hueting, Maks Ovsjanikov, and Niloy J Mitra. 2015. CrossLink: joint understanding of image and 3D model collections through shape and camera pose variations. ACM Transactions on Graphics (ToG) 34, 6 (2015), 1--13.
[29]
Arjun Jain, Thorsten Thormählen, Tobias Ritschel, and Hans-Peter Seidel. 2012. Exploring shape variations by 3d-model decomposition and part-based recombination. In Computer Graphics Forum, Vol. 31. Wiley Online Library, 631--640.
[30]
Waqas Javed and Niklas Elmqvist. 2012. Exploring the design space of composite visualization. In 2012 ieee pacific visualization symposium. IEEE, 1--8.
[31]
Vladimir G Kim, Wilmot Li, Niloy J Mitra, Stephen DiVerdi, and Thomas Funkhouser. 2012. Exploring collections of 3d models using fuzzy correspondences. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1--11.
[32]
Yanir Kleiman, Noa Fish, Joel Lanir, and Daniel Cohen-Or. 2013. Dynamic maps for exploring and browsing shapes. In Computer Graphics Forum, Vol. 32. Wiley Online Library, 187--196.
[33]
Heidi Lam and Tamara Munzner. 2010. A guide to visual multi-level interface design from synthesis of empirical study evidence. Synthesis Lectures on Visualization 1, 1 (2010), 1--117.
[34]
Yong Jae Lee, C Lawrence Zitnick, and Michael F Cohen. 2011. Shadowdraw: real-time user guidance for freehand drawing. ACM Transactions on Graphics (TOG) 30, 4 (2011), 1--10.
[35]
You-En Lin, Yong-Liang Yang, and Hung-Kuo Chu. 2018. Scale-aware black-and-white abstraction of 3D shapes. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1--11.
[36]
Chenxi Liu, Enrique Rosales, and Alla Sheffer. 2018. Strokeaggregator: Consolidating raw sketches into artist-intended curve drawings. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1--15.
[37]
Y-C Liu, Amaresh Chakrabarti, and T Bligh. 2003. Towards an 'ideal'approach for concept generation. Design studies 24, 4 (2003), 341--355.
[38]
Zhen-Bao Liu, Shu-Hui Bu, Kun Zhou, Shu-Ming Gao, Jun-Wei Han, and Jun Wu. 2013. A survey on partial retrieval of 3D shapes. Journal of Computer Science and Technology 28, 5 (2013), 836--851.
[39]
Justin Matejka, Michael Glueck, Erin Bradner, Ali Hashemi, Tovi Grossman, and George Fitzmaurice. 2018. Dream Lens: Exploration and Visualization of Large-Scale Generative Design Datasets. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--12.
[40]
Harry A Nichols. 1954. DESIGN FEATURES OF THE DOUGLAS A3D SKYWARRIOR. Technical Report. SAE Technical Paper.
[41]
Jamie R. Nuñez, Christopher R. Anderton, and Ryan S. Renslow. 2018. Optimizing colormaps with consideration for color vision deficiency to enable accurate interpretation of scientific data. PLOS ONE 13, 7 (08 2018), 1--14.
[42]
Luke Olsen, Faramarz F Samavati, Mario Costa Sousa, and Joaquim A Jorge. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1 (2009), 85--103.
[43]
Günay Orbay, Mehmet Ersin Yümer, and Levent Burak Kara. 2012. Sketch-based aesthetic product form exploration from existing images using piecewise clothoid curves. Journal of Visual Languages & Computing 23, 6 (2012), 327--339.
[44]
Maks Ovsjanikov, Wilmot Li, Leonidas Guibas, and Niloy J Mitra. 2011. Exploration of continuous variability in collections of 3d shapes. ACM Transactions on Graphics (TOG) 30, 4 (2011), 1--10.
[45]
Patrick Paczkowski, Min H Kim, Yann Morvan, Julie Dorsey, Holly E Rushmeier, and Carol O'Sullivan. 2011. Insitu: sketching architectural designs in context. ACM Trans. Graph. 30, 6 (2011), 182.
[46]
Mark Pauly, Niloy J Mitra, Johannes Wallner, Helmut Pottmann, and Leonidas J Guibas. 2008. Discovering structural regularity in 3D geometry. In ACM SIGGRAPH 2008 papers. 1--11.
[47]
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in Python. the Journal of machine Learning research 12 (2011), 2825--2830.
[48]
Miquel Prats. 2007. Shape exploration in product design: assisting transformation in pictorial representations. Open University (United Kingdom).
[49]
Miquel Prats and Chris F Earl. 2006. Exploration through drawings in the conceptual stage of product design. In Design Computing and Cognition'06. Springer, 83--102.
[50]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2021. High-Resolution Image Synthesis with Latent Diffusion Models. arXiv:2112.10752 [cs.CV]
[51]
Ryan M. Schmidt and Karan Singh. 2010. meshmixer: an interface for rapid mesh composition. In SIGGRAPH Talks, James L. Mohler (Ed.). ACM. http://dblp.unitrier.de/db/conf/siggraph/siggraph2010talks.html#SchmidtS10
[52]
Donald A Schon and Glenn Wiggins. 1992. Kinds of seeing and their functions in designing. Design studies 13, 2 (1992), 135--156.
[53]
Adriana Schulz, Ariel Shamir, Ilya Baran, David IW Levin, Pitchaya Sitthi-Amorn, and Wojciech Matusik. 2017. Retrieval on parametric shape collections. ACM Transactions on Graphics (TOG) 36, 1 (2017), 1--14.
[54]
Ben Shneiderman. 1999. User interfaces for creativity support tools. In Proceedings of the 3rd conference on Creativity & cognition. 15--22.
[55]
Tim Smithers. 2001. Is sketching an aid to memory or a kind of thinking? (2001).
[56]
Johan WH Tangelder and Remco C Veltkamp. 2008. A survey of content based 3D shape retrieval methods. Multimedia tools and applications 39, 3 (2008), 441--471.
[57]
Erdogan Taskesen. 2021. Python package clustimage is for unsupervised clustering of images. https://erdogant.github.io/clustimage
[58]
James T Todd. 2004. The visual perception of 3D shape. Trends in cognitive sciences 8, 3 (2004), 115--121.
[59]
Steve Tsang, Ravin Balakrishnan, Karan Singh, and Abhishek Ranjan. 2004. A Suggestive Interface for Image Guided 3D Sketching. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria) (CHI '04). Association for Computing Machinery, New York, NY, USA, 591--598.
[60]
Oliver Van Kaick, Hao Zhang, Ghassan Hamarneh, and Daniel Cohen-Or. 2011. A survey on shape correspondence. In Computer graphics forum, Vol. 30. Wiley Online Library, 1681--1707.
[61]
Dave Pagurek Van Mossel, Chenxi Liu, Nicholas Vining, Mikhail Bessmeltsev, and Alla Sheffer. 2021. StrokeStrip: joint parameterization and fitting of stroke clusters. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1--18.
[62]
Remco C Veltkamp. 2001. Shape matching: Similarity measures and algorithms. In Proceedings International Conference on Shape Modeling and Applications. IEEE, 188--197.
[63]
Aaron Wolin, Brian Eoff, and Tracy Hammond. 2008. ShortStraw: A Simple and Effective Corner Finder for Polylines. SBIM 8 (2008), 33--40.
[64]
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1912--1920.
[65]
Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. 2014a. True2Form: 3D curve networks from 2D sketches via selective regularization. ACM Transactions on Graphics 33, 4 (2014).
[66]
Kai Xu, Rui Ma, Hao Zhang, Chenyang Zhu, Ariel Shamir, Daniel Cohen-Or, and Hui Huang. 2014b. Organizing heterogeneous scene collections through contextual focal points. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1--12.
[67]
Kai Xu, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2012. Fit and diverse: Set evolution for inspiring 3d shape galleries. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1--10.
[68]
Chuan Yan, David Vanderhaeghe, and Yotam Gingold. 2020. A benchmark for rough sketch cleanup. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--14.
[69]
Hui Ye, Kin Chung Kwan, and Hongbo Fu. 2021. 3D curve creation on and around physical objects with mobile AR. IEEE transactions on visualization and computer graphics 28, 8 (2021), 2809--2821.
[70]
Jun-Yan Zhu, Yong Jae Lee, and Alexei A Efros. 2014. Averageexplorer: Interactive exploration and alignment of visual data collections. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1--11.

Cited By

View all
  • (2024)Toward a More Comprehensive Understanding of Visualization LiteracyExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636289(1-7)Online publication date: 11-May-2024
  • (2024)Exploring Interactive Color Palettes for Abstraction-Driven Exploratory Image ColorizationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642223(1-16)Online publication date: 11-May-2024

Index Terms

  1. Juxtaform: interactive visual summarization for exploratory shape design

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Transactions on Graphics
        ACM Transactions on Graphics  Volume 42, Issue 4
        August 2023
        1912 pages
        ISSN:0730-0301
        EISSN:1557-7368
        DOI:10.1145/3609020
        Issue’s Table of Contents
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 26 July 2023
        Published in TOG Volume 42, Issue 4

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. shape exploration
        2. shape design
        3. sketch-based interaction

        Qualifiers

        • Research-article

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)114
        • Downloads (Last 6 weeks)12
        Reflects downloads up to 16 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Toward a More Comprehensive Understanding of Visualization LiteracyExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636289(1-7)Online publication date: 11-May-2024
        • (2024)Exploring Interactive Color Palettes for Abstraction-Driven Exploratory Image ColorizationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642223(1-16)Online publication date: 11-May-2024

        View Options

        Login options

        Full Access

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media