skip to main content
10.1145/2807442.2807448acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Procedural Modeling Using Autoencoder Networks

Published: 05 November 2015 Publication History

Abstract

Procedural modeling systems allow users to create high quality content through parametric, conditional or stochastic rule sets. While such approaches create an abstraction layer by freeing the user from direct geometry editing, the nonlinear nature and the high number of parameters associated with such design spaces result in arduous modeling experiences for non-expert users. We propose a method to enable intuitive exploration of such high dimensional procedural modeling spaces within a lower dimensional space learned through autoencoder network training. Our method automatically generates a representative training dataset from the procedural modeling rule set based on shape similarity features. We then leverage the samples in this dataset to train an autoencoder neural network, while also structuring the learned lower dimensional space for continuous exploration with respect to shape features. We demonstrate the efficacy our method with user studies where designers create content with more than 10-fold faster speeds using our system compared to the classic procedural modeling interface.

Supplementary Material

suppl.mov (uist1227-file4.mp4)
Supplemental video
MP4 File (p109.mp4)

References

[1]
1. I. Alhashim, H. Li, K. Xu, J. Cao, R. Ma, and H. Zhang. 2014. Topology-varying 3d shape creation via structural blending. ACM Trans. Graph. 33, 4 (2014), 158.
[2]
2. Melinos Averkiou, Vladimir G Kim, Youyi Zheng, and Niloy J Mitra. 2014. Shapesynth: Parameterizing model collections for coupled shape exploration and synthesis. In Computer Graphics Forum, Vol. 33. 125--134.
[3]
3. Serge Belongie, Jitendra Malik, and Jan Puzicha. 2000. Shape context: A new descriptor for shape matching and object recognition. In NIPS, Vol. 2. 3.
[4]
4. Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle. 2007. Greedy layer-wise training of deep networks. Advances in neural information processing systems 19 (2007), 153.
[5]
5. A. Bhattacharjee, Z. Shams, and K. Z. Sultana. 2006. New Constraints on Generation of Uniform Random Samples from Evolutionary Trees. In CCECE. 115--118.
[6]
6. S. Chaudhuri, E. Kalogerakis, S. Giguere, and T. Funkhouser. 2013. Attribit: content creation with semantic attributes. In UIST. 193--202.
[7]
7. D. Chen, X. Tian, Y. Shen, and M. Ouhyoung. 2003. On visual similarity based 3D model retrieval. In Computer graphics forum, Vol. 22. 223--232.
[8]
8. George Dahl, Abdel-rahman Mohamed, Geoffrey E Hinton, and others. 2010. Phone recognition with the mean-covariance restricted Boltzmann machine. In Advances in neural information processing systems. 469--477.
[9]
9. Alexey Dosovitskiy, Jost Tobias Springenberg, and Thomas Brox. 2014. Learning to generate chairs with convolutional neural networks. arXiv preprint arXiv:1411.5928 (2014).
[10]
10. Kunihiko Fukushima. 1980. Neocognitron: A self-organizing network model for a mechanism of pattern recognition unaffected by shift in position. Biological cybernetics 36, 4 (1980), 193--202.
[11]
11. T Germer and M Schwarz. 2009. Procedural Arrangement of Furniture for Real-Time Walkthroughs. In Computer Graphics Forum, Vol. 28. 2068--2078.
[12]
12. Geoffrey Hinton, Simon Osindero, and Yee-Whye Teh. 2006. A fast learning algorithm for deep belief nets. Neural computation 18, 7 (2006), 1527--1554.
[13]
13. Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313, 5786 (2006), 504--507.
[14]
14. S. Huang, A. Shamir, C. Shen, H. Zhang, A. Sheffer, S. Hu, and D. Cohen-Or. 2013. Qualitative organization of collections of shapes via quartet analysis. ACM Trans. Graph. 32, 4 (2013), 71.
[15]
15. Arjun Jain, Thorsten Thormählen, Tobias Ritschel, and Hans-Peter Seidel. 2012. Exploring Shape Variations by 3D-Model Decomposition and Part-based Recombination. In Computer Graphics Forum, Vol. 31. 631--640.
[16]
16. Evangelos Kalogerakis, Siddhartha Chaudhuri, Daphne Koller, and Vladlen Koltun. 2012. A probabilistic model for component-based shape synthesis. ACM Trans. Graph. 31, 4 (2012), 55.
[17]
17. P. Kearney, J. I. Munro, and D. Phillips. 2003. Efficient generation of uniform samples from phylogenetic trees. In Algorithms in Bioinformatics. 177--189.
[18]
18. Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2014. Crowd-powered parameter analysis for visual design exploration. In UIST. 65--74.
[19]
19. Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. 1989. Backpropagation applied to handwritten zip code recognition. Neural computation 1, 4 (1989), 541--551.
[20]
20. Stefan Lienhard, Matthias Specht, Boris Neubert, Mark Pauly, and Pascal Müller. 2014. Thumbnail galleries for procedural models. In Computer Graphics Forum, Vol. 33. 361--370.
[21]
21. Aristid Lindenmayer. 1968. Mathematical models for cellular interactions in development. Journal of theoretical biology 18, 3 (1968), 280--299.
[22]
22. Bernd Lintermann and Oliver Deussen. 1999. Interactive modeling of plants. Computer Graphics and Applications 19, 1 (1999), 56--65.
[23]
23. Charles T Loop and Tony D DeRose. 1989. A multisided generalization of Bézier surfaces. ACM Trans. Graph. 8, 3 (1989), 204--234.
[24]
24. Joe Marks, Brad Andalman, and others. 1997. Design galleries: A general approach to setting parameters for computer graphics and animation. In ACM SIGGRAPH. 389--400.
[25]
25. James McCrae and Karan Singh. 2009. Sketch-based Path Design. In Proceedings of Graphics Interface 2009. 95--102.
[26]
26. Radomír Měch and Gavin Miller. 2012. The Deco Framework for Interactive Procedural Modeling. JCGT 1, 1 (2012), 43--99.
[27]
27. Radomír Měch and Przemyslaw Prusinkiewicz. 1996. Visual models of plants interacting with their environment. In ACM SIGGRAPH. 397--410.
[28]
28. Paul Merrell, Eric Schkufza, and Vladlen Koltun. 2010. Computer-generated residential building layouts. In ACM Trans. Graph., Vol. 29. 181.
[29]
29. Pascal Müller, Peter Wonka, Simon Haegler, Andreas Ulmer, and Luc Van Gool. 2006. Procedural modeling of buildings. ACM Trans. Graph. 25, 3 (2006), 614--623.
[30]
30. Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y Ng. 2011. Multimodal deep learning. In ICML. 689--696.
[31]
31. Yoav IH Parish and Pascal Müller. 2001. Procedural modeling of cities. In ACM SIGGRAPH. ACM, 301--308.
[32]
32. Przemyslaw Prusinkiewicz. 1986. Graphical applications of L-systems. In Proceedings of graphics interface, Vol. 86. 247--253.
[33]
33. Przemyslaw Prusinkiewicz, Mark James, and Radomír Měch. 1994. Synthetic topiary. In ACM SIGGRAPH. 351--358.
[34]
34. Przemyslaw Prusinkiewicz and Aristid Lindenmayer. 1990. The Algorithmic Beauty of Plants. Springer-Verlag, New York.
[35]
35. M Ranzato, Fu Jie Huang, Y-L Boureau, and Yann LeCun. 2007. Unsupervised learning of invariant feature hierarchies with applications to object recognition. In IEEE CVPR. 1--8.
[36]
36. Michael Schwarz and Peter Wonka. 2014. Procedural Design of Exterior Lighting for Buildings with Complex Constraints. ACM Trans. Graph. 33, 5 (2014), 166.
[37]
37. Pierre Sermanet, Koray Kavukcuoglu, Soumith Chintala, and Yann LeCun. 2013. Pedestrian detection with unsupervised multi-stage feature learning. In IEEE CVPR. 3626--3633.
[38]
38. Lior Shapira, Ariel Shamir, and Daniel Cohen-Or. 2009. Image Appearance Exploration by Model-Based Navigation. In Computer Graphics Forum, Vol. 28. 629--638.
[39]
39. J. Shotton, J. Winn, C. Rother, and A. Criminisi. 2009. Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. IJCV 81, 1 (2009), 2--23.
[40]
40. Ruben M. Smelik, Tim Tutenel, Rafael Bidarra, and Bedrich Benes. 2014. A Survey on Procedural Modelling for Virtual Worlds. Computer Graphics Forum 33, 6 (2014), 31--50.
[41]
41. Sagi Snir and Satish Rao. 2010. Quartets MaxCut: a divide and conquer quartets algorithm. IEEE TCBB 7, 4 (2010), 704--718.
[42]
42. O Stava, S Pirk, J Kratt, B Chen, R Měch, O Deussen, and B Benes. 2014. Inverse Procedural Modelling of Trees. In Computer Graphics Forum.
[43]
43. George Stiny and James Gips. 1971. Shape Grammars and the Generative Specification of Painting and Sculpture. In IFIP Congress (2). 1460--1465.
[44]
44. Jerry O Talton, Daniel Gibson, Lingfeng Yang, Pat Hanrahan, and Vladlen Koltun. 2009. Exploratory modeling with collaborative design spaces. ACM Trans. Graph. 28, 5 (2009), 167.
[45]
45. Jerry O Talton, Yu Lou, Steve Lesser, Jared Duke, Radomír Měch, and Vladlen Koltun. 2011. Metropolis procedural modeling. ACM Trans. Graph. 30, 2 (2011), 11.
[46]
46. Nobuyuki Umetani, Takeo Igarashi, and Niloy J Mitra. 2012. Guided exploration of physically valid shapes for furniture design. ACM Trans. Graph. 31, 4 (2012), 86.
[47]
47. Carlos A Vanegas, Ignacio Garcia-Dorado, Daniel G Aliaga, Bedrich Benes, and Paul Waddell. 2012. Inverse design of urban procedural models. ACM Trans. Graph. 31, 6 (2012), 168.
[48]
48. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In ICML. 1096--1103.
[49]
49. Jason Weber and Joseph Penn. 1995. Creation and rendering of realistic trees. In ACM SIGGRAPH. 119--128.
[50]
50. Chris White. 2006. King Kong: the building of 1933 New York City. In ACM SIGGRAPH Sketches. 96.
[51]
51. S. J. Willson. 1999. Building phylogenetic trees from quartets by using local inconsistency measures. Molecular Biology and Evolution 16, 5 (1999), 685.
[52]
52. Peter Wonka, Michael Wimmer, Francçois Sillion, and William Ribarsky. 2003. Instant architecture. In ACM Trans. Graph., Vol. 22. 669--667.
[53]
53. Kai Xu, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2012. Fit and diverse: set evolution for inspiring 3D shape galleries. ACM Trans. Graph. 31, 4 (2012), 57.
[54]
54. M. E. Yumer, S. Chaudhuri, J. K. Hodgins, and L. B. Kara. 2015. Semantic Shape Editing Using Deformation Handles. ACM Trans. Graph. 34 (2015). Issue 4.

Cited By

View all
  • (2025)GeoCode: Interpretable Shape ProgramsComputer Graphics Forum10.1111/cgf.15276Online publication date: 12-Feb-2025
  • (2024)Accelerated forest modeling from tree canopy point clouds via deep learningInternational Journal of Applied Earth Observation and Geoinformation10.1016/j.jag.2024.104074132(104074)Online publication date: Aug-2024
  • (2023)Understanding Design Collaboration Between Designers and Artificial Intelligence: A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36102177:CSCW2(1-35)Online publication date: 4-Oct-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
UIST '15: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology
November 2015
686 pages
ISBN:9781450337793
DOI:10.1145/2807442
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 November 2015

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. autoencoders
  2. neural networks
  3. parametric shape design
  4. procedural modeling

Qualifiers

  • Research-article

Conference

UIST '15

Acceptance Rates

UIST '15 Paper Acceptance Rate 70 of 297 submissions, 24%;
Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25
The 38th Annual ACM Symposium on User Interface Software and Technology
September 28 - October 1, 2025
Busan , Republic of Korea

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)39
  • Downloads (Last 6 weeks)1
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)GeoCode: Interpretable Shape ProgramsComputer Graphics Forum10.1111/cgf.15276Online publication date: 12-Feb-2025
  • (2024)Accelerated forest modeling from tree canopy point clouds via deep learningInternational Journal of Applied Earth Observation and Geoinformation10.1016/j.jag.2024.104074132(104074)Online publication date: Aug-2024
  • (2023)Understanding Design Collaboration Between Designers and Artificial Intelligence: A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36102177:CSCW2(1-35)Online publication date: 4-Oct-2023
  • (2023)Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language ModelsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606725(1-14)Online publication date: 29-Oct-2023
  • (2023)TreeSketchNet: From Sketch to 3D Tree Parameters GenerationACM Transactions on Intelligent Systems and Technology10.1145/357983114:3(1-29)Online publication date: 24-Mar-2023
  • (2023)Data‐guided Authoring of Procedural Models of ShapesComputer Graphics Forum10.1111/cgf.1493542:7Online publication date: 5-Nov-2023
  • (2023)User-Centric Synthetic Road Network Generation Using Graph Neural Networks and Graph Autoencoders2023 International Conference on Machine Learning and Applications (ICMLA)10.1109/ICMLA58977.2023.00175(1169-1174)Online publication date: 15-Dec-2023
  • (2022)BO as Assistant: Using Bayesian Optimization for Asynchronously Generating Design SuggestionsProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545664(1-14)Online publication date: 29-Oct-2022
  • (2021)A review of computer graphics approaches to urban modeling from a machine learning perspective机器学习视角下的城市建模计算机图形方法综述Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.200014122:7(915-925)Online publication date: 22-May-2021
  • (2021)Automated tree generation using grammar & particle systemProceedings of the Twelfth Indian Conference on Computer Vision, Graphics and Image Processing10.1145/3490035.3490285(1-9)Online publication date: 19-Dec-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media