skip to main content
research-article

SCORES: shape composition with recursive substructure priors

Authors Info & Claims
Published:04 December 2018Publication History
Skip Abstract Section

Abstract

We introduce SCORES, a recursive neural network for shape composition. Our network takes as input sets of parts from two or more source 3D shapes and a rough initial placement of the parts. It outputs an optimized part structure for the composed shape, leading to high-quality geometry construction. A unique feature of our composition network is that it is not merely learning how to connect parts. Our goal is to produce a coherent and plausible 3D shape, despite large incompatibilities among the input parts. The network may significantly alter the geometry and structure of the input parts and synthesize a novel shape structure based on the inputs, while adding or removing parts to minimize a structure plausibility loss. We design SCORES as a recursive autoencoder network. During encoding, the input parts are recursively grouped to generate a root code. During synthesis, the root code is decoded, recursively, to produce a new, coherent part assembly. Assembled shape structures may be novel, with little global resemblance to training exemplars, yet have plausible substructures. SCORES therefore learns a hierarchical substructure shape prior based on per-node losses. It is trained on structured shapes from ShapeNet, and is applied iteratively to reduce the plausibility loss. We show results of shape composition from multiple sources over different categories of man-made shapes and compare with state-of-the-art alternatives, demonstrating that our network can significantly expand the range of composable shapes for assembly-based modeling.

Skip Supplemental Material Section

Supplemental Material

References

  1. Ibraheem Alhashim, Honghua Li, Kai Xu, Junjie Cao, Rui Ma, and Hao Zhang. 2014. Topology-Varying 3D Shape Creation via Structural Blending. In SIGGRAPH. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Sara Ball. 1985. Croc-gu-phant. Ragged Bears.Google ScholarGoogle Scholar
  3. Martin Bokeloh, Michael Wand, and Hans-Peter Seidel. 2010. A Connection Between Partial Symmetry and Inverse Procedural Modeling. In SIGGRAPH. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Angel X. Chang, Thomas A. Funkhouser, Leonidas J. Guibas, Pat Hanrahan, Qi-Xing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. 2015. ShapeNet: An Information-Rich 3D Model Repository. CoRR abs/1512.03012 (2015).Google ScholarGoogle Scholar
  5. Siddhartha Chaudhuri, Evangelos Kalogerakis, Leonidas Guibas, and Vladlen Koltun. 2011. Probabilistic Reasoning for Assembly-based 3D Modeling. ACM Trans. Graph. 30, 4 (2011), 35:1--35:10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Siddhartha Chaudhuri and Vladlen Koltun. 2010. Data-Driven Suggestions for Creativity Support in 3D Modeling. ACM Trans. Graph. 29, 6 (2010), 183:1--9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Daniel Cohen-Or and Hao Zhang. 2016. From inspired modeling to creative modeling. The Visual Computer 32, 1 (2016), 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Jiajun Wu, Chengkai Zhang, Tianfan Xue, William T. Freeman, and Joshua B. Tenenbaum. 2016. Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling. In NIPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3D ShapeNets: A deep representation for volumetric shapes. In CVPR.Google ScholarGoogle Scholar
  10. Carl Doersch. 2016. Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908 (2016).Google ScholarGoogle Scholar
  11. Noah Duncan, Lap-Fai Yu, and Sai-Kit Yeung. 2016. Interchangeable Components for Hands-on Assembly Based Modelling. ACM Trans. Graph. 35, 6 (2016), 234:1--234:14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Thomas Funkhouser, Michael Kazhdan, Philip Shilane, Patrick Min, William Kiefer, Ayellet Tal, Szymon Rusinkiewicz, and David Dobkin. 2004. Modeling by Example. ACM Trans. Graph. 23, 3 (2004), 652--663. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Rohit Girdhar, David F Fouhey, Mikel Rodriguez, and Abhinav Gupta. 2016. Learning a predictable and generative vector representation for objects. In ECCV.Google ScholarGoogle Scholar
  14. Haibin Huang, Evangelos Kalogerakis, and Benjamin Marlin. 2015. Analysis and synthesis of 3D shape families via deep-learned generative models of surfaces. In SGP. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Arjun Jain, Thorsten Thormählen, Tobias Ritschel, and Hans-Peter Seidel. 2012. Exploring Shape Variations by 3D-Model Decomposition and Part-based Recombination. In Eurographics. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Evangelos Kalogerakis, Siddhartha Chaudhuri, Daphne Koller, and Vladlen Koltun. 2012. A probabilistic model for component-based shape synthesis. ACM Trans. Graph. (SIGGRAPH) 31, 4 (2012). Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Vladislav Kraevoy, Dan Julius, and Alla Sheffer. 2007. Shuffler: Modeling with Interchangeable Parts. In Pacific Graphics.Google ScholarGoogle Scholar
  18. Hamid Laga, Michela Mortara, and Michela Spagnuolo. 2013. Geometry and context for semantic correspondences and functionality recognition in man-made 3D shapes. ACM Trans. Graph. 32, 5 (2013), 150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Phong Le and Willem Zuidema. 2014. The Inside-Outside Recursive Neural Network model for Dependency Parsing.. In on EMNLP. 729--739.Google ScholarGoogle Scholar
  20. Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, and Leonidas Guibas. 2017. GRASS: Generative Recursive Autoencoders for Shape Structures. ACM Trans. Graph. 36, 4 (2017), 52:1--52:14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Niloy Mitra, Michael Wand, Hao Zhang, Daniel Cohen-Or, and Martin Bokeloh. 2013. Structure-aware shape processing. In Eurographics State-of-the-art Report (STAR).Google ScholarGoogle Scholar
  22. Daniel Ritchie, Sarah Jobalia, and Anna Thomas. 2018. Example-based Authoring of Procedural Modeling Programs with Structural and Continuous Variability. Comp. Graph. For. 37, 2 (2018), 401--413.Google ScholarGoogle ScholarCross RefCross Ref
  23. Adriana Schulz, Ariel Shamir, David I. W. Levin, Pitchaya Sitthi-amorn, and Wojciech Matusik. 2014. Design and Fabrication by Example. ACM Trans. Graph. 33, 4 (July 2014), 62:1--62:11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Andrei Sharf, Marina Blumenkrants, Ariel Shamir, and Daniel Cohen-Or. 2006. Snap-Paste: An Interactive Technique for Easy Mesh Composition. In Pacific Graphics.Google ScholarGoogle Scholar
  25. Chao-Hui Shen, Hongbo Fu, Kang Chen, and Shi-Min Hu. 2012. Structure Recovery by Part Assembly. ACM Trans. Graph. (SIGGRAPH Asia) 31, 6 (2012), 180:1--180:11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Richard Socher, Cliff C. Lin, Andrew Y. Ng, and Christopher D. Manning. 2011. Parsing Natural Scenes and Natural Language with Recursive Neural Networks. In ICML. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Minhyuk Sung, Vladimir G. Kim, Roland Angst, and Leonidas Guibas. 2015. Data-driven Structural Priors for Shape Completion. ACM Trans. Graph. 34, 6 (2015). Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Minhyuk Sung, Hao Su, Vladimir G. Kim, Siddhartha Chaudhuri, and Leonidas Guibas. 2017. ComplementMe: Weakly-Supervised Component Suggestions for3DModeling. ACM Trans. Graph. (SIGGRAPH Asia) (2017). Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Kenshi Takayama, Ryan Schmidt, Karan Singh, Takeo Igarashi, Tamy Boubekeur, and Olga Sorkine. 2011. GeoBrush: Interactive Mesh Geometry Cloning. Comp. Graph. For. (Eurographics) 30 (2011), 613--622.Google ScholarGoogle ScholarCross RefCross Ref
  30. Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural Discrete Representation Learning. CoRR abs/1711.00937 (2017).Google ScholarGoogle Scholar
  31. Wikipedia. 2017. Kitbashing --- Wikipedia, The Free Encyclopedia. (2017). https://en.wikipedia.org/wiki/Kitbashing {Online; accessed 23-December-2017}.Google ScholarGoogle Scholar
  32. Kai Xu, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2012. Fit and Diverse: Set Evolution for Inspiring 3D Shape Galleries. ACM Trans. Graph. 31, 4 (2012), 57:1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Youyi Zheng, Daniel Cohen-Or, Melinos Averkiou, and Niloy J Mitra. 2014. Recurring part arrangements in shape collections. Comp. Graph. For. 33, 2 (2014), 115--124. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Youyi Zheng, Daniel Cohen-Or, and Niloy J. Mitra. 2013. Smart Variations: Functional Substructures for Part Compatibility. Comp. Graph. For. 32, 2 (2013).Google ScholarGoogle Scholar
  35. Chenyang Zhu, Renjiao Yi, Wallace Lira, Ibraheem Alhashim, Kai Xu, and Hao Zhang. 2017. Deformation-driven shape correspondence via shape recognition. ACM Trans. Graph. 36, 4 (2017), 51. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. SCORES: shape composition with recursive substructure priors

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 37, Issue 6
      December 2018
      1401 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/3272127
      Issue’s Table of Contents

      Copyright © 2018 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 4 December 2018
      Published in tog Volume 37, Issue 6

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader