Abstract
We present the Shape Part Slot Machine, a new method for assembling novel 3D shapes from existing parts by performing contact-based reasoning. Our method represents each shape as a graph of “slots,” where each slot is a region of contact between two shape parts. Based on this representation, we design a graph-neural-network-based model for generating new slot graphs and retrieving compatible parts, as well as a gradient-descent-based optimization scheme for assembling the retrieved parts into a complete shape that respects the generated slot graph. This approach does not require any semantic part labels; interestingly, it also does not require complete part geometries—reasoning about the slots proves sufficient to generate novel, high-quality 3D shapes. We demonstrate that our method generates shapes that outperform existing modeling-by-assembly approaches regarding quality, diversity, and structural complexity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Averkiou, M., Kim, V., Zheng, Y., Mitra, N.J.: ShapeSynth: parameterizing model collections for coupled shape exploration and synthesis. In: Computer Graphics Forum (Special Issue of Eurographics 2014) (2014)
Chaudhuri, S., Kalogerakis, E., Giguere, S., Funkhouser, T.: ATTRIBIT: content creation with semantic attributes. In: ACM Symposium on User Interface Software and Technology (UIST) (2013)
Chaudhuri, S., Kalogerakis, E., Guibas, L., Koltun, V.: Probabilistic reasoning for assembly-based 3D modeling. ACM Trans. Graphics 30(4), 1–10 (2011)
Funkhouser, T., et al.: Modeling by example. In: ACM SIGGRAPH 2004 Papers (2004)
Gao, L., et al.: SDM-Net: deep generative network for structured deformable mesh. In: SIGGRAPH Asia (2019)
Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. CoRR arXiv:1704.01212 (2017)
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: NeurIPS (2017)
Huang, J., et al.: Generative 3D part assembly via dynamic graph learning. In: The IEEE Conference on Neural Information Processing Systems (NeurIPS) (2020)
Jain, A., Thormählen, T., Ritschel, T., Seidel, H.P.: Exploring shape variations by 3D-model decomposition and part-based recombination. Comput. Graph. Forum 31(2pt3), 631–640 (2012). https://doi.org/10.1111/j.1467-8659.2012.03042.x
Jones, R.K., et al.: ShapeAssembly: learning to generate programs for 3D shape structure synthesis. ACM Trans. Graphics (TOG), SIGGRAPH Asia 2020 39(6), Article 234 (2020)
Jones, R.K., Charatan, D., Guerrero, P., Mitra, N.J., Ritchie, D.: ShapeMOD: macro operation discovery for 3D shape programs. ACM Trans. Graphics (TOG) 40(4), 1–16 (2021)
Kalogerakis, E., Chaudhuri, S., Koller, D., Koltun, V.: A probabilistic model for component-based shape synthesis. ACM Trans. Graphics 31(4), 1–11 (2012)
Kreavoy, V., Julius, D., Sheffer, A.: Model composition from interchangeable components. In: Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, PG 2007, pp. 129–138. IEEE Computer Society, USA (2007). https://doi.org/10.1109/PG.2007.43
Li, J., Xu, K., Chaudhuri, S., Yumer, E., Zhang, H., Guibas, L.: GRASS: generative recursive autoencoders for shape structures. In: SIGGRAPH 2017 (2017)
Mo, K., et al.: StructureNet: hierarchical graph networks for 3D shape generation. In: SIGGRAPH Asia (2019)
Mo, K., et al.: PartNet: a large-scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 909–918 (2019)
Schor, N., Katzir, O., Zhang, H., Cohen-Or, D.: CompoNet: learning to generate the unseen by part synthesis and composition. In: The IEEE International Conference on Computer Vision (ICCV) (2019)
Shen, C.H., Fu, H., Chen, K., Hu, S.M.: Structure recovery by part assembly. ACM Trans. Graphics 31(6) (2012). https://doi.org/10.1145/2366145.2366199
Sung, M., Su, H., Kim, V.G., Chaudhuri, S., Guibas, L.: ComplementMe: weakly-supervised component suggestions for 3D modeling. ACM Trans. Graphics (TOG) 36(6), 226 (2017)
Wang, H., Schor, N., Hu, R., Huang, H., Cohen-Or, D., Huang, H.: Global-to-local generative model for 3D shapes. ACM Trans. Graphics (Proc. SIGGRAPH ASIA) 37(6), 214:1–214:10 (2018)
Wu, R., Zhuang, Y., Xu, K., Zhang, H., Chen, B.: PQ-Net: a generative part seq2seq network for 3D shapes. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Wu, Z., Wang, X., Lin, D., Lischinski, D., Cohen-Or, D., Huang, H.: SAGNet: structure-aware generative network for 3D-shape modeling. ACM Trans. Graphics (Proc. SIGGRAPH 2019) 38(4), 91:1–91:14 (2019)
Xie, X., et al.: Sketch-to-design: context-based part assembly. Comput. Graph. Forum 32(8), 233–245 (2013)
Xu, K., Zhang, H., Cohen-Or, D., Chen, B.: Fit and diverse: set evolution for inspiring 3D shape galleries. ACM Trans. Graphics (Proc. of SIGGRAPH 2012) 31(4), 57:1–57:10 (2012)
Yang, J., Mo, K., Lai, Y.K., Guibas, L.J., Gao, L.: DSM-Net: disentangled structured mesh net for controllable generation of fine geometry (2020)
Yin, K., Chen, Z., Chaudhuri, S., Fisher, M., Kim, V., Zhang, H.: COALESCE: component assembly by learning to synthesize connections. In: Proceedings of 3DV (2020)
Zou, C., Yumer, E., Yang, J., Ceylan, D., Hoiem, D.: 3D-PRNN: generating shape primitives with recurrent neural networks. In: ICCV 2017 (2017)
Acknowledgements
This work was funded in part by NSF award #1907547 and a gift fund from Adobe. Daniel Ritchie is an advisor to Geopipe and owns equity in the company. Geopipe is a start-up that is developing 3D technology to build immersive virtual copies of the real world with applications in various fields, including games and architecture. Minhyuk Sung acknowledges the support of NRF grant (2022R1F1A1068681), NST grant (CRC 21011), and IITP grant (2022-0-00594) funded by the Korea government (MSIT), and grants from Adobe, KT, and Samsung Electronics corporations. Part of this work was done when Kai Wang interned at Adobe.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, K., Guerrero, P., Kim, V.G., Chaudhuri, S., Sung, M., Ritchie, D. (2022). The Shape Part Slot Machine: Contact-Based Reasoning for Generating 3D Shapes from Parts. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13663. Springer, Cham. https://doi.org/10.1007/978-3-031-20062-5_35
Download citation
DOI: https://doi.org/10.1007/978-3-031-20062-5_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20061-8
Online ISBN: 978-3-031-20062-5
eBook Packages: Computer ScienceComputer Science (R0)