Skip to main content
SpringerLink
Log in

Pattern understanding and synthesis based on layout tree descriptor

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Synthesis from existing examples is a promising way to generate new patterns. However, pattern synthesis is challenging because it is difficult to understand and generate complex structures in patterns. In this paper, we propose an approach based on the layout tree descriptor (LTD) to understand and synthesize patterns from existing ones. The LTD is a binary tree that parametrically describes all primitives, layouts, their dependencies and hierarchies in a pattern. The LTD can be constructed automatically with proposed instance grouping, layout recognition, hyper-primitive matching and tree merging algorithms to realize pattern understanding. To meet specialists’ requirements for detailed modification and recombination of patterns, we designed LTD operations including add, remove, replace and grafting operations to allow users to get new patterns by simply adjusting the LTDs. For stylized synthesis, we gave the computing method of LTD similarity. Therefore, the styles of results and input can be compared and users can control generated serialized results by setting the input pattern weights. To meet user’s implicit preferences and provide novelty in creative design, we propose an evolutionary approach to creative synthesis. The system generates new patterns continuously based on LTD grafting, meanwhile user selection of preferred patterns will guide the direction of evolution. Experiments using the developed prototype system show that our approach can synthesize novel and complex patterns effectively, meeting different requirements in practice and providing plenty of digital textures for products.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems (NIPS), Montreal, Canada, 2014, pp. 2672–2680 (2014)

  2. Nguyen, A., Clune, J., Bengio, Y., Dosovitskiy, A., Yosinski, J.: Plug & play generative networks: conditional iterative generation of images in latent space. In: Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, USA, 2017, pp. 3510–3520 (2017)

  3. Reinert, B., Ritschel, T., Seidel, H.: Interactive by-example design of artistic packing layouts. ACM Trans. Graph. 32, 1–7 (2013)

    Google Scholar 

  4. Xu, P., Fu, H., Tai, C.L., Igarashi, T.: GACA: group-aware command-based arrangement of graphic elements, In: ACM Conference on Human Factors in Computing Systems (CHI), Seoul, Republic of Korea, 2015, pp. 2787–2795 (2015)

  5. Fish, N., Averkiou, M., Kaick, O.V., Sorkine-Hornung, O., Cohen-Or, D., Mitra, N.J.: Meta-representation of shape families. ACM Trans. Graph. 33(4), 1–11 (2014)

    MATH  Google Scholar 

  6. O’Donovan, P., Agarwala, A., Hertzmann, A.: Learning layouts for single-pagegraphic designs. IEEE Trans. Vis. Comput. Graph. 20(8), 1200–1213 (2014)

    Google Scholar 

  7. Lee, A.J.T., Chiu, H.P.: 2D Z-string: a new spatial knowledge representation for image databases. Pattern Recognit. Lett. 24(16), 3015–3026 (2003)

    Google Scholar 

  8. Matsakis, P., Wendling, L.: A new way to represent the relative position between areal objects. IEEE Trans. Pattern Anal. Mach. Intell. 21(7), 634–643 (1999)

    Google Scholar 

  9. Matsakis, P., Keller, J.M., Wendling, L., Marjamaa, J., Sjahputera, O.: Linguistic description of relative positions in images. IEEE Trans. Syst. Man Cybern. Part B (Cybern.). 31(4), 573–588 (2001). https://doi.org/10.1109/3477.938261

  10. Matsakis, P., Wawrzyniak, L., Ni, J.: Relative positions in words: a system that builds descriptions around Allen relations. Int. J. Geogr. Inf. Sci. 24(1), 1–23 (2010)

    Google Scholar 

  11. Hiller, B.: Space is the machine: a configurational theory of architecture. J. Urban Des. 3, 288–313 (2007)

    Google Scholar 

  12. Lee, J.H., Ostwald, M.J., Gu, N.: A syntactical and grammatical approach to architectural configuration, analysis and generation. Archit. Sci. Rev. 58(3), 189–204 (2015)

    Google Scholar 

  13. Fisher, M., Savva, M., Hanrahan, P.: Characterizing structural relationships in scenes using graph kernels. ACM Trans. Graph. 30(4), 1–12 (2011)

    Google Scholar 

  14. Ślusarczyk, G.: Graph-based representation of design properties in creating building floorplans. Comput. Aided Des. 95, 24–39 (2017)

    MathSciNet  Google Scholar 

  15. Jaiswal, P., Huang, J., Rai, R.: Assembly-based conceptual 3D modeling with unlabeled components using probabilistic factor graph. Comput. Aided Des. 74, 45–54 (2016)

    Google Scholar 

  16. Nan, L., Sharf, A., Xie, K., Wong, T.T., Deussen, O., Cohenor, D., Chen, B.: Conjoining Gestalt rules for abstraction of architectural drawings. J. Comput. Aided Des. Comput. Graph. 30(6), 1–10 (2012)

    Google Scholar 

  17. Nelson, G.: Juno, a constraint-based graphics system. In: Conference on Computer Graphics and Interactive Techniques, New York, NY, USA, 1985, pp. 235–243 (1985)

  18. Ryall, K., Marks, J., Shieber, S.M.: An interactive system for drawing graphs. In: Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology, Banff, Alberta, Canada, 1997, pp. 387–394 (1997)

  19. Baudisch, P., Cutrell, E., Hinckley, K., Eversole, A.: Snap-and-go: helping users align objects without the modality of traditional snapping. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), Portland, Oregon, USA, 2005, pp. 301–310 (2005)

  20. O’Donovan, P., Agarwala, A., Hertzmann, A.: DesignScape: design with interactive layout suggestions. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI), Seoul, Republic of Korea, 2015, pp. 1221–1224 (2015)

  21. Guerrero, P., Bernstein, G., Li, W., Mitra, N.: PATEX: exploring pattern variations. ACM Trans. Graph. 35(4), 1–13 (2016)

    Google Scholar 

  22. Jesus, D., Coelho, A., Sousa, A.A.: Layered shape grammars for procedural modelling of buildings. Vis. Comput. 32(6), 933–943 (2016)

    Google Scholar 

  23. Jowers, I., Earl, C., Stiny, G.: Shape computations without compositions. Commun. Comput. Inf. Sci. 724, 348–365 (2017)

    Google Scholar 

  24. Bohl, E., Terraz, O., Ghazanfarpour, D.: Modeling fruits and their internal structure using parametric 3Gmap L-systems. Vis. Comput. 31(6), 1–11 (2015)

    Google Scholar 

  25. Cui, J., Tang, M.X.: Integrating shape grammars into a generative system for Zhuang ethnic embroidery design exploration. Comput. Aided Des. 45(3), 591–604 (2013)

    MathSciNet  Google Scholar 

  26. Ruiz-Montiel, M., Belmonte, M.V., Boned, J., Mandow, L., Millán, E., Badillo, A., Pérez-De-La-Cruz, J.L.: Layered shape grammars. Comput. Aided Des. 56, 104–119 (2014)

    Google Scholar 

  27. Kielarova, S.W., Pradujphongphet, P., Bohez, E.L.J.: New interactive-generative design system: hybrid of shape grammar and evolutionary design—an application of jewelry design. In: Advances in Swarm and Computational Intelligence, Beijing, China, 2015, pp. 302–313 (2015)

  28. Dino, I.G.: An evolutionary approach for 3D architectural space layout design exploration. Autom. Constr. 69, 131–150 (2016)

    Google Scholar 

  29. Lu, S., Mok, P.Y., Jin, X.: From design methodology to evolutionary design: an interactive creation of marble-like textile patterns. Eng. Appl. Artif. Intell. 32, 124–135 (2014)

    Google Scholar 

  30. Guo, X., Lin, J., Xu, K., Jin, X.: Creature grammar for creative modeling of 3D monsters. Graph. Models 75(5), 376–389 (2014)

    Google Scholar 

  31. Zhou, J., Chen, X.: Convertible furniture design. Comput. Graph. 70, 165–175 (2017)

    Google Scholar 

  32. Alhashim, I., Li, H., Xu, K., Cao, J., Ma, R., Zhang, H.: Topology-varying 3D shape creation via structural blending. ACM Trans. Graph. 33(4), 1–10 (2014)

    Google Scholar 

  33. Yeh, Y., Měch, R.: Detecting symmetries and curvilinear arrangements in vector art. Comput. Graph. Forum 28(2), 707–716 (2010)

    Google Scholar 

  34. Yeh, Y., Breeden, K., Yang, L., Fisher, M., Hanrahan, P.: Synthesis of tiled patterns using factor graphs. ACM Trans. Graph. 22(1), 1–13 (2013)

    MATH  Google Scholar 

  35. Talton, J.O., Gibson, D., Yang, L., Hanrahan, P., Koltun, V.: Exploratory modeling with collaborative design spaces. ACM Trans. Graph. 28(5), 1–10 (2009)

    Google Scholar 

  36. Kusiak, A., Heragu, S.S.: The facility layout problem. Eur. J. Oper. Res. 29(3), 229–251 (1987)

    MathSciNet  MATH  Google Scholar 

  37. Vitayasak, S., Pongcharoen, P., Hicks, C.: A tool for solving stochastic dynamic facility layout problems with stochastic demand using either a Genetic Algorithm or modified Backtracking Search Algorithm. Int. J. Prod. Econ. 190, 146–157 (2017)

    Google Scholar 

  38. Xu, P., Fu, H., Igarashi, T., Tai, C.: Global beautification of layouts with interactive ambiguity resolution, In: Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology, Honolulu, Hawaii, USA, 2014, pp. 243–252 (2014)

  39. Hua, H.: Irregular architectural layout synthesis with graphical inputs. Autom. Constr. 72, 388–396 (2016)

    Google Scholar 

  40. Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Google Scholar 

  41. Valor, M., Albert, F., Gomis, J.M., Contero, M.: Analysis tool for cataloguing textile and tile pattern designs. Lect. Notes Comput. Sci. 2669, 569–578 (2003)

    Google Scholar 

  42. Huang, H., Zhang, Y., Yu, X.: Study and implementation of ceramic pattern intelligent design. In: International Conference on Computing, Control and Industrial Engineering (CCIE), Wuhan, China, 2010, pp. 132–135 (2010)

  43. Ge, Y.: Compositional rules for graphic system of decorative pattern art based on digital image. Paper Asia 1(8), 52–55 (2018)

    Google Scholar 

  44. Yao, J., Yu, H., Hu, R.: A new sparse representation-based object segmentation framework. Vis. Comput. 33(2), 179–192 (2017)

    Google Scholar 

  45. Qi, J., Xin, F., Yu, L., Haojie, L., Zhongxuan, L., He, G.: Hierarchical projective invariant contexts for shape recognition. Pattern Recognit. 52, 358–374 (2016)

    Google Scholar 

  46. Wei, S., Yuan, J., Gao, W., Dan, Z., Wang, X.: Shape recognition by bag of skeleton-associated contour parts. Pattern Recognit. Lett. 83, 321–329 (2016)

    Google Scholar 

  47. https://github.com/mrdoob/three.js/. Accessed Apr 2019

Download references

Acknowledgements

We wish to acknowledge Dr. Cui of Shandong Normal University and Prof. Tang of Shaanxi Fashion Engineering University for their permission of using embroidery patterns extracted from their paper in this study.

Funding

This study is funded by the National Natural Science Foundation of China (51775492) and Robotics Institute of Zhejiang University (K18-508116-001).

Author information

Authors and Affiliations

  1. State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou, 310027, People’s Republic of China

    Xinwei Zhang, Jin Wang & Guodong Lu

  2. College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, People’s Republic of China

    Xusheng Zhang

Authors
  1. Xinwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guodong Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xusheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This paper is sponsored by Grants from the National Natural Science Foundation of China (51775492) and Robotics Institute of Zhejiang University (K18-508116-001).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Wang, J., Lu, G. et al. Pattern understanding and synthesis based on layout tree descriptor. Vis Comput 36, 1141–1155 (2020). https://doi.org/10.1007/s00371-019-01723-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01723-5

Keywords

Search

Navigation