Abstract
Urban modeling facilitates the generation of virtual environments for various scenarios about cities. It requires expertise and consideration, and therefore consumes massive time and computation resources. Nevertheless, related tasks sometimes result in dissatisfaction or even failure. These challenges have received significant attention from researchers in the area of computer graphics. Meanwhile, the burgeoning development of artificial intelligence motivates people to exploit machine learning, and hence improves the conventional solutions. In this paper, we present a review of approaches to urban modeling in computer graphics using machine learning in the literature published between 2010 and 2019. This serves as an overview of the current state of research on urban modeling from a machine learning perspective.
摘要
城市建模为生成城市不同场景下的虚拟环境提供了便利。城市建模需要专业知识和考虑,并消耗大量时间和计算资源。即便如此,与之相关的任务有时仍以不满意的结果甚至失败告终。这些挑战得到了计算机图形学领域学者的大量关注。同时,人工智能的蓬勃发展激励人们充分利用机器学习以改进现有解决方案。本文回顾了2010至2019年间发表的文献,对计算机图形领域中使用机器学习的城市建模方法进行综述。本文可作为机器学习视角下城市建模研究现状的概述。
Similar content being viewed by others
References
Affara L, Nan LL, Ghanem B, et al., 2016. Large scale asset extraction for urban images. Proc 14th European Conf on Computer Vision, p.437–452. https://doi.org/10.1007/978-3-319-46487-9_27
Aleotti F, Zaccaroni G, Bartolomei L, et al., 2020. Real-time single image depth perception in the wild with handheld devices. https://arxiv.org/abs/2006.05724
AlHalawani S, Yang YL, Liu H, et al., 2013. Interactive facades analysis and synthesis of semi-regular facades. Comput Graph Forum, 32(2pt2):215–224. https://doi.org/10.1111/cgf.12041
Aliaga DG, 2012. 3D design and modeling of smart cities from a computer graphics perspective. Int Schol Res Notic, 2012:728913. https://doi.org/10.5402/2012/728913
Bao F, Schwarz M, Wonka P, 2013a. Procedural facade variations from a single layout. ACM Trans Graph, 32(1):8:1–8:13. https://doi.org/10.1145/2421636.2421644
Bao F, Yan DM, Mitra NJ, et al., 2013b. Generating and exploring good building layouts. ACM Trans Graph, 32(4):122:1–122:10. https://doi.org/10.1145/2461912.2461977
Benes J, Wilkie A, Krivánek J, 2014. Procedural modelling of urban road networks. Comput Graph Forum, 33(6):132–142. https://doi.org/10.1111/cgf.12283
Besuievsky G, Patow G, 2013. Customizable LOD for procedural architecture. Comput Graph Forum, 32(8):26–34. https://doi.org/10.1111/cgf.12141
Bishop CM, 2006. Pattern Recognition and Machine Learning. Springer, New York, USA.
Caruana R, Niculescu-Mizil A, 2006. An empirical comparison of supervised learning algorithms. Proc 23rd Int Conf on Machine Learning, p.161–168. https://doi.org/10.1145/1143844.1143865
Ceylan D, Mitra NJ, Li H, et al., 2012. Factored facade acquisition using symmetric line arrangements. Comput Graph Forum, 31(2pt3):671–680. https://doi.org/10.1111/j.1467-8659.2012.03046.x
Ceylan D, Mitra NJ, Zheng Y, et al., 2014. Coupled structure-from-motion and 3D symmetry detection for urban facades. ACM Trans Graph, 33(1):2. https://doi.org/10.1145/2517348
Cortes C, Vapnik V, 1995. Support-vector networks. Mach Learn, 20(3):273–297. https://doi.org/10.1007/BF00994018
Cover T, Hart P, 1967. Nearest neighbor pattern classification. IEEE Trans Inform Theory, 13(1):21–27. https://doi.org/10.1109/TIT.1967.1053964
Dang M, Ceylan D, Neubert B, et al., 2014. Safe: structure-aware facade editing. Comput Graph Forum, 33(2):83–93. https://doi.org/10.1111/cgf.12313
Dang M, Lienhard S, Ceylan D, et al., 2015. Interactive design of probability density functions for shape grammars. ACM Trans Graph, 34(6):206. https://doi.org/10.1145/2816795.2818069
Demir I, Aliaga DG, Benes B, 2014. Proceduralization of buildings at city scale. Proc 2nd Int Conf on 3D Vision, p.456–463. https://doi.org/10.1109/3DV.2014.31
Emilien A, Bernhardt A, Peytavie A, et al., 2012. Procedural generation of villages on arbitrary terrains. Vis Comput, 28(6–8):809–818. https://doi.org/10.1007/s00371-012-0699-7
Feng T, Yu LF, Yeung SK, et al., 2016. Crowd-driven mid-scale layout design. ACM Trans Graph, 35(4):132. https://doi.org/10.1145/2897824.2925894
Feng T, Truong QT, Nguyen DT, et al., 2018. Urban zoning using higher-order Markov random fields on multi-view imagery data. Proc 15th European Conf on Computer Vision, p.627–644. https://doi.org/10.1007/978-3-030-01237-3_38
Fernandes GD, Fernandes AR, 2018. Space colonisation for procedural road generation. Proc Int Conf on Graphics and Interaction, p.1–8. https://doi.org/10.1109/ITCGI.2018.8602928
Fukushima K, 1980. Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern, 36(4):193–202. https://doi.org/10.1007/BF00344251
Galin E, Peytavie A, Maréchal N, et al., 2010. Procedural generation of roads. Comput Graph Forum, 29(2):429–438. https://doi.org/10.1111/j.1467-8659.2009.01612.x
Galin E, Peytavie A, Guérin E, et al., 2011. Authoring hierarchical road networks. Comput Graph Forum, 30(7):2021–2030. https://doi.org/10.1111/j.1467-8659.2011.02055.x
Garcia-Dorado I, Aliaga D G, Ukkusuri S V, 2014. Designing large-scale interactive traffic animations for urban modeling. Comput Graph Forum, 33(2):411–420. https://doi.org/10.1111/cgf.12329
Garcia-Dorado I, Aliaga DG, Bhalachandran S, et al., 2017. Fast weather simulation for inverse procedural design of 3D urban models. ACM Trans Graph, 36(2):21. https://doi.org/10.1145/2999534
Goldblatt R, Stuhlmacher MF, Tellman B, et al., 2018. Using landsat and nighttime lights for supervised pixel-based image classification of urban land cover. Remot Sens Environ, 205:253–275. https://doi.org/10.1016/j.rse.2017.11.026
Goodfellow I, Bengio Y, Courville A, 2016. Deep Learning. MIT Press, Cambridge, UK.
Goodfellow I, Pouget-Abadie J, Mirza M, et al., 2014. Generative adversarial nets. Proc 27th Int Conf on Neural Information Processing Systems, p.2672–2680.
Guerrero P, Jeschke S, Wimmer M, et al., 2015. Learning shape placements by example. ACM Trans Graph, 34(4):108. https://doi.org/10.1145/2766933
Guo YJ, Chen EX, Guo Y, et al., 2017. Deep highway unit network for land cover type classification with GF-3 SAR imagery. Proc SAR in Big Data Era: Models, Methods and Applications, p.1–6. https://doi.org/10.1109/BIGSARDATA.2017.8124926
Hartmann S, Weinmann M, Wessel R, et al., 2017. Street-GAN: towards road network synthesis with generative adversarial networks. Proc 25th Int Conf in Central Europe on Computer Graphics, Visualization and Computer Vision, p.133–142.
Hassoun MH, 1995. Fundamentals of Artificial Neural Networks. MIT Press, Cambridge, USA.
Hastie T, Tibshirani R, Friedman J, 2009. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York, USA.
Ho TK, 1995. Random decision forests. Proc 3rd Int Conf on Document Analysis and Recognition, p.278. https://doi.org/10.1109/ICDAR.1995.598994
Hopfield JJ, 1982. Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA, 79(8):2554–2558. https://doi.org/10.1073/pnas.79.8.2554
Hu YJ, Gao S, Janowicz K, et al., 2015. Extracting and understanding urban areas of interest using geotagged photos. Comput Environ Urban Syst, 54:240–254. https://doi.org/10.1016/j.compenvurbsys.2015.09.001
Huang HB, Kalogerakis E, Yumer E, et al., 2017. Shape synthesis from sketches via procedural models and convolutional networks. IEEE Trans Vis Comput Graph, 23(8):2003–2013. https://doi.org/10.1109/TVCG.2016.2597830
Ilčík M, Musialski P, Auzinger T, et al., 2015. Layer-based procedural design of façades. Comput Graph Forum, 34(2):205–216. https://doi.org/10.1111/cgf.12553
Isola P, Zhu J, Zhou TH, et al., 2016. Image-to-image translation with conditional adversarial networks. https://arxiv.org/abs/1611.07004
James G, Witten D, Hastie T, et al., 2014. An Introduction to Statistical Learning: with Applications in R. Springer, New York, USA.
Kaelbling LP, Littman ML, Moore AW, 1996. Reinforcement learning: a survey. J Artif Intell Res, 4(1):237–285.
Kelly T, Wonka P, Mueller P, 2015. Interactive dimensioning of parametric models. Comput Graph Forum, 34(2):117–129. https://doi.org/10.1111/cgf.12546
Kelly T, Femiani J, Wonka P, et al., 2017. BigSUR: large-scale structured urban reconstruction. ACM Trans Graph, 36(6):204. https://doi.org/10.1145/3130800.3130823
Kelly T, Guerrero P, Steed A, et al., 2018. FrankenGAN: guided detail synthesis for building mass models using style-synchonized GANs. ACM Trans Graph, 37(6):216. https://doi.org/10.1145/3272127.3275065
Khanum M, Mahboob T, Imtiaz W, et al., 2015. A survey on unsupervised machine learning algorithms for automation, classification and maintenance. Int J Comput Appl, 119(13):34–39. https://doi.org/10.5120/21131-4058
Kim S, Kim D, Choi S, 2020. CityCraft: 3D virtual city creation from a single image. Vis Comput, 36(5):911–924. https://doi.org/10.1007/s00371-019-01701-x
Krecklau L, Pavic D, Kobbelt L, 2010. Generalized use of non-terminal symbols for procedural modeling. Comput Graph Forum, 29(8):2291–2303. https://doi.org/10.1111/j.1467-8659.2010.01714.x
Krizhevsky A, Sutskever I, Hinton GE, 2017. ImageNet classification with deep convolutional neural networks. Commun ACM, 60(6):84–90. https://doi.org/10.1145/3065386
Kuang ZZ, Chan B, Yu YZ, et al., 2013. A compact random-access representation for urban modeling and rendering. ACM Trans Graph, 32(6):172. https://doi.org/10.1145/2508363.2508424
Lafarge F, Mallet C, 2011. Building large urban environments from unstructured point data. Proc Int Conf on Computer Vision, p.1068–1075. https://doi.org/10.1109/ICCV.2011.6126353
Lecun Y, Bottou L, Bengio Y, et al., 1998. Gradient-based learning applied to document recognition. Proc IEEE, 86(11):2278–2324. https://doi.org/10.1109/5.726791
Li HN, Jia YH, Zhou Y, 2018. Urban expansion pattern analysis and planning implementation evaluation based on using fully convolution neural network to extract land range. NeuroQuantology, 16(5):814–822. https://doi.org/10.14704/nq.2018.16.5.1390
Li ML, Wonka P, Nan LL, 2016. Manhattan-world urban reconstruction from point clouds. Proc 14th European Conf on Computer Vision, p.54–69. https://doi.org/10.1007/978-3-319-46493-0_4
Lienhard S, Lau C, Müller P, et al., 2017. Design transformations for rule-based procedural modeling. Comput Graph Forum, 36(2):39–48. https://doi.org/10.1111/cgf.13105
Lin H, Gao JZ, Zhou Y, et al., 2013. Semantic decomposition and reconstruction of residential scenes from LiDAR data. ACM Trans Graph, 32(4):66. https://doi.org/10.1145/2461912.2461969
Lindenmayer A, 1968. Mathematical models for cellular interactions in development I. Filaments with one-sided inputs. J Theor Biol, 18(3):280–299. https://doi.org/10.1016/0022-5193(68)90079-9
Lipp M, Scherzer D, Wonka P, et al., 2011. Interactive modeling of city layouts using layers of procedural content. Comput Graph Forum, 30(2):345–354. https://doi.org/10.1111/j.1467-8659.2011.01865.x
Lynch K, 1964. The Image of the City. MIT Press, Cambridge, UK.
MacQueen J, 1967. Some methods for classification and analysis of multivariate observations. Proc 5th Berkeley Symp on Mathematical Statistics and Probability, p.281–297.
Mathew CDT, Knob PR, Musse SR, et al., 2019. Urban walkability design using virtual population simulation. Comput Graph Forum, 38(1):455–469. https://doi.org/10.1111/cgf.13585
Měch R, Prusinkiewicz P, 1996. Visual models of plants interacting with their environment. Proc 23rd Annual Conf on Computer Graphics and Interactive Techniques, p.397–410. https://doi.org/10.1145/237170.237279
Merrell P, Schkufza E, Koltun V, 2010. Computer-generated residential building layouts. ACM Trans Graph, 29(6):181. https://doi.org/10.1145/1882261.1866203
Mirza M, Osindero S, 2014. Conditional generative adversarial nets. https://arxiv.org/abs/1411.1784
Müller P, Wonka P, Haegler S, et al., 2006. Procedural modeling of buildings. ACM Trans Graph, 25(3):614–623. https://doi.org/10.1145/1141911.1141931
Musialski P, Wimmer M, Wonka P, 2012. Interactive coherence-based façades modeling. Comput Graph Forum, 31(2pt3):661–670. https://doi.org/10.1111/j.1467-8659.2012.03045.x
Nan LL, Sharf A, Zhang H, et al., 2010. SmartBoxes for interactive urban reconstruction. ACM Trans Graph, 29(4):93. https://doi.org/10.1145/1778765.1778830
Nan LL, Jiang CG, Ghanem B, et al., 2015. Template assembly for detailed urban reconstruction. Comput Graph Forum, 34(2):217–228. https://doi.org/10.1111/cgf.12554
Newton D, 2019. Generative deep learning in architectural design. Technol Arch Des, 3(2):176–189. https://doi.org/10.1080/24751448.2019.1640536
Nishida G, Garcia-Dorado I, Aliaga DG, 2016a. Example-driven procedural urban roads. Comput Graph Forum, 35(6):5–17. https://doi.org/10.1111/cgf.12728
Nishida G, Garcia-Dorado I, Aliaga DG, et al., 2016b. Interactive sketching of urban procedural models. ACM Trans Graph, 35(4):130. https://doi.org/10.1145/2897824.2925951
Parish YIH, Müller P, 2001. Procedural modeling of cities. Proc 28th Annual Conf on Computer Graphics and Interactive Techniques, p.301–308. https://doi.org/10.1145/383259.383292
Peng CH, Yang YL, Wonka P, 2014. Computing layouts with deformable templates. ACM Trans Graph, 33(4):99. https://doi.org/10.1145/2601097.2601164
Peng CH, Yang YL, Bao F, et al., 2016. Computational network design from functional specifications. ACM Trans Graph, 35(4):131. https://doi.org/10.1145/2897824.2925935
Rumelhart DE, Hinton GE, Williams RJ, 1986. Learning Internal Representations by Error Propagation. In: Rumelhart DE, McClelland JL (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1: Foundations. MIT Press, Cambridge, MA, USA.
Schapire RE, 1999. A brief introduction to boosting. Proc 16th Int Joint Conf on Artificial Intelligenc, p.1401–1406.
Scholkopf B, Smola AJ, 2001. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge, MA, USA.
Schwarz M, Müller P, 2015. Advanced procedural modeling of architecture. ACM Trans Graph, 34(4):107. https://doi.org/10.1145/2766956
Shen CH, Huang SS, Fu H, et al., 2011. Adaptive partitioning of urban facades. ACM Trans Graph, 30(6):184. https://doi.org/10.1145/2070781.2024218
Smelik RM, Tutenel T, Bidarra R, et al., 2014. A survey on procedural modelling for virtual worlds. Comput Graph Forum, 33(6):31–50. https://doi.org/10.1111/cgf.12276
Smith N, Moehrle N, Goesele M, et al., 2018. Aerial path planning for urban scene reconstruction: a continuous optimization method and benchmark. ACM Trans Graph, 37(6):183. https://doi.org/10.1145/3272127.3275010
United Nations, 2018. 2018 Revision of World Urbanization Prospects. https://un.org/development/desa/publications/2018-revision-of-world-urbanization-prospects.html [Accessed on May 16, 2018]
Vanegas CA, Aliaga DG, Benes B, et al., 2009. Interactive design of urban spaces using geometrical and behavioral modeling. ACM Trans Graph, 28(5):111. https://doi.org/10.1145/1618452.1618457
Vanegas CA, Aliaga DG, Beneš B, 2010. Building reconstruction using Manhattan-world grammars. Proc IEEE Computer Society Conf on Computer Vision and Pattern Recognition, p.358–365. https://doi.org/10.1109/CVPR.2010.5540190
Vanegas CA, Garcia-Dorado I, Aliaga DG, et al., 2012a. Inverse design of urban procedural models. ACM Trans Graph, 31(6):168. https://doi.org/10.1145/2366145.2366187
Vanegas CA, Kelly T, Weber B, et al., 2012b. Procedural generation of parcels in urban modeling. Comput Graph Forum, 31(2pt3):681–690. https://doi.org/10.1111/j.1467-8659.2012.03047.x
Verdie Y, Lafarge F, Alliez P, 2015. LOD generation for urban scenes. ACM Trans Graph, 34(3):30. https://doi.org/10.1145/2732527
Wonka P, Wimmer M, Sillion F, et al., 2003. Instant Architecture. ACM Press, New York, NY, USA.
Wu FZ, Yan DM, Dong WM, et al., 2014. Inverse procedural modeling of facade layouts. ACM Trans Graph, 33(4):121. https://doi.org/10.1145/2601097.2601162
Wu WM, Fan LB, Liu LG, et al., 2018. MIQP-based layout design for building interiors. Comput Graph Forum, 37(2):511–521. https://doi.org/10.1111/cgf.13380
Xiao JX, Fang T, Zhao P, et al., 2009. Image-based street-side city modeling. ACM Trans Graph, 28(5):114. https://doi.org/10.1145/1618452.1618460
Yang YL, Wang J, Vouga E, et al., 2013. Urban pattern: layout design by hierarchical domain splitting. ACM Trans Graph, 32(6):181. https://doi.org/10.1145/2508363.2508405
Yu QZ, Steed A, 2012. Example-based Road Network Synthesis. Eurographics 2012-Short Papers. https://doi.org/10.2312/conf/EG2012/short/053-056
Yumer ME, Asente P, Mech R, et al., 2015. Procedural modeling using autoencoder networks. Proc 28th Annual ACM Symp on User Interface Software and Technology, p.109–118. https://doi.org/10.1145/2807442.2807448
Zhang H, Xu K, Jiang W, et al., 2013. Layered analysis of irregular facades via symmetry maximization. ACM Trans Graph, 32(4):121. https://doi.org/10.1145/2461912.2461923
Zheng Q, Sharf A, Wan GW, et al., 2010. Non-local scan consolidation for 3D urban scenes. ACM Trans Graph, 29(4):94. https://doi.org/10.1145/1778765.1778831
Zhou B, Lapedriza A, Xiao J, et al., 2014. Learning deep features for scene recognition using places database. Proc 27th Int Conf on Neural Information Processing Systems, p.487–495.
Zhu X, Goldberg AB, 2009. Introduction to semi-supervised learning. Synth Lect Artif Intell Mach Learn, 3(1):1–130. https://doi.org/10.2200/S00196ED1V01Y200906AIM006
Author information
Authors and Affiliations
Contributions
Tian FENG designed the review and collected the literature. Tian FENG and Feiyi FAN drafted the manuscript. Tian FENG, Feiyi FAN, and Tomasz BEDNARZ revised and finalized the paper.
Corresponding author
Additional information
Compliance with ethics guidelines
Tian FENG, Feiyi FAN, and Tomasz BEDNARZ declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Feng, T., Fan, F. & Bednarz, T. A review of computer graphics approaches to urban modeling from a machine learning perspective. Front Inform Technol Electron Eng 22, 915–925 (2021). https://doi.org/10.1631/FITEE.2000141
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2000141