Skip to main content
SpringerLink
Log in
Book cover

European Conference on Computer Vision

ECCV 2020: Computer Vision – ECCV 2020 pp 573–588Cite as

Synthesis and Completion of Facades from Satellite Imagery

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12347))

Abstract

Automatic satellite-based reconstruction enables large and widespread creation of urban areas. However, satellite imagery is often noisy and incomplete, and is not suitable for reconstructing detailed building facades. We present a machine learning-based inverse procedural modeling method to automatically create synthetic facades from satellite imagery. Our key observation is that building facades exhibit regular, grid-like structures. Hence, we can overcome the low-resolution, noisy, and partial building data obtained from satellite imagery by synthesizing the underlying facade layout. Our method infers regular facade details from satellite-based image-fragments of a building, and applies them to occluded or under-sampled parts of the building, resulting in plausible, crisp facades. Using urban areas from six cities, we compare our approach to several state-of-the-art image completion/in-filling methods and our approach consistently creates better facade images.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Bokeloh, M., Wand, M., Seidel, H.P.: A connection between partial symmetry and inverse procedural modeling. ACM Trans. Graph. 29 (2010)

    Google Scholar 

  2. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49

    Chapter  Google Scholar 

  3. Cohen, A., Schwing, A.G., Pollefeys, M.: Efficient structured parsing of facades using dynamic programming. In: IEEE Computer Vision and Pattern Recognition, pp. 3206–3213 (2014)

    Google Scholar 

  4. Demir, I., Aliaga, D.G., Benes, B.: Procedural editing of 3D building point clouds. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 2147–2155, December 2015. https://doi.org/10.1109/ICCV.2015.248

  5. Demir, I., Aliaga, D.G., Benes, B.: Coupled segmentation and similarity detection for architectural models. ACM Trans. Graph. 34(4), 1–11 (2015)

    Article  Google Scholar 

  6. Fathalla, R., Vogiatzis, G.: A deep learning pipeline for semantic facade segmentation. In: Proceedings of the British Machine Vision Conference 2016, BMVC 2017, September 2017. c 2017. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms. http://publications.aston.ac.uk/id/eprint/31805/

  7. Gadde, R., Marlet, R., Paragios, N.: Learning grammars for architecture-specific facade parsing. Int. J. Comput. Vis. 117(3), 290–316 (2016). https://doi.org/10.1007/s11263-016-0887-4

    Article  MathSciNet  MATH  Google Scholar 

  8. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015). http://arxiv.org/abs/1512.03385

  9. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: IEEE Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)

    Google Scholar 

  10. Kelly, T., Guerrero, P., Steed, A., Wonka, P., Mitra, N.J.: FrankenGAN: guided detail synthesis for building mass-models using style-synchonized GANs. ACM Trans. Graph. 37(6) (2018). https://doi.org/10.1145/3272127.3275065

  11. Kozinski, M., Gadde, R., Zagoruyko, S., Obozinski, G., Marlet, R.: A MRF shape prior for facade parsing with occlusions. In: IEEE Computer Vision and Pattern Recognition, pp. 2820–2828 (2015)

    Google Scholar 

  12. Koziński, M., Obozinski, G., Marlet, R.: Beyond procedural facade parsing: bidirectional alignment via linear programming. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 79–94. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16817-3_6

    Chapter  Google Scholar 

  13. Leotta, M.J., et al.: Urban semantic 3D reconstruction from multiview satellite imagery. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019

    Google Scholar 

  14. Liu, H., Zhang, J., Zhu, J., Hoi, S.C.H.: DeepFacade: a deep learning approach to facade parsing. In: International Joint Conference on Artificial Intelligence, pp. 2301–2307 (2017)

    Google Scholar 

  15. Martinovic, A., Van Gool, L.: Bayesian grammar learning for inverse procedural modeling. In: IEEE Computer Vision and Pattern Recognition, pp. 201–208 (2013)

    Google Scholar 

  16. Mathias, M., Martinović, A., Van Gool, L.: ATLAS: a three-layered approach to facade parsing. Int. J. Comput. Vis. 118(1), 22–48 (2016). https://doi.org/10.1007/s11263-015-0868-z

    Article  MathSciNet  Google Scholar 

  17. Müller, P., Zeng, G., Wonka, P., Van Gool, L.: Image-based procedural modeling of facades. ACM Trans. Graph. 26(3), 85–es (2007). https://doi.org/10.1145/1276377.1276484

  18. Musialski, P., Wonka, P., Aliaga, D.G., Wimmer, M., Van Gool, L., Purgathofer, W.: A survey of urban reconstruction. Comput. Graph. Forum 32, 146–177 (2013)

    Article  Google Scholar 

  19. Nguatem, W., Mayer, H.: Modeling urban scenes from pointclouds. In: IEEE International Conference on Computer Vision, pp. 3837–3846 (2017)

    Google Scholar 

  20. Nishida, G., Bousseau, A., Aliaga, D.G.: Procedural modeling of a building from a single image. Comput. Graph. Forum 37, 415–429 (2018)

    Article  Google Scholar 

  21. Ozcanli, O.C., Dong, Y., Mundy, J.L., Webb, H., Hammoud, R., Tom, V.: A comparison of stereo and multiview 3-D reconstruction using cross-sensor satellite imagery. In: IEEE Computer Vision and Pattern Recognition Workshops, pp. 17–25 (2015)

    Google Scholar 

  22. Qin, R.: Automated 3D recovery from very high resolution multi-view satellite images. In: ASPRS (IGTF) Annual Conference, p. 10 (2017)

    Google Scholar 

  23. Riemenschneider, H., et al.: Irregular lattices for complex shape grammar facade parsing. In: IEEE Computer Vision and Pattern Recognition, pp. 1640–1647 (2012)

    Google Scholar 

  24. Ritchie, D., Mildenhall, B., Goodman, N.D., Hanrahan, P.: Controlling procedural modeling programs with stochastically-ordered sequential Monte Carlo. ACM Trans. Graph. 34(4), 1–11 (2015)

    Article  Google Scholar 

  25. Sasaki, Y.: The truth of the f-measure. Teach Tutor Mater, January 2007

    Google Scholar 

  26. Talton, J.O., Lou, Y., Lesser, S., Duke, J., Měch, R., Koltun, V.: Metropolis procedural modeling. ACM Trans. Graph. 30(2), 1–14 (2011)

    Article  Google Scholar 

  27. Teboul, O., Kokkinos, I., Simon, L., Koutsourakis, P., Paragios, N.: Shape grammar parsing via reinforcement learning. In: IEEE Computer Vision and Pattern Recognition, pp. 2273–2280 (2011)

    Google Scholar 

  28. Vanegas, C.A., Aliaga, D.G., Beneš, B.: Building reconstruction using manhattan-world grammars. In: IEEE Computer Vision and Pattern Recognition (2010)

    Google Scholar 

  29. Yang, C., Han, T., Quan, L., Tai, C.L.: Parsing façade with rank-one approximation. In: IEEE Computer Vision and Pattern Recognition, pp. 1720–1727 (2012)

    Google Scholar 

  30. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative image inpainting with contextual attention. CoRR abs/1801.07892 (2018). http://arxiv.org/abs/1801.07892

  31. Zhang, H., et al.: Context encoding for semantic segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    Google Scholar 

  32. Zhang, X., May, C., Nishida, G., Aliaga, D.: Progressive regularization of satellite-based 3D buildings for interactive rendering. In: Symposium on Interactive 3D Graphics and Games, I3D 2020. Association for Computing Machinery, New York (2020)

    Google Scholar 

  33. Zheng, C., Cham, T.J., Cai, J.: Pluralistic image completion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1438–1447 (2019)

    Google Scholar 

  34. Zheng, E., Wang, K., Dunn, E., Frahm, J.M.: Minimal solvers for 3D geometry from satellite imagery. In: IEEE International Conference on Computer Vision, pp. 738–746 (2015)

    Google Scholar 

  35. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

    Google Scholar 

Download references

Acknowledgements

This research was supported in part by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/ Interior Business Center (DOI/IBC) contract number D17PC00280. Additional support came from National Science Foundation grants #10001387 and #1835739.

Author information

Authors and Affiliations

  1. Purdue University, West Lafayette, USA

    Xiaowei Zhang, Christopher May & Daniel Aliaga

Authors
  1. Xiaowei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher May
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Aliaga
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaowei Zhang .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 950 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, X., May, C., Aliaga, D. (2020). Synthesis and Completion of Facades from Satellite Imagery. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12347. Springer, Cham. https://doi.org/10.1007/978-3-030-58536-5_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58536-5_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58535-8

  • Online ISBN: 978-3-030-58536-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation