Skip to main content
SpringerLink
Log in

PhotoSketch: a photocentric urban 3D modeling system

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Online mapping services from Google, Apple, and Microsoft are exceedingly popular applications for exploring 3D urban cities. Their explosive growth provides impetus for photorealistic 3D modeling of urban scenes. Although classical algorithms such as multiview stereo and laser range scanners are traditional sources for detailed 3D models of existing structures, they generate heavyweight models that are not appropriate for the streaming data that these navigation applications leverage. Instead, lightweight models as produced by interactive image-based tools are better suited for this domain. The contribution of this work is that it merges the benefits of multiview geometry, an intuitive sketching interface, and dynamic texture mapping to produce lightweight photorealistic 3D models of buildings. We present experimental results from urban scenes using our PhotoSketch system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. http://profs.sci.univr.it/~fusiello/demo/samantha/

  2. http://www.cs.unc.edu/~marc/

  3. http://research.microsoft.com/en-us/um/redmond/groups/ivm/PlanarStereo/supplementary/playhouse.html

  4. Arikan, M., Schwärzler, M., Flöry, S., Wimmer, M., Maierhofer, S.: O-snap: optimization-based snapping for modeling architecture. ACM Trans. Graph. 32(1), 6:1–6:15 (2013)

    Article  MATH  Google Scholar 

  5. Atkinson, K.B.: Close Range Photogrammetry and Machine Vision. Whittles Publishing, Dunbeath, UK (2003)

    Google Scholar 

  6. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110, 346–359 (2008)

    Article  Google Scholar 

  7. Chen, T., Zhu, Z., Shamir, A., Hu, S.M., Cohen-Or, D.: 3-sweep: extracting editable objects from a single photo. ACM Trans. Graph. 32(6), 195:1–195:10 (2013)

    Google Scholar 

  8. Debevec, P.E., Taylor, C.J., Malik, J.: Modeling and rendering architecture from photographs: a hybrid geometry-and image-based approach. Comput. Graph. Proc. SIGGRAPH ’96 30, 11–20 (1996)

    Article  Google Scholar 

  9. El-Hakim, S., Whiting, E., Gonzo, L.: 3D modeling with reusable and integrated building blocks. In: Conference on Optical 3D Measurement Techniques (2005)

  10. Faugeras, O., Laveau, S., Robert, L., Csurka, G., Zeller, C.: 3D reconstruction of urban scenes from sequences of images. In: Gruen, A., Kuebler, O., Agouris, P. (eds.) Automatic Extraction of Man-Made Objects from Aerial and Space Images. Birkhauser, Basel, Switzerland (1995)

    Google Scholar 

  11. Faugeras, O., Luong, Q.T., Papadopoulou, T.: The Geometry of Multiple Images. MIT Press, Cambridge, MA (2001)

    MATH  Google Scholar 

  12. Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1–8 (2007)

  13. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  14. van den Hengel, A., Dick, A.R., Thormählen, T., Ward, B., Torr, P.H.: Videotrace: rapid interactive scene modelling from video. ACM Trans. Graph. Proc. SIGGRAPH ’07 26(3), 86 (2007)

    Article  Google Scholar 

  15. Hou, F., Qin, H., Qi, Y.: Procedure-based component and architecture modeling from a single image. Vis. Comput. 32(2), 151–166 (2016)

    Article  Google Scholar 

  16. Li, M., Nan, L., Liu, S.: Fitting boxes to Manhattan scenes using linear integer programming. Int. J. Digit. Earth 9, 806–817 (2016)

    Article  Google Scholar 

  17. Li, W., Wolberg, G., Zokai, S.: Lightweight 3d modeling of urban buildings from range data. In: 3DIMPVT, pp. 124–131 (2011)

  18. Liu, L., Stamos, I., Yu, G., Wolberg, G., Zokai, S.: Multiview geometry for texture mapping 2D images onto 3D range data. IEEE Conference Computer Vision and Pattern Recognition (CVPR) pp. 2293–2300 (2006)

  19. Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  20. Luhmann, T., Robson, S., Kyle, S., Harley, I.: Close Range Photogrammetry: Principles, Techniques and Applications. Wiley, Hoboken (2006)

    Google Scholar 

  21. Ma, Y., Soatto, S., Kosecka, J., Sastry, S.: An Invitation to 3-D Vision: From Images to Geometric Models. Springer, Berlin (2004)

    Book  MATH  Google Scholar 

  22. Mathias, M., Martinović, A., Weissenberg, J., Van Gool, L.: Procedural 3d building reconstruction using shape grammars and detectors. In: 3DIMPVT, pp. 304–311 (2011)

  23. Moulon, P., Monasse, P., Marlet, R., Others: Openmvg. an open multiple view geometry library. https://github.com/openMVG/openMVG

  24. Müller, P., Wonka, P., Haegler, S., Ulmer, A., Gool, L.V.: Procedural modeling of buildings. ACM Trans. Graph. Proc. SIGGRAPH ’06 25(3), 614–623 (2006)

    Article  Google Scholar 

  25. Müller, P., Zeng, G., Wonka, P., Van Gool, L.: Image-based procedural modeling of facades. ACM Trans. Graph. 26(3), 85 (2007)

    Article  Google Scholar 

  26. Musialski, P., Wonka, P., Aliaga, D.G., Wimmer, M., van Gool, L., Purgathofer, W.: A survey of urban reconstruction. Comput. Graph. Forum 32(6), 146–177 (2013)

    Article  Google Scholar 

  27. Nan, L., Jiang, C., Ghanem, B., Wonka, P.: Template assembly for detailed urban reconstruction. Comput. Graph. Forum 34, 217–228 (2015)

    Article  Google Scholar 

  28. Parish, Y.I.H., Müller, P.: Procedural modeling of cities. Comput. Graph. (Proc. SIGGRAPH ’01) pp. 301–308 (2001)

  29. Remondino, F., El-Hakim, S.: Image-based 3d modelling : a review. Photogramm. Rec. 21, 269–291 (2006)

    Article  Google Scholar 

  30. Samavati, F., Runions, A.: Interactive 3D content modeling for digital earth. Vis. Comput. 32(10), 1293–1309 (2016)

    Article  Google Scholar 

  31. Sinha, S.N., Steedly, D., Szeliski, R., Agrawala, M., Pollefeys, M.: Interactive 3D architectural modeling from unordered photo collections. In: SIGGRAPH Asia ’08, pp. 159:1–159:10 (2008)

  32. Snavely, N.: http://www.cs.cornell.edu/~snavely/bundler/

  33. Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. Proc. SIGGRAPH ’06 25(3), 835–846 (2006)

    Article  Google Scholar 

  34. Stamos, I., Allen, P.K.: Automatic registration of 2D with 3D imagery in urban environments. In: Proceedings of International Conference On Computer Vision (ICCV) pp. 731–737 (2001)

  35. Stamos, I., Allen, P.K.: Geometry and texture recovery of scenes of large scale. Comput. Vis. Image Underst. 88(2), 94–118 (2002)

    Article  MATH  Google Scholar 

  36. Stamos, I., Liu, L., Chen, C., Wolberg, G., Yu, G., Zokai, S.: Integrating automated range registration with multiview geometry for the photorealistic modeling of large-scale scenes. Int. J. Comput. Vis. 78(2), 237–260 (2007)

    Google Scholar 

  37. Toldo, R., Gherardi, R., Farenzena, M., Fusiello, A.: Samantha: Structure-and-motion pipeline on a hierarchical cluster tree. http://www.diegm.uniud.it/fusiello/demo/samantha/

  38. Vanegas, C.A., Aliaga, D.G., Beneš, B.: Building reconstruction using Manhattan-world grammars. Comput. Vis. Pattern Recognit. 0, 358–365 (2010)

    Google Scholar 

  39. Verdie, Y., Lafarge, F., Alliez, P.: Lod generation for urban scenes. ACM Trans. Graph. 34(3), 30:1–30:14 (2015)

    Article  MATH  Google Scholar 

  40. Werner, T., Schaffalitzky, F., Zisserman, A.: Automated architecture reconstruction from close-range photogrammetry. In: Proceedings of CIPA 2001 International Symposium

  41. Wu, C.: Visualsfm. http://ccwu.me/vsfm

  42. Wu, C., Agarwal, S., Curless, B., Seitz, S.M.: Schematic surface reconstruction. In: Proceedings of IEEE CVPR pp. 1498–1505 (2012)

  43. Wu, F., Yan, D.M., Dong, W., Zhang, X., Wonka, P.: Inverse procedural modeling of facade layouts. ACM Trans. Graph. 33(4), 121:1–121:10 (2014)

    Article  Google Scholar 

  44. Zheng, Y., Chen, X., Cheng, M.M., Zhou, K., Hu, S.M., Mitra, N.J.: Interactive images: cuboid proxies for smart image manipulation. ACM Trans. Graph. 31(4), 99:1–99:11 (2012)

    Google Scholar 

Download references

Acknowledgements

This work was supported by a grant from the US Department of Energy (DE-NA0002492).

Author information

Authors and Affiliations

  1. City College of New York, CUNY, New York, NY, 10031, USA

    George Wolberg

  2. Brainstorm Technology LLC, New York, NY, 10001, USA

    Siavash Zokai

Authors
  1. George Wolberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siavash Zokai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George Wolberg.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wolberg, G., Zokai, S. PhotoSketch: a photocentric urban 3D modeling system. Vis Comput 34, 605–616 (2018). https://doi.org/10.1007/s00371-017-1365-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-017-1365-x

Keywords

Search

Navigation