Abstract
This paper presents a technique to simultaneously model 3D urban scenes in the spatial-temporal space using a collection of photos that span many years. We propose to use a middle level representation, building, to characterize significant structure changes in the scene. We first use structure-from-motion techniques to build 3D point clouds, which is a mixture of scenes from different periods of time. We then segment the point clouds into independent buildings using a hierarchical method, including coarse clustering on sparse points and fine classification on dense points based on the spatial distance of point clouds and the difference of visibility vectors. In the fine classification, we segment building candidates using a probabilistic model in the spatial-temporal space simultaneously. We employ a z-buffering based method to infer existence of each building in each image. After recovering temporal order of input images, we finally obtain 3D models of these buildings along the time axis. We present experiments using both toy building images captured from our lab and real urban scene images to demonstrate the feasibility of the proposed approach.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Agarwal, S., Snavely, N., Simon, I., Seitz, S., Szeliski, R.: Building rome in a day. In: Proc. ICCV, pp. 72–79 (2009)
Akhter, I., Sheikh, Y.A., Khan, S., Kanade, T.: Nonrigid structure from motion in trajectory space. In: Proc. NIPS, vol. 1 (2008)
Chuang, Y.Y., Goldman, D., Zheng, K., Curless, B., Salesin, D., Szeliski, R.: Animating pictures with stochastic motion textures. In: Proc. ACM SIGGRAPH, pp. 853–860 (2005)
Dorninger, P., Nothegger, C.: 3d segmentation of unstructured point clouds for building modeling. In: Int. Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences (2007)
Faugeras, O., Bras-Mehlman, E.L., Boissonnat, J.D.: Representing stereo data with the delaunay triangulation. Artificial Intelligence 44, 41–87 (1990)
Filin, S., Pfeifer, N.: Segmentation of airborne laser scanning data using a slope adaptive neighborhood. ISPRS Journal of Photogrammetry and Remote Sensing 60, 71–80 (2006)
Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. IEEE Trans. on PAMI 32, 1362–1376 (2010)
Pani, A., Bhattacharjee, G.: Temporal representation and reasoning in artificial intelligence a review. Mathematical and Computer Modeling 34, 55–80 (2001)
Pollefeys, M., Nister, D., Frahm, J.M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles, H.: Detailed real-time urban 3d reconstruction from video. Int. Journal of Computer Vision 78, 143–167 (2008)
Snavely, N., Seitz, S., Szeliski, R.: Modeling the world from internet photo collections. Int. Journal of Computer Vision 80, 189–210 (2008)
Schindler, G., Dellaert, F., Kang, S.: Inferring temporal order of images from 3d structure. In: Proc. CVPR (2007)
Schindler, G., Dellaert, F.: Probabilistic temporal inference on reconstructed 3d scenes. In: Proc. CVPR (2010)
Simon, I., Seitz, S.M.: Scene segmentation using the wisdom of crowds. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 541–553. Springer, Heidelberg (2008)
Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: Exploring image collections in 3d. ACM Transactions on Graphics 25, 835–846 (2006)
Xiao, J., Wang, J., Quan, L.: Joint affinity propagation for multiple view segmentation. In: Proc. ICCV (2007)
Xiao, J., Fang, T., Zhao, P., Lhuillier, M., Quan, L.: Image-based street-side city modeling. ACM Transaction on Graphics 28, 114 (2009)
Xu, X., Wan, L., Liu, X., Wong, T.T., Wang, L., Leung, C.S.: Animating animal motion from still. ACM Transactions on Graphics 27, 117:1–117:8 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xu, J., Wang, Q., Yang, J. (2011). Modeling Urban Scenes in the Spatial-Temporal Space. In: Kimmel, R., Klette, R., Sugimoto, A. (eds) Computer Vision – ACCV 2010. ACCV 2010. Lecture Notes in Computer Science, vol 6493. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19309-5_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-19309-5_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19308-8
Online ISBN: 978-3-642-19309-5
eBook Packages: Computer ScienceComputer Science (R0)