Abstract
Video provides strong cues for automatic road extraction that are not available in static aerial images. In video from a static camera, or stabilized (or geo-referenced) aerial video data, motion patterns within a scene enable function attribution of scene regions. A “road”, for example, may be defined as a path of consistent motion — a definition which is valid in a large and diverse set of environments. The spatio-temporal structure tensor field is an ideal representation of the image derivative distribution at each pixel because it can be updated in real time as video is acquired. An eigen-decomposition of the structure tensor encodes both the local scene motion and the variability in the motion. Additionally, the structure tensor field can be factored into motion components, allowing explicit determination of traffic patterns in intersections. Example results of a real time system are shown for an urban scene with both well-traveled and infrequently traveled roads, indicating that both can be discovered simultaneously. The method is ideal in urban traffic scenes, which are the most difficult to analyze using static imagery.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
P. Agouris, A. Stefanidis, and S. Gyftakis. “Differential snakes for change detection in road segments,” Photogrammetric Engineering & Remote Sensing, Vol. 67(12):1391–1399, 2001.
M. Bicego, S. Dalfini, G. Vernazza, and V. Murino. “Automatic road extraction from aerial images by probabilistic contour tracking,” in Proceedings of IEEE International Conference on Image Processing (ICIP), vol. III, pp. 585–588, 2003.
X.-T. Dai, L. Lu, and G. Hager. “Realtime video mosaicing with adaptive parameterized warping,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, volume (Demo Program), 2001.
P. Doucette, P. Agouris, A. Stefanidis, and M. Musavi. “Self-organized clustering for road extraction in classified imagery,” ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 55(56):347–358, 2001.
A. Faber and W. Forstner. “Detection of dominant orthogonal structures in small scale imagery,” International Archives of Photogrammetry and Remote Sensing, 33(Part B3/1):274–281, 2000.
H. Farid and E.P. Simoncelli. “Optimally rotationequivariant directional derivative kernels,” in Computer Analysis of Images and Patterns (CAIP), pp. 207–214, 1997.
D. Geman, and B. Jedynak. “An active testing model for tracking roads in satellite images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 18(1):1–14, 1996.
W.E.L. Grimson, C. Stauffer, R. Romano, and L. Lee. “Using adaptive tracking to classify and monitor activities in a site,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 22–27, 1998.
S. Hinz, and A. Baumgartner. “Automatic extraction of urban road networks from multiview aerial imagery.” ISPRS Journal of Photogrammetry and Remote Sensing, 53:83–98, 2003.
B.K.P. Horn. “Robot Vision.” McGraw Hill: New York, 1986.
S. Van Huffel and J. Vandewalle. “The Total Least Squares Problem: Computational Aspects and Analysis,” Society for Industrial and Applied Mathematics, Philadelphia, 1991.
I. Laptev, H. Mayer, T. Lindeberg, W. Eckstein, C. Steger, and A. Baumgartner. “Automatic extraction of roads from aerial images based on scale space and snakes,” Machine Vision and Applications, Vol. 12(1):23–31, 2000.
H.H. Nagel. “Extending the ‘oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow,” in Proceedings of the First European Conference on Computer Vision, Springer: Berlin Heidelberg New York, Inc., pp. 139–148, 1990.
R. Pless, T. Brodsky, and Y. Aloimonos. “Detecting independent motion: The statistics of temporal continuity,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22(8):68–73, 2000.
R. Pless, J. Larson, S. Siebers, and B. Westover. “Evaluation of local models of dynamic backgrounds,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 73–78, 2003.
F.M. Porikli. “Road extraction by pointwise gaussian models,” in SPIE AeroSense Technologies and Systems for Defense and Security, vol. 5093, pp. 758–764, 2003.
K. Price. “Urban street grid description and verification,” in IEEE Workshop on Applications of Computer Vision, pp. 148–154, 2000.
C. Stauffer, and W.E.L. Grimson. “Adaptive background mixture models for realtime tracking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 246–252, 1999.
F. Tupin, H. Maitre, J.F. Mangin, J.M. Nicolas, and E. Pechersky. “Detection of linear features in SAR images: Application to the road network extraction,” IEEE Transactions on Geoscience and Remote Sensing, Vol. 36(2):434–453, Mar. 1998.
S. Wang, Y. Markandey, and A. Reid. “Total least squares fitting spatiotemporal derivatives to smooth optical flow fields,” in Proc. of the SPIE: Signal and Data Processing of Small Targets, vol. 1698, pp. 42–55. SPIE, 1992.
J. Weber, and J. Malik. “Robust computation of optical flow in a multiscale differential framework,” International Journal of Computer Vision, Vol. 14:67–81, 1995.
C. Wiedemann, C. Heipke, H. Mayer, and S. Hinz. “Automatic extraction and evaluation of road networks from moms-2p imagery,” International Archives of Photogrammetry and Remote Sensing, Vol. 32(3):285–291, 1998.
L. Wixson. “Detecting salient motion by accumulating directionallyconsistent flow,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22(8):774–780, 2000.
J. Wright, and R. Pless. “Analysis of persistent motion patterns using the 3d structure tensor,” in Proceedings of the IEEE Workshop on Motion and Video Computing, pp. 14–19, 2005.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Pless, R. Detecting Roads in Stabilized Video with the Spatio-Temporal Structure Tensor. Geoinformatica 10, 37–53 (2006). https://doi.org/10.1007/s10707-005-4885-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10707-005-4885-x