This work is aimed at automatically learning the three-dimensional structure of an outdoor scene observed by a single uncalibrated video camera. In particular, we are proposing to estimate the 3D layout of roads and paths traveled by pedestrians by observing the pedestrians and to estimate the road parameters from their height and position in a video sequence. The developed algorithm has been implemented and was successfully tested in different environment including scene luminance variation during a day, possible mistakes in pedestrian detection, road coverage variation during a year. Proposed algorithm for 3D road map estimation (up to a scale factor) can be used in video surveillance applications to classify people on the scene by their heights, to detect human abnormal trajectories, for human gait analysis, for people traffic analysis and in other applications that require automatic roads estimation and human height prediction. This algorithm can be one of building block for advanced video surveillance systems.