In this paper, a new approach for non rigid obstacle detection and tracking is proposed. Traditionally, this task is performed for each obstacle as a rigid body without considering the local movements of its parts. The presented method combines foreground segmentation techniques for static cameras with nonrigid point set registration algorithms with the objective of having information about the local movements of pedestrians. This information will be used by an electrical unmanned vehicle that will be working inside a closed bioclimatic urbanization in order to perform a more intelligent path planning. This paper has been focused on pedestrian detection, but as no model is used, it can be applied to any type of obstacle. At the end of the paper, results of some tests about the different evaluated algorithms are shown, as well as the final results of all parts of the method working together.