This paper presents a system for pedestrian detection and tracking by using image processing techniques. A very important issue in the field of intelligent transportation system is to prevent pedestrians from being hit by vehicles. Recently, a great number of vision-based techniques have been proposed for this purpose. In this paper, we propose a vision-based method which combines the use of a pedestrian model as well as the walking rhythm of pedestrians. Through integrating these spatial and temporal information grabbed by a vision system, we are able to develop a reliable pedestrian detection and tracking system that can always produce accurate results. Experimental results obtained using the real world cases have demonstrated that the proposed model is indeed superb.