We propose a methodology for learning and using a multiple-goal probabilistic motion model within a particle filter-based target tracking on video streams. In a set of training video sequences, we first extract the locations (coined as “goals”) where the pedestrians either leave the scene or often change directions. Then, we learn one motion prior model per detected goal. Each of these models is learned statistically based on the local motion observed by the camera during the training phase. Given that the initial, empirical distribution may be incomplete or noisy, we regularize it in a second phase. These priors are then used in an Interactive Multiple Model (IMM) scheme for target tracking and goal estimation. We demonstrate the relevance of this methodology with tracking experiments and comparisons done on standard datasets.