Particle filter has been widely applied into many fields in recent years. Combined with the deficiency analysis of particle filter, an improved particle filter based on diversity guidance is proposed. Firstly, the adaptive resampling step in particle filter is tuned based on two diversity measures which are effective sample size and population diversity factor. Moreover, the operation of particle mutation after resampling is integrated into PF so as to assure the diversity of particle sets. Then, a hybrid proposal distribution is adopted to consider current information of the latest observed measurement. At the same time, annealing parameter is utilized to control the proportional of priori function and likelihood function. With the simulation program using matlab 7.0 to track a single target motion from a fixed visual observation points, the validity of the proposed method is verified.