A joint probabilistic face detection and tracking algorithm for combining a likelihood estimation and a prior probability is proposed. Face tracking is achieved by a Bayesian framework. The likelihood estimation scheme is based on statistical training of sets of automatically generated feature points, while the prior probability estimation is based on the fusion of an information theoretic tracking cue and a Gaussian temporal model. The likelihood estimation process is the cone of a multiple face detection scheme used to initialize the tracking process. The resulting system was tested on real image sequences and is robust to significant partial occlusion and illumination changes.