During recent years the subject of mean shift algorithm for object tracking using color information has received much attention. However the use of color information to characterize the tracked object is very sensitive to noisy interference and illumination changes. Thus the flexibility and applicability of conventional color-based mean shift tracking are limited. In this paper, a fuzzy color histogram generated by a self-constructing fuzzy cluster is proposed to reduce the interference from lighting changes for the mean shift tracking algorithm. The experimental results show that the proposed tracking approach is more robust than the conventional mean shift tacking algorithm and the cost of increasing computation time is also moderate.