In this paper, a method for real-time tracking of moving objects is proposed. We applied Kalman particle filter (KPF) to color-based tracking. This KPF is a particle filter including the principle of Kalman filter, and it was adopted to the object contour tracking. We modified this KPF for color-based tracking. This modified KPF can approximate the probabilistic density of the position of the tracked object properly and needs fewer particles for tracking than conventional particle filters. We made experiments to confirm the effectiveness of this method.