In this paper, we propose a new depth image-based rending (DIBR) scheme for 3DTV applications, which is based on temporal hole filling with sprite generation. Dissimilar to traditional methods (e.g., spatial interpolation and inpainting), temporal information is utilized to recover the holes with true background information. The proposed scheme is composed of two parts: sprite generation and virtual view synthesis. To collect sufficiently temporal information, a key-frame-based sprite generation algorithm was developed to incrementally and accurately fuse the background color and depth information from successive frames. Then the holes in the synthesized virtual view can be recovered by referring to the constructed sprite models. Experiments demonstrate that our proposed scheme is capable of achieving good results and outperforms some existing methods subjectively and objectively.