Text detection, localization, and tracking in compressed video

Abstract

Video text information plays an important role in semantic-based video analysis, indexing and retrieval. Video texts are closely related to the content of a video. Usually, the fundamental steps of text-based video analysis, browsing and retrieval consist of video text detection, localization, tracking, segmentation and recognition. Video sequences are commonly stored in compressed formats where MPEG coding techniques are often adopted. In this paper, a unified framework for text detection, localization, and tracking in compressed videos using the discrete cosines transform (DCT) coefficients is proposed. A coarse to fine text detection method is used to find text blocks in terms of the block DCT texture intensity information. The DCT texture intensity of an 8×8 block of an intra-frame is approximately represented by seven AC coefficients. The candidate text block regions are further verified and refined. The text block region localization and tracking are carried out by virtue of the horizontal and vertical block texture intensity projection profiles. The appearing and disappearing frames of each text line are determined by the text tracking. The final experimental results show the effectiveness of the proposed methods.

Introduction

Video sequences are usually integrated with audio, image, graph, text and so on. At present, video sequences become an indispensable part of people's daily life. With the rapid improvements in video compression technology, the expansion of low-cost storage media and the information explosion over the Internet, digital video libraries come into reality and pervade in the near future. Accordingly, the fast video accessing and browsing is required in many areas such as video conferencing, remote video based education, video-on-demand systems and so on [1], [2]. Video texts provide more intuitive information for audiences to grasp the meaning of video content. The clues provided by video texts have been widely used in semantic based video analysis, indexing and retrieval [3], [4], [5], [6], [7], [8], [9]. Video texts in the sport video programs and news always indicate the happening of highlight events, to which the audiences pay more attention.

In general, there are two types of texts in video sequences, namely, the scene texts and the artificial texts. The scene texts came from cameras and are naturally embedded in scenes, such as the text in trademarks, signpost and so on [9]. Artificial texts are purposely added to video frames during video editing [10]. Thus, they are closely related to the content of the video. As addressed in Ref. [11], text-based video retrieval, indexing and abstracting are more reliable than audio- and image-based methods due to the fact that many existing commercial optical character recognition (OCR) systems are far more robust than the speech analysis techniques and visual object analysis systems [3], [4], [5], [7], [8]. The fundamental problem of semantic-based video indexing and retrieval turns to that of video text detection, localization and tracking.

The rest of this paper is organized as follows. In Section 2, brief review is given for the related work on text detection, localization and tracking. In Section 3, our video text detection, localization and tracking methods based on DCT texture are proposed. Experimental results and their discussions are given in Section 4. Conclusions are finally drawn in Section 5.