DynTex: A comprehensive database of dynamic textures

Abstract

We present the DynTex database of high-quality dynamic texture videos. It consists of over 650 sequences of dynamic textures, mostly in everyday surroundings. Additionally, we propose a scheme for the manual annotation of the sequences based on a detailed analysis of the physical processes underlying the dynamic textures. Using this scheme we describe the texture sequences in terms of both visual structure and semantic content. The videos and annotations are made publicly available for scientific research.

Introduction

In recent years we have witnessed a rapid growth of interest in the study of dynamic texture (DT). This new field of research offers an extension of the study of static texture into the temporal domain. Dynamic texture phenomena can be observed all around us in daily life, e.g. moving trees in the wind, rippling water, fluttering sails and flags, or moving crowds of people in a shopping street.

Just like static textures, dynamic textures can be studied in a wide variety of ways. Research on texture synthesis aims to model dynamic textures for realistic rendering (e.g. Szummer and Picard, 1996, Saisan et al., 2001 or Filip et al., 2006). A related area is texture analysis where the problem is to characterize the visual properties of the texture, e.g. its regularity or granularity. This can also be useful for texture retrieval. For texture detection the goal is to detect when a certain type of dynamic texture appears in a video sequence, e.g. to detect fire (Dedeoglu et al., 2005). An interesting problem is also to discern dynamic texture from camera motion (Amiaz et al., 2007). Texture segmentation is about video segmentation and the accurate localization of the textures, in space, and, possibly, in time (e.g. Doretto et al., 2003, Chan and Vasconcelos, 2005). For texture recognition the aim is to recognize the type of dynamic texture, possibly from among several other ones (Péteri and Chetverikov, 2005, Zhao and Pietikainen, 2007); a review on DT description and recognition is presented in (Chetverikov and Péteri, 2005). In most of the cases, works on this topic consider well segmented sequences of dynamic textures. A final topic, irregularity detection has, to our knowledge, not been investigated so far. It aims at detecting irregular motions in sequences consisting of pure dynamic textures (for instance the detection of a piece of wood drifting on a river surface), and can be seen as the counterpart of defect detection for static textures (see Chetverikov and Hanbury, 2002).

With regard to the task of setting up test sets for performance evaluation, each of these areas of study have their own specific requirements. For instance, testing performance of methods for texture synthesis is best done on close-up sequences of the texture, whereas texture detection requires dynamic texture that is shown in context. For texture segmentation it may be beneficial to offer sequences where several dynamic texture phenomena are present in the same sequence.

In this paper we present the DynTex database. It consists of over 650 high-quality sequences of dynamic texture. It aims to serve as a standard database for dynamic texture research and to accommodate the needs of the different research areas mentioned above. So far no other databases suitable for this purpose are available. One interesting pioneer database to mention is a dataset compiled at MIT (Szummer, 1995). It is composed of around 25 black and white segmented sequences of dimension 170 × 115 × 120. However, this collection has a number of drawbacks: video dimensions are small (especially in the temporal direction); there is only a single occurrence per class, and not enough classes are available for practical classification purposes; finally, some of the sequences show undesirable camera motion. Other datasets of dynamic textures used in research papers have been shot by the authors themselves and are not publicly released, which prevents from using them for comparison in other research works.

The need for a standard database is clearly demonstrated by the interest of the research community: at the time of writing of this paper, DynTex has already over 300 registered researchers (about half of which are PhD candidates) that use the database for their studies.

The structure of the paper is as follows. In Section 2 we shortly review our understanding of dynamic textures as extensions of spatial dynamic textures to the temporal domain, and discuss some implications for the compilation of the DynTex database. In Section 3 we describe the acquisition protocol of the texture sequences and the video formats in which the sequences are made available. In Section 4 we propose a video annotation scheme based on the physical processes underlying the dynamical textures. This scheme has been used for the manual annotation of the full collection of DynTex sequences. In Section 5 we discuss the two primary means by which the annotations are made available. We also discuss how users of the database can create their own test subset selections for their specific research purposes. Finally, in Section 6 we present our conclusion and discuss a number of prospects for future extensions.