Multiple description coding of animated meshes
Introduction
With increasing level of realism, animated 3D meshes (or dynamic meshes/mesh sequences) have become more important in areas like computer games, 3DTV, animation movies, physical simulations, etc. As a result, research on dynamic 3D mesh compression has attracted great attention. A common representation for dynamic meshes is series of static triangular meshes, called mesh frames or simply frames where each frame corresponds to a time instant. In this work, we deal with time-consistent dynamic meshes in which the connectivity of meshes in each frame stays constant.
In the field of time consistent dynamic 3D mesh compression, many methods have been proposed recently in the literature [1], [2], [3], [4], [5], [6], [7], [8]. One of the most significant works is FAMC—Frame-based Animated Mesh Compression [9], [10], recently promoted within the MPEG-4 standard as Amendment 2 of part 16 AFX (Animation Framework eXtension), which provides very efficient compression performance. On the other hand, there is not much research on transmission and error resilient coding of 3D dynamic meshes. There is only one previous work [11], where the mesh is spatially partitioned into segments and each segment is encoded and protected by Forward Error Protection (FEC). The disadvantage of the method is that the loss of some parts of the bitstream means complete loss of corresponding spatial areas.
On the other hand, error resilient coding and transmission of video has been widely studied in the literature. Among these studies, multiple description coding (MDC) has emerged as an efficient method for error resilient coding of multimedia data. In MDC, compared to the conventional coding, the source is coded into multiple independent bitstreams or so-called descriptions instead of a single bitstream/description. As a result of the independent coding, each description can be decoded on its own without the need of any other descriptions. This property can be efficiently used in real-time applications, network simplification designs and multi path transmission (MPT) scenarios [12].
Although there are many works related to MDC of video [12], [13], [14], [15], [16], [17] and MDC of static 3D meshes [18], [19], [20], [21], [22], MDC of animated mesh data is at a very early stage. Briefly for the works related to MDC of static meshes, in [18] multiple descriptions are generated by splitting the mesh geometry into submeshes and including the whole connectivity information in each description. The work in [19] is based on multiple description scalar quantization (MDSQ) of wavelet coefficients of a multiresolution compression scheme and the authors of [22] propose improvements with optimal filtering. In [20], wavelet coefficient trees obtained by Progressive Geometry Compression (PGC) [23] algorithm are partitioned into multiple descriptions followed by encoding of each set of trees independently. In [21], the embedded bitstream generated by the PGC algorithm is optimally packetized and protected unequally with forward error protection (FEC) codes.
In this work, we propose three MDC schemes for animated 3D meshes. The main difference between the MDC of static and animated meshes is that the extra dimension of time in animated meshes brings additional challenges. The first proposed scheme is based on partitioning vertices spatially to be encoded independently. This scheme resembles the static mesh methods in [18], [20] in the sense that spatial partitioning of the vertices is common in all the methods. However, the proposed method has the additional challenge of handling reduced compression performance due to inferior temporal prediction. The second proposed scheme is based on subsampling frames temporally by exploiting the hierarchical B frame structure, which makes it a completely a different approach compared to static meshes. The final proposed scheme makes use of the layered scalable structure and is based on duplicating layers in the descriptions. From the bitstream protection point of view, this scheme shows similarity to the static mesh approach in [21] with two descriptions.
The rest of the paper is organized as follows. In Section 2, we give a brief review of MDC. In Section 3, we provide the technical details of the reference dynamic mesh coder on which the proposed MDC methods depend. In Section 4, we describe the proposed MDC methods. In Section 5, we present the experimental results and finally we conclude in Section 6.
Section snippets
MDC review
In this section, we briefly review the concept of MDC and two important related issues: redundancy allocation and mismatch control.
Reference 3D dynamic mesh coder
The single description (SD) dynamic mesh coder on which our MDC method is based is the scalable predictive dynamic mesh coder presented in [8]. The coder can also be viewed as the layered prediction structure of FAMC [10] omitting any other modules like skinning-based motion compensation (SMC). We have chosen this coder as our reference coder because of both efficient compression performance and the layered structure which gives flexibility to the design of MD structures. Additional tools like
Proposed MDC methods
In this section, we describe the proposed MDC methods for 3D dynamic meshes. We propose three methods which are named after the strategies they are based on:
- Vertex partitioning
Based on spatially partitioning the set of vertices into two sets for each frame.
- Temporal subsampling
Based on encoding odd and even frames separately.
- Layer duplication
Based on duplicating the bitstream corresponding to a subset of layers and splitting the remaining bitstream for each description.
Results
In this section, we evaluate and compare the performances of the proposed MDC methods. Since we aim to analyze MDC performance, we do not provide a comparison with the work in [11] where the MDC usage is not considered. In addition, the compression methods employed in [11] are less efficient than the one we employed. Therefore in a transmission scenario, it is very likely that a comparison would favor the method employing a better compressor since the bandwidth can be used more efficiently.
Conclusions and future work
In this paper, we have proposed and evaluated three novel multiple description coding (MDC) methods for reliable transmission of compressed animated meshes. The methods make use of an efficient 3D dynamic mesh coder based on temporal/spatial layer decompositions. We have also proposed necessary modifications to adjust the amount of redundancy to gain resiliency to losses.
We have presented the experimental results with redundancy-rate-distortion curves and visual reconstructions. The
Acknowledgements
This work is supported by EC within FP7 under Grant 216503 with the acronym Mobile3DTV. It is also partially supported by The Scientific and Technological Research Council of Turkey (TUBITAK). The chicken character was created by Andrew Glassner, Tom McClure, Scott Benza, and Mark Van Langeveld. This short sequence of connectivity and vertex position data is distributed solely for the purpose of comparison of geometry compression techniques. We would like to thank Nikolce Stefanoski for
References (28)
- et al.
Compression of soft-body animation sequences
Computers & Graphics
(2004) - M. Sattler, R. Sarlette, R. Klein, Simple and efficient compression of animation sequences, in: Proceedings of the 2005...
- I. Guskov, A. Khodakovsky, Wavelet compression of parametrically coherent mesh sequences, in: Proceedings of the 2004...
- F. Payan, M. Antonini, Wavelet-based compression of 3D mesh sequences, in: Proceedings of IEEE ACIDCA-ICMI2005,...
- K. Muller, A. Smolic, M. Kautzner, P. Eisert, T. Wiegand, Predictive compression of dynamic 3D meshes, in: Proceedings...
- et al.
Compression of 3-D triangle mesh sequences based on vertex-wise motion vector prediction
IEEE Transactions on Circuits and Systems for Video Technology
(2002) - L. Ibarria, J. Rossignac, Dynapack: space-time compression of the 3D animations of triangle meshes with fixed...
- N. Stefanoski, J. Ostermann, SPC: fast and efficient scalable predictive coding of animated meshes, Computer Graphics...
- K. Mamou, T. Zaharia, F. Preteux, Famc: the mpeg-4 standard for animated mesh compression, in: Proceedings of the IEEE...
- N. Stefanoski, J. Ostermann, Spatially and temporally scalable compression of animated 3D meshes with MPEG-4/FAMC, in:...