Multiple description coding of animated meshes

https://doi.org/10.1016/j.image.2010.10.004Get rights and content

Abstract

In this paper, we propose three novel multiple description coding (MDC) methods for reliable transmission of compressed animated meshes represented by series of 3D static meshes with same connectivity. The proposed methods trade off reconstruction quality for error resilience to provide the best expected reconstruction of 3D mesh sequence at the decoder side. The methods are based on layer duplication and partitioning of the set of vertices of a scalable coded animated mesh by either spatial or temporal subsampling. Each partitioned set is encoded separately to generate independently decodable bitstreams or so-called descriptions. In addition, the proposed MDC methods can achieve varying redundancy allocations by including a number of encoded spatial or temporal layers from the other description. The algorithms are evaluated with redundancy-rate-distortion (RRD) curves and per-frame reconstruction analysis. RRD performances show that vertex partitioning-based MDC performs better at low redundancies for especially spatially dense models. Temporal subsampling-based MDC performs better at moderate redundancies as well as low redundancies for spatially coarse models. Layer duplication-based MDC can achieve the lowest redundancies with flexible redundancy allocation capability and can be designed to achieve the smallest variance of reconstruction quality between consecutive frames.

Introduction

With increasing level of realism, animated 3D meshes (or dynamic meshes/mesh sequences) have become more important in areas like computer games, 3DTV, animation movies, physical simulations, etc. As a result, research on dynamic 3D mesh compression has attracted great attention. A common representation for dynamic meshes is series of static triangular meshes, called mesh frames or simply frames where each frame corresponds to a time instant. In this work, we deal with time-consistent dynamic meshes in which the connectivity of meshes in each frame stays constant.

In the field of time consistent dynamic 3D mesh compression, many methods have been proposed recently in the literature [1], [2], [3], [4], [5], [6], [7], [8]. One of the most significant works is FAMC—Frame-based Animated Mesh Compression [9], [10], recently promoted within the MPEG-4 standard as Amendment 2 of part 16 AFX (Animation Framework eXtension), which provides very efficient compression performance. On the other hand, there is not much research on transmission and error resilient coding of 3D dynamic meshes. There is only one previous work [11], where the mesh is spatially partitioned into segments and each segment is encoded and protected by Forward Error Protection (FEC). The disadvantage of the method is that the loss of some parts of the bitstream means complete loss of corresponding spatial areas.

On the other hand, error resilient coding and transmission of video has been widely studied in the literature. Among these studies, multiple description coding (MDC) has emerged as an efficient method for error resilient coding of multimedia data. In MDC, compared to the conventional coding, the source is coded into multiple independent bitstreams or so-called descriptions instead of a single bitstream/description. As a result of the independent coding, each description can be decoded on its own without the need of any other descriptions. This property can be efficiently used in real-time applications, network simplification designs and multi path transmission (MPT) scenarios [12].

Although there are many works related to MDC of video [12], [13], [14], [15], [16], [17] and MDC of static 3D meshes [18], [19], [20], [21], [22], MDC of animated mesh data is at a very early stage. Briefly for the works related to MDC of static meshes, in [18] multiple descriptions are generated by splitting the mesh geometry into submeshes and including the whole connectivity information in each description. The work in [19] is based on multiple description scalar quantization (MDSQ) of wavelet coefficients of a multiresolution compression scheme and the authors of [22] propose improvements with optimal filtering. In [20], wavelet coefficient trees obtained by Progressive Geometry Compression (PGC) [23] algorithm are partitioned into multiple descriptions followed by encoding of each set of trees independently. In [21], the embedded bitstream generated by the PGC algorithm is optimally packetized and protected unequally with forward error protection (FEC) codes.

In this work, we propose three MDC schemes for animated 3D meshes. The main difference between the MDC of static and animated meshes is that the extra dimension of time in animated meshes brings additional challenges. The first proposed scheme is based on partitioning vertices spatially to be encoded independently. This scheme resembles the static mesh methods in [18], [20] in the sense that spatial partitioning of the vertices is common in all the methods. However, the proposed method has the additional challenge of handling reduced compression performance due to inferior temporal prediction. The second proposed scheme is based on subsampling frames temporally by exploiting the hierarchical B frame structure, which makes it a completely a different approach compared to static meshes. The final proposed scheme makes use of the layered scalable structure and is based on duplicating layers in the descriptions. From the bitstream protection point of view, this scheme shows similarity to the static mesh approach in [21] with two descriptions.

The rest of the paper is organized as follows. In Section 2, we give a brief review of MDC. In Section 3, we provide the technical details of the reference dynamic mesh coder on which the proposed MDC methods depend. In Section 4, we describe the proposed MDC methods. In Section 5, we present the experimental results and finally we conclude in Section 6.

Section snippets

MDC review

In this section, we briefly review the concept of MDC and two important related issues: redundancy allocation and mismatch control.

Reference 3D dynamic mesh coder

The single description (SD) dynamic mesh coder on which our MDC method is based is the scalable predictive dynamic mesh coder presented in [8]. The coder can also be viewed as the layered prediction structure of FAMC [10] omitting any other modules like skinning-based motion compensation (SMC). We have chosen this coder as our reference coder because of both efficient compression performance and the layered structure which gives flexibility to the design of MD structures. Additional tools like

Proposed MDC methods

In this section, we describe the proposed MDC methods for 3D dynamic meshes. We propose three methods which are named after the strategies they are based on:

  • Vertex partitioning

    Based on spatially partitioning the set of vertices into two sets for each frame.

  • Temporal subsampling

    Based on encoding odd and even frames separately.

  • Layer duplication

    Based on duplicating the bitstream corresponding to a subset of layers and splitting the remaining bitstream for each description.

A common property in all of the methods is that the compressed mesh connectivity

Results

In this section, we evaluate and compare the performances of the proposed MDC methods. Since we aim to analyze MDC performance, we do not provide a comparison with the work in [11] where the MDC usage is not considered. In addition, the compression methods employed in [11] are less efficient than the one we employed. Therefore in a transmission scenario, it is very likely that a comparison would favor the method employing a better compressor since the bandwidth can be used more efficiently.

Conclusions and future work

In this paper, we have proposed and evaluated three novel multiple description coding (MDC) methods for reliable transmission of compressed animated meshes. The methods make use of an efficient 3D dynamic mesh coder based on temporal/spatial layer decompositions. We have also proposed necessary modifications to adjust the amount of redundancy to gain resiliency to losses.

We have presented the experimental results with redundancy-rate-distortion curves and visual reconstructions. The

Acknowledgements

This work is supported by EC within FP7 under Grant 216503 with the acronym Mobile3DTV. It is also partially supported by The Scientific and Technological Research Council of Turkey (TUBITAK). The chicken character was created by Andrew Glassner, Tom McClure, Scott Benza, and Mark Van Langeveld. This short sequence of connectivity and vertex position data is distributed solely for the purpose of comparison of geometry compression techniques. We would like to thank Nikolce Stefanoski for

References (28)

  • Z. Karni et al.

    Compression of soft-body animation sequences

    Computers & Graphics

    (2004)
  • M. Sattler, R. Sarlette, R. Klein, Simple and efficient compression of animation sequences, in: Proceedings of the 2005...
  • I. Guskov, A. Khodakovsky, Wavelet compression of parametrically coherent mesh sequences, in: Proceedings of the 2004...
  • F. Payan, M. Antonini, Wavelet-based compression of 3D mesh sequences, in: Proceedings of IEEE ACIDCA-ICMI2005,...
  • K. Muller, A. Smolic, M. Kautzner, P. Eisert, T. Wiegand, Predictive compression of dynamic 3D meshes, in: Proceedings...
  • J. Yang et al.

    Compression of 3-D triangle mesh sequences based on vertex-wise motion vector prediction

    IEEE Transactions on Circuits and Systems for Video Technology

    (2002)
  • L. Ibarria, J. Rossignac, Dynapack: space-time compression of the 3D animations of triangle meshes with fixed...
  • N. Stefanoski, J. Ostermann, SPC: fast and efficient scalable predictive coding of animated meshes, Computer Graphics...
  • K. Mamou, T. Zaharia, F. Preteux, Famc: the mpeg-4 standard for animated mesh compression, in: Proceedings of the IEEE...
  • N. Stefanoski, J. Ostermann, Spatially and temporally scalable compression of animated 3D meshes with MPEG-4/FAMC, in:...
  • S. Varakliotis et al.

    Optimally smooth error resilient streaming of 3D wireframe animations

    Proceedings of SPIE

    (2004)
  • Y. Wang et al.

    Multiple description coding using pairwise correlating transforms

    IEEE Transactions on Image Processing

    (2001)
  • V. Goyal

    Multiple description coding: compression meets the network

    IEEE Signal Processing Magazine

    (2001)
  • A. Reibman, H. Jafarkhani, Y. Wang, M. Orchard, R. Puri, Multiple-description video coding using motion-compensated...
  • Cited by (0)

    View full text