Welcome to the InfoSci Platform
Reference Hub2
Multimodal Dance Generation Networks Based on Audio-Visual Analysis

Multimodal Dance Generation Networks Based on Audio-Visual Analysis

Lijuan Duan, Xiao Xu, Qing En
Copyright: © 2021 |Volume: 12 |Issue: 1 |Pages: 16
ISSN: 1947-8534|EISSN: 1947-8542|EISBN13: 9781799860488|DOI: 10.4018/IJMDEM.2021010102
Cite Article Cite Article

MLA

Duan, Lijuan, et al. "Multimodal Dance Generation Networks Based on Audio-Visual Analysis." IJMDEM vol.12, no.1 2021: pp.17-32. http://doi.org/10.4018/IJMDEM.2021010102

APA

Duan, L., Xu, X., & En, Q. (2021). Multimodal Dance Generation Networks Based on Audio-Visual Analysis. International Journal of Multimedia Data Engineering and Management (IJMDEM), 12(1), 17-32. http://doi.org/10.4018/IJMDEM.2021010102

Chicago

Duan, Lijuan, Xiao Xu, and Qing En. "Multimodal Dance Generation Networks Based on Audio-Visual Analysis," International Journal of Multimedia Data Engineering and Management (IJMDEM) 12, no.1: 17-32. http://doi.org/10.4018/IJMDEM.2021010102

Export Reference

Mendeley
Favorite Full-Issue Download

Abstract

3D human dance generation from music is an interesting and challenging task in which the aim is to estimate 3D pose from visual and audio information. Existing methods only use skeleton information to complete this task, which may cause jittering results. In addition, due to lack of appropriate evaluation metrics for this task, it is difficult to evaluate the quality of the generated results. In this paper, the authors explore multi-modality dance generation networks through constructing the correspondence between the visual and the audio cues. Specifically, they propose a 2D prediction module to predict future frames by fusing visual and audio features. Moreover, they propose a 3D conversion module, which is able to generate the 3D skeleton from the 2D skeleton. In addition, some new human dance generation evaluation metrics are proposed to evaluate the quality of the generated results. Experimental results indicate that the proposed modules can meet the requirements of authenticity and diversity.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.