skip to main content
10.1145/3689062acmconferencesBook PagePublication PagesmmConference Proceedingsconference-collections
MuSe'24: Proceedings of the 5th on Multimodal Sentiment Analysis Challenge and Workshop: Social Perception and Humor
ACM2024 Proceeding
Publisher:
  • Association for Computing Machinery
  • New York
  • NY
  • United States
Conference:
MM '24: The 32nd ACM International Conference on Multimedia Melbourne VIC Australia 28 October 2024- 1 November 2024
ISBN:
979-8-4007-1199-2
Published:
28 October 2024
Sponsors:
Recommend ACM DL
ALREADY A SUBSCRIBER?SIGN IN

Reflects downloads up to 17 Feb 2025Bibliometrics
Skip Abstract Section
Abstract

We welcome you to the 5th Multimodal Sentiment Analysis Challenge and Workshop (MuSe 2024). MuSe 2024 is a half-day workshop in conjunction with ACM Multimedia 2024 in Melbourne, Australia. In its fifth edition, MuSe continues its tradition of offering novel multimodal sentiment analysis benchmark tasks within a unified challenge setting. This year, we invited participation in two distinct multimodal sub-challenges: 1) predicting 16 different social attributes of individuals and 2) detecting spontaneous, humorous communication in a cross-cultural context.

The challenge offers a stimulating environment for developing and evaluating novel approaches to multimedia processing and deep learning methods for automatic audio-visual and textual affect modeling, all within a common experimental framework. It is designed to appeal to various research communities, fostering interdisciplinary collaboration. Primarily, MuSe 2024 targets researchers in audio-visual emotion recognition, unimodal and multimodal representation learning, health informatics, and NLP-based emotion and sentiment analysis. All raw recordings, transcriptions, pre-extracted features, and baseline models are available on our website (www.muse-challenge.org).

Skip Table Of Content Section
SESSION: MuSe 2024 Introduction
research-article
Open Access
The MuSe 2024 Multimodal Sentiment Analysis Challenge: Social Perception and Humor Recognition

The Multimodal Sentiment Analysis Challenge (MuSe) 2024 addresses two contemporary multimodal affect and sentiment analysis problems: In the Social Perception Sub-Challenge (MuSe-Perception), participants will predict 16 different social attributes of ...

short-paper
Open Access
MuSe '24: The 5th Multimodal Sentiment Analysis Challenge and Workshop: Social Perception & Humor

The 5th Multimodal Sentiment Analysis Challenge (MuSe), a workshop in conjunction with ACM Multimedia '24, is focused on Multimodal Machine Learning in the domain of Affective Computing. Two different sub-challenges are proposed: Social Perception Sub-...

SESSION: MuSe 2024 Challenge: Social Perception Challenge
research-article
Modality Weights Based Fusion Model for Social Perception Prediction in Video, Audio, and Text

Social perception is a crucial psychological concept that explains how we understand and interpret others and their behaviors. It encompasses the complex process of discerning individual characteristics, intentions, and emotions, significantly ...

research-article
Open Access
Larger Encoders, Smaller Regressors: Exploring Label Dimensionality Reduction and Multimodal Large Language Models as Feature Extractors for Predicting Social Perception

Designing reliable automatic models for social perception can contribute to a better understanding of human behavior, enabling more trustworthy experiences in the multimedia on-line communication environment. However, predicting social attributes from ...

research-article
Open Access
Feature-wise Optimization and Performance-weighted Multimodal Fusion for Social Perception Recognition

Automatic social perception recognition is a new task to mimic the measurement of human traits, which was previously done by humans via questionnaires. We evaluated unimodal and multimodal systems to predict agentive and communal traits from the LMU-ELP ...

research-article
Exploring Multimodal Approaches and Fusion Methods for CEO Social Attribute Prediction in 2024 MuSe-Perception

In this paper, we discuss the way we addressed the 2024 MuSe-Perception challenge, which aims to automatically recognize and quantify social attributes of CEOs using multimodal data. We investigated five different approaches: (1) optimizing a basic RNN ...

research-article
Open Access
LLM-Driven Multimodal Fusion for Human Perception Analysis

The Multimodal Sentiment Analysis Challenge presents two distinct sub-challenges related to human perception characteristics. This paper focuses on the MUSE-PERCEPTION challenge, which aims to predict the perceptual attributes of CEOs from video data. ...

research-article
Open Access
Social Perception Prediction for MuSe 2024: Joint Learning of Multiple Perceptions

In this paper, we present our unique method for the MuSe 2024 Perception sub-challenge. In the Perception sub-challenge, 21 labeled social perceptions data are given, 16 social perceptions are required to be predicted. Joint learning is crucial for our ...

SESSION: MuSe 2024 Challenge: Humor Challenge
short-paper
Open Access
Multimodal Humor Detection and Social Perception Prediction

The parallel audio-visual-text data contains vast amount of information. Thus it is essential to develop machine learning algorithms that can utilise them efficiently. In this work, we investigated unimodal and multimodal solutions for MuSe Humor and ...

short-paper
AMTN: Attention-Enhanced Multimodal Temporal Network for Humor Detection

In this paper, we introduce the Attention-Enhanced Multimodal Temporal Network (AMTN) to address the MuSe 2024 Cross-Cultural Humor Detection Sub-Challenge (MuSe-Humor), which highlights the task of humor detection within a cross-cultural context, ...

research-article
Open Access
DPP: A Dual-Phase Processing Method for Cross-Cultural Humor Detection

Multimodal humor detection has become an active research area in the field of artificial intelligence. This paper presents the solution for the MuSe-Humor sub-challenge of cross-cultural humor detection. The goal of the MuSe-Humor sub-challenge is to ...

Contributors
  • Technical University of Munich
  • University of Augsburg
  • Ludwig-Maximilians-University Munich
  • University of Passau
  • Nanyang Technological University
  • Imperial College London
Index terms have been assigned to the content through auto-classification.

Recommendations

Acceptance Rates

Overall Acceptance Rate 14 of 17 submissions, 82%
YearSubmittedAcceptedRate
MuSe' 22171482%
Overall171482%