Fudan University: hierarchical video retrieval with adaptive multi-modal fusion

• Propose a multiple multi-head attentions network with random missing modalities.
• Present a network to obtain missing modalities contextual information.
• Achieve the best performance, and the method is robust and convergent.

Multi-modal fusion is a hot topic in field of multi-modal learning. Most of the previous multi-modal fusion tasks are based on the complete modality. Existing researches on missing multi-modal fusion fail to consider the random missing of ...

In multi-modal classification tasks, a good fusion algorithm can effectively integrate and process multi-modal data, thereby significantly improving its performance. Researchers often focus on the design of complex fusion operators and have proposed ...

• Multi-modal bioelectrical signal fusion is reviewed.
• Rationality and challenge ...

Multi-modal fusion combines multiple modal information to overcome the limitation of incomplete information expressed by a single modality, so as to realize the complementarity of modal information and enhance feature representation. ...