Abstract
Recent strides in AI research, particularly in computer vision and natural language processing, have significantly advanced the partial automation of data labeling and annotation processes. However, there remains a notable void in applying these cutting-edge techniques to videos portraying human-centric scenarios, with scant exploration of automated solutions for multimedia data. Current research primarily focuses on visual cues, such as on-screen detections, textual cues such as named entity recognition, and auditory cues involved in speech-to-text conversion. This paper proposes a methodology that leverages state-of-the-art deep learning techniques to extract multimedia cues from videos. Through evaluation across various video contexts, our methodology yields promising results, potentially charting a course for future research endeavors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: YOLOv4: optimal speed and accuracy of object detection (2020). https://doi.org/10.48550/arXiv.2004.10934, http://arxiv.org/abs/2004.10934, arXiv:2004.10934 [cs, eess]
Hagberg, A.A., Schult, D.A., Swart, P.J.: Exploring network structure, dynamics, and function using NetworkX. In: Varoquaux, G., Vaught, T., Millman, J. (eds.) Proceedings of the 7th Python in Science Conference, pp. 11–15. Pasadena, CA USA (2008)
Jiang, Y.G., Wu, Z., Tang, J., Li, Z., Xue, X., Chang, S.F.: Modeling multimodal clues in a hybrid deep learning framework for video classification. IEEE Trans. Multimedia 20(11), 3137–3147 (2018)
Kondratyuk, D., et al.: MoViNets: mobile video networks for efficient video recognition. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16015–16025. IEEE, Nashville, TN, USA (2021). https://doi.org/10.1109/CVPR46437.2021.01576, https://ieeexplore.ieee.org/document/9578260/
Kukleva, A., Tapaswi, M., Laptev, I.: Learning interactions and relationships between movie characters. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9849–9858 (2020)
Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I.: Robust speech recognition via large-scale weak supervision (2022). _eprint: 2212.04356
Serengil, S.I., Ozpinar, A.: HyperExtended LightFace: a facial attribute analysis framework. In: 2021 International Conference on Engineering and Emerging Technologies (ICEET), pp. 1–4. IEEE (2021).https://doi.org/10.1109/ICEET53442.2021.9659697
Yamashita, R., Nishio, M., Do, R.K.G., Togashi, K.: Convolutional neural networks: an overview and application in radiology. Insights Imaging 9(4), 611–629 (2018)
Yu, Y., Si, X., Hu, C., Zhang, J.: A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput. 31(7), 1235–1270 (2019). https://doi.org/10.1162/neco_a_01199. https://direct.mit.edu/neco/article-pdf/31/7/1235/1053200/neco_a_01199.pdf
Zhang, B., Fang, Y., Ren, T., Wu, G., : Multimodal analysis for deep video understanding with video language transformer. In: Proceedings of the 30th ACM International Conference on Multimedia (2022). https://dl.acm.org/doi/abs/10.1145/3503161.3551600
Acknowledgements
This work has been supported by the French National Research Agency through the ANR TRACTIVE project ANR-21-CE38-00012-01
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Serouis, I., Sedes, F. (2025). AMDA: Advancing Multimedia Data Annotation for Human-Centric Situations. In: Ide, I., et al. MultiMedia Modeling. MMM 2025. Lecture Notes in Computer Science, vol 15524. Springer, Singapore. https://doi.org/10.1007/978-981-96-2074-6_7
Download citation
DOI: https://doi.org/10.1007/978-981-96-2074-6_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-96-2073-9
Online ISBN: 978-981-96-2074-6
eBook Packages: Computer ScienceComputer Science (R0)