research-article

Efficient Indexing of 3D Human Motions

Authors:

Petra Budikova,

Jan Sedmidubsky,

Pavel ZezulaAuthors Info & Claims

ICMR '21: Proceedings of the 2021 International Conference on Multimedia Retrieval

Pages 10 - 18

https://doi.org/10.1145/3460426.3463646

Published: 01 September 2021 Publication History

Abstract

Digitization of human motion using 2D or 3D skeleton representations offers exciting possibilities for many applications but, at the same time, requires scalable content-based retrieval techniques to make such data reusable. Although a lot of research effort focuses on extracting content-preserving motion features, there is a lack of techniques that support efficient similarity search on a large scale. In this paper, we introduce a new indexing scheme for organizing large collections of spatio-temporal skeleton sequences. Specifically, we apply the motion-word concept to transform skeleton sequences into structured text-like motion documents, and index such documents using an extended inverted-file approach. Over this index, we design a new similarity search algorithm that exploits the properties of the motion-word representation and provides efficient retrieval with a variable level of approximation, possibly reaching constant search costs disregarding the collection size. Experimental results confirm the usefulness of the proposed approach.

References

[1]

Fakhreddine Ababsa, Hicham Hadj-Abdelkader, and Marouane Boui. 2019. 3D Human Tracking with Catadioptric Omnidirectional Camera. In International Conference on Multimedia Retrieval (ICMR). ACM, New York, NY, USA, 73--77. https://doi.org/10.1145/3323873.3325027

Digital Library

[2]

Andreas Aristidou, Daniel Cohen-Or, Jessica K. Hodgins, Yiorgos Chrysanthou, and Ariel Shamir. 2018. Deep Motifs and Motion Signatures. ACM Transactions on Graphics, Vol. 37, 6 (2018), 187:1--187:13. https://doi.org/10.1145/3272127.3275038

Digital Library

[3]

Ricardo Baeza-Yates and Berthier A. Ribeiro-Neto. 2011. Modern Information Retrieval - the concepts and technology behind search, Second edition .Pearson Education Ltd., Harlow, England.

Digital Library

[4]

Christian Beecks and Alexander Grass. 2018. Efficient Point-Based Pattern Search in 3D Motion Capture Databases. In 6th IEEE International Conference on Future Internet of Things and Cloud (FiCloud) . IEEE Computer Society, 230--235. https://doi.org/10.1109/FiCloud.2018.00041

[5]

Petr Byvshev, Pascal Mettes, and Yu Xiao. 2020. Heterogeneous Non-Local Fusion for Multimodal Activity Recognition. In International Conference on Multimedia Retrieval (ICMR). ACM, 63--72. https://doi.org/10.1145/3372278.3390675

Digital Library

[6]

Shuning Chang, Li Yuan, Xuecheng Nie, Ziyuan Huang, Yichen Zhou, Yupeng Chen, Jiashi Feng, and Shuicheng Yan. 2020. Towards Accurate Human Pose Estimation in Videos of Crowded Scenes. In 28th ACM International Conference on Multimedia (MM). ACM, 4630--4634. https://doi.org/10.1145/3394171.3416299

Digital Library

[7]

Myung Geol Choi and Taesoo Kwon. 2019. Motion rank: applying page rank to motion data search. The Visual Computer, Vol. 35, 2 (2019), 289--300. https://doi.org/10.1007/s00371-018--1498--6

[8]

Mubbasir Kapadia, I-Kao Chiang, Tiju Thomas, Norman I. Badler, and Joseph T. Kider Jr. 2013. Efficient motion retrieval in large motion databases. In Symposium on Interactive 3D Graphics and Games (I3D). ACM, 19--28. https://doi.org/10.1145/2448196.2448199

Digital Library

[9]

Bjö rn Krü ger, Jochen Tautges, Andreas Weber, and Arno Zinke. 2010. Fast Local and Global Similarity Searches in Large Motion Capture Databases. In Eurographics/ACM SIGGRAPH Symposium on Computer Animation (SCA) . Eurographics Association, 1--10. https://doi.org/10.2312/SCA/SCA10/001-010

[10]

Jianan Li, Xuemei Xie, Qingzhe Pan, Yuhan Cao, Zhifu Zhao, and Guangming Shi. 2020. SGM-Net: Skeleton-guided multimodal network for action recognition. Pattern Recognition, Vol. 104 (2020), 1--38. https://doi.org/10.1016/j.patcog.2020.107356

[11]

Chunhui Liu, Yueyu Hu, Yanghao Li, Sijie Song, and Jiaying Liu. 2017. PKU-MMD: A Large Scale Benchmark for Skeleton-Based Human Action Understanding. In Workshop on Visual Analysis in Smart and Connected Communities (VSCC@MM). ACM, 1--8. https://doi.org/10.1145/3132734.3132739

Digital Library

[12]

Na Lv, Ying Wang, Zhiquan Feng, and Jingliang Peng. 2021. Deep Hashing for Motion Capture Data Retrieval. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2215--2219. https://doi.org/10.1109/ICASSP39728.2021.9413505

[13]

Vladimir Mic, David Novak, and Pavel Zezula. 2019. Binary Sketches for Secondary Filtering. ACM Transactions on Information Systems, Vol. 37, 1 (2019), 1:1--1:28. https://doi.org/10.1145/3231936

Digital Library

[14]

Meinard Mü ller, Tido Rö der, and Michael Clausen. 2005. Efficient content-based retrieval of motion capture data. ACM Transactions on Graphics, Vol. 24, 3 (2005), 677--685. https://doi.org/10.1145/1073204.1073247

Digital Library

[15]

Konstantinos Papadopoulos, Enjie Ghorbel, Renato Baptista, Djamila Aouada, and Bjö rn E. Ottersten. 2019. Two-Stage RGB-Based Action Detection Using Augmented 3D Poses. In 18th International Conference on Computer Analysis of Images and Patterns (CAIP), Vol. 11678. Springer, 26--35. https://doi.org/10.1007/978--3-030--29888--3_3

[16]

James Philbin, Ondrej Chum, Michael Isard, Josef Sivic, and Andrew Zisserman. 2008. Lost in quantization: Improving particular object retrieval in large scale image databases. In Computer Vision and Pattern Recognition (CVPR) . IEEE Computer Society. https://doi.org/10.1109/CVPR.2008.4587635

[17]

Thanawin Rakthanmanon, Bilson J. L. Campana, Abdullah Mueen, Gustavo E. A. P. A. Batista, M. Brandon Westover, Qiang Zhu, Jesin Zakaria, and Eamonn J. Keogh. 2012. Searching and mining trillions of time series subsequences under dynamic time warping. In 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, 262--270. https://doi.org/10.1145/2339530.2339576

Digital Library

[18]

Cheng Ren, Xiaoyong Lei, and Guofeng Zhang. 2011. Motion Data Retrieval from Very Large Motion Databases. In International Conference on Virtual Reality and Visualization. IEEE, 70--77. https://doi.org/10.1109/ICVRV.2011.50

Digital Library

[19]

Tingxin Ren, Wei Li, Zifei Jiang, Xueqing Li, Yan Huang, and Jingliang Peng. 2020. Video-Based Human Motion Capture Data Retrieval via MotionSet Network. IEEE Access, Vol. 8 (2020), 186212--186221. https://doi.org/10.1109/ACCESS.2020.3030258

[20]

Jan Sedmidubsky, Petra Budikova, Vlastislav Dohnal, and Pavel Zezula. 2020. Motion Words: A Text-like Representation of 3D Skeleton Sequences. In 42nd European Conference on Information Retrieval (ECIR). Springer, 527--541. https://doi.org/10.1007/978--3-030--45439--5_35

Digital Library

[21]

Jan Sedmidubsky, Petr Elias, Petra Budikova, and Pavel Zezula. 2021. Content-Based Management of Human Motion Data: Survey and Challenges. IEEE Access, Vol. 9 (2021), 64241--64255. https://doi.org/10.1109/ACCESS.2021.3075766

[22]

Jan Sedmidubsky, Petr Elias, and Pavel Zezula. 2019. Searching for variable-speed motions in long sequences of motion capture data. Information Systems, Vol. 80 (2019), 148--158. https://doi.org/10.1016/j.is.2018.04.002

[23]

Sijie Song, Cuiling Lan, Junliang Xing, Wenjun Zeng, and Jiaying Liu. 2018. Spatio-Temporal Attention-Based LS™ Networks for 3D Action Recognition and Detection. IEEE Transactions on Image Processing, Vol. 27, 7 (2018), 3459--3471. https://doi.org/10.1109/TIP.2018.2818328

[24]

Yingying Wang and Michael Neff. 2015. Deep signatures for indexing and retrieval in large motion databases. In 8th ACM SIGGRAPH Conference on Motion in Games (MIG). ACM, 37--45. https://doi.org/10.1145/2822013.2822024

Digital Library

[25]

Shuangyuan Wu, Zhaoqi Wang, and Shihong Xia. 2009. Indexing and retrieval of human motion data by a hierarchical tree. In ACM Symposium on Virtual Reality Software and Technology (VRST). ACM, 207--214. https://doi.org/10.1145/1643928.1643974

Digital Library

Cited By

Messina NSedmidubsky JFalchi FRebok TChen HDuh WHuang HKato MMothe JPoblete B(2023)Text-to-Motion Retrieval: Towards Joint Understanding of Human Motion Data and Natural LanguageProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592069(2420-2425)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3592069
Imamura MNakamura T(2023)A Symbolic Representation of Two-Dimensional Time Series for Arbitrary Length DTW Motif2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00125(1067-1072)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00125
Zhu SQi QZhuang ZWang JSun HLiao JOria VSapino MSatoh SKerhervé BCheng WIde ISingh V(2022)FedNKD: A Dependable Federated Learning Using Fine-tuned Random Noise and Knowledge DistillationProceedings of the 2022 International Conference on Multimedia Retrieval10.1145/3512527.3531372(185-193)Online publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1145/3512527.3531372
Show More Cited By

Index Terms

Efficient Indexing of 3D Human Motions
1. Information systems

Recommendations

Text-to-Motion Retrieval: Towards Joint Understanding of Human Motion Data and Natural Language
SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Due to recent advances in pose-estimation methods, human motion can be extracted from a common video in the form of 3D skeleton sequences. Despite wonderful application opportunities, effective and efficient content-based access to large volumes of such ...
Hierarchical indexing structure for 3d human motions
MMM'07: Proceedings of the 13th international conference on Multimedia Modeling - Volume Part I

Content-based retrieval of 3D human motion capture data has significant impact in different fields such as physical medicine, rehabilitation, and animation. This paper develops an efficient indexing approach for 3D motion capture data, supporting ...
Motion Words: A Text-Like Representation of 3D Skeleton Sequences
Advances in Information Retrieval
Abstract
There is a growing amount of human motion data captured as a continuous 3D skeleton sequence without any information about its semantic partitioning. To make such unsegmented and unlabeled data efficiently accessible, we propose to transform them ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '21: Proceedings of the 2021 International Conference on Multimedia Retrieval

August 2021

715 pages

ISBN:9781450384636

DOI:10.1145/3460426

General Chairs:
Wen-Huang Cheng
National Yang Ming Chiao Tung University, Taiwan
,
Mohan Kankanhalli
National University of Singapore, Singapore
,
Meng Wang
Hefei University of Technology, China
,
Program Chairs:
Wei-Ta Chu
National Cheng Kung University, Taiwan
,
Jiaying Liu
Peking University, China
,
Marcel Worring
University of Amsterdam, Netherlands

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Grantová Agentura České Republiky

Conference

ICMR '21

Sponsor:

SIGMM

ICMR '21: International Conference on Multimedia Retrieval

August 21 - 24, 2021

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
148
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)2

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Messina NSedmidubsky JFalchi FRebok TChen HDuh WHuang HKato MMothe JPoblete B(2023)Text-to-Motion Retrieval: Towards Joint Understanding of Human Motion Data and Natural LanguageProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592069(2420-2425)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3592069
Imamura MNakamura T(2023)A Symbolic Representation of Two-Dimensional Time Series for Arbitrary Length DTW Motif2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00125(1067-1072)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00125
Zhu SQi QZhuang ZWang JSun HLiao JOria VSapino MSatoh SKerhervé BCheng WIde ISingh V(2022)FedNKD: A Dependable Federated Learning Using Fine-tuned Random Noise and Knowledge DistillationProceedings of the 2022 International Conference on Multimedia Retrieval10.1145/3512527.3531372(185-193)Online publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1145/3512527.3531372
Zhou BWang X(2022)Feature representation for 3D object retrieval based on unconstrained multi-viewMultimedia Systems10.1007/s00530-022-00939-128:5(1699-1711)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1007/s00530-022-00939-1

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten