Xiaojuan Gu
ABSTRACT
In this paper, we presented a footstep detection method, which could assist sound editors in positioning the character’s footsteps on the timeline of the film. Based on it, a footstep detection system was designed for film sound production. Considering the characteristics of human motion and the needs of film sound production, our method included two parts: data preprocessing and footstep detection modeling. Experiments on various types of shots showed the good generalization and high accuracy of our method. The application evaluation demonstrated the high efficiency of the system.
- Z. Cao, G. Hidalgo, T. Simon, S. Wei, and Y. Sheikh. 2021. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 01 (jan 2021), 172–186. https://doi.org/10.1109/TPAMI.2019.2929257Google Scholar
Digital Library
- J. Carreira and Andrew Zisserman. 2017. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. 4724–4733. https://doi.org/10.1109/CVPR.2017.502Google ScholarCross Ref
- MMPose Contributors. 2020a. OpenMMLab Pose Estimation Toolbox and Benchmark. https://github.com/open-mmlab/mmpose.Google Scholar
- MMAction2 Contributors. 2020b. OpenMMLab’s Next Generation Video Understanding Toolbox and Benchmark. https://github.com/open-mmlab/mmaction2.Google Scholar
- Anastasios Dimou, Olivia Nemethova, and Markus Rupp. 2005. SCENE CHANGE DETECTION FOR H.264 USING DYNAMIC THRESHOLD TECHNIQUES. https://api.semanticscholar.org/CorpusID:16772266Google Scholar
- Hao-Shu Fang, Jiefeng Li, Hongyang Tang, Chao Xu, Haoyi Zhu, Yuliang Xiu, Yong-Lu Li, and Cewu Lu. 2022. AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time. IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).Google Scholar
- Hao-Shu Fang, Shuqin Xie, Yu-Wing Tai, and Cewu Lu. 2017. RMPE: Regional Multi-person Pose Estimation. In 2017 IEEE International Conference on Computer Vision (ICCV). 2353–2362. https://doi.org/10.1109/ICCV.2017.256Google ScholarCross Ref
- Francois Faure, Gilles Debunne, Marie-Paule Cani, and Franck Multon. 2001. Dynamic Analysis of Human Walking. 8th Eurographics Workshop on Computer Animation and Simulation (04 2001). https://doi.org/10.1007/978-3-7091-6874-5_4Google ScholarCross Ref
- Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. 2019. Slowfast networks for video recognition. In Proceedings of the IEEE international conference on computer vision. 6202–6211.Google ScholarCross Ref
- Kamiar Kordari, Benjamin Funk, Jared Napora, Ruchika Verma, Carole Teolis, and Travis Young. 2015. Method for step detection and gait direction estimation.Google Scholar
- Ji Lin, Chuang Gan, and Song Han. 2019. TSM: Temporal Shift Module for Efficient Video Understanding. In Proceedings of the IEEE International Conference on Computer Vision.Google ScholarCross Ref
- Dahua Lin Yue Zhao, Yuanjun Xiong. 2019. MMAction. https://github.com/open-mmlab/mmaction.Google Scholar
- Patryk Łaś and Piotr Wisniowski. 2021. Method of Step Detection and Counting Based on Measurements of Magnetic Field Variations. Sensors 21 (11 2021), 7775. https://doi.org/10.3390/s21237775Google ScholarCross Ref
Index Terms
- Footstep Detection for Film Sound Production
Recommendations
A Notation Based Approach to Film Pre-vis
CVMP '10: Proceedings of the 2010 Conference on Visual Media ProductionThis paper describes ANSWER which is a new approach to the creative process of film production. It is based on a symbolic notation system called Director Notation (DN) through which film directors are free to express their creative intent much like a ...
Immersive auditory display system 'sound cask': three-dimensional sound field reproduction system based on the boundary surface control principle
VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and TechnologySound cask was developed to realize the perfect 3D auditory display that creates 3D sound waves around the listener's head just the same as the primary sound field, based on the boundary surface control (BoSC) principle.
If we consider the sound ...
Film comic reflecting camera-works
MMM'12: Proceedings of the 18th international conference on Advances in Multimedia ModelingWe propose a novel technique for automatically creating film comics reflecting the camera-works of an original movie. Camera-works are one of the most important effects contributing to the mise en scene of the movie. A skilled director can use the ...
Comments