research-article

Topple-free foot strategy applied to real-time motion capture data using kinect sensor

Authors:

Danilo B. da Silva,

Creto A. Vidal,

Joaquim B. Cavalcante-Neto,

Italo N. S. Pessoa,

Rubens F. NunesAuthors Info & Claims

SAC '21: Proceedings of the 36th Annual ACM Symposium on Applied Computing

Pages 98 - 106

https://doi.org/10.1145/3412841.3441890

Published: 22 April 2021 Publication History

Abstract

The hardware evolution has promoted unique possibilities of interaction in virtual environments, attracting much research in the most diverse fields, especially in games. In this work, we explore the animation data captured by Microsoft Kinect in real-time for developing user interaction in a simulated virtual environment. We use physically-based characters with proportional-derivative (PD) controllers to track the user's reference joints and a virtual actuator (VA) control for balance, using internal torques. We apply a simple contact strategy called topple-free foot (TFF) strategy, which provides excellent character stability by artificially compensating torques at the standing feet. Further, we present different support change policies based on three elements: the feet' positions from Kinect data, the character's support configuration during the simulation, and a virtual step strategy inspired by SIMBICON (Simple Biped Locomotion Control). The results show the controller's ability to provide interactivity in real-time without optimizing or inverse dynamics procedures.

References

[1]

[n.d.]. Open Kinect Project. https://openkinect.org/. Accessed: 2020-12-18.

[2]

[n.d.]. OpenNI. https://structure.io/openni. Accessed: 2020-12-18.

[3]

Yeuhi Abe, Marco da Silva, and Jovan Popović. 2007. Multiobjective control with frictional contacts. In ACM SIGGRAPH/Eurographics Symposium on Computer animation (SCA '07). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 249--258. http://dl.acm.org/citation.cfm?id=1272690.1272724

[4]

Jorge Ballester and Chuck Pheatt. 2013. Using the Xbox Kinect sensor for positional data acquisition. American journal of Physics 81, 1 (2013), 71--77.

[5]

Anil Çamci, Matias Vilaplana, and Lusi Wang. 2020. Exploring the Affordances of VR for Musical Interaction Design with VIMEs. In Proceedings of the International Conference on New Interfaces for Musical Expression.

[6]

Jiawen Chen, Shahram Izadi, and Andrew Fitzgibbon. 2012. KinÊtre: animating the world with the human body. In Proceedings of the 25th annual ACM symposium on User interface software and technology. ACM, 435--444.

[7]

Jongin Choi, Sookyun Kim, Sunjeong Kim, and Shinjin Kang. 2019. Generating a Ball Sport Scene in a Virtual Environment. KSII Transactions on Internet and Information Systems (TIIS) 13, 11 (2019), 5512--5526.

[8]

Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2010. Generalized biped walking control. In ACM SIGGRAPH 2010 papers (SIGGRAPH '10). ACM, New York, NY, USA, Article 130, 9 pages.

Digital Library

[9]

D. B. d. Silva, C. A. Vidal, J. B. Cavalcante-Neto, and R. F. Nunes. 2014. A Simplified Contact Model for Treating the Balance of Biped Virtual Characters. In 2014 XVI Symposium on Virtual and Augmented Reality. 11--19.

[10]

D. B. da Silva, R. F. Nunes, C. A. Vidal, J. B. Cavalcante-Neto, P. G. Kry, and V. B. Zordan. 2017. Tunable Robustness: An Artificial Contact Strategy with Virtual Actuator Control for Balance. Computer Graphics Forum 36, 8 (2017), 499--510.

[11]

T. Geijtenbeek, N. Pronost, and A. F. van der Stappen. 2012. Simple data-driven control for simulated bipeds. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '12). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 211--219. http://dl.acm.org/citation.cfm?id=2422356.2422387

[12]

Ahmed Amine Jerraya. 2020. Hardware/Software Interface Codesign for Cyber Physical Systems. In Embedded, Cyber-Physical, and IoT Systems. Springer, 73--77.

[13]

Gizem Kayar. 2019. An Approach to Develop a Motion-Sensitive, Locally Multiplayer-Hybrid (MultiPID) 3D Video Game. Mugla Journal of Science and Technology 5, 1 (2019), 105--113.

[14]

Belinda Lange, Chien-Yen Chang, Evan Suma, Bradley Newman, Albert Skip Rizzo, and Mark Bolas. 2011. Development and evaluation of low cost game-based balance rehabilitation tool using the Microsoft Kinect sensor. In Engineering in medicine and biology society, EMBC, 2011 annual international conference of the IEEE. IEEE, 1831--1834.

[15]

Harry J. M. Lemmens, Jay B. Brodsky, and Donald P. Bernstein. 2005. Estimating Ideal Body Weight - A New Formula. Obesity Surgery 15, 7 (01 Aug 2005), 1082--1083.

[16]

Dake Liu, Guangzheng Fei, and Zanzen Huang. 2020. Immersive Prototyping for Rigid Body Animation. In Journal of Physics: Conference Series, Vol. 1518. IOP Publishing, 012029.

[17]

Adriano Macchietto, Victor Zordan, and Christian R. Shelton. 2009. Momentum control for balance. In ACM SIGGRAPH 2009 papers (SIGGRAPH '09). ACM, New York, NY, USA, Article 80, 8 pages.

Digital Library

[18]

Igor Mordatch, Jack M. Wang, Emanuel Todorov, and Vladlen Koltun. 2013. Animating Human Lower Limbs Using Contact-invariant Optimization. ACM Trans. Graph. 32, 6, Article 203 (Nov. 2013), 8 pages.

Digital Library

[19]

Štěpán Obdržálek, Gregorij Kurillo, Ferda Ofli, Ruzena Bajcsy, Edmund Seto, Holly Jimison, and Michael Pavel. 2012. Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population. In Engineering in medicine and biology society (EMBC), 2012 annual international conference of the IEEE. IEEE, 1188--1193.

[20]

I. N. S. Pessoa, P. H. A. Q. De Sousa, and R. F. Nunes. 2016. Interacting with Physically-Based Character Using Real-Time Motion Capture with Kinect Sensor. In 2016 XVIII Symposium on Virtual and Augmented Reality (SVR). 150--154.

[21]

Jerry E. Pratt, Chee-Meng Chew, Ann Torres, Peter Dilworth, and Gill A. Pratt. 2001. Virtual Model Control: An Intuitive Approach for Bipedal Locomotion. I. J. Robotic Res. 20, 2 (2001), 129--143. https://doi.org/cgi/content/abstract/20/2/129

[22]

John Arthur Roebuck, Karl Heinrich Eberhard Kroemer, and Walter Gary Thomson. 1975. Engineering anthropometry methods. Vol. 3. John Wiley & Sons.

[23]

Hubert Shum and Edmond S.L. Ho. 2012. Real-time Physical Modelling of Character Movements with Microsoft Kinect. In Proceedings of the 18th ACM Symposium on Virtual Reality Software and Technology (VRST '12). ACM, New York, NY, USA, 17--24.

Digital Library

[24]

H. P. H. Shum, E. S. L. Ho, Y. Jiang, and S. Takagi. 2013. Real-Time Posture Reconstruction for Microsoft Kinect. IEEE Transactions on Cybernetics 43, 5 (Oct 2013), 1357--1369.

[25]

C. Sunada, D. Argaez, S. Dubowsky, and C. Mavroidis. 1994. A coordinated Jacobian transpose control for mobile multi-limbed robotic systems. In IEEE ICRA. 1910--1915 vol.3.

[26]

Michiel Van De Panne and Alexis Lamouret. 1995. Guided Optimization for Balanced Locomotion. In 6th Eurographics Workshop on Animation and Simulation, 1995 (Eurographics), Demetri Terzopoulos and Daniel Thalmann (Eds.). Springer, Maastricht, Pays-Bas, 165--177.

[27]

Jack M. Wang, David J. Fleet, and Aaron Hertzmann. 2009. Optimizing walking controllers. In ACM SIGGRAPH Asia 2009 papers (SIGGRAPH Asia '09). ACM, New York, NY, USA, Article 168, 8 pages.

Digital Library

[28]

Jack M. Wang, Samuel R. Hamner, Scott L. Delp, and Vladlen Koltun. 2012. Optimizing locomotion controllers using biologically-based actuators and objectives. ACM Trans. Graph. 31, 4, Article 25 (July 2012), 11 pages.

Digital Library

[29]

Pawel Wrotek, Odest Chadwicke Jenkins, and Morgan McGuire. 2006. Dynamo: Dynamic, Data-driven Character Control with Adjustable Balance. In ACM SIGGRAPH Symposium on Videogames (Sandbox '06). ACM, New York, NY, USA, 61--70.

Digital Library

[30]

KangKang Yin, Kevin Loken, and Michiel van de Panne. 2007. SIMBICON: simple biped locomotion control. In ACM SIGGRAPH 2007 papers (SIGGRAPH '07). ACM, New York, NY, USA, Article 105.

Digital Library

[31]

Victor Brian Zordan and Jessica K. Hodgins. 2002. Motion capture-driven simulations that hit and react. In ACM SIGGRAPH/Eurographics Symposium on Computer animation (SCA '02). ACM, New York, NY, USA, 89--96.

Digital Library

Cited By

Cao ZBao TRen ZFan YDeng KJia W(2022)Real-Time Stylized Humanoid Behavior Control through Interaction and SynchronizationSensors10.3390/s2204145722:4(1457)Online publication date: 14-Feb-2022
https://doi.org/10.3390/s22041457
Yan ZZhou JWu YLiu GLuo DZhou ZMi HSun LChen XTao YZhang YWang G(2022)Shoes++Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346206:2(1-29)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534620

Index Terms

Topple-free foot strategy applied to real-time motion capture data using kinect sensor
1. Computing methodologies
  1. Computer graphics
  2. Modeling and simulation

Recommendations

Multi-sensor data fusion for hand tracking using Kinect and Leap Motion
VRIC '14: Proceedings of the 2014 Virtual Reality International Conference

Often presented as competing products on the market of low cost 3D sensors, the Kinect™ and the Leap Motion™ (LM) can actually be complementary in some scenario. We promote, in this paper, the fusion of data acquired by both LM and Kinect sensors to ...
Force-Based Foot Gesture Navigation in Virtual Reality
VRST '21: Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology

Navigation is a primary interaction in virtual reality. Previous research has explored different forms of artificial locomotion techniques for navigation, including hand gestures and body motions. However, few studies have investigated force-based foot ...
Motion Performance Measurement using the Microsoft Kinect Sensor

In this study, motion performance indices based on the kinematics of upper body have been presented and compared to be used in a home-based rehabilitation device. Microsoft Kinect sensor is used to extract and calculate such indices. A set of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '21: Proceedings of the 36th Annual ACM Symposium on Applied Computing

March 2021

2075 pages

ISBN:9781450381048

DOI:10.1145/3412841

Conference Chairs:
Chih-Cheng Hung
Kennesaw State University
,
Jiman Hong
Soongsil University, South Korea
,
Program Chairs:
Alessio Bechini
University of Pisa, Italy
,
Eunjee Song
Baylor University

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 April 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SAC '21

Sponsor:

SIGAPP

SAC '21: The 36th ACM/SIGAPP Symposium on Applied Computing

March 22 - 26, 2021

Virtual Event, Republic of Korea

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25

Sponsor:
sigapp

The 40th ACM/SIGAPP Symposium on Applied Computing

March 31 - April 4, 2025

Catania , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
60
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cao ZBao TRen ZFan YDeng KJia W(2022)Real-Time Stylized Humanoid Behavior Control through Interaction and SynchronizationSensors10.3390/s2204145722:4(1457)Online publication date: 14-Feb-2022
https://doi.org/10.3390/s22041457
Yan ZZhou JWu YLiu GLuo DZhou ZMi HSun LChen XTao YZhang YWang G(2022)Shoes++Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346206:2(1-29)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534620

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten