skip to main content
10.1145/2933242.2933263acmconferencesArticle/Chapter ViewAbstractPublication PageseicsConference Proceedingsconference-collections
research-article

FusionKit: a generic toolkit for skeleton, marker and rigid-body tracking

Published: 21 June 2016 Publication History

Abstract

We present a toolkit for markerless skeleton tracking and marker-based object tracking utilizing data fusion with an arbitrary number of depth cameras. As depth-camera based skeletal tracking is always inaccurate due to technology limitations, our goal was to be able to preestimate systematic errors for given tracking situations to improve fusion.
Previous work analyzed various aspects of depth camera accuracy, however to our best knowledge, there has been neither systematic error modelling nor an application of such a model for skeletal fusion.
Our paper presents such a model for the Kinect v2 camera, by using statistical modelling on capture datasets using such cameras and a marker-based ground truth capture system. By applying this model, we are able to improve the overall accuracy of the fusion output by 68% by predicting data quality with an error of around 3.2 cm.
Our toolkit is available for use by other researchers to easily create larger capture spaces with higher tracking accuracy based on the error model when compared to single depth cameras.

References

[1]
Stylianos Asteriadis, Anargyros Chatzitofis, Dimitrios Zarpalas, Dimitrios S. Alexiadis, and Petros Daras. 2013. Estimating Human Motion from Multiple Kinect Sensors. In Proceedings of the 6th International Conference on Computer Vision / Computer Graphics Collaboration Techniques and Applications (MIRAGE '13). ACM, New York, NY, USA, 3:1--3:6.
[2]
Nur Aziza Azis, Ho-Jin Choi, and Youssef Iraqi. 2015. Substitutive skeleton fusion for human action recognition. In 2015 International Conference on Big Data and Smart Computing (BigComp). 170--177.
[3]
Paul J Besl and Neil D McKay. 1992. Method for registration of 3-D shapes. In Robotics-DL tentative. International Society for Optics and Photonics, 586--606.
[4]
Maurizio Caon, Yong Yue, Julien Tscherrig, Elena Mugellini, and Omar Abou Khaled. 2011. Context-Aware 3D Gesture Interaction Based on Multiple Kinects. In Proceedings of The First International Conference on Ambient Computing, Applications, Services and Technologies, AMBIENT. 7--12.
[5]
Florian Faion, Simon Friedberger, Antonio Zea, and Uwe D Hanebeck. 2012. Intelligent sensor-scheduling for multi-kinect-tracking. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on. IEEE, 3993--3999.
[6]
Jerome H. Friedman. 2002. Stochastic gradient boosting. Computational Statistics & Data Analysis 38, 4 (2002), 367--378.
[7]
Nicholas Gillian and Joseph A Paradiso. 2014. The gesture recognition toolkit. The Journal of Machine Learning Research 15, 1 (2014), 3483--3487.
[8]
T. Hachaj, M. R. Ogiela, and M. Piekarczyk. 2013. Dependence of Kinect sensors number and position on gestures recognition with Gesture Description Language semantic classifier. In Computer Science and Information Systems (FedCSIS), 2013 Federated Conference on. 571--575.
[9]
Emanuela Haller, Georgiana Scarlat, Irina Mocanu, and Mihai Trăscău. 2013. Human Activity Recognition Based on Multiple Kinects. In Evaluating AAL Systems Through Competitive Benchmarking. Communications in Computer and Information Science, Vol. 386. Springer Berlin Heidelberg, 48--59.
[10]
Jungong Han, Ling Shao, Dong Xu, and Jamie Shotton. 2013. Enhanced computer vision with microsoft kinect sensor: A review. Cybernetics, IEEE Transactions on 43, 5 (2013), 1318--1334.
[11]
Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, and Andrew Fitzgibbon. 2011. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera (UIST '11). ACM, New York, NY, USA, 559--568.
[12]
Brett Jones, Rajinder Sodhi, Michael Murdock, Ravish Mehra, Hrvoje Benko, Andrew Wilson, Eyal Ofek, Blair MacIntyre, Nikunj Raghuvanshi, and Lior Shapira. 2014. Roomalive: Magical experiences enabled by scalable, adaptive projector-camera units. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 637--644.
[13]
S. Kaenchan, P. Mongkolnam, B. Watanapa, and S. Sathienpong. 2013. Automatic multiple Kinect cameras setting for simple walking posture analysis. In Computer Science and Engineering Conference (ICSEC), 2013 International. 245--249.
[14]
Bahador Khaleghi, Alaa Khamis, Fakhreddine O. Karray, and Saiedeh N. Razavi. 2013. Multisensor Data Fusion: A Review of the State-of-the-art. 14, 1 (2013), 28--44.
[15]
Alexandros Kitsikidis, Kosmas Dimitropoulos, Stella Douka, and Nikos Grammalidis. 2014. Dance analysis using multiple kinect sensors. VISAPP2014 (2014).
[16]
Matteo Munaro, Alberto Basso, Andrea Fossati, Luc Van Gool, and Emanuele Menegatti. 2014. 3D reconstruction of freely moving persons for re-identification with a depth sensor. In Robotics and Automation (ICRA), 2014 IEEE International Conference on. IEEE, 4512--4519.
[17]
Matteo Munaro, Filippo Basso, and Emanuele Menegatti. 2016. OpenPTrack: Open source multi-camera calibration and people tracking for RGB-D camera networks. Robotics and Autonomous Systems 75 (2016), 525--538.
[18]
Michael Otto, Philipp Agethen, Florian Geiselhart, and Enrico Rukzio. 2015. Towards ubiquitous tracking: Presenting a scalable, markerless tracking approach using multiple depth cameras, In In Proc. of EuroVR 2015. In Proc. of EuroVR 2015 (European Association for Virtual Reality and Augmented Reality), Best Industrial Paper award (2015).
[19]
Christian Schönauer and Hannes Kaufmann. 2011. Wide Area Motion Tracking Using Consumer Hardware. In ACM Advances in Computer Entertainment Technology conference (ACE 2011), Lisbon, Portugal, Vol. 8.
[20]
Qifei Wang, G. Kurillo, F. Ofli, and R. Bajcsy. 2015. Evaluation of Pose Tracking Accuracy in the First and Second Generations of Microsoft Kinect. In Healthcare Informatics (ICHI), 2015 International Conference on. 380--389.
[21]
B. Williamson, J. LaViola, Tim Roberts, and Pat Garrity. 2012. Multi-kinect tracking for dismounted soldier training. In Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC). 1727--1735.
[22]
Lin Yang, Longyu Zhang, Haiwei Dong, Abdulhameed Alelaiwi, and Abdulmotaleb El Saddik. 2015. Evaluating and improving the depth accuracy of Kinect for Windows v2. Sensors Journal, IEEE 15, 8 (2015), 4275--4285.
[23]
Kwok-Yun Yeung, Tsz-Ho Kwok, and Charlie CL Wang. 2013. Improved Skeleton Tracking by Duplex Kinects: A Practical Approach for Real-Time Applications. Journal of Computing and Information Science in Engineering 13, 4 (2013), 041007.
[24]
Zhengyou Zhang. 2012. Microsoft Kinect Sensor and Its Effect. MultiMedia, IEEE 19, 2 (Feb 2012), 4--10.

Cited By

View all
  • (2024)Mixed-test method for performance evaluation of intelligent collaborative robotic systems2024 IEEE 20th International Conference on Automation Science and Engineering (CASE)10.1109/CASE59546.2024.10711408(1922-1927)Online publication date: 28-Aug-2024
  • (2023)Full-body Human Motion Reconstruction with Sparse Joint Tracking Using Flexible SensorsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/356470020:2(1-19)Online publication date: 25-Sep-2023
  • (2020)An Unsupervised Approach For Gait Phase Detection2020 4th International Conference on Computational Intelligence and Networks (CINE)10.1109/CINE48825.2020.234396(1-5)Online publication date: Feb-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
EICS '16: Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems
June 2016
321 pages
ISBN:9781450343220
DOI:10.1145/2933242
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. kinect error modelling
  2. multi-depth camera fusion
  3. toolkit

Qualifiers

  • Research-article

Conference

EICS'16
Sponsor:

Acceptance Rates

Overall Acceptance Rate 73 of 299 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)15
  • Downloads (Last 6 weeks)2
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Mixed-test method for performance evaluation of intelligent collaborative robotic systems2024 IEEE 20th International Conference on Automation Science and Engineering (CASE)10.1109/CASE59546.2024.10711408(1922-1927)Online publication date: 28-Aug-2024
  • (2023)Full-body Human Motion Reconstruction with Sparse Joint Tracking Using Flexible SensorsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/356470020:2(1-19)Online publication date: 25-Sep-2023
  • (2020)An Unsupervised Approach For Gait Phase Detection2020 4th International Conference on Computational Intelligence and Networks (CINE)10.1109/CINE48825.2020.234396(1-5)Online publication date: Feb-2020
  • (2020)A Low-Cost Pathological Gait Detection System in Multi-Kinect EnvironmentProgress in Optomechatronics10.1007/978-981-15-6467-3_13(97-104)Online publication date: 22-Sep-2020
  • (2019)A Radar Measurement Setup with a Ground Truth System for Micro-Doppler Human Movements2019 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM)10.1109/ICMIM.2019.8726509(1-4)Online publication date: Apr-2019
  • (2018)DynCamProceedings of the Virtual Reality International Conference - Laval Virtual10.1145/3234253.3234299(1-8)Online publication date: 4-Apr-2018
  • (2018)Demo of the Matrix Has You: Realizing Slow Motion in Full-Body Virtual Reality2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR.2018.8446136(773-774)Online publication date: Mar-2018
  • (2018)A Study on Human Gait Kinematic Validation in Multi-Kinect v2 Environment2018 15th IEEE India Council International Conference (INDICON)10.1109/INDICON45594.2018.8987073(1-4)Online publication date: Dec-2018
  • (2018)The digital twin implementation for linking the virtual representation of human-based production tasks to their physical counterpart in the factory-floorInternational Journal of Computer Integrated Manufacturing10.1080/0951192X.2018.152943032:1(1-12)Online publication date: 11-Oct-2018
  • (2017)Real-time ambient fusion of commodity tracking systems for virtual realityProceedings of the 27th International Conference on Artificial Reality and Telexistence and 22nd Eurographics Symposium on Virtual Environments10.5555/3298830.3298832(1-8)Online publication date: 22-Nov-2017
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media