Abstract
The ability to capture human motion based on wearable sensors has a wide range of applications, e.g., in healthcare, sports, well-being, and workflow analysis. This article focuses on the development of an online-capable system for accurately capturing joint kinematics based on inertial measurement units (IMUs) and its clinical application, with a focus on locomotion analysis for rehabilitation. The article approaches the topic from the technology and application perspectives and fuses both points of view. It presents, in a self-contained way, previous results from three studies as well as new results concerning the technological development of the system. It also correlates these with new results from qualitative expert interviews with medical practitioners and movement scientists. The interviews were conducted for the purpose of identifying relevant application scenarios and requirements for the technology used. As a result, the potentials of the system for the different identified application scenarios are discussed and necessary next steps are deduced from this analysis.
About the authors
Gabriele Bleser received her diploma in computer science from the University Koblenz-Landau, in 2004. From 2004 to 2008 she worked at the Department Virtual and Augmented Reality at the Fraunhofer IGD in Darmstadt. In 2008, she joined the Augmented Vision department at the DFKI in Kaiserslautern. Her PhD, which she received from the University of Kaiserslautern in 2009, focuses on visual-inertial SLAM for mobile augmented reality. Since November 2014, she leads the junior research group wearHEALTH at the University of Kaiserslautern, focusing on the design, development and evaluation of mobile and wearable health systems.
Bertram Taetz received his B.Sc. and M.Sc. in applied mathematics, minor subject physics, from the Ruhr University Bochum, in 2009. During his PhD at the Ruhr University he focused on numerical methods for dynamical systems. He finished his PhD in 2012. Since 2013 he works as senior researcher at the DFKI. In 2015 he also joined the junior research group wearHEALTH at the University of Kaiserslautern. His research interests are numerical and statistical methods for motion estimation and dynamical systems.
Markus Miezal studied computer science at the University of Bremen and finished his diploma in 2010. In the same year, he started working as a researcher at DFKI focusing on visual-inertial sensor fusion and inertial human body motion capturing. In January 2015, he joined the junior research group wearHEALTH at the University of Kaiserslautern. He is pursuing a PhD focusing on different models and methods for inertial human motion capturing with medical applications.
Corinna Christmann received her diploma in psychology from the University of Mainz, in 2011. Her PhD, which she received from the University of Kaiserslautern in January 2014, focuses on speech and auditory processing. Afterwards she was the scientific coordinator for the Center for Cognitive Science at the University of Kaiserslautern. In November 2014, she joined the junior research group wearHEALTH. Her research interests are mobile health, human-computer interaction and electroencephalography.
Daniel Steffen holds a diploma degree in computer science from the University of Kaiserslautern. From 2007 to 2014 he was a researcher in the Augmented Vision department at DFKI. Since 2015, he is a member of the junior research group wearHEALTH at the University of Kaiserslautern. His research interests include human centered interaction and visualization. In particular, he is passionate about improving user’s experience in both existing and emerging computing environments.
Katja Regenspurger completed her studies of human medicine in 1998 at the University of Jena and afterwards finished the specialist’s education in the field of physical and rehabilitative medicine. Additional qualifications followed in manual therapy, sports medicine and special pain therapy. She is currently working as a department head of Conservative Orthopedics and Physical Therapy at the Department of Orthopedics, Trauma and Restorative Surgery at the University Hospital Halle (Saale).
A.1 Biomechanical Model Definitions
Four different biomechanical model definitions exemplified with a two-segment model: Solid black lines represent the segments, solid curves indicate known six degrees of freedom transformations (parameters), while dashed curves indicate transformations to be estimated (variables). Note that, here, the IMU-to-segment transformations as well as the segment lengths, are always assumed known.
An intuitive and widely used biomechanical model definition is a kinematic chain. Here, the orientation of each segment is represented relative to the orientation of the previous segment (often in terms of joint angles) and only the chain root is represented with an additional position w.r.t. a global coordinate system (e.g., [24]), see Fig. 5(a). This leads to a minimal parametrization, however, with the problem of singularities. In contrast, [32] described the concept of a free segments model where each segment is defined by a global position and orientation (see Fig. 5(b)). In order to obtain physically plausible solutions, the connections between the segments at the joints as well as restricted degrees of freedom of the joints need to be integrated into the estimation process as constraints. Analogously, a free IMUs model can be defined, which has implications concerning the motion model assumed in the estimation (see Fig. 5(c)). It is also possible to estimate both the global IMU and segment orientations and translations, while additionally modeling the IMU-to-segment transformations as stochastic constraints to the estimation. This is then called free segments and free IMUs model (see Fig. 5(d)) and was proposed in [18], however, without evaluation of its effects. Obviously, the more redundant the model is defined, the higher the dimensionality of the estimation problem and, hence, the computational complexity becomes. However, at the same time, a redundant system with stochastic constraints represents one possibility to address unavoidable errors in the biomechanical model parameters (cf. Section 2.1). This is investigated in Section 3.1.
A.2 Estimation Methods
Typical estimation approaches based on noisy data utilize Bayesian inference and formulate the estimation problem as nonlinear maximum a posteriori problem that can be solved in multiple ways, see [11] for a textbook. A widely used online-capable method for nonlinear systems is the extended Kalman filter (EKF), which works based on a predictor-corrector scheme (see [43] for a textbook). The system states (variables) are predicted for the next time step via a given motion model and are then corrected based on measurements via respective measurement models. Another online-capable approach is the sliding-window (weighted least squares) optimization. This is inspired through [18] who propose an offline optimization-based method for inertial motion capture. The EKF solves the maximum a posteriori problem recursively with rather low computational costs; however, it introduces problem-dependent linearization errors and the inclusion of constraints is usually non-trivial. In contrast, the sliding-window optimization allows handling constraints in a concise way via an optimization framework; however, it has a higher computational cost (depending on the chosen window size). In fact, the EKF can be derived from an optimization point of view by taking only one Newton step on the quadratic cost function [14].
References
[1] Allseits, E.; Lucarevic, J.; Gailey, R.; Agrawal, V.; Gaunaurd, I. & Bennett, C. The development and concurrent validity of a real-time algorithm for temporal gait analysis using inertial measurement units. Journal of Biomechanics, 2017, doi:10.1016/j.jbiomech.2017.02.016.Search in Google Scholar PubMed
[2] Barth, J.; Oberndorfer, C.; Pasluosta, C.; Schülein, S.; Gassner, H.; Reinfelder, S.; Kugler, P.; Schuldhaus, D.; Winkler, J.; Klucken, J. & Eskofier, B. M. Stride segmentation during free walk movements using multi-dimensional subsequence dynamic time warping on inertial sensor data. Sensors, 2015, 15, 6419–6440.10.3390/s150306419Search in Google Scholar PubMed PubMed Central
[3] Bergmann, J. & McGregor, A. Body-worn sensor design: what do patients and clinicians want? Annals of Biomedical Engineering, 2011, 39, 2299–2312.10.1007/s10439-011-0339-9Search in Google Scholar PubMed
[4] Bouvier, B.; Duprey, S.; Claudon, L.; Dumas, R. & Savescu, A. Upper Limb Kinematics Using Inertial and Magnetic Sensors: Comparison of Sensor-to-Segment Calibrations. Sensors, 2015, 15, 18813–18833.10.3390/s150818813Search in Google Scholar PubMed PubMed Central
[5] Brunnekreef, J. J.; van Uden, C. J.; van Moorsel, S. & Kooloos, J. G. Reliability of videotaped observational gait analysis in patients with orthopedic impairments. BMC Musculoskeletal Disorders, 2005, 6.10.1186/1471-2474-6-17Search in Google Scholar PubMed PubMed Central
[6] Chen, S.; Lach, J.; Lo, B. & Yang, G. Z. Towards Pervasive Gait Analysis for Medicine with Wearable Sensors: A Systematic Review for Clinicians and Medical Researchers. IEEE Journal of Biomedical and Health Informatics, 2016, 20, 1521–1537.10.1109/JBHI.2016.2608720Search in Google Scholar PubMed
[7] Cutti, A. G.; Ferrari, A.; Garofalo, P.; Raggi, M.; Cappello, A. & Ferrari, A. ‘Outwalk’: a protocol for clinical gait analysis based on inertial and magnetic sensors. Medical & Biological Engineering & Computing, Springer, 2010, 48, 17–25.10.1007/s11517-009-0545-xSearch in Google Scholar PubMed
[8] Deterding, S.; Björk, S.L.; Nacke, L.E.; Dixon, D. & Lawley, E. Designing gamification: creating gameful and playful experiences. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), 2013, 3263–3266.10.1145/2468356.2479662Search in Google Scholar
[9] Garcia, P.; Domnick; Lodde, G. & Raschke, M. J. Operative Versorgung in Orthopädie und Unfallchirurgie. Der Unfallchirurg, 2016, 1–9.10.1007/s00113-016-0275-0Search in Google Scholar PubMed
[10] Götz-Neumann, K. Gehen verstehen: Ganganalyse in der Physiotherapie. Georg Thieme Verlag, 2015.10.1055/b-003-127005Search in Google Scholar
[11] Gustafsson, F. Statistical Sensor Fusion, Studentlitteratur, 2nd edition, 2012.Search in Google Scholar
[12] Hannink, J.; Kautz, T.; Pasluosta, C.; Barth, J.; Schulein, S.; Gassmann, K. G.; Klucken, J. & Eskofier, B. M. Mobile Stride Length Estimation with Deep Convolutional Neural Networks. IEEE Journal of Biomedical and Health Informatics, 2017.10.1109/JBHI.2016.2636456Search in Google Scholar PubMed
[13] Harle, R. A Survey of Indoor Inertial Positioning Systems for Pedestrians. IEEE Communications Surveys & Tutorials, 2013, 15, 1281–1293.10.1109/SURV.2012.121912.00075Search in Google Scholar
[14] Humpherys, J. & West, J. Kalman Filtering with Newton’s Method. IEEE Control Systems, 2010, 30, 101–106.10.1109/MCS.2010.938485Search in Google Scholar
[15] Iosa, M.; Picerno, P.; Paolucci, S. & Morone, G. Wearable Inertial Sensors for Human Movement Analysis. Expert Review of Medical Devices, Taylor & Francis, 2016.10.1080/17434440.2016.1198694Search in Google Scholar PubMed
[16] Kari, T. Can Exergaming Promote Physical Fitness and Physical Activity?: A Systematic Review of Systematic Reviews. International Journal of Gaming and Computer-Mediated Simulations, 2014, 6(4), 59–77.10.4018/ijgcms.2014100105Search in Google Scholar
[17] Kavanagh, J. J. & Menz, H. B. Accelerometry: a technique for quantifying movement patterns during walking. Gait & Posture, 2008, 28, 1–15.10.1016/j.gaitpost.2007.10.010Search in Google Scholar PubMed
[18] Kok, M.; Hol, J. & Schön, T. An optimization-based approach to human body motion capture using inertial sensors. 19th World Congress of the International Federation of Automatic Control (IFAC), 2014, 79–85.10.3182/20140824-6-ZA-1003.02252Search in Google Scholar
[19] Leardini, A.; Sawacha, Z.; Paolini, G.; Ingrosso, S.; Nativo, R. & Benedetti, M. G. A new anatomically based protocol for gait analysis in children. Gait & Posture, 2007, 26, 560–571.10.1016/j.gaitpost.2006.12.018Search in Google Scholar PubMed
[20] Ligorio, G. & Sabatini, A. M. Dealing with Magnetic Disturbances in Human Motion Capture: A Survey of Techniques. Micromachines, 2016, 7.10.3390/mi7030043Search in Google Scholar PubMed PubMed Central
[21] Ligorio, G.; Zanotto, D.; Sabatini, A. & Agrawal, S. A novel functional calibration method for real-time elbow joint angles estimation with magnetic-inertial sensors. Journal of Biomechanics, 2017, 54, 106–110.10.1016/j.jbiomech.2017.01.024Search in Google Scholar PubMed
[22] Ludwig, O. Ganganalyse in der Praxis: Anwendung in Prävention, Therapie und Versorgung. Maurer, 2. Auflage, 2015.Search in Google Scholar
[23] Maetzler, W.; Klucken, J. & Horne, M. A clinical view on the development of technology-based tools in managing Parkinson’s disease. Movement Disorders, Wiley Online Library, 2016.10.1002/mds.26673Search in Google Scholar PubMed
[24] Miezal, M.; Bleser, G.; Schmitz, N. & Stricker, D. A generic approach to inertial tracking of arbitrary kinematic chains. 8th International Conference on Body Area Networks, 2013.10.4108/icst.bodynets.2013.253608Search in Google Scholar
[25] Miezal, M.; Taetz, B. & Bleser, G. Real-time inertial lower body kinematics and ground contact estimation at anatomical foot points for agile human locomotion. International Conference on Robotics and Automation (ICRA), 2017.10.1109/ICRA.2017.7989371Search in Google Scholar
[26] Miezal, M.; Taetz, B. & Bleser, G. On Inertial Body Tracking in the Presence of Model Calibration Errors. Sensors, 2016, 16.10.3390/s16071132Search in Google Scholar PubMed PubMed Central
[27] Miezal, M.; Taetz, B.; Schmitz, N. & Bleser, G. Ambulatory inertial spinal tracking using constraints. 9th International Conference on Body Area Networks, London, UK, October, 2014.10.4108/icst.bodynets.2014.256955Search in Google Scholar
[28] Müller, P.; Bégin, M. A.; Schauer, T. & Seel, T. Alignment-free, self-calibrating elbow angles measurement using inertial sensors. IEEE Journal of Biomedical and Health Informatics, 2017, 21 (2): 312–319.10.1109/JBHI.2016.2639537Search in Google Scholar PubMed
[29] van den Noort, J. C.; Ferrari, A.; Cutti, A. G.; Becher, J. G. & Harlaar, J. Gait analysis in children with cerebral palsy via inertial and magnetic sensors. Medical & Biological Engineering & Computing, Springer, 2013, 51, 377–386.10.1007/s11517-012-1006-5Search in Google Scholar PubMed
[30] Palermo, E.; Rossi, S.; Marini, F.; Patane, F. & Cappa, P. Experimental evaluation of accuracy and repeatability of a novel body-to-sensor calibration procedure for inertial sensor-based gait analysis. Measurement, 2014, 52, 145–155.10.1016/j.measurement.2014.03.004Search in Google Scholar
[31] Pohl, K. Requirements Engineering. Grundlagen, Prinzipien, Techniken. Dpunkt, 2007.Search in Google Scholar
[32] Roetenberg, D.; Luinge, H.; Slycke, P. Xsens MVN: Full 6DOF Human Motion Tracking Using Miniature Inertial Sensors. Technical Report. Xsens Technologies: Enschede, The Netherlands, 2014.Search in Google Scholar
[33] Salarian, A.; Horak, F. B.; Zampieri, C.; Carlson-Kuhta, P.; Nutt, J. G. & Aminian, K. iTUG, a Sensitive and Reliable Measure of Mobility. IEEE Transactions on Neural Systems and Rehabilitation Engineering., 2010, 18, 303–310.10.1109/TNSRE.2010.2047606Search in Google Scholar PubMed PubMed Central
[34] Seel, T.; Landgraf, L.; Escobar, V. C. & Schauer, T. Online Gait Phase Detection with Automatic Adaption to Gait Velocity Changes Using Accelerometers and Gyroscopes. Biomedical Engineering, 2014, 59, 795–798.Search in Google Scholar
[35] Seel, T.; Schauer, T. & Raisch, J. IMU-Based Joint Angle Measurement for Gait Analysis. Sensors, 2014, 14, 6891–6909.10.3390/s140406891Search in Google Scholar PubMed PubMed Central
[36] Shull, P.; Jirattigalachote, W.; Hunt, M.; Cutkosky, M. & Delp, S. Quantified self and human movement: a review on the clinical impact of wearable sensing and feedback for gait analysis and intervention. Gait & Posture, 2014, 40.10.1016/j.gaitpost.2014.03.189Search in Google Scholar PubMed
[37] Skog, I.; Handel, P.; Nilsson, J.-O. & Rantakokko, J. Zero-velocity detection: An algorithm evaluation. IEEE Transactions on Biomedical Engineering, 2010, 57, 2657–2666.10.1109/TBME.2010.2060723Search in Google Scholar PubMed
[38] Suppé, B. & Bongartz, M. FBL Klein-Vogelbach Functional Kinetics praktisch angewandt: Gehen- Analyse und Intervention. Springer-Verlag, 2013.10.1007/978-3-642-20726-6Search in Google Scholar
[39] Sutherland, D. The evolution of clinical gait analysis. Part II kinematics. Gait & Posture, 2002, 16, 159–179.10.1016/S0966-6362(02)00004-8Search in Google Scholar
[40] Tadano, S.; Takeda, R. & Miyagawa, H. Three dimensional gait analysis using wearable acceleration and gyro sensors based on quaternion calculations. Sensors, 2013, 13, 9321–9343.10.3390/s130709321Search in Google Scholar PubMed PubMed Central
[41] Tadano, S.; Takeda, R.; Sasaki, K.; Fujisawa, T. & Tohyama, H. Gait characterization for osteoarthritis patients using wearable gait sensors (H-Gait systems). Journal of Biomechanics, 2016, 49, 684–690.10.1016/j.jbiomech.2016.01.017Search in Google Scholar PubMed
[42] Taetz, B.; Bleser, G. & Miezal, M. Towards self-calibrating inertial body motion capture. 19th International Conference on Information Fusion, 2016, 1751–1759.Search in Google Scholar
[43] Thrun, S.; Burgard, W. & Fox, D. Probabilistic Robotics (Intelligent Robotics and Autonomous Agents). The MIT Press, 2005.Search in Google Scholar
[44] Toro, B.; Nester, C. & Farren, P. A review of observational gait assessment in clinical practice. Physiotherapy Theory and Practice, 2003, 19, 137–149.10.1080/09593980307964Search in Google Scholar
[45] Vargas-Valencia, L. S.; Elias, A.; Rocon, E.; Bastos-Filho, T. & Frizera, A. An IMU-to-Body Alignment Method Applied to Human Gait Analysis. Sensors, 2016, 16.10.3390/s16122090Search in Google Scholar PubMed PubMed Central
[46] De Vries, W.; Veeger, H.; Cutti, A.; Baten, C. & Van der Helm, F. Functionally interpretable local coordinate systems for the upper extremity using inertial & magnetic measurement systems. Journal of Biomechanics, 2010, 43, 1983–1988.10.1016/j.jbiomech.2010.03.007Search in Google Scholar PubMed
[47] Wenk, F. & Frese, U. Posture from motion. International Conference on Intelligent Robots and Systems (IROS), 2015, 280–285.10.1109/IROS.2015.7353386Search in Google Scholar
[48] Wu, G.; Siegler, S.; Allard, P.; Kirtley, C.; Leardini, A.; Rosenbaum, D.; Whittle, M.; D’Lima, D. D.; Cristofolini, L.; Witte, H.; Schmid, O. & Stokes, I. ISB recommendation on definitions of joint coordinate system of various joints for the reporting of human joint motion–part I: ankle, hip, and spine. Journal of Biomechanics, 2002, 35, 543–548.10.1016/S0021-9290(01)00222-6Search in Google Scholar
[49] Young, A. D. Use of body model constraints to improve accuracy of inertial motion capture. International Conference on Body Sensor Networks, IEEE Computer Society, 2010, 180–186.10.1109/BSN.2010.30Search in Google Scholar
[50] Zheng, Y.; Ding, X.; Poon, C.; Lo, B.; Zhang, H.; Zhou, X.; Yang, G.; Zhao, N. & Zhang, Y. Unobtrusive Sensing and Wearable Devices for Health Informatics. IEEE Transactions on Biomedical Engineering, 2014, 61, 1538–1554.10.1109/TBME.2014.2309951Search in Google Scholar PubMed PubMed Central
© 2017 Walter de Gruyter GmbH, Berlin/Boston
