A dynamic spatiotemporal model for fall warning and protection

Abstract

Early detection of falls is important for reducing fall injuries. However, existing fall detection strategies mostly focus on reducing impact injuries rather than avoiding falls. This study proposed the concept of identifying “Imbalance Point” to warn the body imbalance, allowing sufficient time to recover balance. And if falling cannot be avoided, an impact sign is released by detecting the “Fall Point” prior to the impact. To achieve this goal, motion prediction model and balance recovery model are integrated into a spatiotemporal framework to analyze dynamic and kinematic features of body motion. Eight healthy young volunteers participated in three sets of experiment: Normal trial, Recovery trial and Fall trial. The body motion in the trials was recorded using Microsoft Azure Kinect. The results show that the developed algorithm for Fall Point detection achieved 100% sensitivity and 98.6% specificity, along with an average lead time of 297 ms. Moreover, Imbalance Point was successfully detected in all Fall trials, and the average time interval between Imbalance Point and Fall Point was 315 ms, longer than reported step reaction time for elderly (approximately 270 ms). The experiment results demonstrate that the developed algorithm have great potential for fall warning and protection in the elderly.

Graphical Abstract

Appendix

The model motion governing equations and the detailed derivation during pre-contact phase and contact phase are given in the appendix.

1.1 Basic settings

$$\left\{ \begin{gathered} \tau_{ankle} = \, k_{ankle} (\pi /2 - \alpha ) \hfill \\ \tau_{knee} = \, - k_{knee} \beta \hfill \\ \tau_{hip} = \, k_{hip} (\alpha + \beta + \gamma - \pi /2) \hfill \\ \tau_{sp} = \tau_{ankle} , \, \tau_{sd} = - \tau_{knee} , \, \tau_{tp} = \tau_{knee} \hfill \\ \tau_{td} = - \tau_{hip} ,\tau_{hp} = \tau_{hip} , \, \tau_{hd} = 0 \hfill \\ F_{sd} = - F_{tp} , \, F_{td} = - F_{hp} , \, F_{hd} = [\begin{array}{*{20}c} 0 & 0 & 0 \\ \end{array} ]^{T} \hfill \\ I_{s} = 1/3m_{s} \cdot (0.5l_{s} )^{2} ,I_{t} = 1/3m{}_{t} \cdot (0.5l_{t} )^{2} ,I_{h} = 1/3m_{h} \cdot (0.5l_{h} )^{2} \hfill \\ P_{s} = \left[ {\begin{array}{*{20}c} 0 \\ {l_{s} \cdot \cos (\alpha )} \\ {l_{s} \cdot \sin (\alpha )} \\ \end{array} } \right],P_{t} = \left[ {\begin{array}{*{20}c} 0 \\ {l_{t} \cdot \cos (\alpha + \beta )} \\ {l_{t} \cdot \sin (\alpha + \beta )} \\ \end{array} } \right],P_{h} = \left[ {\begin{array}{*{20}c} 0 \\ {l_{h} \cdot \cos (\alpha + \beta + \gamma )} \\ {l_{h} \cdot \sin (\alpha + \beta + \gamma )} \\ \end{array} } \right] \hfill \\ \end{gathered} \right.$$

(2)

pre-contact phase: ${F}_{leg}={\left[0\quad0\quad0\right]}^{T}$

contact phase:

$$\left\{ \begin{gathered} y_{hip} = l_{s} \cdot \cos (\alpha ) + l_{t} \cdot \cos (\alpha + \beta ) \hfill \\ z_{hip} = l_{s} \cdot \sin (\alpha ) + l_{t} \cdot \sin (\alpha + \beta ) \hfill \\ \theta_{1} = \arctan (z_{hip} /y_{hip} ) \hfill \\ l_{leg1} = (y_{hip}^{2} + z_{hip}^{2} )^{1/2} \hfill \\ l_{leg\_o} = l_{s} + l_{t} \hfill \\ l_{leg\_r} = (l_{leg1}^{2} + sl^{2} - 2l_{leg1} \cdot sl \cdot \cos (\theta_{1} ))^{1/2} \hfill \\ \theta = \arccos (sl^{2} + l_{leg\_r}^{2} - l_{leg1}^{2} )/(2sl \cdot l_{leg\_r} ) \hfill \\ \end{gathered} \right. \Rightarrow \left\{ \begin{gathered} f_{leg} = k_{leg} \cdot (l_{leg\_o} - l_{leg\_r} ) \hfill \\ f_{leg\_y} = - f_{leg} \cdot \cos (\theta ) \hfill \\ \, f_{leg\_z} = f_{leg} \cdot \sin (\theta ) \hfill \\ \end{gathered} \right.$$

(3)

where $\alpha ,\beta ,\gamma ,\theta$ are joint angles and ${k}_{\text{ankle}},{k}_{\text{knee}},{k}_{hip}$ are the spring coefficients of the torque springs acting on each joint, respectively; ${k}_{leg}$ is the spring coefficient of the linear spring Leg; ${F}_{leg}$ is the force generated by the Leg on the recovery side, ${f}_{leg}$ is the magnitude of force; ${l}_{leg\_o},{l}_{leg\_r}$ are the Leg original length and the length during the balance recovery process, respectively. $\tau$ is the joint torque; F is the joint reaction force; I is body segment moment of inertia, m is body segment mass, l is body segment length. The first letter of the subscript indicates the body segment, and the second letter indicates the proximal or distal end of the body segment. s stands for Shank, t stands for Thigh, h stands for Hat, p stands for proximal, d stands for distal. ${P}_{s},{P}_{t},{P}_{h}$ are the coordinates of the center of mass of the Shank, Thigh and HAT, respectively.

1.2 Shank motion equations

$$\begin{gathered} \left\{ \begin{gathered} V_{sd} = \alpha \times P_{s} \hfill \\ V_{sc} = \alpha \times 0.5P \hfill \\ \dot{V}_{sd} = \ddot{\alpha } \times P_{s} + \dot{\alpha } \times (\dot{\alpha } \times P_{s} ) \, \hfill \\ \dot{V}_{sc} = \ddot{\alpha } \times 0.5P_{s} + \dot{\alpha } \times (\dot{\alpha } \times 0.5P_{s} ) \hfill \\ \end{gathered} \right. \hfill \\ \left\{ \begin{gathered} F_{s} = m_{s} \cdot \dot{V}_{sc} \hfill \\ F_{s} = F_{sp} + F_{sd} - [0,0,m_{s} \cdot g]^{T} \hfill \\ \end{gathered} \right. \, \hfill \\ \Rightarrow m_{s} \cdot \dot{V}_{sc} = F_{sp} + F_{sd} - [0,0,m_{s} \cdot g]^{T} \hfill \\ \left\{ \begin{gathered} \tau_{s} = I_{s} \cdot \ddot{\alpha } \hfill \\ \tau_{s} = \tau_{sp} + \tau_{sd} + 0.5P_{s} \times (F_{sd} - F_{sp} ) \hfill \\ \end{gathered} \right. \hfill \\ \Rightarrow I_{s} \cdot \ddot{\alpha } = \tau_{sp} + \tau_{sd} + 0.5P_{s} \times (F_{sd} - F_{sp} ) \hfill \\ \Rightarrow \hfill \\ I_{s} \cdot \ddot{\alpha } + 0.5P_{s} \times (m_{s} \cdot \dot{V}_{sc} ) = \tau_{sp} + \tau_{sd} + 0.5P_{s} \times (2F_{sd} - [0,0,m_{s} \cdot g]^{T} ) \hfill \\ \end{gathered}$$

(4)

1.3 Thigh motion equations

$$\begin{gathered} \left\{ \begin{gathered} V_{td} = V_{sd} + (\dot{\alpha } + \dot{\beta }) \times P_{t} \hfill \\ V_{tc} = V_{sd} + (\dot{\alpha } + \dot{\beta }) \times 0.5P_{t} \hfill \\ \dot{V}_{td} = \dot{V}_{sd} + (\ddot{\alpha } + \ddot{\beta }) \times P_{t} + (\dot{\alpha } + \dot{\beta }) \times [(\dot{\alpha } + \dot{\beta }) \times P_{t} )] \hfill \\ \dot{V}_{tc} = \dot{V}_{sd} + (\ddot{\alpha } + \ddot{\beta }) \times 0.5P_{t} + (\dot{\alpha } + \dot{\beta }) \times [(\dot{\alpha } + \dot{\beta }) \times 0.5P_{t} )] \hfill \\ \end{gathered} \right. \hfill \\ \left\{ \begin{gathered} F_{t} = mt \cdot \dot{V}tc \hfill \\ F_{t} = F_{tp} + F_{td} + F_{leg} - [0,0,m_{t} \cdot g]^{T} \hfill \\ \end{gathered} \right. \hfill \\ \Rightarrow m_{t} \cdot \dot{V}_{tc} = F_{tp} + F_{td} + F_{leg} - [0,0,m_{t} \cdot g]^{T} \hfill \\ \left\{ \begin{gathered} \tau_{t} = I_{t} \cdot \ddot{\beta } \hfill \\ \tau_{t} = \tau_{tp} + \tau_{td} + 0.5P_{t} \times (F_{td} - F_{tp} ) \hfill \\ \end{gathered} \right. \, \hfill \\ \Rightarrow I_{t} \cdot \ddot{\beta } = \tau_{tp} + \tau_{td} + 0.5P_{t} \times (F_{td} - F_{tp} ) \hfill \\ \Rightarrow \hfill \\ I_{t} \cdot \ddot{\beta } + 0.5P_{t} \times (m_{t} \cdot \dot{V}_{tc} ) = \tau_{tp} + \tau_{td} + 0.5P_{t} \times (2F_{td} + F_{leg} - [0,0,m_{t} \cdot g]^{T} ) \hfill \\ \end{gathered}$$

(5)

1.4 HAT motion equations

$$\begin{gathered} \left\{ \begin{gathered} V_{hd} = V_{td} + (\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \times P_{h} \hfill \\ V_{hc} = V_{td} + (\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \times 0.5P_{h} \hfill \\ \dot{V}_{hd} = \dot{V}_{td} + (\ddot{\alpha } + \ddot{\beta } + \ddot{\gamma }) \times P_{h} + (\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \times [(\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \times P_{h} )] \hfill \\ \dot{V}_{hc} = \dot{V}_{td} + (\ddot{\alpha } + \ddot{\beta } + \ddot{\gamma }) \times 0.5P_{h} + (\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \times [(\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \times 0.5P_{h} )] \hfill \\ \end{gathered} \right. \hfill \\ \left\{ \begin{gathered} F_{h} = m_{h} \cdot \dot{V}_{hc} \hfill \\ F_{h} = F_{hp} + F_{hd} - [\begin{array}{*{20}c} 0 & 0 & {m_{h} \cdot g} \\ \end{array} ]^{T} \hfill \\ \end{gathered} \right. \, \hfill \\ \Rightarrow m_{h} \cdot \dot{V}_{hc} = F_{hp} + F_{hd} - [0,0,m_{h} \cdot g]^{T} \hfill \\ \left\{ \begin{gathered} \tau_{h} = I_{h} \cdot \ddot{\gamma } \hfill \\ \tau_{h} = \tau_{hp} + \tau_{hd} + 0.5P_{h} \times (F_{hd} - F_{hp} ) \hfill \\ \end{gathered} \right. \hfill \\ \Rightarrow I_{h} \cdot \ddot{\gamma } = \tau_{hp} + \tau_{hd} + 0.5P_{h} \times (F_{hd} - F_{hp} ) \hfill \\ \Rightarrow \hfill \\ I_{h} \cdot \ddot{\gamma } + 0.5P_{h} \times (m_{h} \cdot \dot{V}_{hc} ) = \tau_{hp} + \tau_{hd} + 0.5P_{h} \times (2F_{hd} - [0,0,m_{h} \cdot g]^{T} ) \hfill \\ \end{gathered}$$

(6)

1.5 Standard motion equation form of balance recovery model

$$\begin{gathered} \left\{ \begin{gathered} I_{s} \cdot \ddot{\alpha } + 0.5P_{s} \times (m_{s} \cdot \dot{V}_{sc} ) = \tau_{sp} + \tau_{sd} + 0.5P_{s} \times (2F_{sd} - [0,0,m_{s} \cdot g]^{T} ) \hfill \\ I_{t} \cdot \ddot{\beta } + 0.5P_{t} \times (m_{t} \cdot \dot{V}_{tc} ) = \tau_{tp} + \tau_{td} + 0.5P_{t} \times (2F_{td} + F_{leg} - [0,0,m_{t} \cdot g]^{T} ) \hfill \\ I_{h} \cdot \ddot{\gamma } + 0.5P_{h} \times (m_{h} \cdot \dot{V}_{hc} ) = \tau_{hp} + \tau_{hd} + 0.5P_{h} \times (2F_{hd} - [0,0,m_{h} \cdot g]^{T} ) \hfill \\ \end{gathered} \right. \hfill \\ \Rightarrow \hfill \\ \left\{ \begin{gathered} D(q) \cdot \ddot{q} + C(q,\dot{q}) \cdot \dot{q} + G(q) = U(q) \hfill \\ \hfill \\ D = \left[ {\begin{array}{*{20}c} {D_{11} } & {D_{12} } & {D_{13} } \\ {D_{21} } & {D_{22} } & {D_{23} } \\ {D_{31} } & {D_{32} } & {D_{33} } \\ \end{array} } \right], \, C = \left[ {\begin{array}{*{20}c} {C_{11} } & {C_{12} } & {C_{13} } \\ {C_{21} } & {C_{22} } & {C_{23} } \\ {C_{31} } & {C_{32} } & {C_{33} } \\ \end{array} } \right] \hfill \\ G = \left[ {\begin{array}{*{20}c} {G_{1} } \\ {G_{2} } \\ {G_{3} } \\ \end{array} } \right], \, U = \left[ {\begin{array}{*{20}c} {U_{1} } \\ {U_{2} } \\ {U_{3} } \\ \end{array} } \right], \, q = \left[ {\begin{array}{*{20}c} \alpha \\ \beta \\ \gamma \\ \end{array} } \right] \hfill \\ \end{gathered} \right. \hfill \\ \end{gathered}$$

(7)

where $D\left(q\right)$ is inertia matrix, $C\left(q,\dot{q}\right)$ is centrifugal force and Coriolis force matrix, $G\left(q\right)$ is gravity matrix, $U\left(q\right)$ is Generalized force matrix.

$D\left(q\right)$:inertia matrix

$$\left\{ \begin{gathered} D_{11} = (4/3m_{s} + 4m_{t} + 4m_{h} ) \cdot (0.5l_{s} )^{2} + (2m_{t} + 4m_{h} ) \cdot 0.5l_{s} \cdot 0.5l_{t} \cdot \cos (\beta ) \hfill \\ \, + 2m_{h} \cdot 0.5l_{s} \cdot 0.5l_{h} \cdot \cos (\beta + \gamma ) \hfill \\ D_{12} = (2m_{t} + 4m_{h} ) \cdot 0.5l_{s} \cdot 0.5l_{t} \cdot \cos (\beta ) + 2m_{h} \cdot 0.5l_{s} \cdot 0.5l_{h} \cdot \cos (\beta + \gamma ) \hfill \\ D_{13} = 2m_{h} \cdot 0.5l_{s} \cdot 0.5l_{h} \cdot \cos (\beta + \gamma ) \hfill \\ D_{21} = (2m_{t} + 4m_{h} ) \cdot 0.5l_{s} \cdot 0.5l_{t} \cdot \cos (\beta ) \hfill \\ \, + (4/3m_{t} + 4m_{h} ) \cdot (0.5l_{t} )^{2} + 2m_{h} \cdot 0.5l_{t} \cdot 0.5l_{h} \cdot \cos (\gamma ) \hfill \\ D_{22} = (4/3m_{t} + 4m_{h} ) \cdot (0.5l_{t} )^{2} + 2m_{h} \cdot 0.5l_{t} \cdot 0.5l_{h} \cdot \cos (\gamma ) \hfill \\ D_{23} = 2m_{h} \cdot 0.5l_{t} \cdot 0.5l_{h} \cdot \cos (\gamma ) \hfill \\ D_{31} = 4/3m_{h} \cdot (0.5l_{h} )^{2} + 2m_{h} \cdot 0.5l_{s} \cdot 0.5l_{h} \cdot \cos (\beta + \gamma ) \hfill \\ \, + 2m_{h} \cdot 0.5l_{t} \cdot 0.5l_{h} \cdot \cos (\gamma ) \hfill \\ D_{32} = 4/3m_{h} \cdot (0.5l_{h} )^{2} + 2m_{h} \cdot 0.5l_{t} \cdot 0.5l_{h} \cdot \cos (\gamma ) \hfill \\ D_{33} = 4/3m_{h} \cdot (0.5l_{h} )^{2} \hfill \\ \end{gathered} \right.$$

(8)

$C\left(q,\dot{q}\right)$:centrifugal force and Coriolis force matrix

$$\left\{ \begin{gathered} C_{11} = - (2m_{t} + 4m_{h} ) \cdot 0.5l_{s} \cdot 0.5l_{t} \cdot \sin (\beta ) \cdot (\dot{\alpha } + \dot{\beta }) \hfill \\ \, - 2m_{h} \cdot 0.5l_{s} \cdot 0.5l_{h} \cdot \sin (\beta + \gamma ) \cdot (\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \hfill \\ C_{12} = - (2m_{t} + 4m_{h} ) \cdot 0.5l_{s} \cdot 0.5l_{t} \cdot \sin (\beta ) \cdot (\dot{\alpha } + \dot{\beta }) \hfill \\ \, - 2m_{h} \cdot 0.5l_{s} \cdot 0.5l_{h} \cdot \sin (\beta + \gamma ) \cdot (\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \hfill \\ C_{13} = - 2m_{h} \cdot 0.5l_{s} \cdot 0.5l_{h} \cdot \sin (\beta + \gamma ) \cdot (\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \hfill \\ C_{21} = - 2m_{h} \cdot 0.5l_{t} \cdot 0.5l_{h} \cdot \sin (\gamma ) \cdot (\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \hfill \\ \, + (2m_{t} + 4m_{h} ) \cdot 0.5l_{s} \cdot 0.5l_{t} \cdot \sin (\beta ) \cdot (\dot{\alpha }) \hfill \\ C_{22} = - 2m_{h} \cdot 0.5l_{t} \cdot 0.5l_{h} \cdot \sin (\gamma ) \cdot (\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \hfill \\ C_{23} = - 2m_{h} \cdot 0.5l_{t} \cdot 0.5l_{h} \cdot \sin (\gamma ) \cdot (\dot{\alpha } + \dot{\beta } + \dot{\gamma }) \hfill \\ C_{31} = 2m_{h} \cdot 0.5l_{s} \cdot 0.5l_{h} \cdot \sin (\beta + \gamma ) \cdot (\dot{\alpha }) \hfill \\ \, + 2m_{h} \cdot 0.5l_{t} \cdot 0.5l_{h} \cdot \sin (\gamma ) \cdot (\dot{\alpha } + \dot{\beta }) \hfill \\ C_{32} = 2m_{h} \cdot 0.5l_{t} \cdot 0.5l_{h} \cdot \sin (\gamma ) \cdot (\dot{\alpha } + \dot{\beta }) \hfill \\ C_{33} = 0 \hfill \\ \end{gathered} \right.$$

(9)

$G\left(q\right)$:gravity matrix

$$\left\{\begin{array}{c}{G}_{1}=\left({m}_{s}+2{m}_{t}+2{m}_{h}\right)\cdot g\cdot {l}_{s}\cdot cos\left(\alpha \right)\\ {G}_{2}=\left({m}_{t}+2{m}_{h}\right)\cdot g\cdot {l}_{t}\cdot cos\left(\alpha +\beta \right)\\ {G}_{3}={m}_{h}\cdot g\cdot {l}_{h}\cdot cos\left(\alpha +\beta +\gamma \right)\end{array}\right.$$

(10)

$U\left(q\right)$:generalized force matrix

$$\left\{\begin{array}{c}{U}_{1}={\tau }_{sp}+{\tau }_{sd}+2{f}_{leg\_z}\cdot {l}_{s}\cdot cos\left(\alpha \right)-2{f}_{leg\_y}\cdot {l}_{s}\cdot sin\left(\alpha \right)\\ {U}_{2}={\tau }_{tp}+{\tau }_{td}+2{f}_{leg\_z}\cdot {l}_{h}\cdot cos\left(\alpha +\beta \right)-2{f}_{leg\_y}\cdot {l}_{h}\cdot sin\left(\alpha +\beta \right)\\ {U}_{3}={\tau }_{hp}+{\tau }_{hd}\end{array}\right.$$

(11)