Abstract
In recent years, there is an increasing trend towards using wearable activity trackers to help monitor and track physical activities (PA) for older adults, with the purpose of motivating regular PA for better health. However, existing activity trackers are frequently abandoned within a short period of time. One of the major reasons is that they do not differentiate individual PA habits and provide PA recommendations based on a unified standard, which may lead to unrealistic suggestions and thus cause frustrations. In order to motivate long-term use of activity trackers and promote PA progression in older adults, PA recommendations should adapt to the changes of an individual’s PA habits. As a step towards achieving this, we introduce in this paper an innovative multi-scale personalized LSTM model that can predict an individual’s daily PA level with satisfied accuracy. This model is verified through a series of experimental studies.
Keywords
Y. Zheng—This work is supported by the Texas A&M University-Corpus Christi Research Grant.
1 Introduction
Obesity and related diseases are threatening the national and global health, especially the health of older adults. Recent data from the National Health and Nutrition Examination Survey (NHANES) show that 71% of adults in the US were overweight and obese between 2015–2016 [1]. Obesity prevalence is 43% among adults aged 40–59 and 41% among those 60 years or older [2]. Intentionally losing weight and regular physical activity (PA) can control obesity and moderate the risk of associated chronic diseases and many types of cancers [3]. It is recommended that adults engage in at least 2.5 h of moderate- or 1.25 h of vigorous-intensity aerobic activity each week [4].
The past few years have witnessed an increasing use of activity trackers to help monitor, track and hopefully motivate regular PA in older adults. However, existing activity trackers are often abandoned by older adults within a short period of time after initial adoption [5], due to issues like functional complexity, high cost, poor usability, etc [6]. Furthermore, existing activity trackers do not differentiate individual PA habits and provide PA recommendations to users based on the same standard, which may lead to unrealistic suggestions and cause frustrations. In order to motivate long-term use of activity trackers and promote PA progression in older adults, new techniques that can provide personalized PA recommendations to users and can automatically adjust the recommendations to adapt to the changes of users’ PA habits are urgently needed. This can be realized by comparing PA histories with future dynamics. As a step towards achieving this, we investigate the PA level prediction problem for older adults in this paper.
Most existing studies on using wearable devices to analyze human activities focus on how to recognize different types of activities such as walking, resting, running, cycling, etc [12,13,14,15]. These studies usually use inertial sensors such as accelerometers to measure the acceleration while users are performing different activities. These acceleration data are then used to extract informative features and train a classifier that can detect and differentiate human activities. As the major challenge faced by existing wearable devices is the battery life, many studies on human activity recognition have been devoted to addressing this challenge through reducing the sensors [8, 9], decreasing the sampling rate [10, 11], or using alternative devices such as the kinetic energy harvesting device that converts kinetic energy from human motion to electrical energy [7].
Despite the abundant works on wearable-based human activity recognition, studies on predicting human activities using wearable devices are very limited, to the best of our knowledge. Most existing studies on human activity prediction use data collected by more advanced tools like cameras [19, 20], social media [17, 18], smart meter [21], etc. Among the limited studies on wearable-based human activity prediction, article [16] introduces a deep learning based model to jointly predict future activities and associated durations using data collected by smartphones. However, this approach is only suitable for predicting activity types, not activity levels.
In this paper, we aim to address the problem of predicting daily PA levels for older adults wearing low-cost activity trackers, as a step towards realizing the next-generation activity trackers that can provide adaptive and personalized PA recommendations to motivate regular PA in older adults. Developing a PA level predictor with satisfied accuracy is not easy. In addition to the challenges shared with general human activity prediction problems including the difficulty in data collection and high uncertainty in human behaviors, using low-cost wearable activity trackers introduces a number of new challenges. For instance, comparing with smartphones or cameras, low-cost activity trackers, such as the Nokia Go Activity and Sleep Tracker adopted in this study, provide less informative data, which are often limited to the variants of accelerations including step counts and walking distance [22, 23]. The lack of informative data makes the development of accurate prediction models extremely difficult.
To conquer aforementioned challenges and capture the individuals’ daily PA habits, we develop a multi-scale personalized LSTM model that adopts the Long Short Term Memory (LSTM) network, a state-of-the-art deep learning model with superior performance on time series data, and explores individual activity data at different temporal scales. Real PA data of older adults are collected using activity trackers. With this dataset, we then conduct extensive experimental studies to evaluate the performance of the proposed method. For comparison, various traditional time series data forecast approaches are also investigated.
In the rest of the paper, we first briefly review the fundamentals of the LSTM model in Sect. 2. We then describe the real PA dataset we acquired for this study in Sect. 3. The proposed multi-scale personalized LSTM model is introduced in Sect. 4. In Sect. 5, we conduct experimental studies to evaluate the performance of the proposed model. In Sect. 6, we conclude the paper with a brief summary.
2 Review of the LSTM Model
The LSTM model [27] has been widely used for time series data analysis, due to its promising capability in discovering hidden time-dependent information. An LSTM model is composed of a set of memory cells and a fully connected network as illustrated in Fig. 1. The memory cells address the gradient vanishing and exploding problems typical for recurrent neural networks [26].
As illustrated in Fig. 1, each memory cell takes an input signal \(x_t\), as well as the hidden state \(h_{t-1}\) and cell state \(c_{t-1}\) from the previous memory cell, and outputs the new hidden state \(h_t\) and cell state \(c_t\). If an LSTM model consists of T memory cells, data from the previous T time steps, i.e., \(\{x_{t-T+1}, x_{t-T+2}, \ldots , x_t\}\), are thus used to perform prediction. To calculate the hidden state \(h_t\) and cell state \(c_t\), the inputs of the memory cell are passed through three gates including the forget, input, and output gates. In particular, the forget gate first decides the information to be thrown away from the cell state \(c_{t-1}\) by the following equation:
where \(f_t\) is the output of the forget gate. \(\sigma (\cdot )\) represents the sigmoid function. \(W_{xf} \in R^{T \times D}\) and \(W_{hf} \in R^{T \times T}\) are weight matrices, and \(b_f\in R^{T}\) is a bias vector. D is the dimension of the input signal.
The input gate and a candidate cell then decide the information to be stored in the cell state through
where \(i_t\) is the output of the input gate. \(\tilde{c}_t\) is a candidate state vector. \(W_{xi} \in R^{T \times D}\), \(W_{hi} \in R^{T \times T}\), \(W_{xc} \in R^{T \times D}\) and \(W_{hc} \in R^{T \times T}\) are weight matrices, and \(b_i\in R^{T}\) and \(b_c\in R^{T}\) are bias vectors. \(\tanh (\cdot )\) is the hyperbolic function. The new cell state \(c_t\) can then be calculated using the following equation:
where \(\circ \) stands for element-wise multiplication.
Finally, the output gate decides the information in the cell state to be outputted through the following equation:
where \(o_t\) is the output of the output gate. \(W_{xo} \in R^{T \times D}\) and \(W_{ho} \in R^{T \times T}\) are weight matrices, and \(b_o\in R^{T}\) is a bias vector. The new hidden state \(h_t\) can then be obtained by
Given \(h_t\), predictions can be made by
where \({\hat{y}}_{t}\) is the predicted value. \(W_y\) and \(b_y\) are weight and bias vectors, respectively.
3 Data Acquisition and Pre-processing
In this section, we briefly describe the activity data we acquired for this study and the pre-processing procedures we applied to clean the raw data.
To acquire the activity data for this study, we recruited 7 older adults aged between 50 and 70 and let them wear Nokia Go Activity and Sleep Trackers for around three months from 05/07/2018 to 07/26/2018. A data fetching application was developed that leverages the Nokia’s public application programming interface to collect participants’ activity data. The collected data have five fields: time, step counts, walking distance, sleeping status, and burned calories. As we are interested in PA levels in this study, the sleeping status is not considered. Furthermore, we found that the walking distance and burned calories are linearly correlated with the step counts, indicating that both the walking distance and burned calories do not provide additional information. Therefore, we end up with the time and step counts as the only data fields that are useful for this study. As the raw data is recorded when moving is detected, we pre-process the data to generate daily or hourly data through aggregation. Missing entries are filled with zeros. An example trajectory of daily step counts for one participant is shown in Fig. 2. This dataset is also used to generate the results shown in the following sections.
4 Multi-scale Personalized LSTM Model
In this study, we consider the problem of predicting the daily step counts, an indicator of PA level, for an individual using the step counts data collected from the activity tracker.
As the daily dataset only contains 81 data points for each participant, which is too small to train a good prediction model, we propose to use the hourly data to predict the daily step counts. The key idea is to use the hourly data to train an LSTM model, and then let the trained LSTM model predict step counts for the next 24 h. The sum of the predicted step counts is the total step counts for the next day. An illustration of the proposed model with \(T=2\) is shown in Fig. 3, where \(x_t\) represents the hourly data point measured at time t (hour), \(\hat{x}_t\) is the hourly data point predicted at time t (hour), and \(\hat{x}_{t_d}\) denotes the daily data point predicted at time \(t_d\) (day).
Before we use the data to train the multi-scale LSTM model, we first conduct an autocorrelation analysis to understand the participant’s PA patterns. As shown in Fig. 4, the PA level shows behavior rhythms. In particular, the hourly PA levels show high autocorrelations (see Fig. 4(a)) at lags 24, 48, 72, etc., indicating a cycle of 24 h. The strongest peaks appear at lags \(24\times 7\), \(24\times 14\), \(24\times 21\), etc., indicating a cycle of 7 days. To better illustrate the 7-day cycle, we plot in Fig. 4(b) the autocorrelation of daily PA levels, which demonstrates high autocorrelations at lags 7, 14, 21, etc., further verifying the existence of a 7-day cycle. To capture these behavior rhythms, we insert two data fields into the hourly data, one is the hour of the day, and the other is the day of the week. Therefore, each data point \(x_t\) has four fields: time, step counts, hour of the day and day of the week.
5 Experimental Evaluation
In this section, we conduct a series of experiments to evaluate the performance of the proposed multi-scale personalized LSTM model for daily PA level prediction.
5.1 Model Training and Configuration
To train the multi-scale personalized LSTM model, we use the dataset visualized in Fig. 2 from a single participant as an example. We then divide the dataset into the training and testing data. The training data include the first 51 days (63%) of activity data and are used to train the model, and the testing data include the rest 30 days (37%) of data and are used to evaluate the prediction performance of the model. To determine the optimal value of the parameter T in the LSTM models, we evaluate the prediction performance of the multi-scale LSTM model at different values of T. The results are shown in Fig. 5, where the number of hidden layers and the number of neurons in each layer are both set to 1. As we can see from the figure, the prediction performance, measured by the root mean square error (RMSE), is optimized at \(T=2\). The small differences between the training and testing errors indicate that the model is free of overfitting, which is an issue frequently encountered when a small dataset is used.
5.2 Comparison Studies
In this section, we conduct various comparison studies to evaluate the performance of the proposed model from different aspects.
Model Types. In the first experiment, we investigate the impact of different baseline models on the prediction performance. In particular, we replace the LSTM model in the proposed multi-scale framework (see Fig. 3) with different time series data forecast models including the Autoregressive Integrated Moving Average (ARIMA) [25] and the Multilayer perceptron (MLP) [24] models. We then compare the prediction performances of the resulting multi-scale ARIMA and multi-scale MLP models with the proposed multi-scale LSTM model. For both benchmark models, feature selection is performed to pick the best set of features, and model parameters are tuned to achieve the best prediction performance. In particular, a (1, 0, 2)-ARIMA model is implemented, which takes lag features \(\{t-1,t-2,t-3\}\). The MLP model takes lag features \(\{t-24, t-48\}\), and has one hidden layer with a single neuron. The comparison results are shown in Fig. 6 with the corresponding RMSE provided in Table 1, which demonstrate the promising performance of the LSTM model.
Feature Types. In the second experiment, we study the impact of different feature types on the prediction performance. In particular, we compare the performance of the prediction models trained using the hourly data with two fields (time, step count) and that with two additional fields, hour of the day and day of the week, inserted. The prediction performance of the multi-scale LSTM model trained using data with different feature types are shown in Fig. 7. Table 2 compares the RMSE of different prediction models. From these results, we can conclude that inserting the hour of the day and day of the week into the hourly data improves the prediction performance, as these two data fields naturally capture the participant’s daily and weekly behavior rhythms.
Temporal Scales. In the third experiment, we compare the proposed multi-scale LSTM model with the traditional single-scale LSTM, ARIMA and MLP models that are trained directly using the daily data of two fields (time, step count). In particular, a (0,0,1)-ARIMA model is implemented, whose features include lags \(\{t_d-1,t_d-2,\ldots , t_d-7\}\) and cycle of 7 days. Both MLP and the traditional LSTM have one hidden layer with 3 neurons. The traditional LSTM model also has a dropout layer with dropout rate of 0.4 to alleviate the overfitting issue. The features taken by the MLP and the traditional LSTM model are lags \(\{t_d-7, t_d-11, t_d-12, t_d-14\}\) and \(\{t_d-1,t_d-2,\ldots , t_d-6\}\), respectively. The comparison results are shown in Fig. 8 and Table 3, which demonstrate the good performance of the proposed multi-scale LSTM model.
We also test the performance of the traditional single-scale prediction models trained using the daily data with an additional field, day of the week, inserted. The results are shown in Table 4. As we can see, the proposed multi-scale LSTM model still achieves the best performance.
Model Generality. In the last experiment, we study the impact of adopting a generalized model on the prediction performance. To obtain the generalized model, we use the datasets from all 7 participants to train the multi-scale LSTM model, with T set to 12 for the best performance. We then use the same testing data described in Sect. 5.1 to evaluate the prediction performance of this generalized model. The results are shown in Fig. 9 and the corresponding RMSE is 4877.771, which is much higher than 2496.3, the RMSE of the personalized model. This study demonstrates the necessity to personalize the prediction models, considering the individual differences in PA habits.
6 Conclusion
This paper explores the problem of how to accurately predict daily PA levels for individuals using data collected by low-cost activity trackers, as a step towards developing intelligent activity trackers that can suggest personalized PA goals based on individuals’ PA habits. To address this problem, a systematic investigation on the characteristics of the data collected by low-cost activity trackers was first conducted. New attribute fields were inserted into the raw data to capture daily and weekly behavior rhythms. A novel multi-scale personalized LSTM model was then developed, which addresses the challenge of lack of informative data by exploring PA patterns at different temporal scales. The series of experimental studies demonstrate the good performance of the proposed model. In the future, we will explore how to set appropriate PA goals for individuals to motivate regular PA effectively.
References
Fryar, C.D., Carroll, M.D., Ogden, C.L.: Prevalence of overweight, obesity, and severe obesity among adults aged 20 and over: United States, 1960–1962 through 2015–2016 (2018)
Hales, C.M., Fryar, C.D., Carroll, M.D., Freedman, D.S., Ogden, C.L.: Trends in obesity and severe obesity prevalence in US youth and adults by sex and age, 2007–2008 to 2015–2016. JAMA 319(16), 1723–1725 (2018)
Lauby-Secretan, B., et al.: Breast-cancer screening viewpoint of the IARC Working Group. N. Engl. J. Med. 372(24), 2353–2358 (2015)
Office of Disease Prevention and Health Promotion: 2008 physical activity guidelines for Americans summary (2008). https://health.gov/paguidelines/2008/summary.aspx
Ledger, D., McCaffrey, D.: Inside wearables: how the science of human behavior change offers the secret to long-term engagement. Endeavour Partners 200(93), 1 (2014)
Kim, K.I., Gollamudi, S.S., Steinhubl, S.: Digital technology to enable aging in place. Exp. Gerontol. 88, 25–31 (2017)
Khalifa, S., Lan, G., Hassan, M., Seneviratne, A., Das, S.K.: Harke: human activity recognition from kinetic energy harvesting data in wearable devices. IEEE Trans. Mobile Comput. 17(6), 1353–1368 (2018)
Zappi, P., et al.: Activity recognition from on-body sensors: accuracy-power trade-off by dynamic sensor selection. In: Verdone, R. (ed.) EWSN 2008. LNCS, vol. 4913, pp. 17–33. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-77690-1_2
Wang, Y., et al.: A framework of energy efficient mobile sensing for automatic user state recognition. In: 7th International Conference on Mobile Systems, Applications, and Services, pp. 179–192. ACM (2009)
Krause, A., et al.: Trading off prediction accuracy and power consumption for context-aware wearable computing. In: Ninth IEEE International Symposium on Wearable Computers, pp. 20–26. IEEE (2005)
Yan, Z., Subbaraju, V., Chakraborty, D., Misra, A., Aberer, K.: Energy-efficient continuous activity recognition on mobile phones: an activity-adaptive approach. In: 16th International Symposium on Wearable Computers (ISWC). IEEE (2012)
Bulling, A., Blanke, U., Schiele, B.: A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput. Surv. (CSUR) 46(3), 33 (2014)
Lara, O.D., Labrador, M.A.: A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutor. 15(3), 1192–1209 (2013)
Ronao, C.A., Cho, S.B.: Human activity recognition using smartphone sensors with two-stage continuous hidden Markov models. In: 10th International Conference on Natural Computation (ICNC). IEEE (2014)
Guan, Y., Plötz, T.: Ensembles of deep LSTM learners for activity recognition using wearables. ACM Interact. Mobile Wearable Ubiquit. Technol. 1(2), 11 (2017)
Krishna, K., Jain, D., Mehta, S.V., Choudhary, S.: An LSTM based system for prediction of human activities with durations. ACM Interact. Mobile Wearable Ubiquit. Technol. 1(4), 147 (2018)
Zhu, Y., Zhong, E., Pan, S.J., Wang, X., Zhou, M., Yang, Q.: Predicting user activity level in social networks. In: 22nd ACM International Conference on Information & Knowledge Management, pp. 159–168. ACM (2013)
Preum, S.M., Stankovic, J.A., Qi, Y.: MAPer: a multi-scale adaptive personalized model for temporal human behavior prediction. In: 24th ACM International on Conference on Information and Knowledge Management, pp. 433–442. ACM (2015)
Li, K., Fu, Y.: Prediction of human activity by discovering temporal sequence patterns. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1644–1657 (2014)
Ma, S., Sigal, L., Sclaroff, S.: Learning activity progression in LSTMs for activity detection and early detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1942–1950 (2016)
Yassine, A., Singh, S., Alamri, A.: Mining human activity patterns from smart home big data for health care applications. IEEE Access 5, 13131–13141 (2017)
Lee, B.C., Ajisafe, T., Vo, T., Xie, J.: Understanding long-term adoption and usability of wearable activity trackers among active older adults. In: Human Computer Interaction (HCI) International Conference (2019, accepted)
Lee, B.C., Xie, J.: How do aging adults adopt and use a new technology? New approach to understand aging service technology adoption. In: Stephanidis, C. (ed.) HCI 2018. CCIS, vol. 851, pp. 161–166. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-92279-9_22
Haykin, S.S.: Neural Networks and Learning Machines, vol. 3. Pearson, Upper Saddle River (2009)
George, B., Gwilym, M.J., Gregory, C.R., Greta, M.L.: Time Series Analysis: Forecasting and Control. Wiley, New York (2015)
Hasim, S., Andrew, S., Francoise, B.: Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition, vol. 402, p. 1128 (2014)
Zhao, Z., et al.: LSTM network: a deep learning approach for short-term traffic forecast. IET Intell. Transp. Syst. 11(2), 68–75 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zheng, Y., Xie, J., Vo, T.V.T., Lee, B.C., Ajisafe, T. (2019). Predicting Daily Physical Activity Level for Older Adults Using Wearable Activity Trackers. In: Zhou, J., Salvendy, G. (eds) Human Aspects of IT for the Aged Population. Social Media, Games and Assistive Environments. HCII 2019. Lecture Notes in Computer Science(), vol 11593. Springer, Cham. https://doi.org/10.1007/978-3-030-22015-0_47
Download citation
DOI: https://doi.org/10.1007/978-3-030-22015-0_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22014-3
Online ISBN: 978-3-030-22015-0
eBook Packages: Computer ScienceComputer Science (R0)