Keywords

1 Introduction

As the population ages there is growing expenditure on managing the health of older people [21, 24], particularly those who are frail [7]. Low-cost, tech-based solutions, including ambient living and remote healthcare management systems are under development to tackle some of the aspects of ageing. Such solutions aim to improve people’s quality of life, improve safety and reduce treatment costs as well as help healthcare systems in assisting people to live at home for as long as possible. Our proposed autonomous remote monitoring and decision support tool is developed based on the use of low-cost, off-the-shelf embedded technologies to be deployed in the home to provide affordable solutions for the elderly. Focusing on frailty we have two objectives: (1) to evaluate the effectiveness of using MS Kinect for remote monitoring of the elderly in the home; (2) to assist the clinicians with the assessment of performance of activities of daily living in the periods between hospital clinic visits [11] based on the historical data collected with the MS Kinect and analysed by our proposed system. In this study, we (1) provide evidence of the effectiveness of the remote monitoring and decision support systems; (2) reduce the time and cost spent on in-clinic examination as clinicians use objective measures instead of semistructured interviews aimed at eliciting an accurate history; (3) improve elderly well-being, and promote independent living.

2 Background

2.1 Technologies Used for Monitoring Health

There are two types of remote monitoring technology used to support elderly people living independently at home. Firstly, wearable devices, such as pendants, can send an alarm signal when pressed, or automatically when incorporating sensors such as accelerometers [30]. Smartphones have been used as wearables for fall detection [14], however this comes with limitations. The participant needs to recharge/replace batteries, remember to have the device with them at all times and keep it charged and ‘on’ [30]. Furthermore, smartphones need to be kept in a fixed position and are susceptible to false positives [14].

The second type of technology used to aid independence is an environmentally embedded (non-wearable) sensor. Many kinds are available, such as infrared motion sensors to detect presence; bed sensors that can detect pulse, respiration and restlessness; and pressure sensors to detect walking and gait. Rantz et al. [25] described the development of ‘TigerPlace’, a purpose-built US retirement community using multiple embedded sensors to support residents. Lin et al. [17] developed a 4-device system for home-based remote detection of frailty. This approach benefits from the sensors being mains-powered and elderly people generally prefer non-wearables as they are less intrusive [30]. Alberdi et al. [1] assessed the possibility of detecting changes in psychological, cognitive and behavioral symptoms of Alzheimer’s Disease (AD) by making use of unobtrusively collected smart home behavior data which is collected by passive infrared (PIR) motion sensors and machine learning techniques.

The Technology Integrated Health Management (TIHM) project in Surrey UK [6] uses a combination of passive environmental sensors, medical devices, wearable technologies, and interactive applications to collect real-time data containing information on environmental conditions (i.e. humidity, temperature, appliance usage), patients physiological parameters (i.e. blood pressure, pulse), and their daily lifestyles.

For measuring gait properties, wearable insole pressure sensors can be used in home settings, but have drawbacks such as needing correct placement, regular calibration, and battery charging/replacement [9]. Optical gait analysis systems require two video cameras for depth information and setup and calibration can be complex [9].

The MS Kinect is a single unit, has a built-in depth sensor plus a camera and is fairly inexpensive. The Kinect captures the movements of 26 body joint angles in an anonymised fashion [29]. The Kinect’s ability to measure gait, inactivity, isolation and falls has been demonstrated in laboratory conditions using healthy subjects [17, 20, 28, 30] and has been found to have high to acceptable accuracy when identifying key elements of movement. Very few studies have involved clinical subjects or home environments [28]. One study [30] used the Kinect as a fall detection system with 16 older adults in an independent living facility. Over one year, six out of seven standing falls could be detected at a false alarm rate of one per week, but two sitting falls were highly occluded and not detectable. The Tiger Place project compared the Kinect to an optical system and a pulse-Doppler radar, in a real world test and found the Kinect to be the most robust [26]. These early results demonstrate the utility of technology-supported independence and the great potential of such systems for remote monitoring in a residential setting. However, a major obstacle to the application of this technology more widely is the lack of evidence on performance at scale in real-world settings.

2.2 Software Architectures Used for Monitoring Health

In the study by Alberdi et al. [1] the daily distance that the subjects moved inside the homes was estimated by computing the distance between areas of the home covered by each passive infrared (PIR) motion sensor as determined from the sensor layout and the floor plan. Note that this approach only provides an approximation of the real covered distance, as it does not consider the existence of walls or other obstacles between the sensors that must be avoided or navigated.

All the sensor and medical devices record and send the data to their corresponding gateways over Wi-Fi, Bluetooth or, in exceptional cases, via auxiliary interfaces of such a device. Gateways relay these data to the companies backend systems over GPRS, SigFox or home broadband as in TIHM [6]. At this stage, all participating companies comply with a common JSON data model for the TIHM project. This is followed by the communication with the TIHM backend system through a publish/subscribe (pub/sub) or message queue (MQ) model. Specifically the advanced message queuing protocol (AMQP) is employed.

In-home assessment of walking speed based on PIR sensors and a wireless network for data collection. The authors assume that the sensors are placed at physical positions in some spatial coordinate system. For a particular walking event, the sensors fire at times, where [15] provides linear walking model and estimates velocity. The linear model has been experimentally verified on a total of 882 walks from the 27 participants.

3 Objectives

This project considers two main challenges. The first is to evaluate the effectiveness of remote monitoring elderly participants using MS Kinect in residential settings. The methodology applied to this research is to analyse participant’s data (collected with Kinect) retrospectively and compare this quantitative data against data collected at interview. The key performance metrics are fall detection, activity monitoring (e.g. duration of sitting per hourday), walking speed, and posture. Long term objectives of this project, once the effectiveness of such system is proven, will be using such system to improve patient lifestyle, promote independent living, improve self-care and remote monitoring, and ultimately reduce the cost of healthcare.

3.1 Setting

Salford Royal NHS Foundation Trust delivers health services in the City of Salford, in Greater Manchester UK. It operates under the umbrella of the Northern Care Alliance NHS Group comprising the Care Organisations of Salford, Bury & Rochdale, Oldham and North Manchester. The Salford care organisation not only provides hospital care, it plays a much broader role in the locality and is supporting the establishment of a new integrated model of care. It is working closely with the local Council to develop an Integrated Care Organisation (ICO) to join together health and care services and shift more care into the community.

This paper focuses on one aspect of the development of system infrastructure as part of the MiiHome project [3] which assists person-centered integrated care in the home. MiiHome involves close working with a care organization delivering integrated health and social care and the housing sector. The MiiHome project aims to prevent problems before they escalate to the detriment of the person by using digital technologies viewed eventually as an integral component of the home, not as infrastructure fitted into a home. It applies a philosophy of co-creation with participants (hosted) and with all stakeholders to develop a product that not only meets clinical requirements but are acceptable to the participants living in the home.

3.2 Attributes Detected

It has long been known that several simple tests of physical performance are strongly associated with the onset of long-term functional decline and disability [10]. Ample evidence supports the potential use of physical performance assessments in risk assessment strategies that can identify subgroups of older persons, initially independent in all ADLs, who are at increased risk for decline into disability or even death. We sought to investigate whether real-time measurement of such physical performance was also reflective of acute decline as a potential signal triggering proactive interventions to prevent or mitigate such decline. The following behaviours and attributes were identified for monitoring using the Kinect. Items were selected because they fulfilled the following criteria: simple to implement, acceptable load on computing power, clinically relevant to a wide range of situations, and potential for giving acceptable predictive values in assessing risk of adverse outcomes.

Gait Speed and Gait Disorders. Gait or walking speed is a common clinical measure and has been described as the sixth vital sign [8]. Timed walking tests are an important measure in comprehensive geriatric assessment [23]. Changes in walking speed mark a critical point in personal performance and the assessment of gait speed has the potential to serve as a key indicator in mapping the trajectory of health and function in ageing and disease [31]. Systematic reviews have shown that it reliably predicts disability, cognitive impairment, institutionalisation, falls, hospital admission and mortality [32, 33]. Furthermore a declining trajectory of gait speed is also associated with adverse events such as mortality [34].

Furniture crawling (cruising in North America) is a classic adaptive response to more severe levels of gait disorder [2]. In response to severe postural or gait instability patients hold on to furniture, walls and door handles in order to stabilise themselves while moving around the home. Acutely it is a marker of severe loss of stability and chronically an adaptive response to neurological disorders such as ataxia. [2]. We considered using this approach after advice from clinical colleagues because the arms are used for support in furniture crawling and this may be easy to detect using Kinect in a real home setting.

Sit-to-Stand (SiSt). Rising to a stand from a sitting position is one of the most common activities one performs in a home everyday. Sit-to-stand and stand-to-sit are two of the most mechanically demanding activities undertaken in daily life. The ability to perform a sit-to-stand (SiSt) is therefore an important skill. In elderly people, without disability the inability to perform this basic skill can lead to institutionalization, impaired functioning in activities of daily living (ADL), and impaired mobility. Objective measures of lower-extremity function such as the SiSt are highly predictive of subsequent disability [12, 13].

Aging causes a loss of skeletal muscle mass and quality, which leads to a deficit in muscle function measured as loss of muscle strength or of muscle power [22]. The SiSt depends on quadriceps femoris and trunk musculature. Muscle strength, which is the ability to generate force and muscle power, which is the ability to generate this force rapidly are associated with falling [19, 27]. There is also evidence for the contribution of both muscle strength and power to the ability of older people to maintain balance and posture [22]. Balance and posture are also key components of SiSt [18] alongside muscle strength and power. SiSt was evaluated using Kinect taking into account biomechanical considerations [16] while also being mindful that assessment was taking place in a real home where variables such as the height of the chair cannot be controlled. Our approach was to use the results as part of a composite marker for general health and not as a marker for falls risk as has been done by others [4, 5].

3.3 Implementation

We collect prospective data over time from elderly participants whilst they perform usual activities of daily living in their own home. By doing so, we show the changes in the walking speed of elderly participants on hourly, daily, weekly and monthly bases. We calculate the maximum walking speed using the SPINE-SHOULDER joint coordinates and the distance travelled using MS Kinect. Falls are measured intuitively by analysing the acceleration on the SPINE-SHOULDER on the y-axis. The reason behind using our approach is to minimise the computational complexity of the AI-based fall detection and classification algorithms. The term ‘furniture crawling’ is used when participants need to hold on to furniture, walls, etc. during walking in order to maintain a vertical posture which occurs due to low strength of leg muscles. We categorise participants furniture crawl by analysing the height of the hands relative to the spine. The observations will be carried out for a minimum of three months and up to six months based on the participant recruitment process.

Fig. 1.
figure 1

System architecture.

Full size image

4 System Architecture

Figure 1 demonstrates the proposed autonomous decision support system called Kinecting Framework. Kinecting Framework is developed in Java and consists of Subject Activity Monitoring, Analyzers, Reporters modules as well as log files and a database.

The system works as follows. MS Kinect runs and collects raw xyz data of joints when it detects a person and assembles skeletons. In this study, we only use skeletal data which includes up to 26 joints position in an x, y and z coordinate system. We use a non-relation database, MongoDB to store raw and processed data as we do not deal with tabular relations based on the type of data used in this study. Windower pushes the raw data into the database per window size (every minute), when there is a skeleton detected. The Windower also pulls the raw data and passes it on to the Subject Activity Monitoring module. The Subject Activity consists of a number of analysis components which includes Speed, Fall Detection, Furniture Crawling, Gait Speed and Posture Activity components. We package these modules under the Subject Activity Monitoring umbrella for simplicity. The Reporters on the other hand pull the outputs of the Subject Activity Monitoring module separately from the database periodically and calculates hourly, daily, weekly outcomes of the components of this module separately including minimum, maximum and average speed, duration of sitting, number of falls, number of furniture crawling, frequency and duration of crawling, etc. All the components of the Subject Activity Monitoring module update the last execution times separately on the Analyzer module, which allows the Reporters to collect information based on the specified timestamp on the last execution time of a each and every component separately. Kinecting Framework is modular and runs concurrently as these individual components are designed to execute independently at different times for different periods.

Log files are used by the Reporters and Analysis modules to log the state of the system including the errors. Windows task schedulers are used to make sure the proposed system runs 24/7 and restarts the entire framework if it is shut down for any reason.

Fig. 2.
figure 2

Database view of a document in the collections.

Full size image
Fig. 3.
figure 3

The participants in their houses.

Full size image
Fig. 4.
figure 4

Histogram of subject speeds over the four days.

Full size image
Fig. 5.
figure 5

Histograms per days.

Full size image

Figure 2(a) shows the fields of a document in the skeletons collection. Each document in the skeletons collection contains timestamp, occuperID, skeletonID, jointPositions and jointOrientations in addition to its unique documentid within the collection. We used J4K library to extract raw jointPositions and jointOrientations in this framework. Figure 2(b) shows the fields of a document in the subject speed collection. Each document in the subject speed collection contains windowstart, windowend, duration, distance, and speed fields in addition to its unique documentid within the collection. The Windower sends a number of frames to the Speed component of the Subject Activity Monitoring module. Hence, the windowstart and windowend fields are filled automatically based on the Windower values of the stored raw data. The variable speed is calculated respective to the distance and duration values. Figure 2(c) shows the fields of a document in the posture detection collection. Each document in the posture detection collection contains windowstart, windowend, duration, sitting and fall fields in addition to its unique documentid within the collection. The values of the windowstart and windowend fields also obtained by the Windower values of the stored raw data and the number of falls and sitting is notified on the database. The document shown in the Fig. 2(c) suggests that the participant was not sitting or fallen within the duration of this document, which draws the conclusion to the fact that the participant was standing up or walking.

5 Outcomes and Conclusion

Figure 3 illustrate two of the participants in their homes and their home settings. This project installed Kinect into participants individual homes who were freely living in the community. This differs from some other studies that took place in retirement facilities (e.g. Tiger Place). We found that installation had to be a compromise between the optimal position to detect activity and the wishes of the participants. The Kinect installed in Fig. 3 (a), (b) and (c) had to be placed such that it can be obscured by a door and it faced a window. This caused numerous artifacts in the data. However it illustrates the difficulty in real-world working. It is the ultimate aim to gather data that could help the participants like the one in Fig. 3 (d), (e) and (f). Success in this objective will come not only from maturing the technical and logistical issues but also in resolving attitudinal and sociological issues. As part of the overall project we are therefore working closely with these and other potential participants in focus-groups to co-create a solution that mutually resolves all of these issues.

Figure 4 illustrates a histogram of all the subject speeds collected from one participant between 17–20 Nov. 2017. The participants walking speed has been measured at his house during an interview with a stopwatch for comparison and the participant’s walking speed noted as 0.162 from the timed up and go test. The red arrow on the x-axis of the Fig. 4 points the measured walking speed.

Figure 5 exhibits histograms per days between 17–20 Nov. 2017. The data on walking speed show that speeds within the house are low and variable. The number of episodes of walking change from day to day (day 3 vs day 4) as can be seen in Fig. 5(c) and (d) reflecting the different usage of this room on a day to day basis. The gait speed within the room is distributed over a range of values lower than the speed measured in a straight line with a stop-watch. This reflects the shorter path length and reduction in speed to negotiate furniture and entrances.

Analysis of the large amount of data accumulated will require a big-data, machine learning approach before it can be analysed for clinical significance and become part of a meaningful medical record [31].