Dementia is a progressive neurological condition affecting cognition and behaviour with a significant impact on activities of daily living. It is one of the major causes of disability and dependency among elderly population with approximately 50 million people with the disease world wide. Alzheimer’s disease (AD) is the most common cause of dementia. Mild cognitive impairment (MCI) refers to a decline that has minimal impact on activities of daily living [grundman2004mild]. Patients with MCI can continue to live independently [mckhann2011diagnosis]. Not all patients with MCI will convert to dementia, but they are at increased risk. MCI has an annual conversion rate to AD of between 5 and 10% [mitchell2009rate].
Making an accurate, reliable diagnosis of dementia is a challenge. Patients may find cognitive tests stressful, which impacts their performance. This may be exacerbated in a new, unknown, clinical environment with an unfamiliar clinician. Performance on cognitive tests also does not give an indication of how someone is managing in the real world. Although diagnosis may include bio-markers measurement, in addition to this being invasive and expensive, these do not have adequate temporal resolution to understand disease progression and deterioration since AD has high variance across patients.
Some symptoms are prevalent across dementias. Confusion can lead to unpredictable behaviour and activity abandonment during normal daily routines. Interestingly, behavioural interactions between persons with dementia (PwD) and their carers/family are also affected by disease progression and efforts by PwD to allay confusion and uncertainty often result in them persistently seeking the company of persons whom they trust. In this work we intend to address shadowing [lilly2011illumination] (persistently staying in the company of a trusted carer/family), wandering [lyketsos2011neuropsychiatric] (going from room to room with nonspecific intent) and disturbed sleep [hita2012disturbed] (a common symptom for PwD).
Progression of dementia and MCI are characterised by an increase in symptoms that will affect normal daily activity and behaviour. It is important to remember that these symptoms affect normal daily activity and behaviour. In understanding the dependence between normal behaviour and the expression of symptoms, an opportunity to understand and quantify disease state indirectly via behavioural change is exposed. Modern sensing technology offers promise in monitoring daily behaviour, extracting symptom expression rates and measurement of state and progression of the disease [de2017feasibility, stack2018identifying]. In contrast to normal clinical evaluation that occur in foreign environments, ours is achieved from the patients’ residences.
This paper outlines novel computational behavioural analysis algorithms for modelling disease state and progression. The potential for detecting unseen behavioural bio-markers of the early stages of dementia is exposed by analysis on the longitudinal, in-home sensor data. Section 2 reviews prior work on smart homes and dementia. Our data collection and modelling procedures are outlined in Section 3. Results are presented in Section 4 and we conclude in Section 5.
2 Related Work
Numerous smart systems have been designed and developed to monitor the well-being and health status of elders, such as the GATOR Tech Smart House [helal2005gator], the AWARE home [abowd2000living], the Microsoft’s EasyLiving project [brumitt2000easyliving] and the MavHome Project [cook2006health]. In the dementia domain, [li2017supporting] introduced a support system to aid doctors diagnosing dementia. Participants only needed to perform a selection of Instrumental Activities of Daily Living (IADL) in a smart home environment. However, the main limitation is that this approach relies on data collected in a laboratory environment and not in a real world setting.
Along these lines, [akl2015autonomous] undertook a study on data acquired by The ORegon Centre for Aging and TECHnology (ORCATECH) at the Oregon Health and Science University, that investigated the walking speed and general activity in the homes of participants in order to detect dementia. The extension presented in [akl2015unobtrusive] modelled the presence of the participants in different rooms using Poisson processes to demonstrate statistical differences between different states of cognition. Localisation has been found to be a common ground for different works. For instance [kearns2008ultra] and [schwarz2005accuracy] use an Ultra Wideband device to try to measure wandering behaviours. However, Radio Frequency (RF) identification may be limited by its detection range since unpowered RF devices operate only over a meter’s distance. Also, [d2012indoor] uses a relatively complex sensor network consisted of RF tracking and motion and heading sensors for monitoring and studying behavioural patterns of patients with dementia.
We build upon the existing literature by focusing on data collected in the wild that, while more challenging to analyse, also provides a richer and pervasive perspective on the subjects’ behaviour. Additionally, our aim is to automatically detect behavioural symptoms that may suggest early dementia. Finally, we want go beyond the analysis of a single participant and incorporate the monitoring of interactions within the home environment as a key element to understand the progression of the disease.
This section introduces our data collection and modelling pipelines.
3.1 Data collection
Ethical approval was secured from the Carmarthen Research Ethics Committee and the Health Research Authority (HRA) to record sensor data from the homes of people who have received a diagnosis of dementia or have MCI for up to 12 months using the SPHERE [zhu2015bridging, woznowski2017sphere, diethe2018releasing] system. Informed consent was obtained to analyse this data to determine links between behavioural patterns, residents of the home and disease state. Participants are recruited from North Bristol NHS Trust, the Dementia Wellbeing Service in Bristol and the Research Institute for the Care of Older People (RICE) Centre in Bath. The SPHERE sensor system is installed into their homes [zhu2015bridging, woznowski2017sphere].
The key sensors that we use in the analysis of this work are accelerometers [elsts2018guide, fafoutis2017designing, vafeas2020wearable]. These are set record at a rate of 20 Hz. This data is broadcast to gateways by means of radio links. Six gateways are installed in every house. These simultaneously record accelerometer readings and the Received Signal Strength Intensity (RSSI).
In-home localisation is a key task in this work that leverages the RSSI data [michal2019, michal2018, michal2017]. Labelled data are acquired from a ‘walkthrough’ script that is performed by the technician upon installation. Each room is visited while holding the wearables and RSSI and wearable data are simultaneously recorded. The real-time location of the technician is recorded with an annotation app.
To regress from sensor data to localisation and activity predictions, we extract basic features from windowed data ( seconds length with overlap of seconds following [twomey2018comprehensive]). Features include: mean, standard deviation (std), max, min, diff (first-order difference operator) and the count of missing values.
We can complement labelled data by introducing domain knowledge into the analysis by suggesting reasonable locations based on calculable context. For example, if predicting ‘sleep’ at 3AM it is likely that the resident is in the bedroom.
3.2 Modelling considerations
3.2.1 Covariate Shift
We account for our expectations of covariate shift (due to disease progression) with Maximum Mean Discrepancy (MMD) [huang2007correcting]. In mathematical terms we assume that training and testing distributions differ, i.e. , but that conditional distributions remain the same . MMD offers an elegant approach to quantify and match distributional shift and we follow the methodology outlined by [huang2007correcting] in our work.
3.2.2 Semi-Supervised Learning
Since only a small portion of labelled data are available, we leverage supervised () semi-supervised () objectives to make efficient use of data. The supervised objective () is the traditional conditional log-likelihood that one optimises for in Conditional Random Fields (CRF) [sutton2012introduction]
. The data and labels for this are obtained from the walkthrough that was introduced in the previous section. CRF model sequence probabilities as:
where are the log-potentials and is the partition function. In our case the log-potentials decompose, to emission function and transition function, as follows: = , where:
is the activity level of the participant at time(calculated as the average absolute jerk of the tri-axial signals) and denotes a threshold, below which we assume there is no movement. This forbids location transitions unless a certain activity potential is surpassed.
The emission potentials are modelled with a 4-layer fully-connected neural network,, i.e. the -th element of . The semi-supervised objective () constructed by iteratively using the most likely label sequence as temporary targets, i.e. [alex2020torchstruct].
In this section we present findings relevant to sleeping disturbance, wandering and shadowing. We illustrate the analysis using different houses that are inhabited by either a PwD, or by someone who has been founds to have MCI, and one partner. Data collection is ongoing, so these are preliminary findings and some houses have significantly more data than others. The analyses below result from localisation and activity level predictions that were trained and validated on the walkthrough data.
The mutual information (MI) between the location of the two residents in each house is measured to estimate shadowing, following a similar approach to that described in[twomey2017unsupervised]. Additionally, in our analysis we stratify MI by time of the day in order to understand whether significant MI is confined to particular intervals, e.g. morning Fig.4.
Temporal lags were introduced in order to allow for short delays in shadowing, with no significant change, and are therefore not included in this work. This is likely due to the fact that the lag interval is significantly shorter than the analysis window. Looking at Fig.4, in the case of house B there is no trend of increasing/decreasing correlation. Even though in the case of house A, there is a slight increase over time, stronger conclusions will be drawn when more data arrives. Our data collection is ongoing, and as it grows we will identify finer, more informative signals of disease state. Time lags are likely to become more relevant with finer and more targeted analysis that may be motivated by changes in behaviour as the disease progresses.
The approach for modelling wandering is motivated by its antonym: non-wandering is typified by low activity levels and persistent location predictions. Thus we jointly compare the activity levels and the location complexity of residents in this section. Specifically, in Fig. 4 we track the total activity per room, as well as the total activity of the day, over the span of the data, and in Fig. 4 we track the complexity of the localisation predictions, derived from the Lempel-Ziv [lempel1976complexity] complexity measure. It is clear in Fig. 1(b) that the activity levels of the patient are higher than that of the other participant in this house, and Fig. 2(b) gives similar results, although in Fig. 1(a) and Fig. 2(a) it is less obvious, for which there might be several reasons. One explanation could be that the data we have so far for house A spans only a short time period (less than 10 days). A second cause might be that the patient in house A only has MCI, whose daily life only gets affected to a mild degree compared to the participant with dementia in house B.
Additionally, Fig. 5 shows in detail the complexity of localisation predictions in different time intervals over the experiment span in house B. On every subplot, the number at the top-right is the average number of times the patient has a higher complexity than the partner. In this analysis, we only include house B, as the span is longer. In the case of house A these averages are not so much in favour of the patient - segment as compared to for house B.
4.3 Sleep Disturbance
In both houses, we observe that during the early hours of the day, 0-6 am, the participants have a higher activity levels, while the control/carers have a more steady, and at most times, lower activity levels. This might be an indication of the patients having a more uneasy sleep. As a number of studies suggest [mander2016sleep, hita2012disturbed], people have MCI or dementia will suffer from different levels of sleep disturbance. And a community-based study [hope1990nature] indicates that nearly one-third of their subjects were reported as having been inappropriately active at night.
As can be seen from Fig. 1(b), the line representing the activity levels in bedroom 2 of the participant with dementia is quite unstable compared to that of the other participant during 0-6 am. Although there is not a significant difference between activity levels in bedroom 1 of the two inhabitants during 0-6 am in Fig. 1(a), it is because that the person with MCI got up earlier than am, especially on July 23rd, 24th and 25th, and started walking in the house while the partner was sleeping, which may suggest the increased sleep disturbance in the person with cognitive disorder from another angle.
For reference, we also include a segment of the RSSI data in Fig. 4, from House A, for both participants, from the early hours of the day. It highlights the difference in sleeping quality, as observed through RSSI. The step changes are a result of rotations during sleep. It also shows how different sleeping positions can lead to missing values in certain gateways, e.g. kitchen_1 at 02:00 and at 03:15.
5 Conclusion and Future Work
This paper presents our preliminary results and shows the potential of detecting early symptoms of cognitive disorders automatically by utilizing data acquired from patients’ daily activities and signal processing methods equipped with machine learning techniques. We build a machine learning pipeline that considers the difference in training and testing distributions usingMMD. We account for the sequential nature of data using a CRF, and use our domain knowledge to enhance our training data with data coming from the participants.
One of the limitations of this study is that our localisation precision can only reach room-level for now due to the configuration and the deployment of our sensor system. Finer-grained localisation will expose behavioural reactions to furniture placements across rooms.
Potential directions for improvements include taking into consideration environmental sensors, such as Passive Infrared and Video, through machine learning techniques such as multi-modal data fusion [diethe2017probabilistic]. Moreover, we will enhance our training data using clinical data through our domain knowledge of activities of daily living.
The paper as it stands establishes baseline activity, sleep and co-localisation dependence that will be used after data collection completed. That we see signals here is promising for our future data collection, analysis and research.
This research was funded by CUBOID (UK MRC Momentum grant MC/PC/16029) and the SPHERE IRC (grant EP/K031910/1).