Wearable consumer products, such as smartwatches and fitness trackers, are gaining popularity every day and the enormous technological advances made in recent years have enabled reliable, unobtrusive and remote personalized collection of numerous behavioral and biometric signals through their sensors [20, 6].
This so called “digital phenotyping”  has enabled significant advances in wearables for health purposes, leading to the fact that next-generation wearable technologies are about to help transform nowadays hospital-centered healthcare practice to proactive, individualized care. Behavioral and biometric indexes have been already used in general medicine and sports and nowadays the evidence indicates that they could be introduced into clinical psychiatry , as well. Despite extensive research over the last 60 years in neurobiology and neurophysiology of psychotic disorders, their cause remains unclear and reliable biometric indexes for the diagnosis and prediction of the course of the psychotic symptomatology have not yet been found. The use of such signals for the detection of early diagnosis and prevention of psychotic relapses is now one of the major research areas in psychiatry [30, 13, 17].
The e-Prevention project111More info can be found at: http://eprevention.gr is an ongoing research and development project with the goal of collecting long-term continuous recordings of biometric and behavioral signals through non-intrusive commercial wearable sensors (i.e., smartwatches), in order to develop innovative, advanced and valuable tools. Such tools would facilitate the effective monitoring, the prediction of clinical symptoms and the identification of biomarkers, which correlate with behavioral changes in patients with psychosis so as to support the relapse prevention. Timely detection of such relapses is in fact of major importance, not only for the clinicians; since patients not often present themselves when the symptoms begin to re-emerge or worsen , but it could also assist in reducing the severity of the relapses or even prevent their occurrences.
In contrast with previous works, which have lasted from some hours to a few weeks [29, 19, 8], with some exceptions (), our ongoing research study has been going on for more than one year, with the goal of achieving two years of continuous monitoring. In addition, previous works have mostly used smartphones , and focused mainly in social features such as text messages, call duration or other such as location data, screen on/off time, and sleep duration. [19, 4, 1]. Compared to smartphones, wearable sensors are unobtrusive, lightweight and can be used for monitoring while the subjects perform daily activities , ensuring this way a safe and sound living environment. Additionally, it has been already shown that people with psychotic illnesses are comfortable, able and willing to use personal digital devices to monitor outcomes in their daily life, supporting the fact that by using wearable sensors we could go beyond feasibility and underscore the novel physiological and activity data that can be easily collected with low cost [24, 27].
In our work we employ a commercial off-the-shelf smartwatch, aimed to have minuscule intrusion in the subject’s life and be worn 24/7 (except during charging). The nature of our long-term study asks for a different data processing approach than previous studies. Inspired by traditional signal processing techniques, we extract common and more complex features using short-time analysis, and study them through their descriptive statistics in order to obtain a rough estimate of how they differentiate between healthy controls and patients with psychotic disorders.The experimental evaluation shows that both the more common, but also some of the novel nonlinear features examined are powerful in discriminating between the two groups. The analysis conducted in this work is a vital step towards developing a method that can leverage physiological and behavioral data from sensors in order to timely predict relapses or adverse drug reactions.
2 Experimental Protocol and Data Collection
2.1 Experimental Protocol
Twenty-three (23) healthy control volunteers and 22 patients with a disorder in the psychotic spectrum (9 with Schizophrenia, 8 with Bipolar Disorder I, 3 with Brief Psychotic Episode and 2 with Schizoaffective disorder) were recruited at the University Mental Health, Neurosciences and Precision Medicine Research Institute “Costas Stefanis” (UMHRI) in Athens, Greece. All volunteers gave written consent for their participation after being fully informed about the project and also written permission for the use of their personal data (anonymized), in accordance with the provisions of the General Regulation (EU) 2016/679. Additionally, all protocols of the research project have been approved by the Ethics Committee of the Institution.
Initially, the controls underwent a clinical evaluation to ensure there was no history of mental disorders or toxic substance usage, while for the recruitment of the patients, the clinicians met with the participants to conduct assessment of symptoms and functioning. At recruitment, patients were in active treatment and stable. The clinical team also conducted follow-up assessments with patients once every month of the study to administer various reliable rating scales (i.e., PANSS - Positive and Negative Syndrome Scale), which measure various psychiatric symptoms associated with their psychosis.
Table 1 contains information on the demographics of the two groups as well as the collected data (described in Sec. 3) at the time of writing this paper. We also include the BMI (Body-Mass index) and the PANSS scale rating at the time of recruitment for the two groups (PANSS only applicable to patients).
|Age (years)||27.8 3.9||31.0 6.21|
|Education (years)||16.9 1.8||14.04 2.3|
|Illness dur. (years)||-||7.45 5.89|
|BMI||22.9 3.2||28.23 5.31|
|PANSS (overall)||-||57.7 13.9|
|# Weeks Recorded||20.17 5.27||17.4 6.6|
|# 10 min. mov (awake)||7780 2385||5963 2646|
|# 1 hour HRV (awake)||858 234||743 367|
|# 10 min. mov (sleep)||3767 1282||4058 1958|
|# 1 hour HRV (sleep)||460 164||551 290|
2.2 Method & Data Collection
The subjects wore a Samsung Gear S3 smartwatch that continuously monitored acceleration (acc), angular velocity (gyr), and the heart rate (via Photoplethysmography ). Due to limits on the number of available devices, each subject was recruited at a different date - controls were recruited between June 2019 and October 2019 while patients have been continuously recruited from November 2019 up to now (Oct. 2020). Controls were continuously monitored for at least days and then returned the watches, while the monitoring of patients is an ongoing process. In the analysis presented in this paper, to mitigate the effect of the CoVID19 Pandemic quarantine lockdown (15/03–10/05/2020 in Greece), we exclude data collected during this period.
Data were collected using an in-house developed application and uploaded every day to a secure cloud server . Accelerometer (acc) and gyroscope data (gyr) were collected at a frequency of 20Hz, while the heart rate and the heart rate variability (RR intervals – time intervals between two successive heart pulses) were collected at a rate of 5Hz (if a new beat was not detected the watch duplicates the last obtained value). Using the Tizen API provided by the smartwatch, we also collected information about the sleep schedule of the subjects, and their steps at aggregated intervals of 10 minutes.
Boxplots for features of controls and patients while awake (top row) and asleep (bottom-row). The bold line represents the median, the boxes extend between the 1st and 3rd quartile, whiskers extend to the lowest and highest datum within 1.5 times the inter-quantile range (IQR) of the 1st and 3rd quartile respectively, and outliers are shown as diamonds.
3 Data Processing and Feature Extraction
3.1 Data Preprocessing
The heart rate variability (HRV) sequence from the 5Hz signal was obtained by dropping identical consecutive values and ensuring that for each 1 hour interval, the obtained sequence of RR intervals was summing up to at least 54 minutes (an empirical threshold corresponding to 90% of valid heart data). We also removed RR intervals larger than 2000ms and smaller than 300ms as artifacts and replaced possible non-detected pulses with linear interpolation. We did not perform any kind of noise reduction inacc and gyr, since we determined that for the examined features the effect of noise is negligible.
3.2 Feature Extraction
Short-time analysis of signals using windowing is a traditional signal processing method. In short-time analysis we assume the process under which the data are generated to be stationary. Drawing power from these techniques, but largely increasing the time scale, we proceeded to perform “short-time” analysis in windows of 10 minutes for movement data (acc, gyr) and 1 hour for HRV. The 10 minutes intervals that were chosen for the analysis of the movement data and the aggregation of the features have been found optimal for distinguishing short-term patterns in a previous study .
The mean and standard deviation of the number of intervals for each user is reported in Table1. We consider the following features:
Energy The energy (STE) of the euclidean norm of acc and gyr is extracted (since they are measured triaxially). We use these features as an objective measure of physical activity and general movement behavior.
Spectral features Medical studies split the HRV spectrum in four frequency bands: ultra-low-frequency (ULF Hz), very-low-frequency (VLF – Hz), low-frequency (LF – Hz), and high-frequency (HF – Hz). Since HRV is by definition a non-uniformly sampled signal we perform spectral analysis using the Lomb-Scargle periodogram , and we extract for each interval the relative power and normalized power in two bands: LF and HF, as well as the ratio LF-to-HF.
Sample Entropy Nonlinear methods treat the extracted time series as the output of a nonlinear system. A typical characteristic of a nonlinear system is its complexity. The first measure of complexity we consider is the sample entropy (SampEn). Sample entropy is a measure of the rate of information generation by the system, considered an improvement over approximate entropy  due to its unbiased nature.
Higuchi Fractal Dimension Multiple algorithms have been proposed for measuring the fractal dimensions of a time series. Here we use the Higuchi fractal dimension , which has been used extensively in neurophysiology due to its simplicity and speed.
Multiscale Fractal Dimension (MFD) is an efficient algorithm  that measures the short-time fractal dimension, based on the Minkowski-Bouligand dimension . Real-world signals do not have the same structure over different time scales; and by measuring the MFD we are able to examine the complexity and fragmentation of the signals at multiple scales, thus creating a profile of local MFDs at each time location. For this reason, we summarized the short-time measured MFD profiles by taking the following statistics: fd (the fractal dimension), min, max, mean, and std for the 1 hour HRV data.
Poincare plot measures The Poincare plot  is a kind of recurrence plot where each sample of a time series is plotted against the previous, and then an ellipse is fitted on this scatter plot. The width of the ellipse (SD1) is a measure of short-term HRV, while the length (SD2) is a measure of long-term HRV.
Feature Aggregation Using the information on the sleep schedule of each subject we split the intervals in two groups – one corresponding to intervals during sleep and one during wakefulness. We then calculated the mean and standard deviation (std) over all its intervals, resulting in 2 values for each subject and feature type; resulting in a total of 28 features.
Sleep/Wake Ratio and Steps In addition to the above features, we also extracted for each subject the mean and standard deviation of his sleep/wake ratio, and mean number of steps each day. Since the number of recorded hours each day fluctuates, for these features only we keep data from subjects that have at least 30 days with 20 hours or more recorded (21 controls with 65 26 days and 14 patients with 56 19 days with no significant difference().
Fig. 1 shows examples of STE, and HRV fractal dimensions and LF/HF frequencies, during one day of monitoring a subject.
4 Experimental Results
|feature||Controls||Patients||p value||Controls||Patients||p value|
|acc STE mean||6.87 (1.24)||
|acc STE std||7.48 (2.05)||5.911 (2.979)||0.02||1.770 (1.168)||
|gyr STE mean||4166 (1190)||
|gyr STE std||5213 (2225)||3686 (2306)||0.02||1058 (876)||
|HRV SampEn mean||1.36 (0.26)||1.081 (0.389)||0.02||1.233 (0.217)||
|HRV SampEn std||0.33 (0.10)||
|HRV Higuchi mean||1.97 (0.01)||
|0.06||1.854 (0.055)||1.901 (0.081)||0.04|
|HRV Higuchi std||0.016 (0.004)||
|HRV SD1 mean||224.54 (18.84)||
|HRV SD1 std||29.39 (8.58)||33.758 (10.080)||0.02||39.104 (23.996)||
|HRV SD2 mean||262.98 (26.38)||
|HRV SD2 std||33.34 (6.54)||39.492 (10.849)||0.02||43.139 (23.254)||
|HRV HF power mean||69.11 (0.12)||
|HRV HF power std||3.125 (0.095)||
|HRV MFD mean||
|HRV MFD std||
|0.14||0.039 (0.010)||0.047 (0.006)||0.03|
|HRV MFD max mean||1.796 (0.034)||1.761 (0.060)||0.05||
|HRV MFD max std||0.051 (0.016)||0.063 (0.019)||0.04||0.049 (0.014)||0.055 (0.009)||0.04|
|HRV MFD min mean||
|HRV MFD min std||
|0.14||0.039 (0.010)||0.047 (0.006)||0.03|
|HRV MFD mean mean||
|HRV MFD mean std||
|0.06||0.044 (0.013)||0.050 (0.010)||0.04|
|HRV MFD std mean||0.045 (0.003)||0.040 (0.008)||0.03||
|HRV MFD std std||0.007 (0.002)||0.008 (0.003)||0.03||0.009 (0.002)||0.010 (0.002)||0.051|
4.1 Wakefulness comparison
In Fig. 2
(top row) we show the boxplots of the features during wakefulness that were deemed more fitting (due to space limitations) to display differences between the two groups. Due to the differences observed perceptually between the distributions in most features, we tested for significant difference between distributions (the null hypothesis being that the two distributions are the same) using two-tailed non-parametric Mann-Whitney U tests. We adjusted for p-values using the Benjamini-Hochberg (BH) procedure . Due to the nature of our explorative study, BH was preferred over more strict Family-Wise Error Rate methods .
Table 2 shows the results of Mann-Whitney U tests for all features, while the subjects are awake (due to limited space we omit the values of LF and HF normalized powers which showed no significant differences). During wakefulness, the features that pertain to movements appear to present more variability in the patient group when compared to controls. The same appears to be true for some nonlinear HRV features (SampEn mean, Higuchi, SD1 mean, SD2, MFD max). The testing showed significant distribution differences in the standard deviation of acc and gyr short-time energy, the standard deviation of SD1 and SD2, the sampen mean and MFD max and std. The other features failed to reject the null hypothesis.
4.2 Sleep comparison
Similarly, Fig. 2 (bottom row) presents the feature distributions for each group during sleeping. It is evident that especially the movement-related features present a significant difference, which is also verified in the Mann-Whitney U test results in Table 2. A similar result is found for the standard deviation in sample entropy of HRV as well as the mean of the Higuchi fractal dimension. MFD features (MFD fd std, max std, min std, and mean std) were also found to differ significantly.
4.3 Sleep-wake ratio and total steps
Finally, Fig. 3 shows the boxplots of the statistics of steps per day and sleep wake ratio for the two groups. We observe a large significant difference between both the distributions of the mean and std of the sleep wake ratio () as well as the distributions of total steps per day ().
Our goal with the statistical analysis in this work is to exploit traditional, but also less-known signal processing techniques to identify common markers/features that differ drastically when a person has a psychotic disorder. These markers could prove useful in predicting potential relapses in these patients.
Our findings have shown that patients tend to behave with greater variability and present large outliers – some behave close to controls, while others might show extreme values. During wakefulness, even though the mean energy did not differ when compared to controls, the standard deviation showed a significant difference, indicating that patients tend to depict large variations in their movement behavior. On the contrary, during sleeping the patients presented a small mean and standard deviation of the energy in each of their sleeping intervals compared to the controls. We should note however that the observed differences in sleep between the two groups could be attributed to medication administered to patients, which possibly causes variability in sleep duration as well.
Some of the nonlinear features that were measured for the HRV data showed significant differences in the distributions between controls and patients, i.e., during sleeping, as seen in Table 2, such features are the standard deviation of the sample entropy, the mean of the Higuchi fractal dimension, as well as various statistics derived from the MFD analysis (i.e., min, max, and mean of the standard deviation measurement). During wakefulness, the mean of the sample entropy, the std of the poincare features sd1 and sd2 and again various statistics of the MFD presented significant results. Spectral analysis using the LS spectrogram did not show any significant differences between controls and patients.
The main merits of our work are two-fold: First, compared to previous similar studies, which have mostly lasted for a few weeks, our study has already been going on for more than a year with the goal to obtain two years of continuous monitoring of patients with psychotic disorders. To do this, we employ a commercial off-the-shelf smartwatch, that has been acknowledged by our volunteers to be comfortable and patients are willing to insert it into their daily lives routine. Second, we show how traditional short-time-analysis combined with common but also more complex and novel features, such as the MFD features that depicted significant differences in both awake and sleeping data, can be employed to identify biomarkers, present large inter-group variabilities between healthy controls and patients, paving a way towards both acquiring clinical insights on psychotic disorders, but also exploring the capabilities of these markers to predict relapses.
In this paper we identified markers that differentiate between healthy controls and people with psychotic disorders. To this end, we have specifically collected a large amount of physical activity and autonomic function data from wearable devices. Statistical analysis between the two groups, through their descriptive statistics, indicated significant differences regarding the movement behavior, as well as in some markers of cardiac function during both wakefulness and sleeping. In future analyses, we also intend to account for the effects of antipsychotics and/or other medications administered to patients, as well as other factors that differ in the two samples, such as smoker/non-smokers percentages. Finally, we aim to explore the capabilities of such markers to predict psychotic relapses and adverse drug effects.
Predicting early warning signs of psychotic relapse from passive sensing data: an approach using encoder-decoder neural networks. JMIR mHealth and uHealth 8 (8), pp. e19962. Cited by: §1.
-  (2007) Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement 28, pp. R1. Cited by: §2.2.
-  (2017) Sensing behavioral symptoms of mental health and delivering personalized interventions using mobile technologies. Depression and Anxiety 34 (7), pp. 603–609. Cited by: §1.
-  (2017) CrossCheck: integrating self-report, behavioral sensing, and smartphone use to identify digital indicators of psychotic relapse.. Psychiatric Rehabilitation J. 40. Cited by: §1.
-  (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. the Royal statistical society: Series B (Methodological) 57 (1), pp. 289–300. Cited by: §4.1.
-  (2015) The use of smartwatches for health monitoring in home-based dementia care. In Proc. Int’l Conf. Human Aspects IT Aged Population, Cited by: §1.
-  (2001) Do existing measures of poincare plot geometry reflect nonlinear features of heart rate variability?. IEEE Transactions on biomedical engineering 48 (11), pp. 1342–1347. Cited by: §3.2.
-  (2018) Using wearable technology to detect the autonomic signature of illness severity in schizophrenia. Schizophrenia Research 195, pp. 537–542. Cited by: §1.
-  (2017) A general introduction to adjustment for multiple comparisons. J. Thoracic Disease 9 (6), pp. 1725. Cited by: §4.1.
-  (1990) From noncompliance to collaboration in the treatment of schizophrenia. Psychiatric Services 41 (11), pp. 1203–1211. Cited by: §1.
-  (2004) Fractal geometry: Mathematical foundations and applications. John Wiley & Sons. Cited by: §3.2.
-  (1988) Approach to an irregular time series on the basis of the fractal theory. Physica D: Nonlinear Phenomena 31 (2), pp. 277–283. Cited by: §3.2.
-  (2011) Early recognition and disease prediction in the at-risk mental states for psychosis using neurocognitive pattern classification. Schizophrenia Bulletin 38, pp. 1200–1215. Cited by: §1.
An intelligent cloud-based platform for effective monitoring of patients with psychotic disorders.
Proc. Int’l Conf. on Artificial Intelligence Applic. and Innovation, Cited by: §2.2.
On a test of whether one of two random variables is stochastically larger than the other. The annals of Mathematical Statistics, pp. 50–60. Cited by: §4.1.
-  (1994) Fractal signal analysis using mathematical morphology. In Advances in electronics and electron physics, Vol. 88, pp. 199–246. Cited by: §3.2.
-  (2014) Biomarkers and clinical staging in psychiatry. World Psychiatry 13, pp. 211–223. Cited by: §1.
-  (2014) Wearable sensors for human activity monitoring: a review. IEEE Sensors J. 15. Cited by: §1.
-  (2018) Relapse prediction in schizophrenia through digital phenotyping: a pilot study. Neuropsychopharmacology. Cited by: §1.
-  (2012) A review of wearable sensors and systems with application in rehabilitation. J. Neuroengineering and Rehabilitation 9 (1), pp. 21. Cited by: §1.
Person identification using deep convolutional neural networks on short-term signals from wearable sensors. In Proc. Int’l Conf. on Acoustics, Speech and Signal Processing (ICASSP), Cited by: §3.2.
-  (2014) Human activity recognition on smartphones with awareness of basic activities and postural transitions. In Proc. Int’l Conf. on Artificial Neural Networks, pp. 177–184. Cited by: §1.
-  (2000) Physiological time-series analysis using approximate entropy and sample entropy. American J. Physiology-Heart and Circulatory Physiology 278 (6), pp. H2039–H2049. Cited by: §3.2.
-  (2016) Do we still have a digital divide in mental health? A five-year survey follow-up. J. Medical Internet Research 18, pp. e309. Cited by: §1.
-  (1982) Studies in astronomical time series analysis. ii-statistical aspects of spectral analysis of unevenly spaced data. The Astrophysical Journal 263, pp. 835–853. Cited by: §3.2.
-  (2017) An overview of heart rate variability metrics and norms. Frontiers Public Health 5. Cited by: §3.2.
-  (2017) A comparison of passive and active estimates of sleep in a cohort with schizophrenia. NPJ schizophrenia 3, pp. 1–6. Cited by: §1.
-  (2016) New tools for new research in psychiatry: a scalable and customizable platform to empower data driven smartphone research. JMIR Mental Health 3 (2), pp. e16. Cited by: §1.
-  (2014) Wearable monitoring for mood recognition in bipolar disorder based on history-dependent long-term heart rate variability analysis. IEEE Jour. Of Biomedical and Health Informatics 18. Cited by: §1.
-  (1995) Rodromes and precursors: epidemiologic data for primary prevention of disorders with slow onset. The American J. psychiatry 152, pp. 967. Cited by: §1.