Using Phone Sensors and an Artificial Neural Network to Detect Gait Changes During Drinking Episodes in the Natural Environment

11/09/2017
by   Brian Suffoletto, et al.
University of Pittsburgh
UPMC
0

Phone sensors could be useful in assessing changes in gait that occur with alcohol consumption. This study determined (1) feasibility of collecting gait-related data during drinking occasions in the natural environment, and (2) how gait-related features measured by phone sensors relate to estimated blood alcohol concentration (eBAC). Ten young adult heavy drinkers were prompted to complete a 5-step gait task every hour from 8pm to 12am over four consecutive weekends. We collected 3-xis accelerometer, gyroscope, and magnetometer data from phone sensors, and computed 24 gait-related features using a sliding window technique. eBAC levels were calculated at each time point based on Ecological Momentary Assessment (EMA) of alcohol use. We used an artificial neural network model to analyze associations between sensor features and eBACs in training (70 datasets. We analyzed 128 data points where both eBAC and gait-related sensor data was captured, either when not drinking (n=60), while eBAC was ascending (n=55) or eBAC was descending (n=13). 21 data points were captured at times when the eBAC was greater than the legal limit (0.08 mg/dl). Using a Bayesian regularized neural network, gait-related phone sensor features showed a high correlation with eBAC (Pearson's r > 0.9), and >95 fall between -0.012 and +0.012 of actual eBAC. It is feasible to collect gait-related data from smartphone sensors during drinking occasions in the natural environment. Sensor-based features can be used to infer gait changes associated with elevated blood alcohol content.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

12/03/2017

An Artificial Neural Network for Gait Analysis to Estimate Blood Alcohol Content Level

Impairments in gait occur after alcohol consumption, and, if detected in...
05/10/2022

Real-Time Wearable Gait Phase Segmentation For Running And Walking

Previous gait phase detection as convolutional neural network (CNN) base...
02/10/2018

Automatic Phone Slip Detection System

Mobile phones are becoming increasingly advanced and the latest ones are...
11/08/2019

BB-MAS: A Very Large Dataset for Typing, Gait and Swipe of the Same Person on Desktop, Tablet and Phone

Behavioral biometrics are key components in the landscape of research in...
11/08/2019

Insights from BB-MAS – A Large Dataset for Typing, Gait and Swipes of the Same Person on Desktop, Tablet and Phone

Behavioral biometrics are key components in the landscape of research in...
06/30/2020

Associations between finger tapping, gait and fall risk with application to fall risk assessment

As the world ages, elderly care becomes a big concern of the society. To...
08/17/2019

Assessment of gait normality using a depth camera and mirrors

This paper presents an initial work on assessment of gait normality in w...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Acute alcohol intoxication is associated with numerous health risks. For example, impaired driving due to alcohol was implicated in 28% of the 38,000 deaths from motor vehicle accidents in the US in 2016 [1]. These consequences largely stem from alcohol’s detrimental effects on psychomotor performance [2]. Compounding this risk are impaired decision-making [3] and lack of awareness of the degree of alcohol-related impairments during drinking episodes [4]. Strategies to measure alcohol-related psychomotor impairments and provide real-time feedback to individuals could deter involvement in activities that require psychomotor function (i.e., driving), thus reducing likelihood of injury [5].

Using alcohol consumption as a surrogate for psychomotor impairment can be prone to either underestimations (e.g. when individuals do not report alcohol consumption accurately [6]) or over-estimations (e.g. in individuals with high tolerance to the effects of alcohol [7]). Therefore, asking individuals to input drinks in real time may not be the most accurate estimate of psychomotor impairment. One measure of psychomotor performance that is particularly sensitive to alcohol is gait. Gait requires coordination of multiple sensory and motor systems. Both postural stability [8] and gait [9] are sensitive to blood alcohol concentration (BAC) levels. Although law enforcement professionals have used subjective performance on a heel-to-toe tandem gait task as a field sobriety test for years, there is no current process to objectively measure aspects of gait during drinking occasions in the natural environment.

The rapid growth of smartphone ownership [10] and standard inclusion of accelerometer and gyroscope sensors within phones suggests that these devices could be useful to objectively measure gait impairment during drinking episodes. Researchers have begun to model the associations between gait abnormalities detected using smartphone sensors and either real or simulated alcohol consumption [11],[12], but no one has determined if it is feasible to collect standardized gait data in the natural drinking environment from at-risk individuals, nor the association of gait features with BAC levels. Answers to these questions are critical if sensor based data will be used to provide feedback that is specific to drinking risk level.

The aim of this project was to determine (1) feasibility of collecting gait-related data during a brief gait task during drinking occasions from at-risk young adults in the natural environment, and (2) how gait-related features measured through phone sensors relate to estimated blood alcohol concentration (eBAC). To accomplish these aims, we designed an iPhone app (DrinkTRAC) to collect smartphone sensor-based data on gait (3-axis accelerometer, gyroscope, magnetometer) and ecological momentary assessment (EMA) measures of self-reported number of drinks consumed each hour, from 8pm to 12am, during weekend evenings (Fridays and Saturdays). We enrolled 10 young adults with a history of heavy drinking in a repeated-measures study to provide smartphone sensor and self-report data over a period of four consecutive weeks. We used a Bayesian regularized neural network (BRNN) to perform regression analysis to examine the association of sensor data with eBACs. Results from this work could be useful in designing effective prevention interventions to reduce risky behaviors during periods of alcohol intoxication.

Ii Methods

Ii-a Participants

A convenience sample of young adults (aged 21––26 years) who presented to an urban Emergency Department (ED) between February 19 and May 9, 2016 were recruited. A total of 28 medically stable ED patients who were not seeking treatment for substance use, not intoxicated, and who were going to be discharged to home, were approached by research staff. Among those eligible to be approached, 23 patients provided consent to complete an alcohol use severity screen. Those who reported recent hazardous alcohol consumption based on an Alcohol Use Disorder Identification Test for Consumption (AUDIT-C) score of 3 for women or 4 for men [13] and who drank primarily on weekends were eligible for participation. We excluded those who reported any medical condition that resulted in impaired thinking or memory or gait, those who reported past treatment for alcohol use disorder, and those without an iOS phone. A total of 10 participants met the study enrolment criteria. All participants completed informed consent protocols prior to study procedures and were provided with resources for alcohol treatment.

Characteristics N=10
Age in years, mean (SD) 23.1 (2.6)
Female, n (%) 7 (70%)
Race, n (%)
African American 2 (20%)
White 6 (60%)
Other 2 (20%)
Hispanic Ethnicity, n (%) 1 (10%)
Education, n (%)
Some college 5 (50%)
College graduate or,post-graduate 5 (50%)
Employment, n (%)
For wages 7 (70%)
Student 3 (30%)
Married, n (%) 1 (10%)
Alcohol Use Severity (AUDIT-C score), mean (SD) 5 (1.3)
Weight in pounds, mean (SD) 179 (35)
TABLE I:

Sample descriptive statistics

Ii-B DrinkTRAC Application

The DrinkTRAC app was developed using Apple’s ResearchKit platform, as it allows for convenient and professional-appearing modular builds that incorporate timed psychomotor tasks. The DrinkTRAC app contained the following components: (1) timed electronic notifications, (2) a home screen, (3) a baseline questionnaire, (4) a calendar displaying completed tasks, (5) timed EMA, and (6) timed psychomotor tasks (including a 5-step gait task). The gait task was taken from the mPower GitHub site (https://github.com/ResearchKit/mPower). We altered the gait task to take less than 45 seconds to optimize completion and reduce potential for disruptions that could interfere with task performance.

Prior to each gait task, participants were instructed to walk in a straight line for 5 steps. We advised participants not to continue if they felt that they could not safely walk 5 steps in a straight line unassisted. If participants clicked “next”, they were shown a picture of a phone and told: “Find a place where you can safely walk unassisted for about 5 steps in a straight line”, followed by the text: “Put the phone in a pocket or bag and follow the audio instructions. If you do not have somewhere to put the phone, keep it in your hand”. When the participant clicked “Get Started”, the app displayed a timer and played an audio recording of a voice counting down from 5 to 1. If the audio option was turned on, participants heard “Walk up to 5 steps in a straight line, then stand still.” When the task was completed, participants were presented with a figure of their completion rates for the day. Figure 1 shows DrinkTRAC app screen shots of the gait task.

(a)
(b)
Fig. 1: DrinkTRAC app screenshots of the tandem gait task
(a)

Ii-C Procedure

In the ED, participants uploaded the DrinkTRAC app to their phone. The research associate was present to ensure understanding and to observe compliance with instructions on the initial trial of the app’s tasks, which were conducted in the ED. We instructed participants to refrain from any non-drinking substance use (excluding cigarette use) during the sampling days. We also informed participants that they would receive $10 for completing the baseline survey and app-based tasks in the ED, $10 for completing the exit survey at four weeks, and $1 per completed EMA (up to an additional $40).

Over four consecutive Fridays and Saturdays, every hour from 8pm to 12am, participants were sent an electronic notification to log in to the DrinkTRAC app and complete the tasks. We chose to sample data on weekend evenings, given that this is a time when young adults typically drink alcohol [14]. We collected data hourly from 8pm to 12am on those nights, with an intention of capturing both the ascending (when eBAC is rising) and descending (when eBAC is decreasing) limbs of alcohol intoxication. We used fixed hourly assessment times, given that they would provide a predictable framework for participants and would allow us to more easily calculate eBAC changes over the course of the evening.

Ii-D Data Processing

Ii-D1 Estimated Blood Alcohol Concentration

We calculated eBAC during each hour when data was available using a formula created by Matthews and Miller [15]. This formula was found to have a significantly stronger intraclass correlation with breath alcohol concentrations (criterion standard) than did other equations when measured after an uncontrolled episode of drinking [16].

Ii-D2 Sensor Data Feature Extraction

We collected 3-axis acceleration and angular velocity data, sampled at a frequency of 100 Hz for 30 seconds during the tandem-gait task. We extracted features from sensors that can describe the properties of gait. We consider four features, i.e., mean, standard deviation, correlation, and energy. Energy is a feature extracted from a signal through a Fast Fourier Transformation (FFT) from time domain into frequency domain. Energy feature can reveal how energy in the signal is distributed over its various frequency components. Energy is the sum of the squared discrete FFT coefficient magnitudes of the signal. The sum was divided by the length of the window for normalization. In a formal notation, if

are the FFT components of the window then . Energy value differs for each activity; therefore, it can be used to determine discrepancies among certain movement patterns [17]. In order to study the properties of each signal, we broke up time into smaller segments using a sliding window, where in each new ”window” has 50% overlap with the previous one, and then calculations are performed on the ”windowed” signal. All measurements are in three dimensions; thus, resulting in a total of 24 possible features. The efficiency of these features for characterizing gait has been discussed in (Bao and Intille 2004 [18], Ravi et al. 2005 [19]).

We removed the effects of gravity in order to measure acceleration of the device (linear acceleration). An accelerometer is subject to dynamic (or external) and static (gravity) accelerations. Therefore, in order to measure the real acceleration of the device, the contribution of the static acceleration which is the effect of Earth’s gravity on the device must be removed. Hence, for the purpose of obtaining linear acceleration, other sensors such as gyroscope and magnetometer were utilized to derive accurate values of linear acceleration by applying a high-pass filter. We also measured the attitude of the device, which is the computed device orientation using the accelerometer, magnetometer, and gyroscope. These values yield the Euler angles of the device.

Ii-E Bayesian Regularized Neural Network (BRNN) for eBAC Regression

We applied a neural network model to estimate the association between sensor-based gait features and eBAC. One of the advantages of using neural networks for regression and predicting values is that it uses a nonlinear sigmoid function in a hidden layer, which enhances its computational flexibility, as compared to a standard linear regression model

[20]

. We first used multilayer perceptron (MLP), a BRNN, to model the nonlinear relationships between the extracted gait features and the output (eBAC value), with nonlinear transfer functions. A schematic diagram of our MLP structure can be seen in Appendix. We then tested the BRNN on a training set, and computed the corresponding parameters of the network, such as weights and learning rate. We used the Levenberg-Marquardt optimization algorithm to find the minimum of the multivariate function

[21]

. We compared the performance of MLP with support vector machine (SVM) and linear regression by examining correlation coefficient, mean absolute error, root mean squared error, relative absolute error, and root relative squared error. We then examined the correlation between MLP model-predicted eBAC values based on sensor features and actual eBAC values. We examined a histogram of errors between predicted eBAC and actual eBAC to determine frequency of outliers and potential misclassification of gait impairment relation to eBAC.

Fig. 2: Error histogram visualized errors between target values and predicted values after training a feedforward neural network with 20 bins

Iii Results

Ten participants provided a total of 128 unique data points. Table I shows the baseline characteristics of the cohort. We captured 38 unique drinking episodes, with each participant reporting at least 3 drinking episodes. Almost half of the gait tasks (n=60, 46.9%) were completed either prior to drinking or on non-drinking evenings, 55 (43.0%) were completed on the ascending eBAC limb, and 13 (10.1%) were completed on the descending eBAC limb. On a drinking day, participants reported consuming a mean of 3.6 (SD=2.2; range: 1-10 drinks). The mean eBAC was 0.04 (SD=0.05), with a peak of 0.23. 21 (15.2%) data points were recorded at times when the eBAC was greater than the legal limit (0.08 mg/dl).

Iii-a Regression Results

Based on comparison results of correlation coefficient, mean absolute error, root mean squared error, relative absolute error, and root relative squared error, MLP outperformed SVM and linear regression models, as shown in Table II. The correlation between MLP model-predicted eBAC values based on gait sensor features and actual eBAC values for training (a), testing (b), and all data (c) are also shown in this table. In the testing dataset, we found moderate variability in predicted-actual eBAC values that did not differ by actual eBAC.

Regression,technique Correlation coefficient Mean absolute error Root mean squared error Relative absolute error Root relative squared error
MLP 0.9009 0.0174 0.0226 40.6458 % 43.8853 %
SVM 0.3939 0.0362 0.0482 84.5348 % 93.6504 %
Linear,Regression 0.4367 0.0378 0.0463 88.2747 % 89.9583 %
TABLE II: Comparison of different regression techniques for BAC estimation

The histogram of errors is shown in Figure 2 where the blue bars represent training data and the red bars represent testing data. The histogram can give an indication of outliers, which are data points where the fit is significantly worse than that of most of the data. In this case, we can see that while 95% of errors between estimated eBAC and actual eBAC fall between -0.012 and +0.012, there are some training points and just a few test points that are outside of that range. These outliers are also visible on the testing regression plot (Figure 3). If the outliers are valid data points but are unlike the rest of the data, then the network is extrapolating for these points. This means that more data similar to the outlier points should be considered in training the model. Very rarely (20%) would an individual who had an actual eBAC value greater than the legal limit (

0.08 mg/dl) be classified as having a eBAC less than the legal limit (eBAC

0.08 mg/dl).

Iv Discussion

In this pilot study, we found that it is feasible to collect gait-related data from smartphone sensors during drinking occasions in the natural environment in a cohort of young adult drinkers. Each participant completed the 5-step gait task on at least 3 different weekend evenings, but not surprisingly, there were many missing data points later in evenings. We also found that neural network algorithms using smartphone accelerometers and gyroscope data can produce accurate prediction models to infer elevated blood alcohol concentration. Based on our final model, although there was statistical variability in predicted eBAC compared to actual eBAC values, there appears to be rare occurrence of mis-classification that would be considered clinically significant. These results contribute to the eventual goal of being able to provide real-time feedback to individuals during drinking occasions on gait impairment, which could deter involvement in activities like driving, thus reducing the likelihood of alcohol-related injury. Our findings concur with early pilot work by Kao et al. [22] who showed good agreement between gait captured by inertial sensors and alcohol consumption (coded Yes or No) and Arnold et al. [11]

who used machine learning algorithms of smartphone inertial sensors to determine the number of drinks (not BAC) in a few healthy volunteers. It also expands upon later research

[12] showing that more advanced sensor-based measures including gait sway can improve algorithm performance when tested against simulated intoxication using visual-altering goggles.

(b)
(a)
(b)
(c)
Fig. 3: The three plots represent the training, test, and all data
(a)

Our approach differs from prior work in several important ways. First, we studied a relevant target population (young adult heavy drinkers) who would benefit most from a prevention-oriented intervention, as opposed to healthy volunteers. Second, we studied feasibility of task completion in real drinking environments. Third, we used a widely available software platform for collecting movement data (Apple ResearchKit), as opposed to an external sensor device or a proprietary app, allowing for replication. Fourth, we calculated eBAC using established formulas, thus providing a more accurate representation of actual blood alcohol concentration than drink counting alone. Fifth, we standardized the gait task, thus removing random variability in naturalistic walking. Sixth, we used not only accelerometers to understand movement data, but also considered gyroscope and magnetometer measurements. Finally, we used a sliding window technique for extracting features and feeding MLP, which outperformed the other evaluated approaches.

This pilot study is limited by the small sample size and by the amount of missing gait task data, particularly for the descending limb of alcohol intoxication. We paid participants to complete tasks, which likely artificially inflated completion rates, suggesting that completion rates would be even lower under more naturalistic conditions. In exit interviews, we found that one major barrier to task completion was the requirement to enter a password every time a task was to be initiated. We understand that it is not practical to expect individuals to complete a walking task while they are socially engaged. Future studies could sample movement-based sensor data during natural gait and examine patterns associated with alcohol consumption. Also, findings may not be applicable to other populations, such as young adults with less severe alcohol use or other age groups such as adolescents. The majority of participants were female and white, limiting generalizability. The DrinkTRAC app was made only for iOS devices, which affected study eligibility, and limits generalizability of results to other mobile devices. Self-report of alcohol use using EMA has demonstrated reliability and validity [23], but may be subject to bias. Future work could use transdermal alcohol sensors to validate findings and EMA schedule flexibility to reduce missing data.

V Conclusion and Future Direction

This work provides initial support for the utility of using movement analysis in the real world to infer elevated blood alcohol concentration. A smartphone application (DrinkTRAC) that deployed a brief tandem gait task was completed by young adults in real-world drinking occasions. Using a Bayesian regularized neural network, we modeled and fit a curve using phone sensor data to accurately predict blood alcohol concentration during drinking episodes in the natural environment.

References

  • [1] National Center for Statistics and Analysis. (2017, October). 2016 fatal motor vehicle crashes: Overview. (Traffic Safety Facts Research Note. Report No. DOT HS 812 456). Washington, DC: National Highway Traffic Safety Administration.
  • [2] Christoforou, Z., Karlaftis, M.G., and Yannis, G., 2013. Reaction times of young alcohol-impaired drivers. Accident Analysis & Prevention, 61, 54–62.
  • [3] Steele, C.M. and Josephs, R.A., 1990. Alcohol myopia: Its prized and dangerous effects. American Psychologist, 45 (8), 921.
  • [4] Morris, D.H., Treloar, H.R., Niculete, M.E., and McCarthy, D.M., 2014. Perceived danger while intoxicated uniquely contributes to driving after drinking. Alcoholism: clinical and experimental research, 38 (2), 521–528.
  • [5] Shults, R.A., Elder, R.W., Sleet, D.A., Nichols, J.L., Alao, M.O., Carande-Kulis, V.G., Zaza, S., Sosin, D.M., Thompson, R.S., on Community Preventive Services, T.F., and others, 2001. Reviews of evidence regarding interventions to reduce alcohol-impaired driving. American journal of preventive medicine, 21 (4), 66–88.
  • [6] Gual A, Ángel Arbesú J, Zarco J, Balcells-Oliveró MLM, López-Pelayo H, Miquel L, Bobes J Risky Drinkers Underestimate their Own Alcohol Consumption. Alcohol Alcohol. 2017 Jul 1;52(4):516-517l.
  • [7] Brumback T, Cao D, King A. Effects of alcohol on psychomotor performance and perceived impairment in heavy binge social drinkers. Drug Alcohol Depend. 2007 Nov 2;91(1):10-7.
  • [8] Nieschalk, M., Ortmann, C., West, A., Schmäl, F., Stoll, W., and Fechner, G., 1999. Effects of alcohol on body-sway patterns in human subjects. International journal of legal medicine, 112 (4), 253–260.
  • [9] Jansen, E.C., Thyssen, H.H., and Brynskov, J., 1985. Gait analysis after intake of increasing amounts of alcohol. International Journal of Legal Medicine, 94 (2), 103–107.
  • [10] Anderson, M. and Rainie, L., 2015. Pew Research Center. Technology Device Ownership: 2015, 29.
  • [11] Arnold, Z., LaRose, D., and Agu, E., 2015. Smartphone Inference of Alcohol Consumption Levels from Gait. Proceedings - 2015 IEEE International Conference on Healthcare Informatics, ICHI 2015, 417–426.
  • [12] Aiello and Agu, 2016. Investigating postural sway features, normalization and personalization in detecting blood alcohol levels of smartphone users. In: 2016 IEEE Wireless Health (WH). IEEE, 1–8.
  • [13] Bradley KA, DeBenedetti AF, Volk RJ, Williams EC, Frank D, Kivlahan DR. AUDIT-C as a brief screen for alcohol misuse in primary care. Alcohol Clin Exp Res. 2007; 31:1208–1217.
  • [14] Del Boca, F.K., Darkes, J., Greenbaum, P.E., and Goldman, M.S., 2004. Up close and personal: Temporal variability in the drinking of individual college students during their first year. Journal of consulting and clinical psychology, 72 (2), 155–164.
  • [15] Matthews, D.B. and Miller, W.R., 1979. Estimating blood alcohol concentration: Two computer programs and their applications in therapy and research. Addictive behaviors, 4 (1), 55–60.
  • [16] Hustad, J.T.P. and Carey, K.B., 2005. Using calculations to estimate blood alcohol concentrations for naturally occurring drinking episodes: a validity study. Journal of Studies on Alcohol, 66 (1), 130–138.
  • [17] Gharani, P. and Karimi, H., 2017. Context-aware obstacle detection for navigation by visually impaired. Image and Vision Computing, 64, 103-115.
  • [18] Bao, L. and Intille, S.S., 2004. Activity recognition from user-annotated acceleration data. In: Pervasive computing. Springer, 1–17.
  • [19] Ravi, N., Dandekar, N., Mysore, P., and Littman, M.L., 2005. Activity recognition from accelerometer data. In: AAAI. 1541–1546.
  • [20] Burden, F. and Winkler, D., 2008. Bayesian Regularization of Neural Networks. 23–42.
  • [21] Lourakis, M.I., 2005. A Brief Description of the Levenberg-Marquardt Algorithm Implemened by levmar. Matrix, 3, 2.
  • [22] Kao, H.L., Ho, B.J., Lin, A.C., and Chu, H.H., 2012. Phone-based gait analysis to detect alcohol usage. Proceedings of the 2012 ACM Conference on Ubiquitous Computing - UbiComp ’12, 661.
  • [23] Shiffman S. Ecological momentary assessment (EMA) in studies of substance use. Psychological assessment. 2009;21(4):486-497.