Classification of Neurological Gait Disorders Using Multi-task Feature Learning

12/08/2016 ∙ by Ioannis Papavasileiou, et al. ∙ 0

As our population ages, neurological impairments and degeneration of the musculoskeletal system yield gait abnormalities, which can significantly reduce quality of life. Gait rehabilitative therapy has been widely adopted to help patients maximize community participation and living independence. To further improve the precision and efficiency of rehabilitative therapy, more objective methods need to be developed based on sensory data. In this paper, an algorithmic framework is proposed to provide classification of gait disorders caused by two common neurological diseases, stroke and Parkinson's Disease (PD), from ground contact force (GCF) data. An advanced machine learning method, multi-task feature learning (MTFL), is used to jointly train classification models of a subject's gait in three classes, post-stroke, PD and healthy gait. Gait parameters related to mobility, balance, strength and rhythm are used as features for the classification. Out of all the features used, the MTFL models capture the more important ones per disease, which will help provide better objective assessment and therapy progress tracking. To evaluate the proposed methodology we use data from a human participant study, which includes five PD patients, three post-stroke patients, and three healthy subjects. Despite the diversity of abnormalities, the evaluation shows that the proposed approach can successfully distinguish post-stroke and PD gait from healthy gait, as well as post-stroke from PD gait, with Area Under the Curve (AUC) score of at least 0.96. Moreover, the methodology helps select important gait features to better understand the key characteristics that distinguish abnormal gaits and design personalized treatment.



There are no comments yet.


page 30

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Aging is an unprecedented, pervasive, profound and enduring process for humanity, and it is currently a global phenomenon JRNL:AgingReport . One major challenge associated with aging is the degenerative conditions of the neuromusculoskeletal system (e.g. osteoporosis, arthritis, Alzheimer’s disease MISC:alz , stroke MISC:stroke , and Parkinson’s disease MISC:pd ). Any dysfunction of the central nervous system, spinal cord, peripheral nerves or muscles can result in an abnormal gait BOOK:delisa1998gait . At the age of 60, 85% of people have a normal gait, while at the age of 85 or older this proportion drops to 18% snijders2007neurological . As a result, an increasing number of people suffer from walking difficulties, and the demand for gait rehabilitative therapy has been increasing rapidly.

In the current practice, gait rehabilitative therapy is provided by therapists who manually stimulate patients’ reflexes and rotate their lower limbs to retrain their central nervous systems with the correct gait patterns. This approach is not only physically demanding for both patients and therapists, but also expensive and time-consuming. Moreover, in the clinic, assessment of gait abnormalities is based on timed tests, visual observations by therapists, retrospective qualitative evaluations of video tapes, and specific physical tests, e.g., strength, range of motion, balance, gait speed, and endurance. As a result, most times gait assessment is based on the subjective judgment of the therapist. More objective methods are desired to quantify the gait assessment and progress evaluation of the rehabilitative training, reduce the chances of biased assessment by therapists, and provide better, targeted treatment to patients.

Significant research efforts have been reported to provide more objective gait assessment. A variety of sensory devices have been employed for gait analysis and impairment diagnosis. For instance, encoders, inertial sensors, and camera-based motion capture systems have been employed for kinematic analysis of human motion jrnl:IMUrehab ; jrnl:camerarehab ; force sensors JRNL:forceplate ; jrnl:mitss and electromyography (EMG) sensors jrnl:EMGrehab have been widely used to study the ground contact forces (GCFs) and muscle activities during walking; electroencephalography (EEG) sensors have been employed to analyze brain signals JRNL:gwin2010removal ; JRNL:sburlea2015detecting and better understand neurological mechanisms of walking. Advanced signal processing methods have been designed and applied on the data recorded from such sensor platforms JRNL:sant2011new ; JRNL:tahir2012parkinson ; JRNL:lee2012parkinson ; mulroy2003use ; kinsella2008gait ; JRNL:chen2005gait . These sensor technologies can be used not only to detect or prognose various human gait disorders, but also for disease monitoring and therapy progress tracking and evaluation snijders2007neurological .

To better quantify the severity of abnormal gait, important sensing features need to be identified from the sensory data to characterize gait disorders. Towards this goal, extensive research efforts have been reported to use machine learning algorithms for gait classification and clustering, to identify such parameters and automate gait disorder diagnosis. For example, post-stroke patients usually experience a very diverse set of gait abnormalities, most common of which is the hemiplegic gait BOOK:delisa1998gait

. For this reason, researchers have applied cluster analysis to identify subgroups of patients with similar sensing features who experience similar gait abnormalities

mulroy2003use ; kinsella2008gait ; JRNL:chen2005gait

. Likewise, other research efforts focus on classifying abnormal gaits between healthy subjects and Parkinson’s disease (PD) patients

JRNL:tahir2012parkinson ; JRNL:lee2012parkinson ; JRNL:wu2010statistical

. Classification methods with feature selection can help the target design of treatment and evaluation of therapy through the identified important gait sensing features 

chantraine2016proposition . Furthermore, such tools can improve the valuable clinical management of the patients, ease communication between clinicians chantraine2016proposition and optimize subject selection for human participant studies ferrante2011biofeedback . Consequently, they reduce the cost of physical therapy and improve the quality of life for patients. Especially, patients living in remote areas can benefit from an enhanced tele-medicine system with these quantitative and diagnostic tools, without necessitating complex apparatus chantraine2016proposition . However, to the best of our knowledge, there is no such quantitative gait diagnostic system for neurological diseases.

In order to bridge this gap and enable objective gait analysis, we propose an integrative framework in this paper to automatically classify gait disorders from two common neurological diseases, stroke and PD, and distinguish abnormal gait caused by these two diseases from the healthy gait. Classifying gait into groups caused by these two major neurological diseases can lead the way to provide diagnostic tools for specific gait disorders caused by these two neurological diseases, which is much needed for assisting objective gait assessment in the clinic and rehabilitation therapy centers. Our integrative framework includes a pair of smart shoes as the sensory device to capture the GCF data and a pipeline of data analytic algorithms for feature extraction, classification and feature selection. Gait features, including mobility, balance, strength and rhythm, are extracted from the sensory data.

Because there is strong correlation between the two neurological diseases and resultant gait disorders, multi-task machine learning strategies can be more feasible to identify similarities and differences of gait patterns than classic multi-class classification algorithms given the latter focus on modeling only the exclusive (or discriminative) features of different gait classes JMLR:v17:15-234 ; obozinski2006multi . An advanced multi-task learning algorithm has been developed to jointly create three classifiers, respectively, for distinguishing stroke-induced gait from healthy gait, PD-induced gait from healthy gait, and PD-induced gait from stroke-induced gait. To evaluate the proposed methodology we use data from a human participant study, which includes five PD patients, three post-stroke patients, and three healthy subjects. In our experiments the classification performance achieved Area Under the Curve (AUC) score of at least 0.96. The advantage of our multi-task learning method is that it can identify features useful for all three classification tasks as well as those predictive of a specific abnormality. We conclude our evaluation with a discussion on the important sensing features identified by the algorithms.

The remainder of the paper is organized as follows. Section LABEL:sec:related reviews the related works in gait quantification and analysis. Section 2 presents the sensory device that we developed to measure the GCF and Section 3 discusses the gait sensing features we extracted based on the data. In Section 4, we introduce the multi-task learning approach and use it to classify gait based on the extracted sensing features. Evaluation results are given based on the recorded data from a human participant study and findings are summarized in Section 5. We conclude the paper and discuss future work in Section 6.

Extensive research efforts have been made towards quantitative gait analysis. In this section, we first discuss the literature studies on improving gait quantification methods for objective gait parameter extraction. We then present a summary on machine learning methods for improving gait analysis, which includes gait pattern classification and cluster analysis for finding subgroups of patients who suffer from the same neurological disease and experience similar gait abnormalities.

1.1 Gait quantification

Gait quantification is important for objective gait assessment and analysis. It relates to the methods used for objectively measuring gait parameters, which can be used to estimate the severity of human gait abnormality. In this subsection we discuss gait quantification with respect to hemiplegic and Parkinsonian gait, which are the two most popular gait disorders caused by stroke and PD respectively

BOOK:delisa1998gait .

Among many gait parameters, symmetry is an important gait characteristic and is defined as a perfect agreement between the actions of the two lower limbs sadeghi2000symmetry . To calculate symmetry, mobility parameters (e.g., single support ratio) and spatiotemporal parameters (e.g., step length) can be used patterson2010evaluation . Symmetry indices (SI) have also been developed, from GCF data sadeghi2000symmetry ; JRNL:sant2011new .

Balance or walking stability is another important parameter that needs to be quantified, and used to predict falls. In JRNL:hubble2015wearable

multiple balance and stability measures are proposed, including RMS acceleration, jerk (time series of first derivative of acceleration), sway (a measure on how much a person leans his/her body), step and stride regularity and variability. Mobility and gait phases are also important gait parameters used to quantify gait. Mobility parameters include general movement characteristics like cadence, step length, single and double support ratio and periodicity 

mizuike2009analysis ; patterson2010evaluation . Gait phases refer to the various states within one walking cycle, and there are typically eight gait phases for a healthy subject particleChase2016 .

Gait quantification can be used to extract gait features for gait pattern classification. In this paper we calculate standard gait parameters based on GCF data for mobility, balance and strength quantification. In addition, new gait phase parameters are introduced based on our previous work particleChase2016 ; wenlongJDSMC , in which a wireless human motion monitoring system was designed, and a real-time data-driven gait phase detection algorithm was developed to capture the gait phases based on the recorded GCF data. The proposed system can objectively quantify the underlying gait phases without any input from a medical professional. These two works lead to some of the gait parameters used in this paper.

1.2 Gait pattern classification

Extensive research efforts have been reported to perform cluster analysis of post-stroke gait patterns and enable targeted treatment. In mulroy2003use

non-hierarchical cluster analysis was used to categorize four subgroups based on the temporal-spatial and kinematic parameters of walking. Similarly, hierarchical cluster analysis of post-stroke gait patterns was conducted in

kinsella2008gait , identifying three groups of patients with homogeneous levels of dysfunction. In ferrante2011biofeedback

, k-means clustering was used to group gait patterns in order to optimize participant selection in a biofeedback pedaling treatment.

Classification of post-stroke and PD gait patterns is another example of using machine learning methods in gait analysis. Classification of post-stroke gait patterns against healthy gait was performed in JRNL:chen2005gait and mizuike2009analysis

, using kinematic and kinetic data. Artificial neural networks (ANN) were used in

kaczmarczyk2009gait to classify post-stroke patient’s gait into three categories based on the types of foot positions on the ground at first contact: forefoot, flatfoot, and heel. The work in chantraine2016proposition classified hemiparetic gait in three groups with two subgroups each, that were defined from clinical knowledge. This classification method had the advantage of great usability in clinical routines without necessitating complex apparatus. Classification of PD gait patterns against healthy gait is also studied JRNL:tahir2012parkinson ; JRNL:lee2012parkinson ; JRNL:wu2010statistical

. Gait features from wavelet analysis and kinematic parameters are extracted, which are passed to support vector machines (SVM) and artificial neural networks (ANN) for classification.

In this paper, we perform classification of gait patterns in three classes, healthy, Parkinson’s and post-stroke. To the best of our knowledge, there is no research work on classification of gait patterns between these three classes. We employ a comprehensive set of gait parameters - including mobility, balance, strength and gait phases - and send them as input features to a classifier. An advanced classification method, MTFL, is used to distinguish between the three gait classes. Before we discuss the details of our algorithmic framework, we first present our smart shoe design for GCF data collection.

2 Smart Shoe Design and Ground Contact Force (GCF) Data

Figure 1: An overview of the smart shoe design. A signal processing unit includes barometric sensors, microcontroller, and Bluetooth chip

In order to better analyze patients’ gaits during walking, we have developed a pair of smart shoes to measure the GCFs on both feet wenlongJDSMC ; JRNL:smartshoes . Fig. 1 gives an overview of the shoe design. Four barometric sensors are employed to measure the GCFs on the toe, the first and second metatarsophalangeal (MTP) joint (Meta12), the fourth and fifth metatarsophalangeal joint (Meta45), and the heel. Silicone tubes are wound into air bladders to connect barometric sensors with measurement ranging from 0 to 250 mbar. Each sensor can measure weight up to 200 lbs with a resolution of 0.2 lbs.

The pressure sensor outputs are read by a microcontroller through analog input channels and the sensor signals are sent out to a laptop or mobile device using a Bluetooth module. The Bluetooth module can reliably transmit signals in a range of 200 feet, which is enough for normal clinical and daily use. A 9-volt alkaline battery is used to power the smart shoes, and it can work consecutively for 90 minutes. The sampling rate of the smart shoes can go up to 100 Hz with the Bluetooth module. In this paper the sampling rate is set at 20 Hz. Fig. 2 presents the representative raw data from a healthy subject, a PD patient and a post-stroke patent, respectively. For the healthy subject, a gait cycle always starts with a strong heel strike, and then the subject moves the center of pressure to the forefoot before toe-off. Moreover, the subject is able to maintain a good balance by allocating equal or more force to the medial boarder (Meta 12) in most of the gait cycles. However, for the PD patient, more force is observed on the lateral boarder (Meta 45) during the stance phase and this will significantly increase the risk of instability and falling. The stroke gait is even more abnormal, primarily due to the lack of heel strike as well as the poor stability shown by the large force on Meta 45. Additionally, the stroke patient walked much slower as it took 7 seconds to complete 3 steps, while the other two groups completed 5 steps in less time.

Figure 2: GCF data from PD and post-stroke patients and a healthy subject

3 Gait Features Extraction

To accurately describe specific human gait disorders is often a difficult task snijders2007neurological . Consequently, it is challenging to devise gait features 333In the remainder of this paper we refer to gait parameters, the term used in most literature studies, as gait features to avoid confusion with the model parameters used in the multi-task learning methods in Section 4. that can be used to classify gait patterns. Furthermore, the GCF data collected from the smart shoes can be noisy due to imperfect sensor dynamics and complexity of human gait. In this section we present a set of gait features that are used to detect the gait abnormalities by capturing the key gait characteristics of post-stroke and PD patients.

In Table 1, fourteen gait features are proposed based on the GCF data collected from the smart shoes. These featues are organized into four categories: gait phases, mobility, balance and strength. Their details will be discussed in the following subsections. Among these features, double support ratio, single support ratio and cadence are comprehensive features, which require bilateral information. All the other features are unilateral, as they can be calculated for each side separately JRNL:gaitpara .

Category Gait Features Laterality
Gait Phases Exp. Num. of Gait Phases Unilateral
Symmetry of Gait Phases Unilateral
Num. of Swing Phases Unilateral
Symmetry of Swing Phases Unilateral
Mobility Cadence (steps/min) Bilateral
Double Support Ratio Bilateral
Single Support Ratio Bilateral
Stance Phase Ratio Unilateral
Max. Force Difference
between Meta12 and 45
Min. Force Difference
between Meta12 and 45
Strength Max. Force of Heel Strike Unilateral
Max. Force of Toe Off Unilateral
Table 1: Proposed twelve gait features in four categories

3.1 Gait Cycles

We first give an overview of what a gait cycle is, since all the gait features are extracted once for each gait cycle in a walking trial. Gait cycle is the time interval between the same repetitive events of walking. The defined cycle can start at any moment, but generally begins when one foot contacts the ground. If it starts with the right foot contacting the ground, the cycle ends when the right foot makes contact again. Fig. 

3 gives an overview of two gait cycles at the lower two horizontal solid lines. The gait cycle can be broadly divided into two phases: stance phase and swing phase BOOK:delisa1998gait . These two phases can then be further divided into sub-phases within the gait cycle, as shown at the top part of Fig. 3. In general, the stance phase takes around 60% of the gait cycle BOOK:delisa1998gait and can be divided into double support and single support. In double support, both feet are in contact with the ground, while in single support only one foot is in contact with the ground. Double or single support ratio refers to the portion of time within a gait cycle someone spends in double or single support respectively. The swing phase is described when the limb is not weight bearing and represents around 40% of a single gait cycle BOOK:delisa1998gait . These percentages can change with the walking speed, as with a higher speed the double support ratio in the gait cycle tends to be reduced. In Fig. 3 the lower depicted cycle starts with right foot initial contact, which leads to the stance phase, while the other starts with left pre-swing phase which leads to swing phase. Indicative percentages are shown to indicate the different phases within the cycle.

In Fig. 3, different gait features are shown for different categories, like mobility, balance, strength. Gait pahses are shown at the top of the figure. In Sections 3.2 and 3.3 we discuss how gait phases are extracted and what gait phase related features are used in this work for gait disorder diagnosis. In Sections 3.4 and 3.5 we discuss other features related to mobility, balance and strength.

Figure 3: An overview of a gait cycle and the gait features from four categories: gait phases, mobility, balance and strength.

3.2 Gait Phase Detection

Gait phases refer to various states within one walking cycle, and there are typically eight gait phases for a healthy subject (as shown at the top of Fig. 3): initial contact, loading response, mid-stance, terminal stance (or initial contact), pre-swing, initial swing (not shown in Fig. 3), mid-swing, and terminal swing wenlongJDSMC ; JRNL:smartshoes . Pathological gait can be unpredictable and complex, thus some gait phases might be missing and the time allocation of gait phases might also be different from a normal gait. This abnormal gait phase allocation provides a powerful tool for abnormal gait detection.

In this work we extract gait phase related features based on our previous work which applies infinite Gaussian mixture modeling, a non-parametric Bayesian method, for gait phase detection 

particleChase2016 . Our approach estimates the unknown number of gait phases that can be best described from the GCF data. Particle filters and the popular chinese restaurant process (CRP) are used for online model parameters estimation. In the rest of this subsection we describe how swing and stance phases are identified from the extracted gait phases.

Identifying swing phases from the unlabeled gait phases is important as many other gait features are based on it. Although it is straightforward to find healthy gait’s swing phase (Fig. 3), the swing phase detection in pathological gait can be challenging for multiple reasons. First of all, the way smart shoes are worn can affect the raw GCF sensor signals. Tight shoe laces will change the raw values recorded by the barometric sensor, leading to different absolute values even for the same person in different sessions. Additionally, the stochastic nature of the sampling, which is used to estimate the distribution of gait phases particleChase2016 , can sometimes introduce new gait phases, which are not eventually represented in the GCF data. Finally, pathological gait can be so complex that sometimes new gait phases are explored from the particle filter algorithm. Apart from that, various conditions of neural or muscular impairments, like foot-drop, can cause fore-foot dragging on the ground BOOK:delisa1998gait . In such cases new gait phases are likely to be discovered and they should be identified as swing phases. Having correctly identified swing phases is very important as many other features are based on them.

As we described earlier, the swing phase ratio (portion of time spent on swing phase) of a healthy gait is typically around 40% of the gait cycle BOOK:delisa1998gait . This may change depending on the walking speed. Pathological gait can have smaller swing phase ratio, as the patient is walking slowly. Also, in the swing phase, GCF measurements will take very small positive values (or zero), as pressure from the body is not present in that limb. Using these two properties we identify the swing phases from all discovered gait phases according to the following steps:

We first calculate the average euclidean distance for all the observations in each gait phase from 0, by taking its 2-norm. We then sort the gait phases in increasing order based on their norms. We create a new swing phase, and add the observations in the sorted gait phase list one by one until the total number of observations in the new swing phase is more than 10% of all the observations. The 10% threshold is empirically chosen and gives the desired swing phase ratio in our dataset. The number of swing phases that were merged is kept as it is used as a gait phase feature (see Sec. 3.3). All the extracted gait phase features are described in the following subsection.

3.3 Gait Phase Features

The gait phase features are calculated from the gait phases that are extracted by our gait phase detection algorithm (see Section 3.2 and for more details, please refer to particleChase2016 ). The expected number of gait phases can be calculated from the particles and their weights returned from the particle filter algorithm as , where is the number of gait phases detected from particle and is the particle’s weight. is a measure of complexity of the human gait. Compared with the eight standard gait phases of a healthy subject, pathological gait is unpredictable and it may have a different number of gait phases. For example, post-stroke patients with affected neurological system may experience foot-drop. This usually increases the stance phase with circumduction to allow toe clearance BOOK:delisa1998gait , which can lead to toe dragging on the ground, and thus causing the gait phase detection algorithm detecting multiple swing phases. The number of swing phases is another gait parameter and has been discussed in the previous Section 3.2.

The symmetry of gait phases (swing phases) is used as a measure to quantify how even the proportion of time spent is in each gait phase in a gait cycle (swing phases). We chose to include this new type of symmetry measure as it can be easily applied on the gait phases that were extracted from our Dirichlet process mixture model particleChase2016 , given the fact that number of gait phases is not known a-priori

for each subject. This single gait parameter can estimate the symmetry for any number of gait phases detected. It is based on the cosine similarity, as described in the following formula:


where is the angle between and , with and is the number of gait phases (swing phases) found. is a vector of size , where each element in counts the number of observations belonging to each gait phase (swing phase) within a gait cycle and is a vector of size with all its elements equal to 1, assuming observations in are evenly distributed. If the number of observations belonging to each gait phase is not evenly distributed and thus there are gait phases with very few observations, the angle between vector and will be higher resulting in lower symmetry. On the other hand, if the number of observations belonging to one gait phase is always more than normal it would also result in lower symmetry.

3.4 Mobility Features

We select four features in the mobility category, cadence, double and single support ratios and stance phase ratio. Cadence is measured in steps per minute and it is calculated by taking the total number of stance phases in one trial divided by the length of the trial in minutes. The double support ratio refers to the proportion of time in a gait cycle that both feet are in the stance phase to support the subject, whereas the single support ratio refers to the proportion of time in a gait cycle that only one foot touches the ground while the other is in the swing phase. Stance phase ratio refers to the proportion of time in a gait cycle that one foot is in the stance phase. All these features are summarized in Fig. 3.

3.5 Balance and Strength Features

We select two features in the balance and strength categories each. In the balance category, the maximum and minimum force differences between the medial (Meta12, Fig. 1) and lateral (Meta45, Fig. 1) sides of the forefoot in a gait cycle can be calculated as


These features can evaluate the capability of maintaining balance. The refers to the set of indices that belong to one gait cycle. Strength is quantified using the maximum force on the heel during heel strike and on the toe during toe off. All balance and strength features are normalized by the body weight to make them comparable among different subjects.

4 Multi-Task Feature Learning for Gait Disorder Diagnosis

Based on the extracted gait features, we diagnose gait disorders by constructing classifiers as functions of these features. In this work, we use an advanced multi-task feature learning (MTFL) classification method JMLR:v17:15-234 to build three classifiers to discriminate gait observations of PD and stroke patients, respectively, from those of healthy adults as well as in between the gaits of PD and stroke patients. The selected learning strategies can be more feasible to identify similarities and differences of gait patterns than classic multi-class classification algorithms given multi-class classification methods focus on modeling only the exclusive (or discriminative) features of the different gait classes. Moreover, the methodology helps in important gait feature selection which may help in better understanding the key characteristics that distinguish abnormal gaits and help design more targeted treatment methods.

MTL is a methodology that can improve the generalization of multiple related classification tasks by exploiting the task relationships, especially when the training set for some or all the tasks is limited. Related tasks are learned in a joint manner, so that knowledge learned from one task may benefit learning for other tasks. For example, in gait disorder diagnosis, the task of deciding if an observation, represented by a vector of gait features, is recorded from a PD patient or healthy subject, may help diagnose if another observation is recorded from a post-stroke patient or a healthy subject. MTL has been shown to be theoretically and practically more effective than learning tasks individually JMLR:v17:15-234 . A widely-used basic assumption is that the related tasks may share a common representation in the feature space, which is investigated by multi-task feature learning (MTFL).

We revisit two of our recently developed MTFL methods that both rely on a multiplicative decomposition of the model parameters used for each task, and hence are referred to as Multiplicative MTFL (MMTFL). Both methods are related to the widely used block-wise joint regularization MTFL method obozinski2006multi , but bring out a significant advantage over it, in terms of selecting relevant features for classification. The new methods can simultaneously select features that are useful across multiple tasks and features that might be only discriminative for a specific classification task.

Given classification tasks in total, let be the sample set for the -th task, where is a matrix containing rows of examples and columns of gait features, is a column vector containing the corresponding labels for each example, is the sample size of task , and is the number of features. We focus on creating linear classifiers , where is the vector of model parameters to be determined. We then define a model parameter matrix where each column contains a task’s parameter vector , and thus each row of this matrix corresponds to a gait feature, i.e., the weights for a gait feature used for each of the tasks, which we denote as , and

. We choose a loss function

which typically measures the discrepancy between the prediction and the observation for task

. In a classification task, the loss function is commonly a logistic regression loss.

The widely used block-wise joint regularization MTFL method solves the following optimization problem for the best :


where is a block-wise regularizer, often called the matrix norm, that computes . Common choices for are , or . Minimizing this regularizer can shrink an entire row of

to zero, thus eliminating or selecting features for all tasks. The hyperparameter

is used to play the trade-off between the loss function and the regularizer. However, a major limitation of the joint regularization MTFL method is that it either selects a feature for all tasks, or eliminates it from all tasks, which can be unnecessarily restrictive. In practice, several tasks may share features but some features may only be useful for a specific task. Hence, we introduce the following multiplicative MTFL that addresses this issue.

A family of MMTFL methods can be derived by factorizing , where computes a vector whose -th component equals the product of and , and in other words, . The vector is applied across tasks, indicating whether certain features are useful to any of the tasks, and is only relevant to task . We relax the indicator vector (i.e., a binary vector) into a non-negative so the optimization problem can be tractable. If , then the -th feature will not be used by any of the models. If , then a specific can still rule out the -th feature from the -th task. We minimize a regularized loss function with separate regularizers for and as follows for the best models:


where and , which are the -norm of to the power of and the -norm of to the power of if and are positive integers. The tuning parameters , and are used to balance the empirical loss and regularizers. According to the different choices of and , we can have different levels of sparsity for and .

The method MMTFL(2,1) refers to the case when and in Eq.(5) and solves a problem as follows:


It is widely known that -norm is not sparsity-inducing, meaning that minimizing it leads to a vector of many small, non-zero entries. On the other hand, the sparsity-inducing -norm creates a vector with many entries equal to zero. In Eq.(6), is regularized by a sparsity-inducing norm, hence tending to eliminate many features from across all of the tasks. This formulation is more suitable for capturing the feature sharing pattern such that there exists a large subset of irrelevant features across tasks, requiring a sparse , but different tasks share a significant amount of features from the selected feature pool as indicated by , thus requiring a non-sparse .

The method MMTFL(1,2) is on the opposite direction when and in Eq.(5), and solves the following problem:


Eq.(7) is suitable to capture a feature sharing pattern where none or only a small portion of the features can be removed because each may be useful for some tasks, thus requiring a non-sparse . However, different tasks share a small amount of these features, thus requiring a sparse . In this case, -norm is applied to and -norm is applied to .

Since it is difficult to prove any relationship between gait features and actual gait problems, we hypothesize that these methods can help us identify the important gait features to recognize abnormal gaits due to the neurological diseases from otherwise healthy gaits, and may further locate features to discriminate between stroke-induced gaits and PD-induced gaits. To validate this hypothesis, in our performance evaluation, we compare the two methods against early MMTFL methods that are most comparable to the proposed methods and two baseline methods - single task learning (STL) methods that either use -norm or -norm to regularize individual , which we referred to as STL-ridge and STL-lasso respectively.

5 Performance Evaluation

We designed two sets of experiments to evaluate the effectiveness of the proposed methods. In the first set of experiments, we examined the area under the curve (AUC) classification performance metric of the models that are created by the different MTFL methods. In the second set of experiments we studied the importance of each proposed gait feature and their relevance to each classification task. In the following, we first describe our human participant study design and then present the experiment details.

5.1 Human Subject Test Design

In order to evaluate the performance of the proposed algorithms, we collected GCF data using the developed smart shoes from healthy subjects without known walking problems and PD and post-stroke patients. Experiments with healthy subjects were conducted in the Mechanical Systems Control Laboratory at the University of California, Berkeley. The clinical study with patients was conducted in the William J. Rutter Center at the University of California, San Francisco (UCSF). The Committee on Human Research (CHR) at UCSF reviewed and approved this study. The original purpose of this human subject study was to examine whether patients could use visual feedback to direct their rehabilitation training and how was the training performance compared to traditional rehabilitation training directed by a physical therapist only. We use these datasets to evaluate the algorithm developed in this paper. Detailed experimental design and statistical analysis of the clinical outcomes are available in byl2015clinical ; wenlongJDSMC .

To collect data for this work, the subjects were asked to walk multiple trials on a flat ground for at least 50 consecutive steps in their normal walking speeds. The data collected from five PD patients, three post-stroke patients, and three healthy subjects are used to test our methodology. The average ages for each of the groups are 69.2, 53 and 23 years old respectively. Representative raw data from each of the three groups are shown in Fig. 2. Gait features are extracted for each gait cycle and average results are taken for each trial. This generates a dataset of 180 observations with 21 features each.

5.2 Classification of Gait Disorders

To classify among stroke, PD and healthy gaits we designed and evaluated 3 classification tasks: healthy v.s. stroke gait, healthy v.s. Parkinson’s gait and stroke v.s. Parkinson’s gait. We compared our two new formulations MMTFL{2,1} and MMTFL{1,2} with two other standard MMTFL methods. They are all summarized as follows:

  • MMTFL{2,1}: formulation (6)

  • MMTFL{1,2}: formulation (7)

  • MMTFL{1,1}: formulation (5) with

  • MMTFL{2,2}: formulation (5) with

In addition, two single task learning (STL) approaches were implemented as baselines and compared with the MTFL algorithms. They can be formulated as folows:


With and the i-th example and example label for task respectively, the parameter vector for task , the hyperparameter used to play the trade-off between the least squares loss and the regularizer and the selected regularizer. They are summarized as follows:

  • STL-lasso: with as the regularizer

  • STL-ridge: with as the regularizer

Before we ran the experiments we used a tuning process to find appropriate values for the hyperparameters, and . Grid search with three-fold cross validation (CV) was performed to select proper hyperparameter values in the range from to . In all the experiments, hyperparameters were fixed to the values that yielded the best performance in the CV.

In the first set of experiments, we partitioned the 180 observations into a training dataset and a testing dataset according to a given partition ratio, which was set to be 16%, 20%, 25%, 33% or 50%, respectively in each experiment. For each partition ratio, 10-fold CV was performed and average results were reported. The classification performance was measured using AUC, which measures the total area under the receiver operating characteristic (ROC) curves. These results are summarized in the left half of table 2. We can observe from the results that MTFL methods always outperform STL methods. Specifically, with the smallest training set of 16%, the MMTFL{2,1} method has the best improvement over the STL methods. When the training partition ratio was increased, the AUC performance of all the methods improved consistently. When it reached 50%, STL or MTFL methods achieved their highest AUC scores, respectively. The advantage of MTFL methods with smaller training set ratios is explained because they can learn the tasks jointly and not exclusively, which is typically done in STL methods. On the other hand, along with the increase of training dataset percentage, more training examples are provided to the classifiers, making the classification easier and thus STL methods performed closer to MTFL when the partition rate increases.

Random Partition
Method 16% 20% 25% 33% 50%
MMTFL{2,2} 0.930.03 0.970.02 0.970.01 0.980.01 0.990.01
MMTFL{1,1} 0.940.04 0.960.02 0.980.01 0.980.01 0.990.01
MMTFL{2,1} 0.950.03 0.970.02 0.980.01 0.980.01 0.990.01
MMTFL{1,2} 0.930.04 0.960.03 0.980.01 0.980.01 0.990.01
STL-ridge 0.900.03 0.940.03 0.950.02 0.970.01 0.980.01
STL-lasso 0.920.03 0.960.02 0.970.02 0.980.01 0.990.00
Table 2: AUC performance of different methodologies

Following that, we tested how well the classification generalizes when a new subject’s gait was tested against a model built by gaits of other patients and healthy subjects. Specifically, the same classification tasks were performed with the same classification methods, but the testing data were from a single subject and all the data from the rest of subjects were used to train the corresponding model. We repeated this for each individual patient and healthy subject and the performance results are summarized in the right half of Table 3, where average AUC is reported across all tasks and per task separately. PD, ST and H refer to the gait from PD patients, post-stroke patients and healthy subjects, respectively.

Method All tasks PD vs H ST vs H ST vs PD
MMTFL{2,2} 0.949 0.880 0.994 0.967
MMTFL{1,1} 0.979 0.982 0.993 0.960
MMTFL{2,1} 0.978 0.960 0.994 0.979
MMTFL{1,2} 0.975 0.983 0.983 0.967
STL-ridge 0.916 0.831 0.971 0.940
STL-lasso 0.944 0.893 0.977 0.961
Table 3: Per task average AUC scores when a new subject is tested in a model trained by the rest subjects

As can be observed from Table 3, MTFL methods performed better than STL methods consistently. We also observe that there were some easier tasks (e.g., stroke vs healthy), where STL AUC scores were almost as good as MTFL ones, and some more challenging tasks (e.g., PD vs healthy), where STL AUC scores were worse compared to any other task.

To further study how the two new MTFL formulations perform on each task we report the confusion matrices of all the three tasks for MMTFL{1,2} and MMTFL{2,1} in Table 4 and 5 respectively. Each row in the matrix corresponds to which gait class was tested, while a column corresponds to which gait class the algorithm predicted. Between these two new formulations, MMTFL{1,2} performed better with PD, as out of the 83 tested gaits, MMTFL{1,2} predicted 5 of them to be healthy gaits, i.e. false negatives, compared to 11 healthy gaits that were predicted by MMTFL{2,1}. MMTFL{2,1} performed better with post-stroke, as out of the 31 tested stroke gaits MMTFL{2,1} predicted 3 of them to be healthy gaits, compared to 11 healthy gaits that were predicted by MMTFL{1,2}. Overall, MMTFL{2,1} performed better, as it also achieved beter false positive rates. Specifically MMTFL{2,1} predicted only 2 PD gaits out of 64 healthy gaits and 3 stroke gaits out of the 83 PD gaits, compared to 5 and 7 predicted by MMTFL{1,2} in the same tasks respectively.

PD Healthy
PD 78 5
Healthy 5 59
Stroke Healthy
Stroke 20 11
Healthy 0 64
Stroke PD
Stroke 25 6
PD 7 76
Table 4: Confusion Matrices of MMTFL{1,2} for the 3 tasks, true labels in rows, predicted in columns
PD Healthy
PD 72 11
Healthy 2 62
Stroke Healthy
Stroke 28 3
Healthy 0 64
Stroke PD
Stroke 24 7
PD 3 80
Table 5: Confusion Matrices of MMTFL{2,1} for the 3 tasks, true labels in rows, predicted in columns

The last set of experiments aimed to report the prediction results per patient, in order to give complete information of the performance of each subject’s gait. Table 6 summarizes the per patient confusion matrices generated from MMTFL{2,1} for the three classification tasks. The first column indicates each subject’s disease or healthy condition and their identification numbers (ID) are given in the second column. The last two columns give number of times a trial was predicted to be PD, stroke or healthy subjects. The summation of these two numbers in each row corresponds to the total number of trials that were recorded for each subject. From the table we observe that stroke patient 4 was almost always predicted either healthy subject or PD patient, which means that her gait patterns were much different from the other post-stroke patients. This patient was a 33 year old female with minor stroke, which explains the similarity of her gait to a healthy, when compared to other older stroke patients. This wrong prediction may also be related to the limited number of stroke paitents that participated in this study.

Subject Predicted
Disease ID PD Healthy
PD 1 16 0
PD 2 11 6
PD 3 13 5
PD 5 19 0
PD 6 13 0
Healthy 7 1 22
Healthy 8 1 21
Healthy 9 0 19
Subject Predicted
Disease ID Stroke Healthy
Stroke 4 4 3
Stroke 10 8 0
Stroke 11 16 0
Healthy 7 0 23
Healthy 8 0 22
Healthy 9 0 19
Subject Predicted
Disease ID Stroke PD
PD 1 0 16
PD 2 0 17
PD 3 1 17
PD 5 2 17
PD 6 0 13
Stroke 4 1 6
Stroke 10 7 1
Stroke 11 16 0
Table 6: Confusion Matrices of MMTFL{2,1} for the 3 tasks per patient

Given that MMTFL{2,1} performs best in general, the tested data seem to follow the assumption under which MMTFL{2,1} was designed. Specifically, across all three tasks there exists a large subset of irrelevant sensing features, requiring a sparce , but different tasks share a significant amount of features from the selected fetature pool as indicated by . In other words, there are some specific sensing features that help identify the neurological disorders. In the next subsection we are going to present the selected features for each method used in this paper.

5.3 Identification of Important Gait Features

Important gait features identified from gait disorder classification may help better understand the key characteristics that distinguish abnormal gait patterns among different gait disorders and healthy gait. They may also help the target design of treatment and evaluation of rehabilitative progress. In this subsection we present the important gait features that were identified by the used methods in our experiments, for each of the three classification tasks that were evaluated in subsection 5.2. With the important gait features we can understand which of the proposed gait features are more important to classify GCF data from post-stroke or PD patients and healthy subjects. As described in section 4 for the MMTFL methods, we have . Vector is the vector of model parameters for task , vector is used across all tasks, indicating if a feature is useful for any of the tasks, and vector is only for task . In Fig. 4 we plot all vectors for each MMTFL model as progress bars to show the importance of each feature. In Figures 56 and 7 we plot the absolute values of the learned task parameter vectors for each MMTFL and STL method for each of the three classification tasks.

Figure 4: Feature selection vector from all MMTFL methods
Figure 5: Absolute value of task parameter vector in the PD vs. healthy gait classification task.
Figure 6: Absolute value of task parameter vector in the Stroke vs. healthy gait classification task.
Figure 7: Absolute value of task parameter vector in the Stroke vs. PD gait classification task.

Based on the general characteristics of Hemiplegic gait, most commonly seen in stroke, and Parkinsonian gait BOOK:delisa1998gait we have the following observations:

  • The most important feautre is the maximum force at the right toe and the second most selected feature is maximum force at the left heel. These two are strength indicators during toe off and heel strike gait phases. Patients with neurological related diseases, like stroke and PD, may experience weak muscle strength BOOK:delisa1998gait . Circumduction of the affected leg in stroke can also produce different toe contact force signatures. Additionally, slow walking (Bradykinesia) which is characteristic of both stroke and PD gait can have reduced force levels at the toe during push-off BOOK:delisa1998gait ; JRNL:wu2010statistical .

  • Minimum force difference between medial and lateral sides of the metatarsophalangeal joints at the forefoot (see Sec. 3.5) at the left foot is another important feature, which is an indicator of balance. Rigidity, meaning stiff or inflexible muscles, is one of the main symptoms of PD, alongside tremor and slowness of movement. There is usually little or no arm swing to help in balancing the individual BOOK:delisa1998gait . PD patients usually have reduced balance and the algorithm has identified this as an important feature.

  • Cadence and double support ratio are mobility gait parameters and they are also important in distinguishing healthy vs pathological gait. As discussed before, a common characteristic of stroke and PD subjects is bradykinesia. This in turn affects the double support ratio.

  • Symmetry of swing phases is found to be another important factor to distinguish pathological gaits for some models. As discussed before, this parameter captures how evenly the swing gait phases are represented in the subject’s gait. Circumduction of the affected leg can introduce additional gait phases and thus uneven representation of the detected swing phases.

All the rest features are not important and discarded by most of the models, except MMTFL{2,2}, which shows reduced sparsity. These findings are consistent with the literature about the characteristics of PD and stroke patient’s gait BOOK:delisa1998gait .

6 Conclusion

In this work, we presented the design of an integrative framework for gait disorder diagnosis and advance smart gait rehabilitation. Gait features were developed for different categories including gait phases, mobility, balance and strength. MTFL, an advanced classification method, was used to train the different classification tasks that can classify subject’s gait. Data from PD and post-stroke patients, along with healthy subjects were used to evaluate the proposed methods.

The proposed gait features successfully captured the underlying properties of each disease. MTFL was able to construct accurate classifiers based on the given gait parameters to distinguish abnormal gaits. Also it selected the most important gait parameters for this classification task, ignoring the rest. Selected features captured the characteristics of each disease as described in the literature. This study demonstrated the potential to automate gait analysis of multiple common gait disorders which can benefit the medical professionals and patients with improved and targeted treatment plans for rehabilitation.

As future work, we intend to provide more comprehensive gait disorder diagnostic tools for more complex gait disorders that are difficult for the clinicians to detect. We plan to assist their assessment process in the clinic, evaluate these analytic systems with properly designed clinical studies, and design new methods for rehabilitation progress evaluation and treatment plan development.

7 Acknowledgment

The authors want to acknowledge Dr. Nancy Byl and Ms. Sophia Coo at UCSF for organizing the human subject study, and the patients who participated in the study for their cooperation. The authors would like to also thank Dr. Masayoshi Tomizuka at UC Berkeley for his help developing the smart shoes.



  • (1) J. M. Ortman, V. A. Velkoff, H. Hogan, An aging nation: the older population in the united states, Washington, DC: US Census Bureau (2014) 25–1140.
  • (2) Alzheimer’s facts and figures,
  • (3) Stroke statistics,
  • (4) Statistics on Parkinson’s,
  • (5) J. A. DeLisa, Gait analysis in the science of rehabilitation, Vol. 2, Diane Publishing, 1998.
  • (6) A. H. Snijders, B. P. Van De Warrenburg, N. Giladi, B. R. Bloem, Neurological gait disorders in elderly people: clinical approach and classification, The Lancet Neurology 6 (1) (2007) 63–74.
  • (7) B.-C. Lee, S. Chen, K. Sienko, A wearable device for real-time motion error detection and vibrotactile instructional cuing 19 (4) (2011) 374–381.
  • (8) A. Schmitz, M. Ye, R. Shapiro, R. Yang, B. Noehren, Accuracy and repeatability of joint angles measured using a single camera markerless motion capture system, Journal of biomechanics 47 (2) (2014) 587–591.
  • (9) T. Liu, Y. Inoue, K. Shibata, K. Shiojima, A mobile force plate and three-dimensional motion analysis system for three-dimensional gait assessment, IEEE Sensors J. 12 (5) (2012) 1461–1467.
  • (10) S. Bamberg, A. Benbasat, D. Scarborough, D. Krebs, J. Paradiso, Gait analysis using a shoe-integrated wireless sensor system, IEEE Transactions on Information Technology in Biomedicine 12 (4) (2008) 413–423.
  • (11) D. H. Sutherland, The evolution of clinical gait analysis part l: kinesiological EMG, Gait & posture 14 (1) (2001) 61–70.
  • (12) J. T. Gwin, K. Gramann, S. Makeig, D. P. Ferris, Removal of movement artifact from high-density eeg recorded during walking and running, Journal of neurophysiology 103 (6) (2010) 3526–3534.
  • (13) A. I. Sburlea, L. Montesano, R. C. de la Cuerda, I. M. A. Diego, J. C. Miangolarra-Page, J. Minguez, Detecting intention to walk in stroke patients from pre-movement eeg correlates, Journal of neuroengineering and rehabilitation 12 (1) (2015) 1.
  • (14) A. Sant’Anna, A. Salarian, N. Wickstrom, A new measure of movement symmetry in early parkinson’s disease patients using symbolic processing of inertial sensor data, IEEE Transactions on biomedical engineering 58 (7) (2011) 2127–2135.
  • (15) N. M. Tahir, H. H. Manap, Parkinson disease gait classification based on machine learning approach, Journal of Applied Sciences 12 (2) (2012) 180.
  • (16) S.-H. Lee, J. S. Lim, Parkinson’s disease classification using gait characteristics and wavelet-based feature extraction, Expert Systems with Applications 39 (8) (2012) 7338–7344.
  • (17) S. Mulroy, J. Gronley, W. Weiss, C. Newsam, J. Perry, Use of cluster analysis for gait pattern classification of patients in the early and late recovery phases following stroke, Gait & posture 18 (1) (2003) 114–125.
  • (18) S. Kinsella, K. Moran, Gait pattern categorization of stroke participants with equinus deformity of the foot, Gait & posture 27 (1) (2008) 144–151.
  • (19) G. Chen, C. Patten, D. H. Kothari, F. E. Zajac, Gait differences between individuals with post-stroke hemiparesis and non-disabled controls at matched speeds, Gait & posture 22 (1) (2005) 51–56.
  • (20) Y. Wu, S. Krishnan, Statistical analysis of gait rhythm in patients with parkinson’s disease, IEEE Transactions on Neural Systems and Rehabilitation Engineering 18 (2) (2010) 150–158.
  • (21) F. Chantraine, P. Filipetti, C. Schreiber, A. Remacle, E. Kolanowski, F. Moissenet, Proposition of a classification of adult patients with hemiparesis in chronic phase, PloS one 11 (6) (2016) e0156726.
  • (22) S. Ferrante, E. Ambrosini, P. Ravelli, E. Guanziroli, F. Molteni, G. Ferrigno, A. Pedrocchi, A biofeedback cycling training to improve locomotion: a case series study based on gait pattern classification of 153 chronic stroke patients, Journal of neuroengineering and rehabilitation 8 (1) (2011) 1.
  • (23) X. Wang, J. Bi, S. Yu, J. Sun, M. Song, Multiplicative multitask feature learning, Journal of Machine Learning Research 17 (80) (2016) 1–33.
  • (24) G. Obozinski, B. Taskar, M. Jordan, Multi-task feature selection, Statistics Department, UC Berkeley, Tech. Rep 2.
  • (25) H. Sadeghi, P. Allard, F. Prince, H. Labelle, Symmetry and limb dominance in able-bodied gait: a review, Gait & posture 12 (1) (2000) 34–45.
  • (26) K. K. Patterson, W. H. Gage, D. Brooks, S. E. Black, W. E. McIlroy, Evaluation of gait symmetry after stroke: a comparison of current methods and recommendations for standardization, Gait & posture 31 (2) (2010) 241–246.
  • (27) R. P. Hubble, G. A. Naughton, P. A. Silburn, M. H. Cole, Wearable sensor use for assessing standing balance and walking stability in people with parkinson?s disease: a systematic review, PloS one 10 (4) (2015) e0123705.
  • (28) C. Mizuike, S. Ohgi, S. Morita, Analysis of stroke patient walking dynamics using a tri-axial accelerometer, Gait & posture 30 (1) (2009) 60–64.
  • (29) I. Papavasileiou, W. Zhang, S. Han, Real-time data-driven gait phase detection using infinite gaussian mixture model and parallel particle filter, in: 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), 2016, pp. 302–311. doi:10.1109/CHASE.2016.25.
  • (30) W. Zhang, M. Tomizuka, N. Byl, A wireless human motion monitoring system for smart rehabilitation, ASME. J. Dyn. Sys., Meas., Control 138 (11) (2016) 111004–1–11004–9.
  • (31) K. Kaczmarczyk, A. Wit, M. Krawczyk, J. Zaborski, Gait classification in post-stroke patients using artificial neural networks, Gait & posture 30 (2) (2009) 207–210.
  • (32) K. Kong, M. Tomizuka, A gait monitoring system based on air pressure sensors embedded in a shoe, IEEE/ASME Transactions on Mechatronics 14 (3) (2009) 358–370.
  • (33) H. P. Von Schroeder, R. D. Coutts, P. D. Lyden, E. Billings, V. L. Nickel, Gait parameters following stroke: a practical assessment, J. Rehabilitation Research and Development 32 (1995) 25–25.
  • (34) N. Byl, W. Zhang, S. Coo, M. Tomizuka, Clinical impact of gait training enhanced with visual kinematic biofeedback: Patients with parkinson?s disease and patients stable post stroke, Neuropsychologia 79 (2015) 332–343.