Decoding index finger position from EEG using random forests

12/14/2015 ∙ by Sebastian Weichwald, et al. ∙ Max Planck Society Universitätsklinikum Freiburg 0

While invasively recorded brain activity is known to provide detailed information on motor commands, it is an open question at what level of detail information about positions of body parts can be decoded from non-invasively acquired signals. In this work it is shown that index finger positions can be differentiated from non-invasive electroencephalographic (EEG) recordings in healthy human subjects. Using a leave-one-subject-out cross-validation procedure, a random forest distinguished different index finger positions on a numerical keyboard above chance-level accuracy. Among the different spectral features investigated, high β-power (20-30 Hz) over contralateral sensorimotor cortex carried most information about finger position. Thus, these findings indicate that finger position is in principle decodable from non-invasive features of brain activity that generalize across individuals.



There are no comments yet.


page 2

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Invasive recordings of brain activity in human subjects have been shown to provide rich information on various aspects of motor control. For instance, single-unit recordings in primary motor cortex enable the control of end-effectors in multiple dimensions [1, 2]. Subdural electrocorticography (ECoG) allows for the decoding of arm movement direction [3] as well as the laterality of index finger movement [4], and can distinguish between movements of individual fingers of the same hand [5, 6]. It has even been shown that ECoG recordings provide information on the time course of the flexion of individual fingers [7, 8]. It is at present unclear, however, to which extent such information can also be decoded from non-invasive measures of brain activity. To date, empirical evidence has been presented that electroencephalography (EEG) and magnetoencephalography (MEG) recordings provide information on the laterality of index finger movements [9, 10, 11] as well as on arm movement trajectories [12, 13, 14]. Furthermore, predictive features in MEG as well as EEG recordings for discriminating movements of individual fingers on one hand have been reported [15].

Here, this line of work is extended by investigating whether index finger position can be decoded on a single-trial basis from EEG recordings. This is a particularly challenging task, as there exists an overlap of neurons in primary motor cortex controlling individual finger movements, and the encoding of muscle activations and joint positions does not follow a strict somatotopic organization

[16]. This raises the question whether EEG recordings provide sufficient information to differentiate between different index finger positions.

Using a random forest classifier empirical evidence is presented that different index finger positions can indeed be differentiated on a single-trial basis from EEG recordings. The random forest classifier was trained with a leave-one-subject-out cross-validation procedure involving 20 healthy subjects, indicating that the representation of index finger position is shared across subjects. The classification was primarily based on changes in high

-band power (20–30 Hz) over the contralateral motor cortex.

The remainder of this paper is structured as follows: In section II the experimental setup and paradigm, the recorded data, the computation of features, the classification procedure and the feature importance scores are described. Section III presents the results of the applied procedure while section IV closes with a discussion of the results and points to some interesting questions for further investigation.

2 Methods

2.1 Experimental Paradigm

During the experiment subjects were seated in a comfortable chair approximately 1.25 m in front of a computer screen. A numeric 3x3-keyboard was attached to the chair’s right arm rest (cf. fig. 1). Subjects placed their right arm comfortably on the arm rest, so that they were able to easily reach all nine keys of the keyboard with their index finger and without moving their arm or wrist. To keep the hand position fixed throughout the experiment, subjects were asked to keep their thumb in a tube attached to the side of the keyboard. The keyboard was blocked from the subjects’ view by a black wooden board.

The experimental paradigm consisted of three different states: a five minute resting state, during which the subjects were asked to fixate a gray cross displayed centrally on the screen; a trial state of approximately five minutes, during which subjects were familiarized with the experimental paradigm; and the actual testing state, which consisted of 15 sessions of 90 trials each.

All visual stimuli (cf. fig. 2) were presented within a visual angle of less than 2°. The target key was indicated by the position of a blue dot superimposed on a grey circle (cf. fig. 2). Figure 2 shows the mapping of class labels and key positions to the different visual stimuli. The grey circle served as a fixation cross, and lit up in red if the wrong key was pressed (cf. fig. 2).

Within each trial the target key needed to be pressed for a duration between three and four seconds, randomly drawn from a uniform distribution. If the subject released the target key too early, the timer was restarted. If the target key was pressed sufficiently long, the next target key was presented, prompting the subject to release the current- and press the new target key.

Each of the 15 sessions consisted of 90 randomly permuted trials, avoiding repetitions of the same target key and ensuring ten trials per target key within each session. This resulted in 150 trials for each target key per subject.

2.2 Subjects

Twenty healthy subjects (mean age of 28.4 years with a standard deviation of 4.1 years) participated in this study, all of which indicated that they were right-handed (laterality scores according to the Edinburgh inventory

[17] between 0 and +100, mean +68, standard deviation 26). Each subject gave informed consent in agreement with guidelines set by the Max Planck Society.

2.3 Experimental Data

During the experiment EEG was recorded at 121 active electrodes, placed according to the extended international 10-20 system with a sampling rate of 2 kHz using a QuickAmp amplifier (Brain Products GmbH, Gilching, Germany) with built-in common average reference. Impedances were ensured to be below 25 k at the beginning of each recording. Channels exceeding this threshold were switched off by manually connecting them to the ground channel of the amplifier.

To prepare the data for classification the following procedure was employed. First, the data of each subject was spatially filtered using a Laplacian setup [18]. Then, channels that were not present in every individual recording were removed, such that 106 channels common to all subjects remained. The data was then highpass filtered at a cut-off frequency of 3 Hz using a phase-preserving third-order Butterworth filter, and a common average reference filter was applied.

Only the first three seconds, i. e. 6k samples, of each trial were used for further analysis. Due to quitting the experiment early, there is one trial missing for one subject and 300 trials missing for another subject. Thus a total of 26819 trials were recorded.

Figure 1: Experimental setup: The index finger was used to press keys on a 3x3-keyboard attached to an arm rest. Thumb position is fixed by a tube. Visual cover is removed here.
Figure 2: Visual stimuli: 2 indicates the current target key; 2 indicates that the currently pressed key is incorrect. Figure 2 shows how the class labels/key positions are mapped to the different visual stimuli.

2.4 Feature Computation

For each subject, the -band was identified manually by selecting the frequency band (width across subjects between 1 Hz and 3 Hz) with the highest bandpower in the 8–15 Hz range of channels over the left and right sensorimotor cortex using the resting state data only.

Then for each trial, the log-bandpower of the individual -band and the log-bandpower of a general -band (20–30 Hz) was computed for the whole trial as well as for 41 sliding windows of one second length, with the center of the window shifted in steps of 50ms, using a FFT with a Hann window. This yielded 84 bandpower features per trial and channel, resulting in 8904 features per trial in total.

For each subject feature values exceeding three standard deviations were marked as outliers. Specifically, the following step was iterated until no further outliers were found: Mark feature values which differ more than three standard deviations from the mean as outliers, where standard deviation and mean are calculated ignoring already marked outliers. Features marked as outliers were set to non-informational values in the training procedure (c. f. section 


Finally, the features not marked as outliers of each subject were normalised by subtracting the mean and dividing by the standard deviation over trials.

2.5 Classification Procedure

A random forest classifier (RF) is used for decoding in this work [19]

. A RF combines multiple decision trees in an ensemble. The predicition is obtained by a voting scheme, i. e. a RF predicts the class most trees vote for. Every tree is grown on a random subset of training samples. At each tree node the best split, determined by the class separability, of a random selection of features is chosen. Thus, correlation between trees in a RF is reduced using bagging and random selection of features.

Using the features described in the previous section the following leave-one-subject-out cross-validation was employed. In contrast to the commonly employed within-subject cross-validation, this approach searched for features that generalize across subjects. In each fold, only the trials of 19 subjects were used to train a RF model [19] while the trials of the remaining subject were used as a test set. Values, that where previously marked as outliers, were set to non-informational values, i. e. to the mean of the particular feature in all training trials.

For each RF model 900 trees were grown using bagging, i. e. for every tree a training subset (63.2% of the whole training set) was drawn by sampling with replacement; for each tree the remaining training data is called out-of-bag (OOB). While growing the trees, at each node the best split of (as suggested in [20]

) randomly chosen features was used. For each RF model an OOB error estimate can be obtained by voting on the training data, while every tree is only allowed to vote on the corresponding OOB training data (cf. 


Each RF model was used to decode the target keys of the held-out subject and the prediction accuracy (PA) for each fold was calculated. In case of an ambiguous majority vote, a class with the maximum number of votes was assigned to this trial randomly.

The overall prediction accuracy of the cross-validation procedure was then estimated by dividing the total number of hits by the total number of predicted trials. A binomial test was employed to test the statistical significance of the cross-validated prediction result.

2.6 Importance Scores

For every RF model, feature importance scores were computed as described in [19], i. e. the average percentual increase of the OOB error estimate when permuting one feature along trials is the importance score of that particular feature. Averaging these scores across all 20 RF models gives an overall feature importance score (FIS) for every feature.

To obtain an importance score for each channel (channel importance score, CIS) the FISs were mapped to the corresponding channels and added up. For each channel two scores, CIS and CIS, were calculated by either only using the FISs of -band features or -band features, respectively.

To investigate dynamic changes in the importance of the -band and -band over the course of a trial, the FISs belonging to the channel with the highest channel importance score were used. This resulted in importance scores of the -band (WIS) and the -band (WIS) for the 41 time windows, and importance scores of the -band (IS) and the -band (IS) for the whole trial.

3 Results

Subject 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
PA in % 11.33 12.74 10.89 12.81 11.70 13.48 10.96 12.37 12.89 11.26 12.52 14.67  9.91 11.78 12.01 13.41 12.89 13.48 11.70 12.59
Table 1: Prediction accuracy (PA) for each subject of the leave-one-subject-out cross-validation.

Classification accuracies achieved for each subject are shown in table 1. In total, 3295 of 26819 trials were correctly predicted (12.29%, , ).

Figure 3: Confusion matrix showing the distribution of decoded positions for each actual target key. Values indicate percentage of trials. The colormap is adjusted so that green is indicating the chance level of 11.11%.
Figure 4: Topographic map of channel importance scores of the -band (CIS, cf. 2.6). Magenta dots denote EEG electrode positions. Maximum of the map is at C3.
Figure 5: Topographic map of channel importance scores of the -band (CIS, cf. 2.6). Magenta dots denote EEG electrode positions.
Figure 6: Importance scores of the 41 sliding windows obtained from the corresponding -band feature (WIS) and -band feature (WIS) importance scores; importance scores of the whole-trial -band feature (IS) and -band feature (IS) are shown as constant dashed lines; importance scores correspond to feature importance scores at C3.

The confusion matrix in figure 3 shows which keys were predicted for each actual target key: target 5 (16.71%) and target 9 (15.94%) were most accurately predicted. For most target keys a preference for predicting the trial as a target 5 or target 9 trial can be observed. Table 2 shows for each key the true positive rate, i. e. correctly decoding trials with that target key, and the true negative rate, i. e. correctly rejecting trials with other target keys as not belonging to that class.

The topographic map of CIS (cf. fig. 4) identifies channels on the contralateral sensorimotor cortex as most predictive ones. CIS for C3 exceeds the mean of CIS by more than seven standard deviations and is higher than all CIS and CIS. Therefore the importance scores WIS, WIS and IS, IS correspond to the FISs at C3 as described in section 2.6 and are shown in figure 6.

The first windows centered at 0.5 to 1 (which are the windows from 0–1 seconds to 1–2 seconds) have up to 15 times higher WIS than the later windows. IS is higher than the mean of all WIS and higher than IS.

In general the importance scores are smaller for the -band than for the -band; for comparison figures 4 and 5 show topographic maps of CIS and CIS on the same color scale. In addition to the contralateral channels, CIS shows predictive channels on the ipsilateral sensorimotor cortex (cf. fig. 5). CIS for C4 is more than four standard deviations higher than the mean. The whole trial score IS is much higher than all WIS, which in contrast to WIS do not show any prominent peak (cf. fig. 6).

Key 1 2 3 4 5 6 7 8 9
tp in % 10.94 9.80 10.40 11.14 16.71 12.01 13.22 10.40 15.94
tn in % 89.48 90.77 89.89 89.58 86.06 89.28 89.07 90.27 86.92
Table 2: True positive rate (tp) and true negative rate (tn) for each target key.

4 Discussion

In this paper, empirical evidence has been presented that discrete index finger positions are decodable from brain activity recorded by means of non-invasive EEG. -band features strongly focalized over the contralateral sensorimotor cortex during the first second of each trial were found to be most predictive.

During the task employed in this study, a stable force needed to be maintained to press down the current target key and keep the finger in a fixed posture, which involves continuous tonic muscle activity. Therefore, it is unclear if the brain activity related to position-dependent differences in tonic muscle activity was decoded, or if in fact the current index finger position itself was decoded. As the WIS showed a strong preference for the first second of a trial, it can be argued that it is not a continuous brain activation pattern that is being decoded, but rather that a spatial updating after a change in position, as manifested in a -rebound [21], provided information on the current target key. Thus, the decoding results described here on the level of finger movements may have a similar basis as previous findings on arm movements, as in both cases there was a prominent contribution from differences between a central resting position and the remaining peripheral positions [22]. Further investigation is necessary to differentiate between these explanations, e. g. by training models on selected subsets of features. In addition, these findings raise the question whether the dynamics of -activity or the -rebound enable position decoding of other body parts as well.

Interestingly, significant classification accuracy was achieved within a leave-one-subject-out setup using a random forest. This indicates that a pattern was successfully decoded that was consistent across subjects. Furthermore, no parameter tuning, e. g. optimizing the number of trees or the number of variables to select the best split at each node, was carried out. It is likely that prediction accuracies can be further increased by tuning these parameters or training models on a subset of most important features.

An interesting feature of RF models, being ensemble methods, is their ability to handle large feature spaces. This enabled the extension of the feature space to multiple time windows, providing interesting insights into the dynamics of brain rhythms associated with index finger positioning. To date, non-linear classifiers have not been shown to outperform linear methods in brain-state decoding [23]. It remains to be seen whether the non-linearity of RF models contributed to the present decoding results, or if similar results can also be achieved by linear (ensemble) methods.


  • [1] L. R. Hochberg, M. D. Serruya, G. M. Friehs, J. A. Mukand, M. Saleh, A. H. Caplan, A. Branner, D. Chen, R. D. Penn, and J. P. Donoghue, “Neuronal ensemble control of prosthetic devices by a human with tetraplegia,” Nature, vol. 442, no. 7099, pp. 164–171, 2006.
  • [2] L. R. Hochberg, D. Bacher, B. Jarosiewicz, N. Y. Masse, J. D. Simeral, J. Vogel, S. Haddadin, J. Liu, S. S. Cash, P. van der Smagt, et al., “Reach and grasp by people with tetraplegia using a neurally controlled robotic arm,” Nature, vol. 485, no. 7398, pp. 372–375, 2012.
  • [3] E. C. Leuthardt, G. Schalk, J. R. Wolpaw, J. G. Ojemann, and D. W. Moran, “A brain–computer interface using electrocorticographic signals in humans,” Journal of Neural Engineering, vol. 1, no. 2, pp. 63, 2004.
  • [4] T. Ball, M. Nawrot, T. Pistohl, A. Aertsen, A. Schulze-Bonhage, and C. Mehring, “Towards an implantable brain-machine interface based on epicortical field potentials,” Biomedizinische Technik, vol. 49, pp. 756–759, 2004.
  • [5] S. Zanos, K. J. Miller, and J. G. Ojemann, “Electrocorticographic spectral changes associated with ipsilateral individual finger and whole hand movement,” in Engineering in Medicine and Biology Society, 2008. EMBS 2008. 30th Annual International Conference of the IEEE. IEEE, 2008, pp. 5939–5942.
  • [6] K. J. Miller, S. Zanos, E. E. Fetz, M. Den Nijs, and J. G. Ojemann, “Decoupling the cortical power spectrum reveals real-time representation of individual finger movements in humans,” The Journal of Neuroscience, vol. 29, no. 10, pp. 3132–3137, 2009.
  • [7] S. Acharya, M. S. Fifer, H. L. Benz, N. E. Crone, and N. V. Thakor, “Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand,” Journal of Neural Engineering, vol. 7, no. 4, pp. 046002, 2010.
  • [8] J. Kubanek, K. J. Miller, J. G. Ojemann, J. R. Wolpaw, and G. Schalk, “Decoding flexion of individual fingers using electrocorticographic signals in humans,” Journal of Neural Engineering, vol. 6, no. 6, pp. 066001, 2009.
  • [9] Y. Li, X. Gao, H. Liu, and S. Gao, “Classification of single-trial electroencephalogram during finger movement,” Biomedical Engineering, IEEE Transactions on, vol. 51, no. 6, pp. 1019–1025, 2004.
  • [10] J. Lehtonen, P. Jylanki, L. Kauhanen, and M. Sams, “Online classification of single EEG trials during finger movements,” Biomedical Engineering, IEEE Transactions on, vol. 55, no. 2, pp. 713–720, 2008.
  • [11] L. Kauhanen, T. Nykopp, and M. Sams, “Classification of single MEG trials related to left and right index finger movements,” Clinical Neurophysiology, vol. 117, no. 2, pp. 430–439, 2006.
  • [12] A. P. Georgopoulos, F. J. Langheim, A. C. Leuthold, and A. N. Merkle, “Magnetoencephalographic signals predict movement trajectory in space,” Experimental Brain Research, vol. 167, no. 1, pp. 132–135, 2005.
  • [13] K. Jerbi, J.-P. Lachaux, N. Karim, D. Pantazis, R. M. Leahy, L. Garnero, S. Baillet, et al., “Coherent neural representation of hand speed in humans revealed by MEG imaging,” Proceedings of the National Academy of Sciences, vol. 104, no. 18, pp. 7676–7681, 2007.
  • [14] S. Waldert, H. Preissl, E. Demandt, C. Braun, N. Birbaumer, A. Aertsen, and C. Mehring, “Hand movement direction decoded from MEG and EEG,” The Journal of Neuroscience, vol. 28, no. 4, pp. 1000–1008, 2008.
  • [15] F. Quandt, C. Reichert, H. Hinrichs, H. J. Heinze, R. T. Knight, and J. W. Rieger, “Single trial discrimination of individual finger movements on one hand: A combined MEG and EEG study,” NeuroImage, vol. 59, no. 4, pp. 3316–3324, 2012.
  • [16] M. H. Schieber, “Constraints on somatotopic organization in the primary motor cortex,” Journal of Neurophysiology, vol. 86, no. 5, pp. 2125–2143, 2001.
  • [17] R. C. Oldfield, “The assessment and analysis of handedness: The Edinburgh inventory,” Neuropsychologia, vol. 9, no. 1, pp. 97–113, 1971.
  • [18] D. J. McFarland, L. M. McCane, S. V. David, and J. R. Wolpaw, “Spatial filter selection for EEG-based communication,” Electroencephalography and Clinical Neurophysiology, vol. 103, no. 3, pp. 386–394, 1997.
  • [19] L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001.
  • [20] L. Breiman, “Manual On Setting Up, Using, And Understanding Random Forests V3.1,” 2002,; accessed 2013-03-22.
  • [21] G. Pfurtscheller, A. Stancak Jr, and C. Neuper, “Post-movement beta synchronization. A correlate of an idling motor area?,” Electroencephalography and Clinical Neurophysiology, vol. 98, no. 4, pp. 281–293, 1996.
  • [22] E. Demandt, C. Mehring, K. Vogt, A. Schulze-Bonhage, A. Aertsen, and T. Ball, “Reaching movement onset-and end-related characteristics of EEG spectral power modulations,” Frontiers in Neuroscience, vol. 6, 2012.
  • [23] K.-R. Müller, C. W. Anderson, and G. E. Birch, “Linear and nonlinear methods for brain-computer interfaces,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 11, no. 2, pp. 165–169, 2003.