Decoding ECoG signal into 3D hand translation using deep learning

10/05/2021
by   Maciej Śliwowski, et al.
12

Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship. In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron (MLP), convolutional neural networks (CNN), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity. Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60 for the left and right hand, respectively.

READ FULL TEXT

page 4

page 6

page 8

research
10/05/2022

Deep learning for ECoG brain-computer interface: end-to-end vs. hand-crafted features

In brain signal processing, deep learning (DL) models have become common...
research
10/23/2018

Using Deep Learning for price prediction by exploiting stationary limit order book features

The recent surge in Deep Learning (DL) research of the past decade has s...
research
09/08/2022

Impact of dataset size and long-term ECoG-based BCI usage on deep learning decoders performance

In brain-computer interfaces (BCI) research, recording data is time-cons...
research
07/12/2020

Deep Learning modeling of Limit Order Book: a comparative perspective

The present work addresses theoretical and practical questions in the do...
research
06/24/2022

An Intensity and Phase Stacked Analysis of Phase-OTDR System using Deep Transfer Learning and Recurrent Neural Networks

Distributed acoustic sensors (DAS) are effective apparatus which are wid...
research
09/14/2017

Deep Learning for Automatic Stereotypical Motor Movement Detection using Wearable Sensors in Autism Spectrum Disorders

Autism Spectrum Disorders are associated with atypical movements, of whi...
research
03/09/2021

HemCNN: Deep Learning enables decoding of fNIRS cortical signals in hand grip motor tasks

We solve the fNIRS left/right hand force decoding problem using a data-d...

Please sign up or login with your details

Forgot password? Click here to reset