ECGBERT: Understanding Hidden Language of ECGs with Self-Supervised Representation Learning

06/10/2023
by   Seokmin Choi, et al.
0

In the medical field, current ECG signal analysis approaches rely on supervised deep neural networks trained for specific tasks that require substantial amounts of labeled data. However, our paper introduces ECGBERT, a self-supervised representation learning approach that unlocks the underlying language of ECGs. By unsupervised pre-training of the model, we mitigate challenges posed by the lack of well-labeled and curated medical data. ECGBERT, inspired by advances in the area of natural language processing and large language models, can be fine-tuned with minimal additional layers for various ECG-based problems. Through four tasks, including Atrial Fibrillation arrhythmia detection, heartbeat classification, sleep apnea detection, and user authentication, we demonstrate ECGBERT's potential to achieve state-of-the-art results on a wide variety of tasks.

READ FULL TEXT
research
03/23/2021

Self-supervised representation learning from 12-lead ECG data

We put forward a comprehensive assessment of self-supervised representat...
research
11/05/2022

Small Language Models for Tabular Data

Supervised deep learning is most commonly applied to difficult problems ...
research
08/28/2023

Diversified Ensemble of Independent Sub-Networks for Robust Self-Supervised Representation Learning

Ensembling a neural network is a widely recognized approach to enhance m...
research
06/15/2021

Multivariate Business Process Representation Learning utilizing Gramian Angular Fields and Convolutional Neural Networks

Learning meaningful representations of data is an important aspect of ma...
research
04/13/2023

In-Distribution and Out-of-Distribution Self-supervised ECG Representation Learning for Arrhythmia Detection

This paper presents a systematic investigation into the effectiveness of...
research
11/15/2022

Pretraining ECG Data with Adversarial Masking Improves Model Generalizability for Data-Scarce Tasks

Medical datasets often face the problem of data scarcity, as ground trut...
research
01/03/2020

On the comparability of Pre-trained Language Models

Recent developments in unsupervised representation learning have success...

Please sign up or login with your details

Forgot password? Click here to reset