Clinical Language Understanding Evaluation (CLUE)

09/28/2022
by   Travis R. Goodwin, et al.
0

Clinical language processing has received a lot of attention in recent years, resulting in new models or methods for disease phenotyping, mortality prediction, and other tasks. Unfortunately, many of these approaches are tested under different experimental settings (e.g., data sources, training and testing splits, metrics, evaluation criteria, etc.) making it difficult to compare approaches and determine state-of-the-art. To address these issues and facilitate reproducibility and comparison, we present the Clinical Language Understanding Evaluation (CLUE) benchmark with a set of four clinical language understanding tasks, standard training, development, validation and testing sets derived from MIMIC data, as well as a software toolkit. It is our hope that these data will enable direct comparison between approaches, improve reproducibility, and reduce the barrier-to-entry for developing novel models or methods for these clinical language understanding tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/03/2021

The Catalan Language CLUB

The Catalan Language Understanding Benchmark (CLUB) encompasses various ...
research
06/13/2019

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Inspired by the success of the General Language Understanding Evaluation...
research
12/30/2020

Robustness Testing of Language Understanding in Dialog Systems

Most language understanding models in dialog systems are trained on a sm...
research
06/15/2021

CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark

Artificial Intelligence (AI), along with the recent progress in biomedic...
research
09/27/2021

FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding

The few-shot natural language understanding (NLU) task has attracted muc...
research
10/30/2018

Spoken Language Understanding on the Edge

We consider the problem of performing Spoken Language Understanding (SLU...
research
07/27/2020

EffiCare: Better Prognostic Models via Resource-Efficient Health Embeddings

Recent medical prognostic models adapted from high data-resource fields ...

Please sign up or login with your details

Forgot password? Click here to reset