Does the Magic of BERT Apply to Medical Code Assignment? A Quantitative Study

03/11/2021
by   Shaoxiong Ji, et al.
27

Unsupervised pretraining is an integral part of many natural language processing systems, and transfer learning with language models has achieved remarkable results in many downstream tasks. In the clinical application of medical code assignment, diagnosis and procedure codes are inferred from lengthy clinical notes such as hospital discharge summaries. However, it is not clear if pretrained models are useful for medical code prediction without further architecture engineering. This paper conducts a comprehensive quantitative analysis of various contextualized language models' performance, pretrained in different domains, for medical code assignment from clinical notes. We propose a hierarchical fine-tuning architecture to capture interactions between distant words and adopt label-wise attention to exploit label information. Contrary to current trends, we demonstrate that a carefully trained classical CNN outperforms attention-based models on a MIMIC-III subset with frequent codes. Our empirical findings suggest directions for improving the medical code assignment application.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/17/2020

Multi-label natural language processing to identify diagnosis and procedure codes from MIMIC-III inpatient notes

In the United States, 25 spending accounts for administrative costs that...
research
09/30/2020

Dilated Convolutional Attention Network for Medical Code Assignment from Clinical Text

Medical code assignment, which predicts medical codes from clinical text...
research
05/09/2023

Effective Medical Code Prediction via Label Internal Alignment

The clinical notes are usually typed into the system by physicians. They...
research
10/29/2020

Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisation

Diagnostic or procedural coding of clinical notes aims to derive a coded...
research
03/04/2022

AutoMap: Automatic Medical Code Mapping for Clinical Prediction Model Deployment

Given a deep learning model trained on data from a source site, how to d...
research
10/14/2020

Medical Code Assignment with Gated Convolution and Note-Code Interaction

Medical code assignment from clinical text is a fundamental task in clin...
research
07/01/2023

Hierarchical Pretraining for Biomedical Term Embeddings

Electronic health records (EHR) contain narrative notes that provide ext...

Please sign up or login with your details

Forgot password? Click here to reset