BERT for Long Documents: A Case Study of Automated ICD Coding

11/04/2022

∙

Transformer models have achieved great success across many NLP problems. However, previous studies in automated ICD coding concluded that these models fail to outperform some of the earlier solutions such as CNN-based models. In this paper we challenge this conclusion. We present a simple and scalable method to process long text with the existing transformer models such as BERT. We show that this method significantly improves the previous results reported for transformer models in ICD coding, and is able to outperform one of the prominent CNN-based methods.

READ FULL TEXT

BERT for Long Documents: A Case Study of Automated ICD Coding

Evaluating Transformer based Semantic Segmentation Networks for Pathological Image Segmentation

Towards BERT-based Automatic ICD Coding: Limitations and Opportunities

Introducing Vision Transformer for Alzheimer's Disease classification task with 3D input

Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Models

Transformer-based Models for Long-Form Document Matching: Challenges and Empirical Analysis

Techniques to Improve Q A Accuracy with Transformer-based models on Large Complex Documents

Transformer models: an introduction and catalog

BERT for Long Documents: A Case Study of Automated ICD Coding

Related Research

Evaluating Transformer based Semantic Segmentation Networks for Pathological Image Segmentation

Towards BERT-based Automatic ICD Coding: Limitations and Opportunities

Introducing Vision Transformer for Alzheimer's Disease classification task with 3D input

Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Models

Transformer-based Models for Long-Form Document Matching: Challenges and Empirical Analysis

Techniques to Improve Q A Accuracy with Transformer-based models on Large Complex Documents

Transformer models: an introduction and catalog