DeepAI AI Chat
Log In Sign Up

BertGCN: Transductive Text Classification by Combining GCN and BERT

by   Yuxiao Lin, et al.

In this work, we propose BertGCN, a model that combines large scale pretraining and transductive learning for text classification. BertGCN constructs a heterogeneous graph over the dataset and represents documents as nodes using BERT representations. By jointly training the BERT and GCN modules within BertGCN, the proposed model is able to leverage the advantages of both worlds: large-scale pretraining which takes the advantage of the massive amount of raw data and transductive learning which jointly learns representations for both training data and unlabeled test data by propagating label influence through graph convolution. Experiments show that BertGCN achieves SOTA performances on a wide range of text classification datasets. Code is available at


page 1

page 2

page 3

page 4


VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification

Much progress has been made recently on text classification with methods...

InducT-GCN: Inductive Graph Convolutional Networks for Text Classification

Text classification aims to assign labels to textual units by making use...

Understanding Graph Convolutional Networks for Text Classification

Graph Convolutional Networks (GCN) have been effective at tasks that hav...

ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information

Recent pretraining models in Chinese neglect two important aspects speci...

When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing

The rapid development of quantum computing has demonstrated many unique ...

Domain-Adaptive Text Classification with Structured Knowledge from Unlabeled Data

Domain adaptive text classification is a challenging problem for the lar...

SSMix: Saliency-Based Span Mixup for Text Classification

Data augmentation with mixup has shown to be effective on various comput...