An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text

05/09/2023
by   Yova Kementchedjhieva, et al.
0

Standard methods for multi-label text classification largely rely on encoder-only pre-trained language models, whereas encoder-decoder models have proven more effective in other classification tasks. In this study, we compare four methods for multi-label classification, two based on an encoder only, and two based on an encoder-decoder. We carry out experiments on four datasets – two in the legal domain and two in the biomedical domain, each with two levels of label granularity – and always depart from the same pre-trained model, T5. Our results show that encoder-decoder methods outperform encoder-only methods, with a growing advantage on more complex datasets and labeling schemes of finer granularity. Using encoder-decoder models in a non-autoregressive fashion, in particular, yields the best performance overall, so we further study this approach through ablations to better understand its strengths.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset