Dependency Learning for Legal Judgment Prediction with a Unified Text-to-Text Transformer

12/13/2021
by   Yunyun Huang, et al.
0

Given the fact of a case, Legal Judgment Prediction (LJP) involves a series of sub-tasks such as predicting violated law articles, charges and term of penalty. We propose leveraging a unified text-to-text Transformer for LJP, where the dependencies among sub-tasks can be naturally established within the auto-regressive decoder. Compared with previous works, it has three advantages: (1) it fits in the pretraining pattern of masked language models, and thereby can benefit from the semantic prompts of each sub-task rather than treating them as atomic labels, (2) it utilizes a single unified architecture, enabling full parameter sharing across all sub-tasks, and (3) it can incorporate both classification and generative sub-tasks. We show that this unified transformer, albeit pretrained on general-domain text, outperforms pretrained models tailored specifically for the legal domain. Through an extensive set of experiments, we find that the best order to capture dependencies is different from human intuitions, and the most reasonable logical order for humans can be sub-optimal for the model. We further include two more auxiliary tasks: court view generation and article content prediction, showing they can not only improve the prediction accuracy, but also provide interpretable explanations for model outputs even when an error is made. With the best configuration, our model outperforms both previous SOTA and a single-tasked version of the unified transformer by a large margin.

READ FULL TEXT
research
02/13/2022

Transformer-based Approaches for Legal Text Processing

In this paper, we introduce our approaches using Transformer-based model...
research
04/15/2021

Sublanguage: A Serious Issue Affects Pretrained Models in Legal Domain

Legal English is a sublanguage that is important for everyone but not fo...
research
11/15/2022

Exploiting Contrastive Learning and Numerical Evidence for Improving Confusing Legal Judgment Prediction

Given the fact description text of a legal case, legal judgment predicti...
research
08/31/2021

Sentence Bottleneck Autoencoders from Transformer Language Models

Representation learning for text via pretraining a language model on a l...
research
04/18/2021

When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset

While self-supervised learning has made rapid advances in natural langua...
research
10/09/2021

Vector-quantized Image Modeling with Improved VQGAN

Pretraining language models with next-token prediction on massive text c...
research
05/10/2019

Legal Judgment Prediction via Multi-Perspective Bi-Feedback Network

The Legal Judgment Prediction (LJP) is to determine judgment results bas...

Please sign up or login with your details

Forgot password? Click here to reset