An Evaluation Framework for Legal Document Summarization

05/17/2022
by   Ankan Mullick, et al.
3

A law practitioner has to go through numerous lengthy legal case proceedings for their practices of various categories, such as land dispute, corruption, etc. Hence, it is important to summarize these documents, and ensure that summaries contain phrases with intent matching the category of the case. To the best of our knowledge, there is no evaluation metric that evaluates a summary based on its intent. We propose an automated intent-based summarization metric, which shows a better agreement with human evaluation as compared to other automated metrics like BLEU, ROUGE-L etc. in terms of human satisfaction. We also curate a dataset by annotating intent phrases in legal documents, and show a proof of concept as to how this system can be automated. Additionally, all the code and data to generate reproducible results is available on Github.

READ FULL TEXT
research
05/06/2022

Fine-grained Intent Classification in the Legal Domain

A law practitioner has to go through a lot of long legal case proceeding...
research
10/14/2022

Legal Case Document Summarization: Extractive and Abstractive Methods and their Evaluation

Summarization of legal case judgement documents is a challenging problem...
research
11/20/2015

Conducting sparse feature selection on arbitrarily long phrases in text corpora with a focus on interpretability

We propose a general framework for topic-specific summarization of large...
research
05/15/2023

Legal Extractive Summarization of U.S. Court Opinions

This paper tackles the task of legal extractive summarization using a da...
research
06/08/2021

Definitions of intent suitable for algorithms

Intent modifies an actor's culpability of many types wrongdoing. Autonom...
research
12/19/2022

What to Read in a Contract? Party-Specific Summarization of Important Obligations, Entitlements, and Prohibitions in Legal Documents

Legal contracts, such as employment or lease agreements, are important d...
research
06/02/2019

Plain English Summarization of Contracts

Unilateral contracts, such as terms of service, play a substantial role ...

Please sign up or login with your details

Forgot password? Click here to reset