Passing the Brazilian OAB Exam: data preparation and some experiments

12/14/2017
by   Pedro Delfino, et al.
0

In Brazil, all legal professionals must demonstrate their knowledge of the law and its application by passing the OAB exams, the national bar exams. The OAB exams therefore provide an excellent benchmark for the performance of legal information systems since passing the exam would arguably signal that the system has acquired capacity of legal reasoning comparable to that of a human lawyer. This article describes the construction of a new data set and some preliminary experiments on it, treating the problem of finding the justification for the answers to questions. The results provide a baseline performance measure against which to evaluate future improvements. We discuss the reasons to the poor performance and propose next steps.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/12/2023

FALQU: Finding Answers to Legal Questions

This paper presents a new test collection for Legal IR, FALQU: Finding A...
research
02/11/2023

A Brief Report on LawGPT 1.0: A Virtual Legal Assistant Based on GPT-3

LawGPT 1.0 is a virtual legal assistant built on the state-of-the-art la...
research
08/20/2023

LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models

The advent of large language models (LLMs) and their adoption by the leg...
research
08/26/2021

A Statutory Article Retrieval Dataset in French

Statutory article retrieval is the task of automatically retrieving law ...
research
09/21/2020

The Next Era of American Law Amid the Advent of Autonomous AI Legal Reasoning

Legal scholars have postulated that there have been three eras of Americ...
research
11/24/2011

The Network of French Legal Codes

We propose an analysis of the codified Law of France as a structured sys...
research
01/17/2022

Data-Centric Machine Learning in the Legal Domain

Machine learning research typically starts with a fixed data set created...

Please sign up or login with your details

Forgot password? Click here to reset