Technology Assisted Reviews: Finding the Last Few Relevant Documents by Asking Yes/No Questions to Reviewers

10/12/2018
by   Jie Zou, et al.
0

The goal of a technology-assisted review is to achieve high recall with low human effort. Continuous active learning algorithms have demonstrated good performance in locating the majority of relevant documents in a collection, however their performance is reaching a plateau when 80%-90% of them has been found. Finding the last few relevant documents typically requires exhaustively reviewing the collection. In this paper, we propose a novel method to identify these last few, but significant, documents efficiently. Our method makes the hypothesis that entities carry vital information in documents, and that reviewers can answer questions about the presence or absence of an entity in the missing relevance documents. Based on this we devise a sequential Bayesian search method that selects the optimal sequence of questions to ask. The experimental results show that our proposed method can greatly improve performance requiring less reviewing effort.

READ FULL TEXT
research
08/30/2019

Learning to Ask: Question-based Sequential Bayesian Product Search

Product search is generally recognized as the first and foremost stage o...
research
01/28/2022

Probably Reasonable Search in eDiscovery

In eDiscovery, a party to a lawsuit or similar action must search throug...
research
06/18/2021

Heuristic Stopping Rules For Technology-Assisted Review

Technology-assisted review (TAR) refers to human-in-the-loop active lear...
research
07/30/2020

Is there something I'm missing? Topic Modeling in eDiscovery

In legal eDiscovery, the parties are required to search through their el...
research
03/23/2018

Evaluating Sentence-Level Relevance Feedback for High-Recall Information Retrieval

This study uses a novel simulation framework to evaluate whether the tim...
research
11/28/2017

Semantic Technology-Assisted Review (STAR) Document analysis and monitoring using random vectors

The review and analysis of large collections of documents and the period...
research
09/16/2021

FOMO: Topics versus documents in legal eDiscovery

In the United States, the parties to a lawsuit are required to search th...

Please sign up or login with your details

Forgot password? Click here to reset