ExaRanker: Explanation-Augmented Neural Ranker

01/25/2023
by   Fernando Ferraretto, et al.
0

Recent work has shown that inducing a large language model (LLM) to generate explanations prior to outputting an answer is an effective strategy to improve performance on a wide range of reasoning tasks. In this work, we show that neural rankers also benefit from explanations. We use LLMs such as GPT-3.5 to augment retrieval datasets with explanations and train a sequence-to-sequence ranking model to output a relevance label and an explanation for a given query-document pair. Our model, dubbed ExaRanker, finetuned on a few thousand examples with synthetic explanations performs on par with models finetuned on 3x more examples without explanations. Furthermore, the ExaRanker model incurs no additional computational cost during ranking and allows explanations to be requested on demand.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/04/2019

Learning to Explain: Answering Why-Questions via Rephrasing

Providing plausible responses to why questions is a challenging but crit...
research
11/25/2022

Complementary Explanations for Effective In-Context Learning

Large language models (LLMs) have exhibited remarkable capabilities in l...
research
09/07/2021

On the Challenges of Evaluating Compositional Explanations in Multi-Hop Inference: Relevance, Completeness, and Expert Ratings

Building compositional explanations requires models to combine two or mo...
research
02/03/2021

When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data

Many methods now exist for conditioning model outputs on task instructio...
research
11/17/2017

Grounding Visual Explanations (Extended Abstract)

Existing models which generate textual explanations enforce task relevan...
research
04/28/2023

Interpreting Vision and Language Generative Models with Semantic Visual Priors

When applied to Image-to-text models, interpretability methods often pro...
research
02/04/2014

Reasoning about Explanations for Negative Query Answers in DL-Lite

In order to meet usability requirements, most logic-based applications p...

Please sign up or login with your details

Forgot password? Click here to reset