Slice-Aware Neural Ranking

10/07/2020
by   Gustavo Penha, et al.
0

Understanding when and why neural ranking models fail for an IR task via error analysis is an important part of the research cycle. Here we focus on the challenges of (i) identifying categories of difficult instances (a pair of question and response candidates) for which a neural ranker is ineffective and (ii) improving neural ranking for such instances. To address both challenges we resort to slice-based learning for which the goal is to improve effectiveness of neural models for slices (subsets) of data. We address challenge (i) by proposing different slicing functions (SFs) that select slices of the dataset—based on prior work we heuristically capture different failures of neural rankers. Then, for challenge (ii) we adapt a neural ranking model to learn slice-aware representations, i.e. the adapted model learns to represent the question and responses differently based on the model's prediction of which slices they belong to. Our experimental results (the source code and data are available at https://github.com/Guzpenha/slice_based_learning) across three different ranking tasks and four corpora show that slice-based learning improves the effectiveness by an average of 2 slice-aware.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/04/2021

Learning Slice-Aware Representations with Mixture of Attentions

Real-world machine learning systems are achieving remarkable performance...
research
09/01/2023

On the Localization of Ultrasound Image Slices within Point Distribution Models

Thyroid disorders are most commonly diagnosed using high-resolution Ultr...
research
09/13/2019

Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices

In real-world machine learning applications, data subsets correspond to ...
research
12/18/2019

Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking

Neural ranking models are traditionally trained on a series of random ba...
research
04/26/2021

Handling Long-Tail Queries with Slice-Aware Conversational Systems

We have been witnessing the usefulness of conversational AI systems such...
research
05/05/2020

AlignShift: Bridging the Gap of Imaging Thickness in 3D Anisotropic Volumes

This paper addresses a fundamental challenge in 3D medical image process...
research
03/13/2019

SymPas: Symbolic Program Slicing

Program slicing is a technique for simplifying programs by focusing on s...

Please sign up or login with your details

Forgot password? Click here to reset