repro_eval: A Python Interface to Reproducibility Measures of System-oriented IR Experiments

01/19/2022
by   Timo Breuer, et al.
0

In this work we introduce repro_eval - a tool for reactive reproducibility studies of system-oriented information retrieval (IR) experiments. The corresponding Python package provides IR researchers with measures for different levels of reproduction when evaluating their systems' outputs. By offering an easily extensible interface, we hope to stimulate common practices when conducting a reproducibility study of system-oriented IR experiments.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/26/2020

How to Measure the Reproducibility of System-oriented IR Experiments

Replicability and reproducibility of experimental results are primary co...
research
01/03/2017

Pyndri: a Python Interface to the Indri Search Engine

We introduce pyndri, a Python interface to the Indri search engine. Pynd...
research
03/03/2021

Simplified Data Wrangling with ir_datasets

Managing the data for Information Retrieval (IR) experiments can be chal...
research
11/26/2021

Streamlining Evaluation with ir-measures

We present ir-measures, a new tool that makes it convenient to calculate...
research
11/18/2022

Novelpy: A Python package to measure novelty and disruptiveness of bibliometric and patent data

Novelpy (v1.2) is an open-source Python package designed to compute bibl...
research
04/02/2023

An Intrinsic Framework of Information Retrieval Evaluation Measures

Information retrieval (IR) evaluation measures are cornerstones for dete...
research
08/21/2023

Evaluating Temporal Persistence Using Replicability Measures

In real-world Information Retrieval (IR) experiments, the Evaluation Env...

Please sign up or login with your details

Forgot password? Click here to reset