RMITB at TREC COVID 2020

11/09/2020
by   Rodger Benham, et al.
0

Search engine users rarely express an information need using the same query, and small differences in queries can lead to very different result sets. These user query variations have been exploited in past TREC CORE tracks to contribute diverse, highly-effective runs in offline evaluation campaigns with the goal of producing reusable test collections. In this paper, we document the query fusion runs submitted to the first and second round of TREC COVID, using ten queries per topic created by the first author. In our analysis, we focus primarily on the effects of having our second priority run omitted from the judgment pool. This run is of particular interest, as it surfaced a number of relevant documents that were not judged until later rounds of the task. If the additional judgments were included in the first round, the performance of this run increased by 35 rank positions when using RBP p=0.5, highlighting the importance of judgment depth and coverage in assessment tasks.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

04/12/2012

Learning to Rank Query Recommendations by Semantic Similarities

Logs of the interactions with a search engine show that users often refo...
03/28/2018

On Learning Graphs with Edge-Detecting Queries

We consider the problem of learning a general graph G=(V,E) using edge-d...
07/14/2020

Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Dataset

We present Covidex, a search engine that exploits the latest neural rank...
07/12/2021

COPER: a Query-adaptable Semantics-based Search Engine for Persian COVID-19 Articles

With the surge of pretrained language models, a new pathway has been ope...
10/01/2020

RRF102: Meeting the TREC-COVID Challenge with a 100+ Runs Ensemble

In this paper, we report the results of our participation in the TREC-CO...
11/03/2020

Participation in TREC 2020 COVID Track Using Continuous Active Learning

We describe our participation in all five rounds of the TREC 2020 COVID ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.