Reproducing Personalised Session Search over the AOL Query Log

by   Sean MacAvaney, et al.

Despite its troubled past, the AOL Query Log continues to be an important resource to the research community – particularly for tasks like search personalisation. When using the query log these ranking experiments, little attention is usually paid to the document corpus. Recent work typically uses a corpus containing versions of the documents collected long after the log was produced. Given that web documents are prone to change over time, we study the differences present between a version of the corpus containing documents as they appeared in 2017 (which has been used by several recent works) and a new version we construct that includes documents close to as they appeared at the time the query log was produced (2006). We demonstrate that this new version of the corpus has a far higher coverage of documents present in the original log (93 often differs substantially. Given these differences, we re-conduct session search experiments that originally used the 2017 corpus and find that when using our corpus for training or evaluation, system performance improves. We place the results in context by introducing recent adhoc ranking baselines. We also confirm the navigational nature of the queries in the AOL corpus by showing that including the URL substantially improves performance across a variety of models. Our version of the corpus can be easily reconstructed by other researchers and is included in the ir-datasets package.


page 1

page 2

page 3

page 4


Driving the Herd: Search Engines as Content Influencers

In competitive search settings such as the Web, many documents' authors ...

CODEC: Complex Document and Entity Collection

CODEC is a document and entity ranking benchmark that focuses on complex...

TripClick: The Log Files of a Large Health Web Search Engine

Click logs are valuable resources for a variety of information retrieval...

Interactive Query Clarification and Refinement via User Simulation

When users initiate search sessions, their queries are often unclear or ...

ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search

Users of Web search engines reveal their information needs through queri...

Using Image Captions and Multitask Learning for Recommending Query Reformulations

Interactive search sessions often contain multiple queries, where the us...

Please sign up or login with your details

Forgot password? Click here to reset