Rethinking Search: Making Experts out of Dilettantes

by   Donald Metzler, et al.

When experiencing an information need, users want to engage with an expert, but often turn to an information retrieval system, such as a search engine, instead. Classical information retrieval systems do not answer information needs directly, but instead provide references to (hopefully authoritative) answers. Successful question answering systems offer a limited corpus created on-demand by human experts, which is neither timely nor scalable. Large pre-trained language models, by contrast, are capable of directly generating prose that may be responsive to an information need, but at present they are dilettantes rather than experts - they do not have a true understanding of the world, they are prone to hallucinating, and crucially they are incapable of justifying their utterances by referring to supporting documents in the corpus they were trained over. This paper examines how ideas from classical information retrieval and large pre-trained language models can be synthesized and evolved into systems that truly deliver on the promise of expert advice.



There are no comments yet.


page 1

page 2

page 3

page 4


On-The-Fly Information Retrieval Augmentation for Language Models

Here we experiment with the use of information retrieval as an augmentat...

How Context Affects Language Models' Factual Predictions

When pre-trained on large unsupervised textual corpora, language models ...

ReQA: An Evaluation for End-to-End Answer Retrieval Models

Popular QA benchmarks like SQuAD have driven progress on the task of ide...

Answer Interaction in Non-factoid Question Answering Systems

Information retrieval systems are evolving from document retrieval to an...

Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering

Open Domain Question Answering (ODQA) on a large-scale corpus of documen...

Patapasco: A Python Framework for Cross-Language Information Retrieval Experiments

While there are high-quality software frameworks for information retriev...

Measuring Conversational Productivity in Child Forensic Interviews

Child Forensic Interviewing (FI) presents a challenge for effective info...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.