DeepAI AI Chat
Log In Sign Up

Unsupervised Domain Adaptation of Contextual Embeddings for Low-Resource Duplicate Question Detection

by   Alexandre Rochette, et al.

Answering questions is a primary goal of many conversational systems or search products. While most current systems have focused on answering questions against structured databases or curated knowledge graphs, on-line community forums or frequently asked questions (FAQ) lists offer an alternative source of information for question answering systems. Automatic duplicate question detection (DQD) is the key technology need for question answering systems to utilize existing online forums like StackExchange. Existing annotations of duplicate questions in such forums are community-driven, making them sparse or even completely missing for many domains. Therefore, it is important to transfer knowledge from related domains and tasks. Recently, contextual embedding models such as BERT have been outperforming many baselines by transferring self-supervised information to downstream tasks. In this paper, we apply BERT to DQD and advance it by unsupervised adaptation to StackExchange domains using self-supervised learning. We show the effectiveness of this adaptation for low-resource settings, where little or no training data is available from the target domain. Our analysis reveals that unsupervised BERT domain adaptation on even small amounts of data boosts the performance of BERT.


page 1

page 2

page 3

page 4


Domain Adaptation for Question Answering via Question Classification

Question answering (QA) has demonstrated impressive progress in answerin...

Abductive Reasoning as Self-Supervision for Common Sense Question Answering

Question answering has seen significant advances in recent times, especi...

Towards Domain Adaptation from Limited Data for Question Answering Using Deep Neural Networks

This paper explores domain adaptation for enabling question answering (Q...

Adversarial Domain Adaptation for Duplicate Question Detection

We address the problem of detecting duplicate questions in forums, which...

QA Domain Adaptation using Hidden Space Augmentation and Self-Supervised Contrastive Adaptation

Question answering (QA) has recently shown impressive results for answer...

Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Survey

Dense retrieval (DR) approaches based on powerful pre-trained language m...