SemEval-2017 Task 3: Community Question Answering

12/02/2019
by   Preslav Nakov, et al.
11

We describe SemEval-2017 Task 3 on Community Question Answering. This year, we reran the four subtasks from SemEval-2016:(A) Question-Comment Similarity,(B) Question-Question Similarity,(C) Question-External Comment Similarity, and (D) Rerank the correct answers for a new question in Arabic, providing all the data from 2015 and 2016 for training, and fresh data for testing. Additionally, we added a new subtask E in order to enable experimentation with Multi-domain Question Duplicate Detection in a larger-scale scenario, using StackExchange subforums. A total of 23 teams participated in the task, and submitted a total of 85 runs (36 primary and 49 contrastive) for subtasks A-D. Unfortunately, no teams participated in subtask E. A variety of approaches and features were used by the participating systems to address the different subtasks. The best systems achieved an official score (MAP) of 88.43, 47.22, 15.46, and 61.16 in subtasks A, B, C, and D, respectively. These scores are better than the baselines, especially for subtasks A-C.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

12/03/2019

SemEval-2016 Task 3: Community Question Answering

This paper describes the SemEval–2016 Task 3 on Community Question Answe...
11/26/2019

SemEval-2015 Task 3: Answer Selection in Community Question Answering

Community Question Answering (cQA) provides new interesting research dir...
09/08/2021

Cross-Policy Compliance Detection via Question Answering

Policy compliance detection is the task of ensuring that a scenario conf...
06/28/2021

Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

Advancing the state-of-the-art in large-scale biomedical semantic indexi...
06/16/2020

Results of the seventh edition of the BioASQ Challenge

The results of the seventh edition of the BioASQ challenge are presented...
09/12/2019

NSURL-2019 Shared Task 8: Semantic Question Similarity in Arabic

Question semantic similarity (Q2Q) is a challenging task that is very us...
06/28/2021

Overview of BioASQ 2020: The eighth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

In this paper, we present an overview of the eighth edition of the BioAS...

Code Repositories

nlp_tasks

Natural Language Processing Tasks and References


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.