Predicting Triple Scoring with Crowdsourcing-specific Features - The fiddlehead Triple Scorer at WSDM Cup 2017

12/22/2017
by   Masahiro Sato, et al.
0

The Triple Scoring Task at the WSDM Cup 2017 involves the prediction of the relevance scores between persons and professions/nationalities. The ground truth of the relevance scores was obtained by counting the vote of seven crowdworkers. I confirmed that features related to task difficulty correlate with the discrepancy among crowdworkers' judgement. This means such features are useful for predicting whether a score is in the middle or not. Hence, the features were incorporated into the prediction model of the crowdsourced relevance scores. The introduced features improve the average score difference of the prediction. The final ranking of my prediction was 4th for average score difference and 12th for both accuracy and Kendall's tau.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/21/2017

Overview of the Triple Scoring Task at the WSDM Cup 2017

This paper provides an overview of the triple scoring task at the WSDM C...
research
12/22/2017

Relevance Scoring of Triples Using Ordinal Logistic Classification - The Celosia Triple Scorer at WSDM Cup 2017

In this paper, we report our participation in the Task 2: Triple Scoring...
research
12/22/2017

Predicting Relevance Scores for Triples from Type-Like Relations using Neural Embedding - The Cabbage Triple Scorer at WSDM Cup 2017

The WSDM Cup 2017 Triple scoring challenge is aimed at calculating and a...
research
11/18/2017

Prediction Scores as a Window into Classifier Behavior

Most multi-class classifiers make their prediction for a test sample by ...
research
05/23/2017

How hard can it be? Estimating the difficulty of visual search in an image

We address the problem of estimating image difficulty defined as the hum...
research
12/22/2017

Triple Scoring Using Paragraph Vector - The Gailan Triple Scorer at WSDM Cup 2017

In this paper we describe our solution to the WSDM Cup 2017 Triple Scori...

Please sign up or login with your details

Forgot password? Click here to reset