Identifying Products in Online Cybercrime Marketplaces: A Dataset for Fine-grained Domain Adaptation

by   Greg Durrett, et al.

One weakness of machine-learned NLP models is that they typically perform poorly on out-of-domain data. In this work, we study the task of identifying products being bought and sold in online cybercrime forums, which exhibits particularly challenging cross-domain effects. We formulate a task that represents a hybrid of slot-filling information extraction and named entity recognition and annotate data from four different forums. Each of these forums constitutes its own "fine-grained domain" in that the forums cover different market sectors with different properties, even though all forums are in the broad domain of cybercrime. We characterize these domain differences in the context of a learning-based system: supervised models see decreased accuracy when applied to new forums, and standard techniques for semi-supervised learning and domain adaptation have limited effectiveness on this data, which suggests the need to improve these techniques. We release a dataset of 1,938 annotated posts from across the four forums.



There are no comments yet.


page 1

page 2

page 3

page 4


Fine-grained Recognition in the Wild: A Multi-Task Domain Adaptation Approach

While fine-grained object recognition is an important problem in compute...

Fine-Grained Named Entity Recognition using ELMo and Wikidata

Fine-grained Named Entity Recognition is a task whereby we detect and cl...

Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling

As an essential task in task-oriented dialog systems, slot filling requi...

Crowdsourcing Learning as Domain Adaptation: A Case Study on Named Entity Recognition

Crowdsourcing is regarded as one prospective solution for effective supe...

Domain Adaptation and Active Learning for Fine-Grained Recognition in the Field of Biodiversity

Deep-learning methods offer unsurpassed recognition performance in a wid...

FDMT: A Benchmark Dataset for Fine-grained Domain Adaptation in Machine Translation

Previous domain adaptation research usually neglect the diversity in tra...

Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners

Unsupervised domain adaptation (UDA) and semi-supervised learning (SSL) ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.