Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition

by   Ninareh Mehrabi, et al.

We study the bias in several state-of-the-art named entity recognition (NER) models—specifically, a difference in the ability to recognize male and female names as PERSON entity types. We evaluate NER models on a dataset containing 139 years of U.S. census baby names and find that relatively more female names, as opposed to male names, are not recognized as PERSON entities. We study the extent of this bias in several NER systems that are used prominently in industry and academia. In addition, we also report a bias in the datasets on which these models were trained. The result of this analysis yields a new benchmark for gender bias evaluation in named entity recognition systems. The data and code for the application of this benchmark will be publicly available for researchers to use.



There are no comments yet.


page 1

page 2

page 3

page 4


Assessing Demographic Bias in Named Entity Recognition

Named Entity Recognition (NER) is often the first step towards automated...

Dutch Named Entity Recognition and De-identification Methods for the Human Resource Domain

The human resource (HR) domain contains various types of privacy-sensiti...

Knowledge-Augmented Language Model and its Application to Unsupervised Named-Entity Recognition

Traditional language models are unable to efficiently model entity names...

MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective

NER model has achieved promising performance on standard NER benchmarks....

Exploiting Lists of Names for Named Entity Identification of Financial Institutions from Unstructured Documents

There is a wealth of information about financial systems that is embedde...

A Realistic Study of Auto-regressive Language Models for Named Entity Typing and Recognition

Despite impressive results of language models for named entity recogniti...

Named Entity Recognition with stack residual LSTM and trainable bias decoding

Recurrent Neural Network models are the state-of-the-art for Named Entit...

Code Repositories

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.