DeepAI AI Chat
Log In Sign Up

On the Branching Bias of Syntax Extracted from Pre-trained Language Models

by   Huayang Li, et al.

Many efforts have been devoted to extracting constituency trees from pre-trained language models, often proceeding in two stages: feature definition and parsing. However, this kind of methods may suffer from the branching bias issue, which will inflate the performances on languages with the same branch it biases to. In this work, we propose quantitatively measuring the branching bias by comparing the performance gap on a language and its reversed language, which is agnostic to both language models and extracting methods. Furthermore, we analyze the impacts of three factors on the branching bias, namely parsing algorithms, feature definitions, and language models. Experiments show that several existing works exhibit branching biases, and some implementations of these three factors can introduce the branching bias.


page 1

page 2

page 3

page 4


Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Models

Numerous works have analyzed biases in vision and pre-trained language m...

Identifying and Measuring Token-Level Sentiment Bias in Pre-trained Language Models with Prompts

Due to the superior performance, large-scale pre-trained language models...

Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers

Large pre-trained language models have shown remarkable performance over...

Inducing anxiety in large language models increases exploration and bias

Large language models are transforming research on machine learning whil...

A Disability Lens towards Biases in GPT-3 Generated Open-Ended Languages

Language models (LM) are becoming prevalent in many language-based appli...

HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Models

Fairness has become a trending topic in natural language processing (NLP...