Improved Semantic Role Labeling using Parameterized Neighborhood Memory Adaptation

11/29/2020
by   Ishan Jindal, et al.
0

Deep neural models achieve some of the best results for semantic role labeling. Inspired by instance-based learning that utilizes nearest neighbors to handle low-frequency context-specific training samples, we investigate the use of memory adaptation techniques in deep neural models. We propose a parameterized neighborhood memory adaptive (PNMA) method that uses a parameterized representation of the nearest neighbors of tokens in a memory of activations and makes predictions based on the most similar samples in the training data. We empirically show that PNMA consistently improves the SRL performance of the base model irrespective of types of word embeddings. Coupled with contextualized word embeddings derived from BERT, PNMA improves over existing models for both span and dependency semantic parsing datasets, especially on out-of-domain text, reaching F1 scores of 80.2, and 84.97 on CoNLL2005, and CoNLL2009 datasets, respectively.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/30/2019

LSTM Easy-first Dependency Parsing with Pre-trained Word Embeddings and Character-level Word Embeddings in Vietnamese

In Vietnamese dependency parsing, several methods have been proposed. De...
research
10/19/2019

Keyphrase Extraction from Scholarly Articles as Sequence Labeling using Contextualized Embeddings

In this paper, we formulate keyphrase extraction from scholarly articles...
research
05/24/2016

Neural Semantic Role Labeling with Dependency Path Embeddings

This paper introduces a novel model for semantic role labeling that make...
research
05/23/2017

Second-Order Word Embeddings from Nearest Neighbor Topological Features

We introduce second-order vector representations of words, induced from ...
research
08/28/2018

Towards Semi-Supervised Learning for Deep Semantic Role Labeling

Neural models have shown several state-of-the-art performances on Semant...
research
09/28/2021

Instance-Based Neural Dependency Parsing

Interpretable rationales for model predictions are crucial in practical ...
research
05/01/2020

Structured Tuning for Semantic Role Labeling

Recent neural network-driven semantic role labeling (SRL) systems have s...

Please sign up or login with your details

Forgot password? Click here to reset