Structured Prediction of Sequences and Trees using Infinite Contexts

03/09/2015
by   Ehsan Shareghi, et al.
0

Linguistic structures exhibit a rich array of global phenomena, however commonly used Markov models are unable to adequately describe these phenomena due to their strong locality assumptions. We propose a novel hierarchical model for structured prediction over sequences and trees which exploits global context by conditioning each generation decision on an unbounded context of prior decisions. This builds on the success of Markov models but without imposing a fixed bound in order to better represent global phenomena. To facilitate learning of this large and unbounded model, we use a hierarchical Pitman-Yor process prior which provides a recursive form of smoothing. We propose prediction algorithms based on A* and Markov Chain Monte Carlo sampling. Empirical results demonstrate the potential of our model compared to baseline finite-context Markov models on part-of-speech tagging and syntactic parsing.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/13/2016

Ensemble preconditioning for Markov chain Monte Carlo simulation

We describe parallel Markov chain Monte Carlo methods that propagate a c...
research
04/11/2020

Construction and Random Generation of Hypergraphs with Prescribed Degree and Dimension Sequences

We propose algorithms for construction and random generation of hypergra...
research
10/05/2020

Unbounded Slice Sampling

Slice sampling is an efficient Markov Chain Monte Carlo algorithm to sam...
research
07/15/2013

Bayesian Structured Prediction Using Gaussian Processes

We introduce a conceptually novel structured prediction model, GPstruct,...
research
11/08/2018

Variational Bayesian hierarchical regression for data analysis

Collected data, which is used for analysis or prediction tasks, often ha...
research
07/23/2021

A hierarchical prior for generalized linear models based on predictions for the mean response

There has been increased interest in using prior information in statisti...
research
06/30/2011

On Prediction Using Variable Order Markov Models

This paper is concerned with algorithms for prediction of discrete seque...

Please sign up or login with your details

Forgot password? Click here to reset