DeepAI AI Chat
Log In Sign Up

Learning with Latent Structures in Natural Language Processing: A Survey

by   Zhaofeng Wu, et al.
University of Washington

While end-to-end learning with fully differentiable models has enabled tremendous success in natural language process (NLP) and machine learning, there have been significant recent interests in learning with latent discrete structures to incorporate better inductive biases for improved end-task performance and better interpretability. This paradigm, however, is not straightforwardly amenable to the mainstream gradient-based optimization methods. This work surveys three main families of methods to learn such models: surrogate gradients, continuous relaxation, and marginal likelihood maximization via sampling. We conclude with a review of applications of these methods and an inspection of the learned latent structure that they induce.


Modeling Hierarchical Structures with Continuous Recursive Neural Networks

Recursive Neural Networks (RvNNs), which compose sequences according to ...

Discrete Latent Structure in Neural Networks

Many types of data from fields including natural language processing, co...

Continual Lifelong Learning in Natural Language Processing: A Survey

Continual learning (CL) aims to enable information systems to learn from...

Towards Dynamic Computation Graphs via Sparse Latent Structure

Deep NLP models benefit from underlying structures in the data---e.g., p...

Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning

Latent structure models are a powerful tool for modeling language data: ...

Learning Rewards from Linguistic Feedback

We explore unconstrained natural language feedback as a learning signal ...

SparseMAP: Differentiable Sparse Structured Inference

Structured prediction requires searching over a combinatorial number of ...