Towards Shockingly Easy Structured Classification: A Search-based Probabilistic Online Learning Framework

03/29/2015
by   Xu Sun, et al.
0

There are two major approaches for structured classification. One is the probabilistic gradient-based methods such as conditional random fields (CRF), which has high accuracy but with drawbacks: slow training, and no support of search-based optimization (which is important in many cases). The other one is the search-based learning methods such as perceptrons and margin infused relaxed algorithm (MIRA), which have fast training but also with drawbacks: low accuracy, no probabilistic information, and non-convergence in real-world tasks. We propose a novel and "shockingly easy" solution, a search-based probabilistic online learning method, to address most of those issues. This method searches the output candidates, derives probabilities, and conduct efficient online learning. We show that this method is with fast training, support search-based optimization, very easy to implement, with top accuracy, with probabilities, and with theoretical guarantees of convergence. Experiments on well-known tasks show that our method has better accuracy than CRF and almost as fast training speed as perceptron and MIRA. Results also show that SAPO can easily beat the state-of-the-art systems on those highly-competitive tasks, achieving record-breaking accuracies.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/02/2017

A Generic Online Parallel Learning Framework for Large Margin Models

To speed up the training process, many existing systems use parallel tec...
research
12/22/2017

Adaptive Stochastic Dual Coordinate Ascent for Conditional Random Fields

This work investigates training Conditional Random Fields (CRF) by Stoch...
research
03/07/2020

RNN-based Online Learning: An Efficient First-Order Optimization Algorithm with a Convergence Guarantee

We investigate online nonlinear regression with continually running recu...
research
07/04/2021

Autoencoder based Randomized Learning of Feedforward Neural Networks for Regression

Feedforward neural networks are widely used as universal predictive mode...
research
03/16/2017

Efficient Online Learning for Optimizing Value of Information: Theory and Application to Interactive Troubleshooting

We consider the optimal value of information (VoI) problem, where the go...
research
03/06/2023

Online Learning and Optimization for Queues with Unknown Demand Curve and Service Distribution

We investigate an optimization problem in a queueing system where the se...
research
02/05/2021

Active Slices for Sliced Stein Discrepancy

Sliced Stein discrepancy (SSD) and its kernelized variants have demonstr...

Please sign up or login with your details

Forgot password? Click here to reset