Statistically adaptive learning for a general class of cost functions (SA L-BFGS)

08/31/2012
by   Stephen Purpura, et al.
0

We present a system that enables rapid model experimentation for tera-scale machine learning with trillions of non-zero features, billions of training examples, and millions of parameters. Our contribution to the literature is a new method (SA L-BFGS) for changing batch L-BFGS to perform in near real-time by using statistical tools to balance the contributions of previous weights, old training examples, and new training examples to achieve fast convergence with few iterations. The result is, to our knowledge, the most scalable and flexible linear learning system reported in the literature, beating standard practice with the current best system (Vowpal Wabbit and AllReduce). Using the KDD Cup 2012 data set from Tencent, Inc. we provide experimental results to verify the performance of this method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/19/2011

A Reliable Effective Terascale Linear Learning System

We present a system and a set of techniques for learning linear predicto...
research
08/21/2018

Demonstrating PAR4SEM - A Semantic Writing Aid with Adaptive Paraphrasing

In this paper, we present Par4Sem, a semantic writing aid tool based on ...
research
06/22/2011

Learning When Training Data are Costly: The Effect of Class Distribution on Tree Induction

For large, real-world inductive learning problems, the number of trainin...
research
12/31/2017

Using Deep Neural Network Approximate Bayesian Network

We present a new method to approximate posterior probabilities of Bayesi...
research
05/11/2014

Learning from networked examples

Many machine learning algorithms are based on the assumption that traini...
research
02/09/2018

Information Planning for Text Data

Information planning enables faster learning with fewer training example...
research
07/28/2022

Efficient Model Finetuning for Text Classification via Data Filtering

As model finetuning is central to the modern NLP, we set to maximize its...

Please sign up or login with your details

Forgot password? Click here to reset