DeepAI AI Chat
Log In Sign Up

Statistically adaptive learning for a general class of cost functions (SA L-BFGS)

by   Stephen Purpura, et al.
Context Relevant, Inc.

We present a system that enables rapid model experimentation for tera-scale machine learning with trillions of non-zero features, billions of training examples, and millions of parameters. Our contribution to the literature is a new method (SA L-BFGS) for changing batch L-BFGS to perform in near real-time by using statistical tools to balance the contributions of previous weights, old training examples, and new training examples to achieve fast convergence with few iterations. The result is, to our knowledge, the most scalable and flexible linear learning system reported in the literature, beating standard practice with the current best system (Vowpal Wabbit and AllReduce). Using the KDD Cup 2012 data set from Tencent, Inc. we provide experimental results to verify the performance of this method.


page 1

page 2

page 3

page 4


A Reliable Effective Terascale Linear Learning System

We present a system and a set of techniques for learning linear predicto...

Demonstrating PAR4SEM - A Semantic Writing Aid with Adaptive Paraphrasing

In this paper, we present Par4Sem, a semantic writing aid tool based on ...

Learning When Training Data are Costly: The Effect of Class Distribution on Tree Induction

For large, real-world inductive learning problems, the number of trainin...

Using Deep Neural Network Approximate Bayesian Network

We present a new method to approximate posterior probabilities of Bayesi...

Learning from networked examples

Many machine learning algorithms are based on the assumption that traini...

Information Planning for Text Data

Information planning enables faster learning with fewer training example...

Efficient Model Finetuning for Text Classification via Data Filtering

As model finetuning is central to the modern NLP, we set to maximize its...