
Lower Bounds for Linear Decision Lists
We demonstrate a lower bound technique for linear decision lists, which ...
read it

Adaptive Margin Ranking Loss for Knowledge Graph Embeddings via a Correntropy Objective Function
Translationbased embedding models have gained significant attention in ...
read it

Understanding the Loss Surface of Neural Networks for Binary Classification
It is widely conjectured that the reason that training algorithms for ne...
read it

The Optimality of Polynomial Regression for Agnostic Learning under Gaussian Marginals
We study the problem of agnostic learning under the Gaussian distributio...
read it

Towards the optimal construction of a loss function without spurious local minima for solving quadratic equations
The problem of finding a vector x which obeys a set of quadratic equatio...
read it

On the Randomized Complexity of Minimizing a Convex Quadratic Function
Minimizing a convex, quadratic objective is a fundamental problem in mac...
read it

The satisfiability threshold for random linear equations
Let A be a random m× n matrix over the finite field F_q with precisely k...
read it
DistributionIndependent Evolvability of Linear Threshold Functions
Valiant's (2007) model of evolvability models the evolutionary process of acquiring useful functionality as a restricted form of learning from random examples. Linear threshold functions and their various subclasses, such as conjunctions and decision lists, play a fundamental role in learning theory and hence their evolvability has been the primary focus of research on Valiant's framework (2007). One of the main open problems regarding the model is whether conjunctions are evolvable distributionindependently (Feldman and Valiant, 2008). We show that the answer is negative. Our proof is based on a new combinatorial parameter of a concept class that lowerbounds the complexity of learning from correlations. We contrast the lower bound with a proof that linear threshold functions having a nonnegligible margin on the data points are evolvable distributionindependently via a simple mutation algorithm. Our algorithm relies on a nonlinear loss function being used to select the hypotheses instead of 01 loss in Valiant's (2007) original definition. The proof of evolvability requires that the loss function satisfies several mild conditions that are, for example, satisfied by the quadratic loss function studied in several other works (Michael, 2007; Feldman, 2009; Valiant, 2010). An important property of our evolution algorithm is monotonicity, that is the algorithm guarantees evolvability without any decreases in performance. Previously, monotone evolvability was only shown for conjunctions with quadratic loss (Feldman, 2009) or when the distribution on the domain is severely restricted (Michael, 2007; Feldman, 2009; Kanade et al., 2010)
READ FULL TEXT
Comments
There are no comments yet.