
Learning Onehiddenlayer Neural Networks with Landscape Design
We consider the problem of learning a onehiddenlayer neural network: w...
read it

Learning Twolayer Neural Networks with Symmetric Inputs
We give a new algorithm for learning a twolayer neural network under a ...
read it

Supermodularity and valid inequalities for quadratic optimization with indicators
We study the minimization of a rankone quadratic with indicators and sh...
read it

Learning Nonparametric Human Mesh Reconstruction from a Single Image without Ground Truth Meshes
Nonparametric approaches have shown promising results on reconstructing ...
read it

Learning ReLU Networks via Alternating Minimization
We propose and analyze a new family of algorithms for training neural ne...
read it

Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guarantees
Graph representation learning is a ubiquitous task in machine learning w...
read it

Independence Tests Without Ground Truth for Noisy Learners
Exact ground truth invariant polynomial systems can be written for arbit...
read it
Learning TwoLayer Residual Networks with Nonparametric Function Estimation by Convex Programming
We focus on learning a twolayer residual neural network with preactivation by ReLU (preReLUTLRN): Suppose the input π± is from a distribution with support space β^d and the groundtruth generative model is a preReLUTLRN, given by π² = B^β[(A^βπ±)^+ + π±], where groundtruth network parameters A^βββ^dΓ d is a nonnegative fullrank matrix and B^βββ^mΓ d is fullrank with m β₯ d. We design layerwise objectives as functionals whose analytic minimizers sufficiently express the exact groundtruth network in terms of its parameters and nonlinearities. Following this objective landscape, learning a preReLUTLRN from finite samples can be formulated as convex programming with nonparametric function estimation: For each layer, we first formulate the corresponding empirical risk minimization (ERM) as convex quadratic programming (QP), then we show the solution space of the QP can be equivalently determined by a set of linear inequalities, which can then be efficiently solved by linear programming (LP). Experiments show the robustness and sample efficiency of our methods.
READ FULL TEXT
Comments
There are no comments yet.