-
Learning One-hidden-layer Neural Networks with Landscape Design
We consider the problem of learning a one-hidden-layer neural network: w...
read it
-
Learning Two-layer Neural Networks with Symmetric Inputs
We give a new algorithm for learning a two-layer neural network under a ...
read it
-
Supermodularity and valid inequalities for quadratic optimization with indicators
We study the minimization of a rank-one quadratic with indicators and sh...
read it
-
Learning Nonparametric Human Mesh Reconstruction from a Single Image without Ground Truth Meshes
Nonparametric approaches have shown promising results on reconstructing ...
read it
-
Learning ReLU Networks via Alternating Minimization
We propose and analyze a new family of algorithms for training neural ne...
read it
-
Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guarantees
Graph representation learning is a ubiquitous task in machine learning w...
read it
-
Independence Tests Without Ground Truth for Noisy Learners
Exact ground truth invariant polynomial systems can be written for arbit...
read it
Learning Two-Layer Residual Networks with Nonparametric Function Estimation by Convex Programming
We focus on learning a two-layer residual neural network with preactivation by ReLU (preReLU-TLRN): Suppose the input π± is from a distribution with support space β^d and the ground-truth generative model is a preReLU-TLRN, given by π² = B^β[(A^βπ±)^+ + π±], where ground-truth network parameters A^βββ^dΓ d is a nonnegative full-rank matrix and B^βββ^mΓ d is full-rank with m β₯ d. We design layerwise objectives as functionals whose analytic minimizers sufficiently express the exact ground-truth network in terms of its parameters and nonlinearities. Following this objective landscape, learning a preReLU-TLRN from finite samples can be formulated as convex programming with nonparametric function estimation: For each layer, we first formulate the corresponding empirical risk minimization (ERM) as convex quadratic programming (QP), then we show the solution space of the QP can be equivalently determined by a set of linear inequalities, which can then be efficiently solved by linear programming (LP). Experiments show the robustness and sample efficiency of our methods.
READ FULL TEXT
Comments
There are no comments yet.