
The Ridge Path Estimator for Linear Instrumental Variables
This paper presents the asymptotic behavior of a linear instrumental var...
read it

Bayesian Regularization: From Tikhonov to Horseshoe
Bayesian regularization is a central tool in modernday statistical and ...
read it

The phase transition for the existence of the maximum likelihood estimate in highdimensional logistic regression
This paper rigorously establishes that the existence of the maximum like...
read it

Structure Learning in Inverse Ising Problems Using ℓ_2Regularized Linear Estimator
Inferring interaction parameters from observed data is a ubiquitous requ...
read it

A Maximum Entropy Procedure to Solve Likelihood Equations
In this article we provide initial findings regarding the problem of sol...
read it

On resampling methods for model assessment in penalized and unpenalized logistic regression
Penalized logistic regression methods are frequently used to investigate...
read it

Does generalization performance of l^q regularization learning depend on q? A negative example
l^qregularization has been demonstrated to be an attractive technique i...
read it
Takeuchi's Information Criteria as a form of Regularization
Takeuchi's Information Criteria (TIC) is a linearization of maximum likelihood estimator bias which shrinks the model parameters towards the maximum entropy distribution, even when the model is misspecified. In statistical machine learning, L_2 regularization (a.k.a. ridge regression) also introduces a parameterized bias term with the goal of minimizing outofsample entropy, but generally requires a numerical solver to find the regularization parameter. This paper presents a novel regularization approach based on TIC; the approach does not assume a data generation process and results in a higher entropy distribution through more efficient sample noise suppression. The resulting objective function can be directly minimized to estimate and select the best model, without the need to select a regularization parameter, as in ridge regression. Numerical results applied to a synthetic high dimensional dataset generated from a logistic regression model demonstrate superior model performance when using the TIC based regularization over a L_1 and a L_2 penalty term.
READ FULL TEXT
Comments
There are no comments yet.