
QuasiNewton Optimization Methods For Deep Learning Applications
Deep learning algorithms often require solving a highly nonlinear and n...
read it

Newtonbased maximum likelihood estimation in nonlinear state space models
Maximum likelihood (ML) estimation using Newton's method in nonlinear st...
read it

A Fast Algorithm for Maximum Likelihood Estimation of Mixture Proportions Using Sequential Quadratic Programming
Maximum likelihood estimation of mixture proportions has a long history ...
read it

Faster independent component analysis by preconditioning with Hessian approximations
Independent Component Analysis (ICA) is a technique for unsupervised exp...
read it

Modeling Hessianvector products in nonlinear optimization: New Hessianfree methods
In this paper, we suggest two ways of calculating interpolation models f...
read it

Distance Majorization and Its Applications
The problem of minimizing a continuously differentiable convex function ...
read it

Fast Linear Convergence of Randomized BFGS
Since the late 1950's when quasiNewton methods first appeared, they hav...
read it
Accelerating likelihood optimization for ICA on real signals
We study optimization methods for solving the maximum likelihood formulation of independent component analysis (ICA). We consider both the the problem constrained to white signals and the unconstrained problem. The Hessian of the objective function is costly to compute, which renders Newton's method impractical for large data sets. Many algorithms proposed in the literature can be rewritten as quasiNewton methods, for which the Hessian approximation is cheap to compute. These algorithms are very fast on simulated data where the linear mixture assumption really holds. However, on real signals, we observe that their rate of convergence can be severely impaired. In this paper, we investigate the origins of this behavior, and show that the recently proposed Preconditioned ICA for Real Data (Picard) algorithm overcomes this issue on both constrained and unconstrained problems.
READ FULL TEXT
Comments
There are no comments yet.