Risk and parameter convergence of logistic regression
share
The logistic loss is strictly convex and does not attain its infimum; consequently the solutions of logistic regression are in general off at infinity. This work provides a convergence analysis of gradient descent applied to logistic regression under no assumptions on the problem instance. Firstly, the risk is shown to converge at a rate O((t)^2/t). Secondly, the parameter convergence is characterized along a unique pair of complementary subspaces defined by the problem instance: one subspace along which strong convexity induces parameters to converge at rate O((t)^2/√(t)), and its orthogonal complement along which separability induces parameters to converge in direction at rate O((t) / (t)).
READ FULL TEXT