Six Lectures on Linearized Neural Networks

08/25/2023
by   Theodor Misiakiewicz, et al.
0

In these six lectures, we examine what can be learnt about the behavior of multi-layer neural networks from the analysis of linear models. We first recall the correspondence between neural networks and linear models via the so-called lazy regime. We then review four models for linearized neural networks: linear regression with concentrated features, kernel ridge regression, random feature model and neural tangent model. Finally, we highlight the limitations of the linear theory and discuss how other approaches can overcome them.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset