On the validity of kernel approximations for orthogonally-initialized neural networks

04/13/2021

∙

by   James Martens, et al.

∙

0

∙

share

In this note we extend kernel function approximation results for neural networks with Gaussian-distributed weights to single-layer networks initialized using Haar-distributed random orthogonal matrices (with possible rescaling). This is accomplished using recent results from random matrix theory.