  Recently, neural tangent kernel (NTK) has been used to explain the dynamics
of learning parameters of neural networks, at the large width limit.
Quantitative analyses of NTK give rise to network widths that are often
impractical and incur high costs in time and energy in both training and
deployment. Using a matrix factorization technique, we show how to obtain
similar guarantees to those obtained by a prior analysis while reducing
training and inference resource costs. The importance of our result further
increases when the input points' data dimension is in the same order as the
number of input points. More generally, our work suggests how to analyze large
width networks in which dense linear layers are replaced with a low complexity
factorization, thus reducing the heavy dependence on the large width.
