Towards a Theoretical Understanding of Neural Networks

What kind of functions do neural networks learn? Why can neural networks perform well in high-dimensional settings? What is the right way to define the “norm” of a neural network?

This talk will answer these questions and provide mathematical explanations of existing design and training strategies that have evolved largely through experiment and practice.  This includes new insights into the importance of weight decay, linear layers, and skip connections, and a deeper understanding of sparsity and the curse of dimensionality, The theory also suggests new neural network architectures and regularization methods that may lead to improvements in practice. (Joint work with Rahul Parhi).