Les Houches Lectures on Deep Learning at Large & Infinite Width
Abstract
These lectures, presented at the 2022 Les Houches Summer School on Statistical Physics and Machine Learning, focus on the infinitewidth limit and largewidth regime of deep neural networks. Topics covered include various statistical and dynamical properties of these networks. In particular, the lecturers discuss properties of random deep neural networks; connections between trained deep neural networks, linear models, kernels, and Gaussian processes that arise in the infinitewidth limit; and perturbative and nonperturbative treatments of large but finitewidth networks, at initialization and after training.
 Publication:

arXiv eprints
 Pub Date:
 September 2023
 DOI:
 10.48550/arXiv.2309.01592
 arXiv:
 arXiv:2309.01592
 Bibcode:
 2023arXiv230901592B
 Keywords:

 Statistics  Machine Learning;
 Computer Science  Artificial Intelligence;
 Computer Science  Machine Learning;
 High Energy Physics  Theory;
 Mathematics  Probability
 EPrint:
 These are notes from lectures delivered by Yasaman Bahri and Boris Hanin at the 2022 Les Houches Summer School on Statistics Physics and Machine Learning and a first version of them were transcribed by Antonin Brossollet, Vittorio Erba, Christian Keup, Rosalba Pacelli, James B. Simon