In this Episode we talked about the deep neural networks and the spectral density of each layer's weights. It turns out, you can predict the accuracy ( and many more) with the WeigthWatcher application. We talk about the 5+1 Phases of learning and Heavy Tailed Self Regularization.

Charles Martin, PhD on LinkedIn:
https://www.linkedin.com/in/charlesmartin14/

During the episode we talked about these
VC Theory:
https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_theory

Why Deep Learning works? post by Charles Martin, 2015
https://calculatedcontent.com/2015/03/25/why-does-deep-learning-work/

Presentation at Berkeley: Why Deep Learning works? by Charles Martin, 2016
https://www.youtube.com/watch?v=fHZZgfVgC8U

Several Papers written by Charles Martin and Michael Mahoney:
https://arxiv.org/search/?query=%22Charles+H.+Martin%22&searchtype=author&abstracts=show&order=-announced_date_first&size=50

Newest blog post about weightwatcher, 2019 December:
https://calculatedcontent.com/2019/12/03/towards-a-new-theory-of-learning-statistical-mechanics-of-deep-neural-networks/

WeightWatcher on GitHub:
https://github.com/CalculatedContent/WeightWatcher
easy installation for python users:
pip install weightwatcher

How to reach out:
https://calculationconsulting.com/
charles@calculationconsulting.com

To access their slack channel please contact Charles first.

---Copyright Info---
Music is from https://filmmusic.io , intro first part is by Miklos Toth and some free garage band loops. :) intro second part: "Aces High" by Kevin MacLeod, outro "Acid Trumpet" by Kevin MacLeod (https://incompetech.com), License: CC BY (http://creativecommons.org/licenses/by/4.0/)