Deep networks can discover abstractions that shallow models miss because language and images are built from hierarchical parts, and depth lets networks recover coarse-grained variables and escape the curse of dimensionality

Why can deep networks discover abstractions that shallow models miss? Statistical physicist Matthieu Wyart joins Tim Scarfe to argue that the answer lies in the hidden hierarchy of data. Language and images are built from parts within parts; depth lets a network recover those coarse-grained variables and escape the curse of dimensionality.

https://podcasters.spotify.com/pod/show/machinelearningstreettalk/episodes/AI-Is-Learning-at-the-Wrong-Level-of-Abstraction--Matthieu-Wyart-e3n81u2