Title: Gradient span algorithms make predictable progress in high dimension Authors:  Felix Benning - University of Mannheim (Germany) [presenting]
Leif Doering - University of Mannheim (Germany)
Abstract: The aim is to prove that all gradient span algorithms have asymptotically deterministic behavior on Gaussian random objective functions as the dimension tends to infinity. This result explains the counterintuitive phenomenon that different training runs of many large machine learning models result in approximately equal optimization-progress curves despite random initialization on a complicated non-convex landscape.