文章基本信息

标题：Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions
本地全文：下载
作者：Ioannis Panageas ; Georgios Piliouras
期刊名称：LIPIcs : Leibniz International Proceedings in Informatics
电子版ISSN：1868-8969
出版年度：2017
卷号：67
页码：2:1-2:12
DOI：10.4230/LIPIcs.ITCS.2017.2
出版社：Schloss Dagstuhl -- Leibniz-Zentrum fuer Informatik
摘要：Given a twice continuously differentiable cost function f, we prove that the set of initial conditions so that gradient descent converges to saddle points where \nabla^2 f has at least one strictly negative eigenvalue, has (Lebesgue) measure zero, even for cost functions f with non-isolated critical points, answering an open question in [Lee, Simchowitz, Jordan, Recht, COLT 2016]. Moreover, this result extends to forward-invariant convex subspaces, allowing for weak (non-globally Lipschitz) smoothness assumptions. Finally, we produce an upper bound on the allowable step-size.
关键词：Gradient Descent; Center-stable manifold; Saddle points; Hessian